A BatonX pilot aims to evaluate how a coding agent works within your organisation’s boundaries during a real development workflow.
Keep the initial scope small and measurable: start with one representative repository, a limited group of developers, an approved model endpoint and clear success criteria.
This guide covers preparation before the pilot. Features and integrations supported by the selected BatonX version should be confirmed when agreeing the pilot scope.
1. Choose a representative project
Select a repository that reflects everyday development work without requiring access to critical production systems for the initial evaluation.
Document its setup steps, dependencies and test commands. Choose clearly bounded tasks, such as fixing a bug, implementing a small feature or adding tests.
Prepare:
- The pilot repository and access permissions.
- The working branch and code review process.
- Required package sources and development services.
- Sample configuration and test data with sensitive information removed.
Do not assume production credentials or real customer data are necessary to get started.
2. Identify an approved model endpoint
Establish which model provider and endpoint the pilot agent will use. This may be an organisation-approved provider service or a private model endpoint supported by the pilot version.
Agree the model identifier, access method, data-handling conditions and usage budget in advance. Validate a private endpoint against the agent’s required capabilities, including tool calling, rather than relying on API compatibility alone.
Decide who will provision provider credentials, how they will be protected and how they can be revoked. Do not make automatic fallback to an unapproved model or provider part of the default pilot behaviour.
3. Define the developer workflow
The pilot should be realistic enough to evaluate the developer’s everyday working experience.
Establish a simple workflow:
- The developer starts a session for an authorised project.
- They give the agent a clearly scoped task.
- The agent operates within defined permissions and required approvals.
- The developer reviews changes and test results.
- The developer submits the changes through the existing code review process.
Agree which actions are permitted, which require approval and which are prohibited. Explain where developers should seek help when an action is blocked.
4. Prepare the environment and access boundaries
Define the workspace the agent may access, the operations it may perform and the destinations it may connect to.
Alongside model connectivity, consider dependency downloads, test execution and access to development services. Identify required connections individually rather than enabling unrestricted access to resolve setup issues.
Plan to validate boundaries covering:
- Unauthorised repositories and files outside the permitted workspace.
- Disallowed commands and network connections.
- Access to credentials.
- Session termination and access revocation.
5. Clarify operational responsibilities
For a customer-managed pilot, ownership of the hosting environment and day-to-day operations must be clear.
Assign customer-side owners for infrastructure, security and developer coordination. Separately agree the scope of setup assistance, troubleshooting and updates with the BatonX team.
Before starting, answer these questions:
- Who approves user and repository access?
- Who manages model credentials and usage budgets?
- Who authorises policy changes?
- Who reviews audit records, and how long are they retained?
- Who can stop a session if a problem occurs?
- How will access, workspaces and records be handled when the pilot ends?
6. Agree success criteria in advance
Do not measure success solely by the amount of code produced.
Evaluate completion of the selected development tasks alongside enforcement of access boundaries, approval handling, visibility into model usage and clear reporting of problems.
Record unnecessary interruptions to the developer workflow as well. At the end of the pilot, you should be able to identify which controls were validated, which limitations remain and what is required for the next stage.