AI governance for developers and configurators
This page explains how Stack9 AI Agents are governed in production, written for the developers who integrate them and the AI Agent configurators who tune them in Stack9 AI Studio. It covers the design positions and controls you will work within, and why they exist.
Governance sits inside April9's ISO 27001-certified Information Security Management System and a documented AI Policy. The policy's principles include human-in-the-loop oversight of all AI-assisted outputs and named ownership for every AI system in scope.
No training or fine-tuning on client data
April9 does not train or fine-tune any model on client data. This is a deliberate design position, not a gap:
A fine-tuned model absorbs the client's content into weights that cannot be cited, cannot be version-stamped against a change to the source, and cannot be corrected without retraining. Client content is instead what the agents read at the moment a question is asked, so every answer reflects the source as it currently stands and cites the provision and version it used.
What follows from it:
- Client data enters no training path. Foundation models are consumed as a service. Capability comes from agent tools, skills and permissioned retrieval at the time of the enquiry.
- Isolation. The client's records stay in the isolated AWS environment provisioned for that client, are not pooled with any other client's data, and are not used to improve any service offered to another party.
- Customised, not trained. What is tuned is agent behaviour (prompts, tool access, skills, retrieval scope, guardrails) against an evaluation set built on the client's own material.
- Currency is automatic. Live sources are queried at the time of the enquiry; document content is re-indexed and version-stamped, with superseded versions retained under audit.
Every agent change is a Request for Change
The versioned agent configuration is the unit of change. Every change to an agent (system prompt, skills, tool and MCP grants, retrieval scope, guardrails, model version) is raised as a Request for Change (RFC), scored against the client-approved baseline evaluation sets, and approved by the client's named approver. AI configuration is not exempt from change control, and once in service the change also flows through the client's own change process.
For developers this means:
- MCP grants are part of the agent's reviewable scope. Adding a tool to an MCP config widens what an agent can do (for the users authorised to run that tool), so treat it as an agent change, not just a config edit.
- Runtime enforcement uses Amazon Bedrock AgentCore Policies, Identity, Guardrails and Evaluations: permission scoping per agent, guardrails on prohibited and out-of-scope output, and the evaluation threshold gating promotion.
- Every agent has a named owner, a documented purpose, a defined set of permitted sources and an approved use boundary.
- Decision authority stays with the client. A back-office user approves every draft before it has any effect, and the client's system of record remains the system of record.
Model pinning and regression scoring
The foundation model behind each agent is pinned to a version. A new version, including one forced by a provider deprecation, is treated as a change and regression-scored against the baseline sets before adoption. The client is never moved to a new model silently.
Named risks and their controls
| Risk | Control |
|---|---|
| Wrong or unsupported answer | Agents answer only from approved sources and decline and escalate where those sources do not support an answer. A separate verification agent checks the assembled answer against what was retrieved |
| Drift after a source change | Baseline evaluation sets are re-run on every content refresh, so the effect is measured before release |
| Unannounced model change | Model pinned; new versions regression-scored before adoption |
| Inequitable service | Consistency across phrasing and location is a release gate: a materially different answer to a materially equivalent question is treated as a defect |
| Data leakage | Per-client isolated environment; no training path; AI processing in the Australian region |
| Over-reliance by staff | Override and disagreement rates on AI drafts are tracked. Staff ceasing to override is a signal to investigate, not a success |
No demographic attribute reaches any agent, and no person is scored, ranked, approved or refused by a model.
What is captured per interaction
Every prompt and response (for end users, back-office assistance and agent-to-agent steps alike) is captured as a record, not a log line:
| Captured | Why |
|---|---|
| Prompt and response | What was asked and what was said |
| User or anonymous session identifier | Who it was for |
| Timestamp | When |
| Agent version and model version | Which configuration and model produced it |
| Each function and tool call, with its result | What the agent did |
| Sources retrieved, with their versions | What the answer rested on |
| Any refusal or escalation | Where the agent declined |
| Correlation identifier | Links the interaction to the business record and the integration calls behind it |
Decisions are captured in three layers: the agent's (which skill, which sources, where it declined), the person's (where a back-office user accepts, edits or overrides a draft, captured against the same record), and the configuration's (each RFC, the evaluation result that gated it, the named approver). The configuration in force (agent version, system prompt, skills, tool and source grants) is retained as versioned artefacts alongside the interactions it produced, so a past answer is examined against the instructions actually in force at the time.
These records are domain entities, so each carries the native Stack9 audit trail: versioned, user- and time-stamped, with removal by flag under audit rather than destruction. See Entities. The client can query and export them at any time, and chooses at design whether they are retained in the Australian hosting environment or streamed to the client's own systems (for example a SIEM).
On the Stack9 Core side, MCP tool calls add their own trail: the endpoint logs refused tool calls and withheld tools with the MCP key, tool key and user id, and automations run as the calling user. See MCP Server.
Explainability, per audience
| Audience | What they see |
|---|---|
| End user | The source behind every answer, linked and version-stamped. Where live data was used: what was retrieved, from which service, and when |
| Back-office user | A draft as a structured chain (provision applied, record data matched, resulting finding) alongside the source text, agreed or overridden finding by finding. Behind any finding: the agent's reasoning, tool calls and evidence, with the option to question the responsible agent |
| Administrators and assurance staff | The full execution record: each request and response, tools called, identity used, agent-to-agent calls. For any past answer, the agent version, system prompt, skills, permitted tools and sources, and model version that produced it |
Any interaction can be replayed step by step in Stack9 AI Studio.
The AI lifecycle: five loops
None of these loops involves training or retraining a model.
| Loop | Cadence and control |
|---|---|
| Content | Live sources need no refresh. Document content is re-indexed and version-stamped on schedule and on change, with superseded versions retained. A source change triggers a baseline re-run |
| Configuration | The versioned agent configuration is the unit of change. Every change is an RFC, re-scored before release |
| Model | Pinned version. New versions regression-scored before adoption |
| Monitoring | Per-agent telemetry: accuracy against the baseline sets, citation coverage, retrieval success, refusal and escalation rates, latency and cost per interaction. Reported per agent, not blended into solution-wide metrics. Re-evaluated on every content refresh and at least monthly |
| Feedback | Staff accept, edit and override patterns are reviewed with the client and fed into the configuration loop. New question types are added to the baseline sets with approval. User ratings are written against the specific interaction |
Guardrails
Agents run under Amazon Bedrock AgentCore Guardrails, which block categories of harmful content regardless of how a request is framed:
| Category | Examples |
|---|---|
| Harmful content | Hate, harassment, self-harm, violent or sexual content |
| Manipulation | Manipulation and prompt-injection attempts |
| Out-of-scope advice | Advice outside the agent's approved use boundary; an agent stays scoped to its purpose and does not attempt counselling or medical advice |
| Crisis signals | Where a session shows signals of self-harm or crisis, the agent can stop engaging on the substance, surface a crisis-support reference and hand off to the client's contact channel flagged as urgent |
Adversarial and over-reliance scenarios run alongside the accuracy benchmark as part of the same release gate, so guardrails are tested, not only configured.
Limits
- ISO 42001 is a target (Q2 2027), not held.
- No external AI-ethics reviewer is claimed. Governance is April9's own, under its ISMS and AI Policy.
- Capture at this depth has a real storage cost. Prompt-and-response capture over a multi-year retention obligation is a material, compounding cost that must be planned for, not assumed away.
- Fairness cohorts, scenario sets and thresholds are set per engagement. There is no pre-built universal benchmark, and evaluation datasets are authored per client.
- AI answer latency can exceed transactional targets. Responses are streamed and AI timings are reported separately.
- A draft finding is never a decision.
Related
- Stack9 AI Agents
- Stack9 AI Studio
- MCP Server: per-tool authorisation on the Stack9 Core side
- Release and upgrade model: how the same RFC process governs platform releases