Governance & Oversight
Periodic: weekly, monthly, quarterlyPeriodic oversight, analysis and improvement of the system.
Costs (FinOps)
FinOps: managing and optimising the cost of AI infrastructure.
- Tracking API and token costs by agent and department (token costs: the cost of processing requests to AI models)
- Alerts on abnormal spikes in usage
- Monthly report with optimisation recommendations
- Caching recommendations
- Choosing a suitable model for simpler tasks
Logs and data
- Weekly or monthly review of audit logs
- Checking what data has passed through the agents
- Checking for unmasked personal and financial data
- Checking whether the agent followed the defined sequence of steps
Rules and logic
- Prompt updates (the agent’s instructions)
- Rule updates
- State machine updates: the defined sequence of steps and checks
Reviews and documentation
- Quarterly health check
- Tests with edge and unusual cases
- Compliance readiness: preparation for questions from clients, partners and auditors
- Documentation for a client, partner or audit
