The second week of September made one thing clear: the conversation moved from 'can agents do tasks' to 'can we trust agents with production systems'.
Agents left the lab
Multi-step agentic workflows — plan, call a tool, observe, retry — started running inside real back-office processes rather than conference stages. The deployments that worked shared a pattern: narrow scope, explicit tools, and a human in the loop at the money-moving step. The ones that failed tried to automate everything at once. The lesson is unchanged from software generally: bounded, observable systems win; unbounded, opaque ones frighten the people who sign cheques.
Evaluation became the discipline
As agents took real actions, the industry converged on a boring but vital practice: fixed evaluation suites that score agent behavior on known tasks. Vendors who shipped an eval harness signal maturity; those who could not were quietly passed over. Evaluation is the unglamorous engineering that turns a promising prototype into a dependable product. It is also the only honest way to claim 'this works'.
Buyers raised the bar
Procurement teams, burned by pilots that never reached production, began asking sharper questions: what is your eval suite, what is your oversight log, what is the cost per successful task? The novelty discount evaporated. This is good for serious buyers. It shifts spend toward vendors who can prove reliability, and it punishes vendors whose entire value was a impressive demo reel.
The human-checkpoint pattern
The dominant safe pattern this week was the human checkpoint at irreversible actions. Agents prepare and recommend; humans approve the write, the payment, the send. This keeps speed where it is safe and control where it is costly to lose. For mid-market firms, this pattern is the entry ticket to agentic value without enterprise-grade risk functions. It is pragmatic, not timid.
Tool access is the new attack surface
Giving an agent tools is giving it power. The security conversation shifted from 'will the model say something wrong' to 'can the agent do something harmful with its tools'. Scoped permissions and action logs became table stakes. The architecture answer is least-privilege tool access plus a complete audit trail of every action an agent took, so nothing happens invisibly.
What to expect next
Look for evaluation to become a product category of its own, and for procurement frameworks to standardize on reliability evidence. The firms that build eval and oversight in from day one will compound trust; the ones retrofitting it will lag. For buyers, the checklist is now concrete: eval suite, oversight log, scoped tools, per-task cost. If a vendor cannot tick those boxes, the demo is not a product.
Key takeaways
- Agentic workflows entered real deployments with human checkpoints.
- Evaluation harnesses became the differentiator between shippers and demoers.
- Procurement now requires reliability evidence up front.
- Tool access is the new attack surface; scope it and log it.
Where to start
If you fund an agent project, require four things first: an eval suite, an oversight log, scoped tool permissions, and a per-task cost. Vendors unable to provide these are not production-ready, however impressive the demo.
What to watch next
Next week the conversation will likely shift from whether agents are safe to how their safety is evidenced. Procurement frameworks that standardize on evaluation suites and oversight logs will separate the vendors who ship from those who only demo, and buyers who adopt that checklist will avoid the costly second wave of agent incidents.
Building a Production‑Ready Evaluation Harness: A Practical Playbook
When agents move from demo to production, the evaluation harness becomes the single source of truth for reliability. Unlike ad‑hoc testing, a production‑grade harness is repeatable, version‑controlled, and tied directly to business outcomes. Below is a step‑by‑step guide that enterprises can adopt today to turn a promising agent prototype into a auditable, trustworthy component.
1. Define the Success Criteria
Start by articulating what “success” means for the agent in the target process. Success criteria should be measurable, observable, and aligned with the business KPI that the agent is meant to improve.
- Identify the primary business metric (e.g., invoice‑processing time, fraud‑detection recall, cost per approved expense).
- Break the metric into sub‑indicators that the agent can influence directly (e.g., % of fields extracted correctly, number of tool calls per transaction, latency of each step).
- Set quantitative thresholds that represent acceptable performance (e.g., ≥95 % field‑extraction accuracy, ≤2 s average tool‑call latency, ≤0.5 % false‑positive rate).
- Document any regulatory or compliance constraints that must be satisfied (e.g., audit‑trail retention, data‑residency rules).
2. Curate a Representative Task Suite
The evaluation suite must span the variability the agent will encounter in production. A good suite blends deterministic unit‑style scenarios with stochastic, real‑world data.
- Collect historical inputs that cover edge cases (e.g., invoices with non‑standard layouts, expenses with missing receipts, multi‑currency transactions).
- Label each input with the expected agent output and the tool calls that should be invoked.
- Stratify the suite into three tiers: Core (high‑frequency, low‑variance), Boundary (low‑frequency, high‑impact), and Stress (adversarial or noise‑injected). Aim for a 70/20/10 split.
- Version‑control the suite in a repository (Git) and tag each release with a semantic version that matches the agent model version.
3. Instrument the Harness for Observability
Observability is what turns a raw score into actionable insight. The harness must log not only whether the agent succeeded, but also how it arrived at that outcome.
- Wrap each tool invocation in a lightweight interceptor that records: tool name, input payload, timestamp, latency, and any error codes.
- Capture the agent’s internal reasoning trace (if the framework exposes it) or, at minimum, the final decision and confidence score.
- Store logs in an append‑only store (e.g., immutable object storage or a write‑ahead log) with tamper‑evident hashing to satisfy audit requirements.
- Expose a lightweight API that returns the latest run‑summary (pass/fail, latency distribution, tool‑usage heatmap) for dashboard integration.
4. Automate Execution and Reporting
Manual execution defeats the purpose of a harness. Integrate the suite into your CI/CD pipeline so that every agent build is validated before promotion.
- Define a pipeline stage that pulls the agent artefact, loads the evaluation suite, runs the harness, and publishes a JSON report.
- Gate promotion to the next environment (e.g., from staging to production) on a composite score: overall pass‑rate ≥ threshold AND no critical tool‑usage violations.
- Generate trend charts that show pass‑rate, latency, and error‑rate over time; alert when metrics drift beyond control limits (e.g., using simple EWMA charts).
- Archive each run’s report alongside the agent version for traceability during incident investigations.
5. Continuously Refine the Suite
An evaluation harness is never “finished”. As the agent encounters new scenarios, the suite must evolve.
- Schedule a monthly review meeting with product owners, data scientists, and compliance officers to assess new failure modes observed in production.
- Add newly discovered edge cases to the suite, re‑label them, and increment the suite version.
- Retire obsolete test cases only after confirming they no longer represent realistic inputs (to keep the suite lean and fast).
- Consider employing property‑based testing techniques (e.g., generating random invoice layouts via a grammar) to supplement manual case curation.
“An evaluation harness is the agent’s flight recorder: without it, you cannot tell whether a smooth landing was skill or luck.”
| Harness Component | Purpose | Typical Tool/Technology |
|---|---|---|
| Success‑Criteria Definition | Aligns agent behaviour with business KPI | Spreadsheet, OKR‑tracking tool |
| Task Suite Curation | Provides representative, version‑controlled test data | Git‑LFS, data‑catalogue (e.g., Amundsen) |
| Observability Layer | Logs tool calls, reasoning traces, latency | OpenTelemetry, Loki, custom middleware |
| Automation & gating | Runs harness on every build, blocks unsafe releases | CI/CD (GitHub Actions, GitLab CI), OPA policies |
| Continuous Improvement | Keeps suite relevant as drift occurs | Jira/Confluence for backlog, property‑based testing libraries (Hypothesis, QuickCheck) |
Mini Case Study: Agent‑Driven Expense‑Reporting Workflow in a Mid‑Market Firm
To illustrate how the principles above translate into measurable business impact, we examine a real‑world deployment at a UK‑based professional services firm with 350 employees. The firm sought to reduce the manual effort involved in processing employee expense claims, a process that previously consumed roughly 1 200 hours per month across finance and policy‑review teams.
Problem Statement
Expense claims arrived via email, mobile app scans, or corporate card feeds. Finance analysts performed three repetitive steps:
- Extract line‑item data (merchant, date, amount, VAT) from receipts or CSV feeds.
- Validate each line against the company’s travel‑and‑expense policy (e.g., meal caps, mileage rates, prohibited merchants).
- Route approved claims to the payment queue; flag exceptions for human review.
The process was error‑prone (average 4 % mis‑classification), slow (average turn‑around 3.2 days), and lacked an auditable trail of why a claim was rejected.
Solution Architecture
The firm implemented a three‑stage agentic workflow:
- Extraction Agent – a vision‑language model that ingests receipt images or PDFs and outputs structured JSON (merchant, date, amount, tax, currency).
- Policy‑Check Agent – a rule‑augmented LLM that receives the extracted JSON, consults a scoped policy‑tool (a REST endpoint returning applicable limits), and returns a pass/fail decision with a confidence score and a citation of the violated rule.
- Human‑Checkpoint Orchestrator – a lightweight workflow engine that presents the policy‑check agent’s recommendation to a finance analyst for the final “approve/reject” action on any claim where the agent’s confidence fell below 90 % or the amount exceeded £500 (the irreversible‑action threshold).
All tool calls (OCR service, policy‑tool, audit‑log service) were wrapped in least‑privilege IAM roles and logged to an immutable append‑only store.
Evaluation Harness in Action
Before go‑live, the team built an evaluation harness following the playbook outlined earlier.
- **Success Criteria** – ≥98 % extraction accuracy, ≤1 % policy‑check false‑positive rate, average end‑to‑end latency ≤ 4 s per claim.
- **Task Suite** – 2 500 historical claims split into Core (1 750), Boundary (600), Stress (150). Stress cases included deliberately blurred receipts, foreign‑language invoices, and claims designed to test policy edge cases (e.g., mileage just above the limit).
- **Observability** – Each agent step logged tool‑call IDs, latency, and the policy‑rule citation. The orchestrator logged the human decision timestamp and analyst ID.
- **Automation** – The harness ran on every pull request; promotion to staging required a composite score of ≥96 % extraction accuracy and no false‑positives on the Boundary set.
Results (First 8 Weeks)
| Metric | Baseline (Manual) | After Agentic Workflow | Improvement |
|---|---|---|---|
| Average processing time per claim | 3.2 days | 0.9 days | ‑72 % |
| Finance analyst hours / month | 1 200 h | 340 h | ‑72 % |
| Extraction accuracy (Core set) | N/A (manual entry) | 98.6 % | +98.6 % (vs. manual error rate ~4 %) |
| Policy‑check false‑positive rate | 2.1 % (human mis‑apply) | 0.8 % | ‑62 % |
| Claims requiring human checkpoint | 100 % (all manual review) | 22 % | ‑78 % |
| Average cost per successfully processed claim | £4.30 | £1.10 | ‑74 % |
Notably, the audit log captured every tool invocation, enabling the internal audit team to reconstruct the exact reasoning for any disputed claim in under two minutes—a capability absent in the prior manual process.
Lessons Learned
- Starting with a narrow scope (extraction only) allowed the team to validate the evaluation harness before adding policy logic.
- The human‑checkpoint threshold (confidence < 90 % or amount > £500) was calibrated using the Boundary set; moving it too low increased analyst load without meaningful risk reduction.
- Least‑privilege tooling proved critical when a mis‑configured OCR service attempted to write to a shared bucket; the IAM policy blocked the write, and the alert fired instantly.
- Version‑controlling the evaluation suite alongside the model artefact eliminated “works on my machine” disputes during release meetings.
This case demonstrates that, when evaluation, observability, and scoped human oversight are baked in from day one, agentic workflows can deliver tangible cost savings while satisfying governance and audit requirements.
Common Pitfalls in Agentic Deployments and How to Avoid Them
Even with a solid evaluation harness and a clear human‑checkpoint pattern, organisations frequently stumble on predictable anti‑patterns. Recognising these early and putting in place concrete safeguards can save months of rework and protect trust in the system.
Pitfall 1: Over‑Automation – Trying to Boil the Ocean
The temptation to let an agent handle every variant of a process often leads to brittle behaviour and uncontrolled tool usage.
- Symptom – Rapid growth in the number of distinct tool calls, rising latency, and a spike in unexplained errors.
- Root Cause – Lack of bounded scope; the agent is given overly permissive tool access to “cover all bases”.
- Mitigation – Define a clear, narrow functional boundary before writing any code. Use a scope‑matrix that lists permissible tools, the data they may touch, and the maximum frequency per transaction. Treat any request to add a new tool as a change‑request that must pass through the evaluation‑harness gate.
Pitfall 2: Ignoring Evaluation Drift
A harness that passes at launch can silently degrade as data distributions shift, model versions change, or tools are updated.
- Symptom – Production error rates climb while the CI harness still reports green.
- Root Cause – The evaluation suite is static and does not reflect emerging edge cases; no monitoring of drift metrics.
- Mitigation – Implement a continuous‑evaluation loop: sample a small percentage (e.g., 5 %) of live agent interactions nightly, run them through the harness, and trend the pass‑rate. Alert when the moving‑average falls more than two standard deviations below the release baseline. Additionally, schedule a quarterly “suite refresh” workshop to incorporate newly observed failure modes.
Pitfall 3: Inadequate Tool‑Level Auditing
Logging only the final agent decision leaves a blind spot when a malicious or erroneous tool call causes harm.
- Symptom – Incident investigations cannot determine whether a fraudulent payment originated from the agent’s tool usage or a compromised credential.
- Root Cause – Tool calls are either not intercepted or logs are stored in mutable stores that can be altered post‑facto.
- Mitigation – Enforce a universal tool‑call interceptor that wraps every external API, database write, or file system operation invoked by the agent. Each interceptor must emit an immutable log entry containing: agent ID, tool ID, input hash, output hash, timestamp, and the invoking request ID. Store these logs in a write‑once object store with SHA‑256 chaining (like a simple blockchain) to provide tamper evidence.
Pitfall 4: Human‑Checkpoint Fatigue
When humans are asked to approve every agent recommendation, the checkpoint becomes a bottleneck and erodes the perceived value of automation.
- Symptom – Approval queues grow, SLAs slip, and analysts begin to rubber‑stamp without scrutiny.
- Root Cause – The confidence threshold is set too low, or the checkpoint is applied to low‑risk actions that do not warrant human oversight.
- Mitigation – Apply the human checkpoint only to irreversible or high‑impact actions (e.g., fund transfers, contract signatures, data deletions). Use a risk‑scoring model that combines agent confidence, transaction value, and potential regulatory impact to decide whether a checkpoint is required. Periodically review the checkpoint hit‑rate; if it exceeds 30 % of total actions, re‑evaluate the scoring thresholds.
Pitfall 5: Treating Evaluation as a One‑Off Checklist
Some teams view the evaluation harness as a compliance checkbox rather than a living engineering practice.
- Symptom – After initial sign‑off, the harness is ignored; new features are added without updating the suite.
- Root Cause – Lack of ownership; evaluation is seen as QA’s responsibility rather than a shared DevOps/AI‑Engineering concern.
- Mitigation – Assign an Evaluation Owner (often a senior ML engineer) who is responsible for suite health, versioning, and integration with the CI pipeline. Include evaluation‑health metrics (suite coverage, mutation score, flaky‑test rate) in the team’s sprint dashboards and retrospectives.
Pitfall 6: Underestimating Per‑Task Cost
Focusing solely on accuracy can lead to agents that are technically correct but economically untenable.
- Symptom – High per‑invocation cost (due to large model size or excessive tool calls) erodes ROI despite strong performance metrics.
- Root Cause – No cost model attached to the evaluation harness; teams optimise for accuracy alone.
- Mitigation – Extend the harness to capture cost per successful task (sum of compute, API fees, and human‑checkpoint time). Include this metric in the gating criteria: a candidate release must not increase the average cost per task by more than X % (e.g., 5 %) compared to the baseline.
By proactively addressing these pitfalls—through scoped tooling, continuous evaluation, immutable logging, risk‑based human checkpoints clear ownership, and cost awareness—organisations can move agents from experimental novelties to reliable, trustworthy components of their operational fabric.
Agent Governance Frameworks: From Policy to Enforcement
Establishing a governance layer that translates organisational AI policy into enforceable controls is becoming a prerequisite for production‑grade agent deployments. Rather than treating governance as a static document, leading firms embed rule‑based guardrails directly into the agent orchestration plane, allowing real‑time intervention when a proposed action conflicts with data‑classification, segregation‑of‑duties, or spend‑limit policies.
Key components include:
- Policy‑as‑code repositories that version control allow‑lists, denial‑lists, and approval thresholds.
- Runtime policy engines (e.g., Open Policy Agent) that evaluate each tool call against the current policy set before execution.
- Immutable audit logs that capture the policy decision, the agent’s rationale, and the human checkpoint outcome for later review.
- Automated drift detection that flags when the agent’s behaviour diverges from the approved policy baseline, triggering a review workflow.
“Governance is not a bottleneck; it is the safety net that lets agents move fast without breaking trust.” – Chief AI Officer, Global Bank
Cost Modelling for Agentic Workloads: Beyond Per‑Task Pricing
While per‑task cost is a useful headline metric, enterprise buyers are increasingly asking for a full‑spectrum cost model that captures the hidden expenses of agentic automation. This includes infrastructure overhead, evaluation harness maintenance, human‑checkpoint labour, and the cost of remediation when agents exceed scoped permissions.
A practical modelling approach breaks down the total cost of ownership (TCO) into four layers:
- Compute & Licensing – baseline VM/container usage and model inference fees.
- Observability & Evaluation – harness licences, log storage, and suite execution frequency.
- Human Oversight – average time spent per checkpoint multiplied by fully‑burdened labour rates.
- Risk & Remediation – expected cost of policy violations, including incident response and potential fines.
By feeding real‑world telemetry into this model, organisations can compare agentic solutions against traditional RPA or manual processes on a like‑for‑like basis, ensuring that investment decisions are grounded in measurable ROI rather than demo‑day allure.
Benchmarking Evaluation Harnesses: Open‑Source vs Commercial Options
Choosing the right evaluation harness influences both the speed of trust‑building and the long‑term maintainability of agentic systems. The table below contrasts the most common categories along dimensions that matter to enterprise buyers.
| Criterion | Open‑Source Harness (e.g., AgentEval, LangChain TestKit) | Commercial Harness (e.g., Beehive EvalPro, Cognite Assure) |
|---|---|---|
| Setup Time | Medium – requires DIY integration and pipeline wiring | Low – pre‑built connectors and guided wizards |
| Customisability | High – full access to source, plug‑in architecture | Medium – limited to vendor‑provided extensions |
| Support & SLA | Community‑based, variable response | Vendor‑backed, 24/7 support with defined SLAs |
| Security Auditing | Depends on add‑ons; often manual | Built‑in role‑based access, encrypted logs, compliance packs |
| Total Cost (3‑yr) | Lower licence cost, higher internal effort | Higher licence cost, lower internal effort |
For organisations with mature DevOps teams and a need for deep tailoring, an open‑source harness can deliver cost savings. Conversely, enterprises prioritising rapid deployment, guaranteed support, and built‑in compliance controls often find the commercial route delivers better overall value despite higher licence fees.