Coding agents went from curiosity to default in under two years. Every engineering team we work with now runs them; almost none can answer the CFO's question: what do we get back? The vendors quote 55% faster task completion. The ledger in most organisations shows something murkier - real gains in some lanes, and a slow accumulation of review debt and security exposure in others. Both are true. The skill is telling them apart.
The hype vs the ledger
The headline studies measure task completion speed in controlled settings: greenfield features, isolated functions, developers who know they are being observed. Production engineering is mostly none of those things. It is reading code nobody remembers writing, understanding why a workaround exists, and changing a system without breaking the three teams downstream. When we instrument real teams, the pattern is consistent: agents compress the writing of code dramatically and compress the understanding of code not at all - and understanding was most of the job.
Where the ROI is real
The gains are genuine and large in specific lanes. After a year of deployments, these consistently clear the bar:
- 01Greenfield scaffolding and prototypesNew services, CRUD APIs, UI shells, migration scripts. Agents are superb at code with no history. Teams report 3-5x on this lane and we believe them.
- 02Test generation and coverage backfillWriting tests for existing code is unglamorous, well-specified and endlessly postponable - exactly the work agents finish. Coverage gains of 20-40 points in a quarter are common.
- 03Mechanical migrationsFramework upgrades, dependency bumps, API-version churn. Repetitive, verifiable, and previously a morale tax on senior engineers.
- 04Documentation and code explanationOnboarding time drops measurably when an agent can answer 'what does this module do and why' against the actual repository.
- 05Incident forensicsSearching logs, diffs and deploy history to shortlist causes. The agent does not fix the incident; it gets the human to the right file in minutes instead of hours.
Agents compress the writing of code, not the understanding of it - and in production systems, understanding was most of the job.
Where it is a mirage
The mirage lanes look identical in the demo. Deep changes to legacy cores, where the agent lacks the institutional memory of why the code is weird. Cross-cutting refactors, where the blast radius spans teams and the agent cannot attend the meetings. Security-sensitive paths - authentication, payments, data access - where "looks right" is not a verification standard. And anything where the specification is the hard part: an agent accelerates a clear spec enormously and a vague one not at all, because the bottleneck was never typing. Teams that deploy agents broadly and measure nothing discover this a year later as a quality trend nobody can explain.
The review-debt problem
The quietest cost is review debt. Agents raise pull requests faster than humans can thoughtfully review them, and the review is where understanding was traditionally transferred. Two failure patterns follow. First, rubber-stamping: reviewers overwhelmed by volume start approving agent code on skim, and defects per release climb. Second, skill atrophy: junior engineers who orchestrate agents brilliantly but cannot debug the output when it fails in production at 2am. The countermeasures are unglamorous: agent-authored PRs get stricter review, not looser; agents write the tests that prove their own changes; and juniors rotate through agent-free debugging duty. The security dimension - agents that ingest untrusted repository content - deserves the same threat model we apply in our prompt-injection field report.
The measurement framework
"Feels faster" is not a number and vendor benchmarks are not your codebase. The framework we install measures four things, per team, per quarter: cycle time from ticket to merge (the real speed metric), change-failure rate and mean time to restore (the quality counterweight), review load per engineer (the hidden cost), and agent spend per merged PR (the unit economics, using the tagging discipline from our AI FinOps report). A healthy deployment shows cycle time down, failure rate flat or down, review load manageable. If speed is up and failures are up, you have not bought productivity - you have bought future incidents, on credit.
The rollout that works
The orgs getting durable returns share a rollout shape. They start with the high-ROI lanes above as explicit, named use cases - not "everyone gets a licence, good luck". They instrument before they expand, so the second-quarter decision is made on their own numbers. They set lane-specific rules: agents may open PRs against scaffolding and tests freely, legacy cores with senior review, security paths not at all. And they treat the agent configuration - instructions, context, tool access - as engineered infrastructure, the same discipline as context engineering for any other production AI. The result is boring, measurable and compounding - which is what ROI looks like when it is real.
Where RanWebs fits
We help engineering orgs adopt agentic development on our software engineering track: baseline measurement first, then lane selection, guardrails, agent configuration and the review-process changes that keep quality flat while speed climbs. Typical engagement: four weeks to an instrumented pilot with your own ROI numbers.
Running coding agents without knowing what they return? Email info@ranwebs.com or use the contact page. First call is free and goes to a senior engineer.