As a platform team, we’re getting more and more pressure to give AI greater autonomy – partly ungated PRs, allowing non-engineers to push code to production, or building entire features through AI agents and skills – all in the name of improving productivity (metrics).
And there’s good reason for the enthusiasm. AI got better and better on solving unfamiliar problems, inferring missing requirements, and producing functional implementations even when the specifications are incomplete.
However, non-functional requirements and specifying the non-obvious have always been part of what makes the difference between “it works on my machine” and good engineering. And they still do.
The cost of underspecifying requirements was, and still is, often hidden until the feature reaches production: regressions, failures outside the happy path, degraded performance, increased operational costs, scalability issues, security & privacy vulnerabilities, compromised data integrity or broken backward compatibility.
And while there are solutions to these risks, they require careful engineering and monitoring themselves. Meanwhile, I see two concerning tendencies:
Many codebases aren’t prepared for this level of autonomy. They lack the necessary guardrails: clearly defined and tested architectural boundaries, system invariants and dependencies, comprehensive regression tests (including integration and performance tests), and the infrastructure to run all the appropriate checks automatically for each change. Skills and instructions help, but they’re not the same as enforceable constraints.
We’re becoming increasingly comfortable trusting AI-generated results. As the output gets better and the volume of generated code increases, reviewing it properly requires patience we’re increasingly unwilling to invest. We’re tempted to just trust the result, or worse, pass the responsibility to someone else – the architect, the code reviewer, the on-call engineer. I’ve noticed this tendency in myself as well, especially as the amount of content to review – specs, solution designs, code – keeps growing.
There’s also the question of efficiency
I’ve been looking into how coding agents operate, particularly Codex – currently the one I’m most pleased with – and one thing stood out: they can iteratively explore a problem, make changes, run tests, interpret failures, and try again until they reach a working result that seems to match the initial specifications.
It’s a remarkable capability, but not necessarily an efficient one.
When requirements are ambiguous or verification is weak, an agent can spend considerable resources iterating towards a solution that satisfies the visible functional requirements while violating assumptions it was never told about.
And while this may still look economically efficient compared to human development time, the actual computational cost is difficult to evaluate when hidden behind a subscription or testing locally with one user. We tend to measure how quickly a feature was delivered, not how many iterations were needed, what resources were consumed, or how much rework followed.
A recent example from a personal project
For my votam.eu project, which analyzes Romanian politicians’ activity, I use a crawler to collect public data and the OpenAI API to interpret some of it.
From time to time, the structure of the source data changes slightly, requiring adjustments to the analysis pipeline.
Until recently, I used AI to help with these changes while maintaining a proper understanding of what was changed and why.
Last time, I delegated the responsibility almost entirely to an agent, with what I thought were good guardrails.
The result?
1. Silent data corruption. To avoid repeatedly paying for AI analysis of historical activity, the pipeline preserves previously processed data. A reasonable optimization, except that politicians were identified using their position in an alphabetical listing rather than a stable identifier.
When those positions changed, existing IDs could become associated with different politicians while historical activity remained attached to them. The pipeline continued working, and the results looked plausible – but some were wrong.
The underlying architectural flaw went undetected by the agent and automated checks. I only discovered it during manual exploratory testing, after the changes had been implemented.
2. Around 10× higher AI API costs. Retries and rework caused a significant increase in the cost of data interpretation, without a corresponding improvement in the results.
I couldn’t reliably evaluate the coding agent’s own token consumption because I was using a subscription.
3. Significantly more of my own time. What should have been a relatively simple maintenance change turned into a much larger effort: reviewing architectural changes, understanding their implications, and performing exploratory testing across potentially corrupted historical data. Instead of saving time by delegating the work, I ended up spending considerably more time verifying and fixing the result. Arguably I won’t have to do it next time.
Fortunately, the analysis pipeline is separate from the website, so there was no observed impact on user-facing performance. But the experience has certainly made me more careful about reviewing agent-generated changes.
So, should we give AI less autonomy?
Not necessarily.
But I think there’s a shared responsibility here – from investors and senior management pushing for productivity gains, to engineering leaders, developers, and non-engineers adopting these tools. We all need to understand both their capabilities and their limitations, and ensure that the necessary guardrails are in place before increasing autonomy. Otherwise, we risk incentivizing exactly the wrong behaviors: prioritizing speed and output over correctness, maintainability, and long-term cost.
The problems above aren’t entirely new. Underspecified requirements, poor architectural decisions, insufficient testing, and superficial code reviews existed long before AI. But AI can amplify their consequences by dramatically increasing the speed and volume of changes.
And there’s an important distinction between telling an agent what to do and having a system that can verify it did the right thing.
Soperhaps it’s time to rethink our engineering productivity metrics accordingly. After all, what we measure influences what we optimize for.
Are we adapting how we measure engineering productivity to the AI era, or are we just using AI to optimize for the same metrics – and potentially the wrong behaviors?
