Chasing Perfection: Knowing When to Ship

There has always been a point in software development where you have to stop. It’s not because the software is perfect, and it’s not because there are no more bugs to find. You stop because, at some point, the remaining imperfection and uncertainty become acceptable. The product is good enough, the known problems are understood, and another week of testing is less valuable than putting the thing in front of actual users.

That boundary has always been uncomfortable, whether you’re operating solo or at the scale of a multi-team organization. Artificial intelligence is making it much harder to find.

The Old Limit Was Human Attention

For most of software history, quality assurance had a natural constraint: people.

A developer would review a piece of code, then another developer reviewed it again, and finally a QA team exercised the product. They worked through test plans, probed edge cases, and spent time thinking creatively about how it might break.

Given enough time, they would keep finding things, but time wasn’t unlimited. There were other features to build, other bugs to fix, other releases waiting their turn in the pipeline. Eventually the cost of continuing to inspect the software outweighed the value of whatever additional problems might be discovered.

So it shipped.

That isn’t carelessness but quite the opposite. Experienced engineers became good at deciding where scrutiny and attention mattered most, developing an intuition about which pieces of code are fragile, which edge cases are plausible, which warnings deserve investigation, and which imperfections are unlikely to really matter. “Done” is never a statement that nothing else can be found but a judgment that we've looked hard enough.

AI Changes What “Hard Enough” Means

I’ve approached essentially this same claim from multiple different angles now, but I’m still finding it accurate: large language models have altered the equation. They make scrutiny cheap and effectively unlimited.

Give an AI system a codebase and ask it to look for race conditions, then authentication problems, then error-handling failures, unexpected state transitions, and compatibility issues. It will keep going.

A developer’s knowledge is bounded by individual experience. Even an exceptional engineer has encountered only some architectures, some failure modes, and some security mistakes. An LLM, in contrast, has been trained on artifacts produced by enormous numbers of people. It can approach the same code from perspectives that previously would have required many different people with vastly different expertise.

That does not make it omniscient, and it does not make every conclusion correct, but it gives us something new: an inexpensive mechanism for applying an amalgamation of accumulated industry experience to a single codebase.

The result can be extraordinary, but it can also be paralyzing to sift through.

There Is Always Another Question

Software shares a frustrating trait with the natural world: the closer you look, the more there is to see. What appeared simple at one scale reveals another layer of complexity at the next.

At one level, a feature works. Look closer and there is an edge case. Look closer again and there is a race condition. Closer still and there’s a compatibility issue that could make the race condition more likely on one unusual configuration.

It goes on endlessly like a Mandelbrot set — there’s always something nested deeper. These can all be real problems, and some may be worth fixing, but their existence alone does not mean the software should be held up.

This creates an important distinction that AI-assisted development increasingly forces us to make: the ability to find more problems is not the same as proof that the product is not ready.

That distinction sounds straightforward, but in practice, it is surprisingly difficult to accept. Once an issue has been identified, ignoring it feels irresponsible. A previously unknown theoretical weakness becomes a visible item sitting in a ticket, and now someone has to consciously decide not to fix it. It requires a triage flow and the fortitude to make hard choices.

For the type of mind that excels at testing and identifying problems like these, AI tooling can be addicting. It will find new issues and opportunities for improvement for as long as you let it, subtly expanding scope beyond any initial plans. The curious mind, in turn, wants to keep following those threads, chasing an impossible state of perfection.

The Quality Ratchet

We’re entwined in what might be thought of as a quality ratchet. Better tools allow us to detect more imperfections, and because we can detect them, we begin to expect ourselves to address them.

This is mostly good — static analysis, automated testing, and continuous integration all enabled improvements in baseline quality. Memory-safe languages, sanitizers, fuzzers, dependency scanners, and countless other tools have progressively raised that baseline too.

AI is the step being taken now, but there is a fundamental difference. Most earlier tools inspect software within a reasonably defined and narrow domain. An LLM can literally be asked: what else are we missing?

That question has no natural stopping condition, and neither does the answer. Ask it a dozen times, and you’ll receive a dozen answers.

The Flagship Product Problem

This is especially visible late in the development of a flagship product. The feature set may be complete, the known blocking bugs may be gone, and the release may already be substantially better tested than its predecessors. Yet each new AI-assisted review produces another plausible concern, which triggers another fix, which creates another surface to review.

The more consequential the release, the stronger the temptation to keep polishing. A flagship product carries reputation with it, and its failures are more visible. Decisions made now can become constraints that future developers inherit, so additional scrutiny feels justified. Often it is.

But importance can morph the objective from ship a high-quality product to eliminate every rough facet that can be imagined before shipping it. Those are very different goals: the first is difficult, but the second is impossible.

A capable critic can always propose another scenario:

• What if this API behaves differently under load?
• What if a plugin modifies this state unexpectedly?
• What if the network fails at exactly this point?
• What happens under clock skew?
• What if two requests arrive simultaneously?
• What if an old database contains a value we no longer generate?
• What if an administrator has configured something non-standard?

These are good questions. The trap is assuming that every good question must have a perfect answer before release.

AI Has No Opportunity Cost

There is another truth worth recognizing: the AI does not care whether you ship. The machine can help enumerate risks, but it cannot define the organization’s tolerance for them. It has no roadmap, no customers waiting for the feature, and it does not have engineers who need to move on to the next project.

It does not experience the cost of a three-month delay. Ask it to find another weakness and it will happily find one, or ask it to critique the fix and it may find three more.

Its incentives, so to speak, are bounded by the task in the prompt. Yours are not. As AI becomes more deeply integrated into engineering, that distinction becomes increasingly important: someone still has to decide what matters.

Finding a Problem Is Not a Shipping Decision

AI is very good at producing possible failure modes, but that is only the beginning of risk analysis. A meaningful shipping decision also asks:

• How likely is it?
• How bad is the outcome?
• How detectable is the failure?
• How reversible is it?
• How difficult is the fix?
• Can it be mitigated after release?
• Does the affected path represent a core workflow or an exotic configuration?

The relevant comparison isn’t bug versus no bug. It is the expected cost of the remaining risk versus the expected cost of delaying the release.

A bug that corrupts customer data once in a thousand operations deserves very different treatment from a layout defect that appears only when a window is resized to an unusual width. Both are bugs, but they do not and should not have the same effect on the release date.

AI-assisted QA becomes much more useful when it moves from endlessly discovering issues to helping classify them. Finding things is cheap, but prioritizing them is the actual work. It’s the latter that is increasingly the domain of human judgment, and it’s where experienced engineers earn their keep.

The Infinite 1 Percent

It’s been a popular aphorism in software for decades that the final 10% of a project takes 90% of the effort. AI introduces an even stranger corollary: the final one percent can have no end at all.

Every improvement creates another surface to inspect, and every refactor creates another implementation to review. This does not mean we should stop asking questions, but it does mean the stopping condition cannot be running out of questions.

There is another problem too: the target moves. Dependencies change, operating systems change, APIs change, customer expectations change, and threat models change. The environment into which software ships is dynamic, increasingly so, and a product held for another six months of refinement is not simply a more polished version of the product that existed six months earlier. It is now in a somewhat different world.

Meanwhile, holding software back also postpones production feedback. AI can help interpret that feedback once it exists, but it cannot supply it: real users behave differently from imagined users.

Their workloads expose different bottlenecks from synthetic ones, and their environments contain combinations no test lab anticipated. Real usage tells you which imperfections actually matter. That feedback is as much an ingredient in quality software as the code or the testing.

Good Enough Is Not Mediocrity

The phrase good enough has an unfortunate connotation — it can sound like lowering standards. In engineering, it’s more literal and often means precisely the opposite. Good enough is a deliberate judgment that the product satisfies the requirements appropriate to its context and risk.

It’s important to truly grok that requirement, though. Each project has its own threshold for “good enough,” and those thresholds are often very different. Aircraft software has one while a medical device has another, both likely quite high. A photo organizer has its own much lower threshold.

Treating all software as though every conceivable defect has equal consequences is not rigor. It is a failure to reason about risk. The goal is not to produce software with no imperfections — that’s an impossible task — but to produce software whose remaining imperfections are acceptable.

That requires judgment, and paradoxically, AI makes that human judgment more important rather than obsolete.

Shipping Is a Decision Under Uncertainty

Ultimately, releasing software has always been a decision made without complete information. AI hasn’t changed that, but it has changed how much information we can generate before making the decision.

As tools become extraordinarily good at finding problems, one skill is particularly valuable: knowing which problems deserve attention. AI can tell us there are seventeen more things you might want to consider. The important word is might — someone still has to decide whether number fourteen is worth delaying the release.

That does not mean dismissing warnings outright. It means evaluating them against probability, severity, product goals, and the cost of delaying release. The engineer becomes less of a solitary author or inspector and more of an arbiter among an unlimited number of possible concerns.

There will always be another edge case, another hypothetical interaction, another obscure configuration, another architectural concern, another clever critique. The temptation is to believe that enough analysis will eventually produce certainty. It won’t, it can’t, and at some point, the job changes from finding uncertainty to accepting it. That is the moment a product ships.

A product is ready when the important questions have been answered well enough that the remaining uncertainty is an engineering risk you are consciously willing to take. AI can help us get much closer to that boundary than we could before, but it cannot tell us when to cross it. That part is still ours.