There is a strange misunderstanding at the centre of the current AI boom.
We have built machines that are astonishingly good at taking ambiguity and turning it into something plausible. Give a large language model a vague product idea, a half-written specification, a messy set of notes, or a confusing piece of code, and it can often produce something coherent, structured, and useful. That is genuinely impressive.
But somewhere along the way, many people started treating this ability as if it were the same thing as correctness.
It is not.
Modern AI, at least in the form that dominates today’s conversation (large language models, multimodal models, and the agentic systems built around them) is probabilistic in nature. It predicts, infers, generalises, approximates, and proposes. That makes it extraordinary at dealing with uncertainty. It also makes it fundamentally unsuitable as the final authority in situations where the outcome needs to be consistent, repeatable, verifiable, and objectively correct.
This does not mean AI is a fad. It means the current overuse of AI is.
The real opportunity is not to replace deterministic systems with probabilistic ones. It is to combine them properly.
LLMs should help us move from ambiguity to structure. Deterministic systems should turn that structure into reliable outcomes.
That distinction matters more than almost anything else in the next phase of AI adoption.
The power of probabilistic AI is ambiguity reduction
Most human work does not begin with a perfectly specified problem.
It begins with something vague:
“We need a better onboarding flow.”
“Customers are confused by the pricing model.”
“Can you build a reporting dashboard?”
“This reconciliation process is taking too long.”
“The product feels clunky.”
These are not deterministic inputs. They are bundles of intention, context, assumptions, taste, politics, historical baggage, and missing information.
This is where probabilistic AI shines.
An LLM can take an ambiguous request and produce a plausible structure. It can suggest requirements, draft a specification, identify likely edge cases, propose user stories, rewrite a messy document, summarise research, compare options, critique an idea, or generate a first version of something that previously required a human to stare at a blank page.
That is not a small achievement. It is a major shift in the cost of moving from vague thought to structured artefact.
In product and strategy work, this is incredibly valuable. The output does not need to be objectively correct in the same way a financial calculation or database migration needs to be correct. It needs to be useful, coherent, provocative, comprehensive, persuasive, or directionally right.
Those are subjective standards. LLMs are very good at subjectively good work.
They are good at producing candidate answers where many acceptable answers may exist.
They are good at exploring the possibility space.
They are good at making the implicit explicit.
They are good at helping humans think.
But that is not the same as being good at guaranteeing correctness.
Subjectively good is not objectively correct
The difference between subjective and objective quality is where much of the confusion begins.
A product specification can be “good” in many ways. It may be clear, persuasive, thorough, practical, or aligned with a team’s intent. Two strong product managers might write different specs and both could be good.
A user interface can also be good in more than one way. It might be minimal, expressive, playful, enterprise-grade, accessible, conversion-focused, or optimised for speed. There is no single mathematically correct interface.
A marketing message, strategy document, design direction, research synthesis, or executive narrative often lives in this territory. There are better and worse answers, but rarely one provably correct answer.
Coding is different.
Not all of coding, of course. Architecture, naming, decomposition, API design, and user experience all involve judgement. But large parts of software engineering eventually collapse into objective questions:
- Does it compile?
- Does it pass the tests?
- Does it preserve the data?
- Does it enforce the permission model?
- Does it calculate the right amount?
- Does it handle the edge case?
- Does it remain compatible with the contract?
- Does it scale within the required limit?
- Does it avoid the security vulnerability?
- Does it produce the same result every time for the same input?
In these domains, plausibility is dangerous.
A plausible answer that is wrong is worse than no answer at all, because it creates false confidence.
That is the core limitation of LLMs. They can generate something that looks right without it being right. And the closer we get to domains requiring objective correctness, the less acceptable that becomes.
The wasteful pattern: asking the guesser to become the judge
The current workaround is familiar.
Ask the LLM to generate code. Run tests. Feed back the errors. Ask it to fix the code. Run tests again. Ask it to reflect. Ask another model to critique it. Run a linter. Ask it to patch again. Repeat until the system appears to pass.
This can work. Sometimes it works impressively well.
But intellectually, it is a strange architecture.
We are using a probabilistic model to generate a plausible answer, then spending additional tokens, compute, latency, and money trying to constrain that same probabilistic system into behaving like a deterministic one.
That does not mean the loop is useless. Human developers also iterate against feedback. But it does mean we should be honest about what is happening.
The LLM is not becoming deterministic. It is being boxed in by external checks.
The reliability comes from the tests, compilers, type systems, linters, schemas, contracts, static analysers, benchmarks, and human review. It does not come from the model itself.
This is why “agent loops until green” can become wasteful. If the deterministic boundary is weak, the model may simply optimise for passing insufficient checks. If the deterministic boundary is strong, the model becomes more useful, but much of the value is now coming from the boundary rather than the model.
The problem is not that LLMs are bad. The problem is that we are often using them too deep in the stack.
We are asking the creative engine to become the correctness engine.
That is the wrong division of labour.
The missing architecture: probabilistic front-end, deterministic back-end
A better model is simple:
Use probabilistic AI to transform ambiguity into structured intent. Use deterministic automation to transform structured intent into reliable output.
The structured spec is the boundary. Everything left of it is interpretation; everything right of it is execution you can trust.
In software, this means the LLM should not be treated as the compiler. It should be treated as a fuzzy front-end that helps humans define what should be compiled, generated, checked, or executed.
The mature pipeline looks more like this:
Messy human intent
↓
LLM-assisted clarification
↓
Structured requirements, examples, constraints, contracts
↓
Deterministic generators, templates, compilers, schemas, rules
↓
Deterministic checks and validation
↓
Human judgement where taste, risk, and trade-offs remain
This is where so-called “scaffolding” techniques become important.
We already have many deterministic tools that produce consistent outputs when given strong inputs:
- Code generators
- Templates
- Low-code platforms
- Compilers
- Domain-specific languages
- Protocol generators
- GraphQL generators
- OpenAPI generators
- Terraform
- Kubernetes manifests
- Schema-driven form builders
- Rules engines
- Workflow engines
- Static analysis tools
- Type systems
- Design systems
- Component libraries
- Policy-as-code frameworks
These tools are not fashionable in the same way LLMs are fashionable, but they solve a different and crucial problem: repeatability.
They are not creative. That is their strength.
Give them the same valid input and they produce the same valid output.
The catch, of course, is that they need good inputs.
And that is where most organisations struggle.
The real bottleneck is not generation. It is specification.
The reason deterministic generation has not taken over software development is not that it lacks value. It is that most teams do not have deterministic inputs.
They have vague tickets. Partial requirements. Tribal knowledge. Inconsistent naming. Weak tests. Missing documentation. Legacy code. Ambiguous ownership. Edge cases living in someone’s memory. Product decisions embedded in Slack threads. Architecture decisions that were never written down.
A deterministic generator cannot do much with that.
A probabilistic model can.
That is why LLMs feel magical. They can operate in the mess. They can consume ambiguity and produce a plausible structure. They can infer the missing context, even if imperfectly. They can help turn vague intent into something closer to an executable specification.
But the key phrase is “closer to.”
The LLM should help create the deterministic input. It should not replace the deterministic process.
This is the stronger pairing:
LLM:
"Here is what I think the requirement means.
Here are the edge cases.
Here is a proposed contract.
Here are acceptance criteria.
Here are examples."
Human:
"Yes, no, adjust this, remove that, this edge case matters."
Deterministic system:
"Given that approved structure, I will generate,
validate, enforce, deploy, or reject consistently."
That is where the payoff is.
Not pure AI. Not pure automation. The combination.
A practical comparison: pure LLM loops versus constrained generation
Consider three scenarios.
Scenario 1: Building a CRUD admin feature
A team needs an internal admin screen for managing providers, staff, invoices, and payment records.
The pure LLM approach might look like this:
- Ask an agent to build the feature.
- The agent writes database migrations, backend routes, frontend screens, validation, tests, and styling.
- The test suite fails.
- The agent patches the implementation.
- The UI is inconsistent with the rest of the app.
- The agent patches again.
- The validation differs between frontend and backend.
- More prompts, more test runs, more review.
This may still be faster than writing everything manually. But the model is doing too much. It is inventing patterns that should already be defined.
A constrained generation approach looks different:
- The LLM helps define the data model, field descriptions, validation rules, permissions, and user flows.
- A human approves the structure.
- A deterministic generator creates the migration, API routes, form components, table views, validation schemas, and standard tests.
- The LLM is used only for genuinely bespoke logic or unclear edge cases.
The second approach is less glamorous, but it scales better. The more CRUD features you need, the more the deterministic path compounds.
Scenario 2: Implementing business rules
Imagine a budgeting or eligibility system.
A pure LLM agent could read the rules and write code. But every rule change risks interpretation drift. The model may phrase a rule convincingly while encoding it incorrectly.
A deterministic approach would instead express the rules in a structured format:
rule: monthly_contribution_due
when:
day_of_month: 1
then:
create_task: "Review and record monthly contribution"
notify: "account_owner"
The LLM can help draft this structure from natural language. But once approved, the rule engine should execute it.
The model should not be reinterpreting the policy every time the rule runs.
Scenario 3: Designing UI
This applies beyond coding.
In UX and UI design, LLMs and image models are useful for exploration: “What could this page look like?”, “What are three different dashboard concepts?”, “How might we simplify this workflow?”
But once a design system exists, much of the output should be deterministic.
A mature design system has explicit tokens, components, layout rules, interaction patterns, accessibility standards, and content guidelines. If the product needs a standard settings page, table, form, modal, confirmation flow, alert, dashboard card, or empty state, the system should not need a model to invent it from scratch.
It should assemble the design programmatically from known components.
The LLM is useful when a new concept appears that the system does not yet understand. It can propose a new component, pattern, or content structure. But once that pattern is accepted, it should become part of the deterministic system.
Otherwise every design task becomes a fresh act of invention, which is inefficient and inconsistent.
Good design systems are deterministic infrastructure for creativity.
They allow humans and AI to focus novelty where novelty is actually needed.
Constrained generation: where should the boundary sit?
The central design question is not whether to use AI. It is where to constrain it.
There are several possible boundaries.
Option 1: Let the LLM generate everything, then check it
This is the dominant agentic coding pattern.
It is flexible. It works with messy inputs. It can be surprisingly productive. But it is also expensive, unpredictable, and highly dependent on the quality of the verification harness.
It is best for prototypes, one-off scripts, exploratory work, unfamiliar codebases, and tasks where the cost of being wrong is low.
It is risky for security, finance, compliance, permissions, data migrations, and core business logic.
Option 2: Let the LLM generate structured specifications
Here, the model produces schemas, acceptance criteria, examples, contracts, state machines, rules, or design briefs. A human reviews them. Deterministic systems then generate or validate the output.
This is often the sweet spot.
It uses the LLM for what it is good at, ambiguity reduction, while moving correctness into systems designed for repeatability.
The downside is that it requires investment in tooling, conventions, and internal platforms.
Option 3: Let deterministic systems generate most outputs, using LLMs only for exceptions
This is the most scalable approach in mature domains.
If the organisation has strong schemas, design systems, templates, component libraries, policy engines, and workflow engines, the LLM becomes an assistant at the edges rather than the engine at the centre.
This is efficient and consistent. But it is harder to start, because it requires disciplined foundations.
Option 4: Make the LLM operate only inside a constrained grammar or DSL
This is one of the most interesting patterns.
Instead of allowing the model to write arbitrary code, it writes a constrained intermediate representation: a workflow definition, state machine, query plan, rule set, UI schema, or deployment manifest.
The deterministic engine then executes that representation.
This narrows the model’s freedom and reduces the surface area for errors. But it also limits expressiveness. The DSL must be well designed, and novel cases may still require human extension.
The four boundaries, side by side:
| Boundary | What the LLM produces | Best for | Main risk |
|---|---|---|---|
| 1 · Generate, then check | Finished code | Prototypes, one-offs, unfamiliar code | Only as safe as the verification harness |
| 2 · Generate the spec | Schemas, contracts, acceptance criteria | The common sweet spot | Needs tooling to turn specs into output |
| 3 · Systems generate, LLM at the edges | Just the exceptions | Mature domains at scale | Hard to start; needs strong foundations |
| 4 · LLM inside a DSL/grammar | A constrained representation | High-stakes, repeatable execution | Limited expressiveness; DSL must be well-designed |
The right boundary depends on the cost of being wrong.
Where correctness matters, constrain earlier.
Where exploration matters, constrain later.
The cost argument: when loops become waste
The economics matter.
A pure LLM loop consumes tokens and compute every time it reads context, proposes changes, analyses errors, patches files, and re-runs checks. As models become more capable, people often give them more context and more autonomy, which increases the cost per task.
In low-risk work, that may be fine.
If an LLM spends a few extra cycles helping draft a memo or prototype a throwaway tool, the cost is acceptable.
But in repeated production workflows, waste compounds.
Imagine a team generating ten similar admin screens. If each screen requires an agent to rediscover the app’s patterns, rewrite boilerplate, fail tests, adjust styling, and correct validation drift, the team is paying repeatedly for work that should have been encoded once.
By contrast, a deterministic generator has a higher setup cost but lower marginal cost.
The first generated screen may be slower because the team needs to define the schema, templates, tokens, routes, and conventions. But the tenth screen is dramatically cheaper and more consistent.
That is the core economic distinction:
| Approach | Setup cost | Repeated cost | Quality | Flexibility |
|---|---|---|---|---|
| Pure LLM loop | Low | High (per task) | Variable | Flexible but inconsistent |
| Deterministic scaffolding | Higher | Low (marginal) | Consistent | Limited until extended |
| Combined | Medium | Low | Consistent and adaptable | LLM evolves the scaffold; systems execute; humans govern the boundary |
This is why the overuse of AI is likely to be corrected. Not because AI will disappear, but because organisations will realise they are spending too much money asking probabilistic systems to repeatedly solve problems that should have been encoded into deterministic infrastructure.
What does this mean for labour?
The labour impact is real, but it is often described too crudely.
The question is not simply “will AI replace developers, designers, analysts, or product managers?”
The better question is:
Which parts of their work are ambiguity-heavy, and which parts are repeatable once structured?
LLMs threaten work that consists mostly of producing plausible first drafts from ambiguous inputs. They also accelerate people who are already good at judging, editing, structuring, and validating those drafts.
But deterministic systems threaten a different kind of work: repeated execution of known patterns.
If a company can encode its design system, API conventions, infrastructure patterns, reporting templates, test strategy, and business rules into reusable machinery, then many manual implementation tasks shrink.
That does not eliminate humans. It changes where humans are needed.
Humans remain essential for:
- deciding what matters
- setting goals
- resolving trade-offs
- choosing between plausible options
- defining taste
- approving acceptance criteria
- identifying risk
- understanding context
- governing values
- extending the deterministic system when reality changes
The more deterministic the machinery becomes, the more important human judgement becomes upstream.
This is also why fully self-governing LLM systems are unlikely to be trustworthy in any serious sense.
A probabilistic system cannot be its own final authority for objective correctness. It can propose acceptance criteria, but it cannot make them objectively true. It can critique its own output, but that critique is another probabilistic output. It can choose among options, but the choice reflects patterns, preferences, and probabilities, not independent truth.
Humans are not perfect judges either. No one, human or AI, should govern themselves without external constraints. That is why serious systems need separation of duties: creators, checkers, approvers, rules, audits, tests, and enforcement mechanisms.
In that sense, deterministic systems are not merely technical tools. They are governance tools.
They define what must be true regardless of what the model finds plausible.
AI is not the fad. Unbounded AI is.
The current AI wave is not going away. The ability to turn ambiguity into structured possibility is too valuable. It will reshape software, design, operations, research, customer support, strategy, education, and management.
But the idea that LLMs should sit at the centre of every workflow, making every decision, generating every output, checking their own work, and retrying until reality accepts them: that is the part that deserves scepticism.
The future should not be “AI everywhere.”
It should be “AI where ambiguity lives.”
And where consistency, repeatability, compliance, security, calculation, or objective correctness matters, we should prefer deterministic systems.
The most valuable organisations will not be the ones that use the most tokens. They will be the ones that know when not to.
They will use LLMs to clarify intent, explore options, draft structures, and accelerate thinking. Then they will convert the best of that thinking into durable systems: schemas, tests, rules, templates, contracts, components, generators, and compilers.
That is how AI becomes value rather than waste.
The real question is not whether AI is a fad.
It is whether we are currently intoxicated by the wrong part of it.
The magic is not that a machine can guess.
The magic is what happens when we use that guessing machine to help us build systems that no longer need to guess.
Written June 2026, a working note, true as I see it today. Some of this will age badly, and I’m genuinely curious which parts. If you think I’ve got it wrong, come and tell me: that’s rather the point.