GENERATIVE ATTITUDE

The next enterprise AI boundary is not a cleverer chat interface. It is the point at which a probabilistic system receives authority to change the world.

The side effect changes the problem

Imagine two systems handling the same customer-service request.

The first reads the case history and drafts a response. A person checks the facts, decides what to do and updates the system of record. That is an AI copilot: useful, assistive and deliberately short of execution authority.

The second reads the case, selects a resolution, calls a tool, changes the account and tells the customer the work is complete. The interface may look identical. The operational problem is not. The system has crossed from producing content into creating a side effect.

That distinction is more important than the label “agent”. Enterprise AI is gaining an execution layer, but this is not the end of the copilot. It is the beginning of a mixed operating model in which copilots, conventional automation, supervised agents and bounded autonomous systems each have a legitimate place.

The decisive variable is authority. Who may choose the next action? Who may execute it? Against which resource, under whose identity, within what limit, and with what evidence that the intended result actually occurred?

For senior technology leaders, this changes the investment question. The task is not to maximise autonomy. It is to design the smallest safe boundary within which AI-led execution produces a better verified outcome than either human assistance or deterministic code.

Stop governing the product label

“Copilot” and “agent” are convenient product terms, but poor units of governance. Authority varies by action, even within one workflow.

A useful operating spectrum looks like this:

• Deterministic automation follows predefined rules and transitions. It is usually the best choice when inputs, paths and exceptions can be specified economically.
• An assistive copilot proposes, summarises or drafts. A person remains responsible for interpreting the output and taking action.
• A supervised agent can prepare or select an action, but waits for approval at a defined gate.
• A bounded agent may execute pre-authorised actions inside an explicit envelope of tools, resources, values, time and policy.
• A broad autonomy claim implies materially more discretion over goals, sequences and actions, with correspondingly greater verification and accountability demands.

These are not rungs on a maturity ladder. A bank should not aspire to turn every copilot into an autonomous agent. A stable reconciliation rule belongs in code. A relationship manager may benefit from an excellent briefing copilot. A variable document-remediation process may justify bounded execution. A high-value beneficiary change may always require a human decision.

The same workflow can combine all four modes. Consider an illustrative payment-instruction exception. A copilot might summarise the issue. A supervised agent could retrieve authorised records and prepare a correction. A bounded agent might fix a low-risk reference field when identity, policy and value thresholds pass. Releasing funds or changing a beneficiary would remain human-owned.

The governance unit is therefore not “the agent”. It is an authority tuple:

Actor + action + resource + scope + approval + evidence + recovery

That formulation forces a concrete conversation. It also exposes a common category error: connectivity is not authority. The Model Context Protocol and other tool interfaces can make executable functions available to a model, but availability says nothing about whether a business action is permitted, sensible or safe. Authentication, authorisation and semantic policy still need enforcement outside the model.

This is consistent with Anthropic’s distinction between predefined workflows and model-directed agents. The model may decide how to pursue a task, but enterprise authority still belongs to the surrounding system.

The harness is the execution system

A model plus tools can be an impressive demonstration. It is not yet a reliable business system.

Production execution depends on a harness that constrains and observes the model. The exact implementation varies, but a credible design normally includes the following elements.

A behaviour contract. The system needs a machine-testable definition of the intended outcome, permitted actions, forbidden actions, limits and escalation conditions. “Resolve this case” is an objective. It is not an adequate operating contract.

An orchestrator with durable state. The orchestrator holds the workflow state and determines which transitions are possible. This state should live outside the conversation. Approval waits, network failures and retries can last longer than a model session. A transcript is not a transaction log.

Narrow, typed tools. A tool should do one comprehensible thing, accept validated parameters and reveal its likely effect before execution. Broad shell access or an all-purpose administrative API creates a larger blast radius than a narrow function such as “update address after verified approval”.

A real identity. Each call needs an authenticated principal. That might be a dedicated workload identity or the user’s delegated identity, but the choice must be explicit. The target system should enforce least privilege. A prompt saying “do not access payroll” is not an access control.

Policy and approval gates. Deterministic checks should decide whether a proposed action is allowed. Approval is valuable at consequential points, not as a blanket ritual. A reviewer should see the proposed change, affected resource, policy basis, authoritative evidence and reversible alternative, rather than approving the model’s own summary of its reasoning.

A deterministic executor. Once authorised, state-changing work should use transactional safeguards such as idempotency keys, timeouts and well-defined failure responses. Otherwise a retry can duplicate a payment, issue two refunds or partially update a customer record.

Independent postcondition checks. The system must verify the resulting business state in the authoritative system. A fluent “done” message is not evidence. False completion, where the system reports success although the intended state is absent, partial, stale or applied to the wrong entity, deserves its own failure category.

Trace, recovery and escalation. Operators need to reconstruct the request, evidence, proposed action, identity, policy decision, tool call, result, latency, cost and final state. Recovery logic should distinguish retryable faults from terminal failures, compensating actions and human escalation.

Security becomes more consequential at this boundary. Indirect prompt injection exploits the fact that retrieved data and instructions can arrive through the same context. When the system can only draft text, a bad instruction may produce a bad answer. When it has write-capable tools, the same weakness can create a real-world side effect. OWASP’s excessive-agency guidance usefully locates the risk in the combination of excessive functionality, permissions and autonomy.

The appropriate response is architectural: treat retrieved material as untrusted, separate read and write capabilities, constrain egress, enforce policy at gateways and target systems, cap time, spend and action counts, require independent approval for consequential changes and test the full workflow adversarially.

A deterministic spine with agentic branches

The strongest near-term pattern is a deterministic spine with agentic branches.

Known states, permissions, approval points, transaction boundaries and recovery rules remain explicit code. The model handles work that is expensive to enumerate: interpreting an ambiguous request, gathering context, classifying an exception, choosing among permitted tools or drafting a candidate resolution.

This is not a temporary compromise on the road to unrestricted autonomy. It is how probabilistic inference becomes compatible with enterprise control.

It also provides a rational stopping rule. If a branch becomes predictable, move it into deterministic code. If observed failures increase, narrow the branch or demote it to supervised execution. If the system repeatedly succeeds within a measured envelope, widen that envelope deliberately. Autonomy becomes a reversible operating decision rather than a one-way technology programme.

Production is real. Proof is still uneven

The evidence does not support either extreme. It is wrong to say agents are merely demos. It is equally wrong to infer that public production counts establish general reliability or return on investment.

The strongest independent productivity evidence still concerns assistance. A peer-reviewed field study of 5,172 customer-support workers found a 15 per cent average increase in issues resolved per hour, with the largest gains among less-experienced workers. The AI suggested responses while people retained responsibility. The study in the Quarterly Journal of Economics is good evidence for copilot value in one important context, not proof that the same work should be automated end to end.

Evidence can also run in the other direction. A METR randomised controlled trial involving 16 experienced open-source developers and 246 tasks found that access to early-2025 AI tools increased completion time by 19 per cent, even though participants believed they had become faster. The sample and setting are narrow, and the tools have since changed. The result still makes an important measurement point: perception, adoption and generated output are weak proxies for completed value.

Historical agent benchmarks add another warning. Early WebArena and OSWorld baselines recorded large gaps between agents and human participants on execution-based tasks. Those scores should not be presented as current frontier performance. More recent evaluation work shows rapid progress. Yet capability benchmarks remain conditional on the model, scaffold, environment and reliability threshold. They do not include every enterprise complication: fragmented context, policy conflicts, outages, adversaries, unclear ownership and expensive exceptions.

The correct enterprise question is not whether a model can complete a task once. It is whether the exact model-plus-harness release completes the correct task at the required service level, handles exceptions, fails safely and does so at an acceptable total cost.

Public production disclosures are nevertheless meaningful:

• Amazon reports tens of thousands of production application upgrades, an estimated 4,500 developer-years saved and $260 million in annual infrastructure savings. Its first-party account explains elements of the estimation method and notes that developers review plans before implementation.
• Salesforce reports thousands of autonomously resolved internal support cases and early savings. These are first-party case studies without an independent outcome audit.
• BNY’s 2025 annual report states that the bank had 160 enterprise AI solutions and 134 multi-agent “digital employees” in production. Public material does not provide workflow-level accuracy, exception, loss or cost figures for those digital employees.
• Klarna’s Form F-1 says its AI assistant handled 62 per cent of service chats and delivered approximately $39 million in 2024 savings. Later independent reporting described the continued need for people in difficult cases such as identity theft.

These accounts show that execution systems are operating at scale. They also show why evidence maturity matters. A demonstration, vendor-reported production count, primary filing and independently measured outcome are not interchangeable.

ROI should therefore be measured at the workflow outcome. A useful scorecard includes straight-through completion, first-pass quality, false completion, exception and escalation rates, recontact, rework, cycle time, unit cost, risk loss, customer outcome and recovery time. Compare the bounded agent against both relevant alternatives: human work with a copilot and conventional automation.

That comparison may reveal that a copilot captures most of the benefit with less risk, or that deterministic automation is cheaper and easier to audit. That is a sound portfolio decision, not a failure of innovation.

In banking, accountability does not transfer

“Digital employee” can be a helpful operating metaphor. It does not create a new accountable legal person.

For a financial institution, every execution system still needs a sponsor, owner, identity, permissions, inventory entry, validation approach, monitoring, continuity plan and override. The precise duties depend on the entity, jurisdiction, role, system classification and use case.

In the UK, the PRA’s SS1/23 model-risk expectations apply to defined firms and models and cover identification, governance, development and use, independent validation and mitigants. The FCA’s current approach to AI relies on existing outcome-focused frameworks, including Consumer Duty and senior-management accountability, rather than a separate AI rulebook.

In the European Union, 2 August 2026 was the AI Act’s general date of application, but provisions apply progressively. Regulation (EU) 2026/1744, adopted on 8 July 2026 and published in the Official Journal on 24 July 2026, amended the classification-specific schedule for high-risk requirements in Chapter III, Sections 1–3. Those provisions apply from 2 December 2027 for Article 6(2)/Annex III systems and from 2 August 2028 for Article 6(1)/Annex I product-safety systems. This does not mean every enterprise agent is a high-risk system. Classification and role still need analysis.

The Digital Operational Resilience Act has applied to in-scope EU financial entities since 17 January 2025, bringing relevant ICT risk, incident, resilience-testing and third-party expectations.

This is not a generic legal checklist, and it is not legal advice. The strategic point is narrower: an institution cannot delegate accountability simply because a model selected the next action. The control design must preserve a named owner, a defensible authority chain and evidence that the workflow operated as intended.

That is also why claims of autonomous capital-markets execution deserve particular caution. The accepted research found no sufficiently detailed, independently audited example of an autonomous agent executing capital-markets trades end to end. Research, coding and payment-instruction validation are not the same as unsupervised trading authority.

Five tests before an agent is allowed to act

Leaders can turn the architecture into a practical selection exercise. For each candidate workflow, apply five tests.

1. Is the path genuinely variable?

If rules, states and exceptions can be enumerated economically, use deterministic automation. Agentic reasoning is justified where inputs or paths vary enough that maintaining exhaustive rules becomes the dominant cost.

Document the source of variability. “The process is complex” is insufficient. Is the problem ambiguous language, fragmented evidence, open-ended planning or a changing set of permitted tools?

2. Can success be verified independently?

Define the authoritative postcondition before building the agent. The answer should be observable without asking the model whether it succeeded.

For a ticket, this might be a valid state transition and an accepted resolution code. For a code change, it might be tests, deployment health and the absence of a prohibited diff. For a financial workflow, it may require ledger agreement, reconciled balances and evidence of approval.

If success cannot be verified, keep the system assistive.

3. Can permissions be scoped tightly enough?

List every action and resource. Give the executing identity the minimum rights for the minimum duration. Separate reading from writing. Apply value and frequency limits. Identify actions that are prohibited regardless of model confidence.

If the only available tool is an over-privileged administrative interface, fix the interface before increasing autonomy.

4. Is failure recoverable?

Classify likely failures as retryable, compensatable, terminal or uncertain. Design idempotency, rollback or compensation where possible. Set action and cost ceilings. Define the point at which a human receives the case and the evidence they need.

Where an error is irreversible or materially harmful, use approval before execution, or do not delegate the action.

5. Do measured economics beat the alternatives?

Run the workflow against a representative scenario set, including exceptions and adversarial cases. Measure completed outcomes, quality, rework, escalation, risk and total cost. Compare it with a copilot-assisted baseline and a deterministic baseline.

Promote the agent only when the observed value survives verification overhead and operational risk. Define demotion criteria at the same time. A system that cannot be narrowed, paused or returned to supervised mode is not well governed.

These tests produce a clearer portfolio:

• Choose a copilot when judgement or relationship value is central, outputs are easy for experts to inspect and authority should remain human.
• Choose deterministic automation when rules and exceptions are stable and auditable.
• Choose a bounded agent when paths are variable, outcomes are verifiable, permissions are narrow, failures are recoverable and measured economics are favourable.

The winners will know where autonomy ends

The next generation of enterprise AI will execute more work. That much is already visible. But the durable advantage will not come from granting the broadest autonomy to the most capable model.

It will come from designing the execution boundary better than competitors do: precise authority, durable state, narrow tools, independent policy, verifiable outcomes, observable traces and rehearsed recovery.

Copilots will remain valuable because many important decisions should stay human. Deterministic automation will remain valuable because probabilistic reasoning is unnecessary for predictable work. Bounded agents will earn their place where variability is real and control is possible.

The future is not a succession from copilot to agent. It is a portfolio of operating modes, governed action by action. The winners will not be the systems that claim the most autonomy. They will be the systems that know exactly where autonomy ends.