GENERATIVE ATTITUDE

Part 1 of From AI Governance to Governed Execution: Lessons from the FCA’s 2026 AI Findings

When an AI system went wrong at one financial firm, several teams assumed that oversight belonged somewhere else. There was no clear route to escalate concerns or challenge decisions. Governance had arrived after deployment, and the attempt to retrofit it left gaps in accountability.

That is case study J in the Financial Conduct Authority’s October 2026 review of AI practice. It is also a useful test for every bank that can point to an AI policy, an ethics committee and a list of named owners.

The difficult question is not whether those structures exist. It is whether they can govern a consequential decision at the moment it is made.

Can the firm identify who may decide, on what evidence, within what limit, subject to whose challenge, with which escalation and in what durable record? If not, accountability remains an organisational claim rather than an operating capability.

This is the central lesson from the FCA’s findings. Policies and committees matter, but they do not by themselves allocate authority, create an emergency stop, test human judgement or preserve evidence of reasonable steps. For that, a bank needs decision rights that run through the lifecycle of the AI-enabled service.

The FCA did not write a new AI rulebook

The FCA is careful about the status of its publication. Its examples are illustrative, non-exhaustive and expected to evolve. They do not create new requirements or prescribe a single governance model. Firms remain responsible for deciding what is appropriate and proportionate for their use cases, risks and potential impact.

That caveat is important. The examples should not be recast as a checklist with the force of rules. Nor should Consumer Duty examples be transferred unqualified to wholesale trading. The applicable obligations still depend on the firm, activity, product, customer, legal entity and jurisdiction.

The publication is valuable for a different reason. Its case studies show what familiar obligations and control principles look like when applied to AI systems. The Senior Managers Regime, for example, already requires senior managers to take reasonable steps to control their area effectively. Delegation must be reasonable, go to an appropriate person and be overseen effectively. Delegation does not remove accountability.

The evidence also reveals a gap between formal governance and operational capability. In the 2026 Bank of England and FCA survey, 94% of respondents reported firm-wide AI policies, principles or standards, and 84% reported centralised AI or model-risk governance forums. Business owners or users were identified as accountable by 94%, while executive leadership was identified by 90%.

Yet fewer than half reported incident-management and response playbooks. Ongoing monitoring, leakage testing and periodic review were each reported by around half. Only 39% monitored customer-impact measures and 34% monitored risk metrics focused on downstream impact.

This does not mean that banks have no governance. The survey is self-reported and does not test whether controls work. It suggests a narrower and more useful conclusion: policy adoption is ahead of operating evidence.

Accountability should follow the business outcome

One common response to AI complexity is to assign ownership to the model team. Another is to centralise it in an AI committee or chief AI officer. Both can confuse expertise with accountability.

The FCA’s case study L offers a better distinction. Ultimate accountability sat with the Senior Management Function responsible for the business area in which the tool would operate. A separate model owner oversaw the technical build. Both roles were documented before deployment.

The senior manager does not need to become a model engineer. The model owner should not be made accountable for customer, market or operational outcomes that sit beyond their authority. A deployed AI service includes more than a model: data, prompts, tools, interfaces, human reviewers, business rules, permissions, fallbacks and third parties all affect the result.

The practical model is layered:

• one accountable owner for the affected business outcome;
• technical, product, data and service owners responsible for producing and operating evidence;
• control functions with defined rights to advise, require evidence, escalate and, for specified conditions, block;
• independent assurance that does not become a substitute for management ownership.

The distinction matters because committees coordinate. They can set standards, compare exposures and challenge proposals. They rarely operate the service or own its outcome. If a committee makes a reserved decision, its mandate should specify the chair, quorum, decision rule and treatment of dissent. Collective membership should not make the final owner unidentifiable.

The same logic appears in the PRA’s model-risk management principles, where an accountable senior manager owns the overall framework without displacing responsibilities across business, risk and control functions. The scope of that supervisory statement must be respected, but the organisational lesson travels well: accountability and specialist ownership are complementary, not interchangeable.

A human in the loop is not a control specification

Case study O is the FCA’s sharpest warning. A firm’s human-review role became a tick-box exercise. Reviewers increasingly accepted AI recommendations without sufficient scrutiny. Unclear escalation criteria allowed some low-confidence or uncertain outputs to proceed.

Naming a reviewer proves only that a person appears somewhere in the workflow. Meaningful oversight requires a purpose, competence, sufficient information, manageable workload, independence, authority to intervene and a clear escalation route.

The Information Commissioner’s Office human-review guidance is specific on these operating conditions. It calls for reviewers with appropriate knowledge, experience, authority and independence, supported by structured testing, documented tolerances, override logs and reporting to senior management. The ICO notes that the guidance is under review following the Data (Use and Access) Act, so its current status should travel with any reliance on it.

Research on human-AI decision making also argues against a simple promise that people will catch machine errors. The literature does not establish a universal automation-bias effect. It does show that context matters. In a series of experiments across perceptual, emotional and social tasks, Glickman and Sharot found that repeated interaction with biased AI could amplify subsequent human bias. In five preregistered personnel-selection experiments, Cecil and colleagues found that explanations did not reliably prevent participants from following incorrect AI advice. These are experimental settings outside banking, so they identify possible failure mechanisms rather than estimate harm in a regulated financial workflow.

The assurance target should therefore be the joint human-AI workflow. Test representative users with correct, incorrect, uncertain and abstaining outputs. Observe whether they challenge the system, use the escalation path, record a reason and reach the right outcome under realistic workload and time pressure.

Review counts and training completion are weak proxies. More useful evidence includes override reasons, error-catching rates, inappropriate acceptance, time to escalate, unresolved exceptions and outcomes after intervention. Those metrics belong in the next instalment of this series. The accountability point comes first: a reviewer cannot be responsible for a decision they lack the information, time or authority to change.

Risk appetite needs an executable form

Boards often approve statements about responsible AI, acceptable risk and human control. The statements become operational only when someone translates them into limits and predetermined actions.

In case study M, the relevant senior manager and business areas approved an incident playbook before deployment. It covered service interruption, unexpected outliers and performance below agreed thresholds. Elsewhere, case study P describes red, amber and green thresholds with predefined actions.

An incident playbook is an accountability instrument because it allocates authority before pressure distorts the decision. It should answer:

• Who can declare an incident or pause the service?
• Which threshold triggers enhanced monitoring, restricted operation or shutdown?
• Who must be notified, and when?
• Which fallback process is viable?
• What evidence must be preserved?
• Who can approve restart, on what validation and with which open conditions?

This is where risk appetite stops being an aspiration. Every material threshold needs an owner, data source, review frequency, breach action and exception authority. The person who can stop a system at 02:00 should not have to discover whether commercial leadership permits it.

An AI Accountability and Decision Rights Matrix

The following matrix is an original synthesis of the evidence, not an FCA-prescribed template. It should be adapted to the firm’s legal entities, Statements of Responsibilities, materiality framework and committee mandates.

Lifecycle decision	Final accountable owner	Evidence producers	Mandatory challenge	Trigger or limit	Required record
Admit use case	Business SMF or authorised business executive	Product or service owner	Risk, compliance, privacy, security, data and architecture as relevant	Purpose, affected outcome, preliminary materiality and prohibited use	Use-case charter and owner acceptance
Approve classification and controls	Business SMF for material uses; delegate within appetite for lower tiers	Service owner and risk coordinator	Independent risk or compliance; model risk where applicable	Impact, autonomy, uncertainty, sensitive data, customer or market effect	Classification rationale, perimeter and control plan
Accept technical validation	Model or system owner, subject to independent validation authority	Development and validation teams	Model risk, security, data and domain specialists	Performance, robustness, failure modes and unresolved limits	Versioned evaluation and validation opinion
Approve deployment	Business SMF for high materiality; authorised forum or delegate otherwise	Product, service and operations owners	Risk, compliance, resilience and legal where needed	Residual risk within appetite; fallback and playbook tested	Deployment decision, conditions and review date
Override, abstain or escalate a case	Named operational decision owner within mandate	Trained reviewer or automated workflow	Supervisor, quality assurance and specialist escalation	Confidence, impact, exception or prohibited output	Exact system version, reviewer action, reason and outcome
Contain an incident or pause	Incident commander with emergency authority; accountable SMF notified	Operations, security and model owners	Risk, compliance, legal, resilience and privacy	Red or amber threshold, harm, outlier, provider or control failure	Incident record, evidence, notification and fallback
Restart after an incident	Business SMF or pre-authorised risk forum	Service owner and remediation lead	Independent validation and affected control functions	Root cause understood, remediation tested, residual risk accepted	Restart decision, validation and open conditions
Approve material change	Same level as original approval where risk changes materially	Technical and service owners	Risk, compliance, procurement and model risk	New provider, capability, data, permission or failed regression	Change assessment and regression evidence
Retire the system	Business or service owner; SMF for material services	Operations, data and records owners	Legal, compliance, resilience and privacy	Replacement, unsupported model, risk breach or strategic exit	Decommission plan and retention actions
Provide independent assurance	Board or audit committee commissions; internal audit remains independent	Internal audit or qualified assurance	Relevant specialists	Risk-based plan, incidents and control change	Scope, evidence, findings and closure

Three design rules prevent the matrix becoming another static RACI chart.

First, assign one final owner per decision. Several specialists may produce evidence, but accountability should not dissolve across them. If authority crosses legal entities, document the joint decision and escalation.

Second, match accountability to authority. Do not name an owner who lacks budget, access, stop power or control over the business process. Control functions need explicit challenge rights. Challenge does not always mean veto, but it must mean more than optional advice.

Third, scale the control altitude by materiality. Senior managers should not approve every low-risk productivity tool. They should approve the framework, high-impact uses and exceptions, then receive evidence that delegated decisions remain within appetite. Proportionality changes control intensity, not the need for ownership.

Every decision record should identify the exact model, prompt, policy, tool, data and provider version that mattered. It should capture alternatives, limitations, challenge, dissent, residual risk, conditions, expiry, escalation and the later outcome. That record is not administrative residue. It is evidence of how the accountable person understood and managed the risk.

What the decision record must prove

A useful decision record should let an informed reviewer reconstruct the choice without being present in the meeting. It needs more than an approval date and a green status.

At minimum, it should identify the intended use and affected outcome, the accountable owner, the exact system version, the materiality and autonomy classification, and the regulatory perimeter considered. It should link to the evidence reviewed, record known limitations and show which specialists challenged the proposal. Where challengers disagreed, the record should preserve the dissent and the accountable owner’s response.

The approval itself should state the residual risk, any link to risk appetite or a formal exception, and the conditions under which use may continue. Conditions need owners, deadlines and consequences. A promise to improve testing later is not equivalent to a deployment condition unless someone must verify it by a named date and can restrict or stop the service if it remains unresolved.

The record should also point forward. It should name thresholds, review or expiry dates, fallback arrangements and the incident playbook. Later changes, breaches and outcomes should refer back to the original decision. That creates a traceable chain rather than a folder of isolated approvals.

This does not require a vast document for every use case. A low-materiality internal tool may need a concise record. A high-materiality system affecting customers, markets or an important business service should carry a commensurately stronger evidence bundle. The test is whether the record is sufficient for the decision’s impact and uncertainty.

Two traps are worth avoiding. The first is documentary theatre: producing extensive artefacts that do not change who can decide or intervene. The second is retrospective rationalisation: updating the record after an incident until it resembles a decision process that did not actually occur. Version history, contemporaneous challenge and electronic approval matter because accountability depends on what was understood and decided at the time.

The same discipline improves organisational learning. When a threshold is breached or a reviewer overrides the system, the firm can compare the outcome with the assumptions in the approval. It can then decide whether to adjust the model, workflow, limits, training, supplier arrangement or risk appetite. Accountability becomes a feedback mechanism, not simply a defence file.

Third-party AI widens the accountability-information gap

Outsourcing makes this discipline more important. The 2026 survey reports that third-party implementations represented 53% of high-materiality AI systems. Only 24% of respondents reported deep knowledge of high-materiality third-party systems, compared with 77% for internally developed systems. For third-party systems, 81% instead relied on vendor assurance and contractual arrangements.

The answer is not an impossible demand for total transparency. It is a materiality-based decision about the evidence the firm needs to remain accountable: behaviour and outcome testing, constrained autonomy, change notification, regression rights, incident cooperation, rollback, substitution and exit. An internal business owner remains necessary even when the supplier owns the model.

This is also where the matrix exposes a choice that a policy can hide. If the firm cannot obtain enough evidence to approve a material use, the accountable decision may be to narrow the use, add independent testing, build a viable fallback or not deploy.

Five questions for the board

The board should not take over management’s AI decisions. It should demand evidence that management’s accountability system works. Five questions make that concrete:

1. For our most material AI systems, where does decision authority actually sit, including emergency stop and restart? 2. What operating evidence shows that human oversight works when AI is wrong, uncertain or persuasive? 3. Which thresholds translate risk appetite into action, who owns them and what happened in the latest amber or red case? 4. Which material systems and providers have tested incident, fallback and restart playbooks? 5. Which third-party or shared dependencies create the largest gap between accountability and control, and what decision has management taken about it?

The FCA’s case studies do not prescribe one answer. They make it harder to accept a vague one.

Clear accountability tells the organisation who must act. The next question is whether its monitoring gives that person a timely, outcome-focused signal. That is the subject of Part 2.

Sources

• FCA, AI in financial services: good practice and areas for improvement
• Bank of England and FCA, Artificial intelligence in UK financial services 2026
• FCA, Senior Managers Regime
• FCA Handbook, COCON 4.2
• PRA, Model risk management principles for banks
• ICO, Human review
• Glickman and Sharot, How human-AI feedback loops alter human perceptual, emotional and social judgements
• Cecil et al., Explainability does not mitigate the negative impact of incorrect AI advice in a personnel selection task
• NIST, AI Risk Management Framework Core
• Financial Stability Board, The Financial Stability Implications of Artificial Intelligence
• Institute of Internal Auditors, Three Lines Model