What is AI risk?
Governance, risk and assurance
AI risk is the possibility that an AI system causes harm, loss, or unacceptable uncertainty for a business, its customers, staff, or wider society. It includes technical failure, data and privacy issues, cyber threats, legal exposure, bias and fairness concerns, reputational damage, financial waste, and organisational misuse. AI needs special attention because its behaviour can be probabilistic, data-dependent, hard to explain, fast to scale, and in some cases able to act with limited human control.
Reviewed by Jackie, Head of Learning & Development, Levellers · Last reviewed 8 June 2026
What this means
Most business leaders already understand technology risk. Servers fail, software has bugs, projects run late, and suppliers disappoint. AI includes all of that, but adds a different layer of uncertainty. Many AI systems do not follow a fixed rule set from input to answer. They infer patterns from data, produce variable responses, and can behave credibly while still being wrong.
That matters because the system's quality is tied to the quality of the data, the way the task is framed, the prompts or instructions it receives, and the environment in which it is used. A model that looks strong in a demo can perform poorly in live work, especially when people ask unusual questions, when data shifts, or when the system is given access to tools and live business processes.
AI risk is also more socio-technical than ordinary software risk. Harm often appears through interaction between model behaviour, human judgement, policy, incentives, and process design. A flawed model on its own may not cause much trouble. The same model placed inside hiring, lending, customer advice, security triage, or agent-driven automation can create legal, commercial, and human effects very quickly.
Why it matters
AI is now moving from experiments into customer service, drafting, search, analytics, workflow automation, decision support, and agent-led task execution. That means AI risk is no longer only a concern for data teams or innovators. It has become a leadership issue because it affects revenue, cost, compliance, resilience, and trust.
Handled badly, AI can produce false advice, expose sensitive information, automate poor decisions, deepen bias, create security weaknesses, or lock a business into expensive tools that staff do not trust. Handled well, risk management does the opposite. It clarifies where AI is appropriate, where stronger controls are needed, and where a business should stop, redesign, or keep a human in charge. Good AI risk management is not a brake on adoption. It is what makes adoption credible and repeatable.
How it works
The main categories of AI risk
Operational and performance risk
Operational and performance risk is the risk that the system simply does not work well enough for the job it has been given. This includes low accuracy, inconsistent answers, hallucination, poor calibration, brittle behaviour on edge cases, unreliable retrieval, weak summarisation, and model drift after deployment. In classic machine learning, drift happens when the world changes and the model's learned patterns no longer match reality. In generative systems, reliability problems can also come from prompt changes, retrieval errors, context-window limits, or changes made by the vendor to a hosted model.
For business leaders, the key point is that AI performance is not a single score. A system can look excellent on average and still fail badly on the cases that matter most, such as vulnerable customers, unusual contracts, safety-critical events, or low-frequency fraud patterns. Operational risk therefore means asking whether the system is good enough for this specific task, under this specific level of scrutiny, with clear failure thresholds and fallback paths when it goes wrong.
Data risk
Data risk sits underneath much of AI risk because data shapes what the system learns, what it can see, and how it behaves in production. Problems include inaccurate, stale, incomplete, unrepresentative, or poorly labelled data. In generative AI, data risk also includes weak source selection for retrieval, poor document permissions, ambiguous provenance, and the silent reuse of personal or confidential material in places it should not appear.
Bias often begins here. If the data used to train, tune, or feed a system reflects historic inequity, skewed sampling, or unexamined proxy variables, the system may reproduce or amplify those patterns. Privacy and confidentiality risks sit here too. Teams can collect more personal data than they need, keep it too long, mix data across purposes, or allow prompts and files to be retained by external providers. At the more hostile end, data poisoning is the deliberate manipulation of training, fine-tuning, or retrieval data so the model behaves badly, becomes biased, or hides a backdoor.
Security risk
AI security risk has moved from a niche concern to a mainstream one. Prompt injection is the clearest example. An attacker can craft text that causes a model to ignore prior instructions, disclose information, or misuse connected tools. When the hostile instruction is hidden in retrieved content, emails, webpages, PDFs, or documents rather than in the user's visible prompt, the danger grows because the attack can travel through a seemingly normal workflow.
Other security risks include adversarial attacks that manipulate model behaviour, model theft, extraction of sensitive training information, insecure model files, poisoned third-party datasets, dependency and supply-chain compromise, weak plugin or tool permissions, and misuse of APIs at scale. The practical lesson is that the model is only one part of the attack surface. The surrounding system, including prompts, retrieval layer, model hosting, connectors, identities, secrets, logs, and user permissions, is usually where business harm is created or prevented.
Legal and regulatory risk
Legal and regulatory risk is broader than "might we get fined?" It includes whether the system's design, data use, automation logic, and deployment mode fit the laws and duties that already apply to the business. Data protection law, confidentiality, employment law, equality law, consumer protection, sector rules, product safety law, intellectual property, record-keeping duties, contract terms, and professional standards can all be relevant at once.
Training data and copyright remain especially active areas. Questions about lawful access to copyrighted material, text and data mining exceptions, licensing, opt-outs, and responsibility for generated content are still evolving across jurisdictions. The EU AI Act adds a new layer through a risk-based regime with obligations that depend on use and system type. The UK's current approach remains more principles-based and regulator-led, so firms often need to comply through existing law rather than wait for a single AI-specific statute. For many organisations, the operational consequence is simple: legal review has to happen before procurement and before deployment, not after a public problem.
Ethical and societal risk
Ethical and societal risk concerns whether the system treats people fairly, respects dignity and autonomy, and can be understood and challenged in the contexts where it is used. Bias and discrimination sit at the centre, but the category is wider than that. It also covers opacity, weak explainability, inaccessible design, manipulation, surveillance, exclusion of minority groups, and removal of meaningful human judgement where judgement still matters.
This is one reason AI risk cannot be reduced to model metrics alone. A tool can be statistically strong and still ethically weak if it is used for the wrong purpose, applied to the wrong population, or embedded in a process that gives affected people no route to question or correct it. Human oversight matters here, but only if the oversight is real. A nominal reviewer who cannot see enough information, does not understand the model's limits, or has no authority to intervene is not a control. It is theatre.
Reputational and trust risk
Reputational and trust risk is often the first risk leaders feel, even when the root cause sits elsewhere. Customers rarely separate a model error from the company that deployed it. Staff do not distinguish between a weak vendor control and internal governance failure. If an AI chatbot invents policy, if a recruitment tool appears unfair, or if a staff member leaks sensitive material into an unapproved system, the reputational damage lands on the organisation.
Trust risk is not only external. Internal trust matters just as much. If employees do not believe the tools are safe, fair, or useful, adoption stalls or becomes covert. If leaders over-claim what AI can do, credibility falls when reality catches up. Businesses often discover too late that trust is cumulative and fragile. It is built through clear use boundaries, transparency about limits, consistent human judgement, and visible correction when something fails. Without that, every new deployment carries more friction than the last.
Financial and strategic risk
Financial and strategic risk is the risk that AI consumes money, management attention, or strategic flexibility without earning its keep. This includes inflated business cases, hidden inference and integration costs, poor procurement, duplicated tooling, weak adoption, unmanaged experimentation, sunk cost attachment to failing projects, and vendor lock-in that limits bargaining power or exit options later.
There is also opportunity cost. Some organisations spend too long debating abstract AI futures while competitors improve service, speed, and efficiency in bounded, lower-risk areas. Others rush into expensive programmes with no clear operating model and discover that the real constraint was process design, data discipline, or staff capability rather than model capability. Strategic AI risk therefore cuts both ways. It is the risk of doing the wrong thing, scaling the wrong thing, or waiting so long that the right thing becomes harder and more expensive.
Human and organisational risk
Human and organisational risk is frequently underestimated because it does not look technical. Yet many AI failures begin in incentives, ownership, and behaviour. Teams may over-rely on fluent answers, accept weak drafts without checking them, use personal accounts or unapproved tools, or assume that buying from a large supplier transfers accountability elsewhere. Staff can also become less skilled if critical reasoning, writing, analysis, or judgement are repeatedly handed off without deliberate safeguards.
Shadow AI is part of this picture. When people feel pressure to move quickly but have no approved tooling, policy, or support, they will often use whatever is easiest. That can create fragmented data handling, inconsistent records, legal exposure, and conflicting versions of the truth. Organisational risk also includes change failure. A technically sound system can still fail if the business has not defined who owns it, how it fits existing controls, what staff should trust it for, and when they must ignore or override it.
Agentic and autonomous system risk
Agentic and autonomous systems increase the risk profile because they do not only generate content. They can plan, choose tools, call other systems, write or modify records, send messages, trigger transactions, and keep working across multiple steps with reduced human intervention. This creates a sharper version of familiar risks and introduces new ones around scope, action, and control.
An ordinary chatbot that invents an answer may waste time. An agent with write access to finance, CRM, procurement, email, or code repositories can turn the same underlying error into a serious incident. The core risks are over-privileged access, unintended actions, cascading errors across systems, prompt injection through retrieved content, runaway loops, unauthorised spending, weak auditability, and humans losing the ability to understand or stop what the system is doing. For this reason, agentic AI should be treated as a different control class. It needs tighter scope, stronger approval gates, least-privilege access, richer logging, clearer ownership, and faster kill-switches than a read-only assistant.
How the risks interact and compound
Compound failure chains
AI risks rarely arrive one at a time. They move through the system as chains. A data defect becomes a performance issue. That performance issue becomes a fairness issue because the errors hit one group harder than another. Complaints then turn it into a reputational problem, while the same facts create legal and regulatory exposure. If leaders treat each category in isolation, they miss the compound nature of the failure.
Generative AI makes this even more obvious because model behaviour, retrieval quality, user behaviour, interface design, and tool permissions interact constantly. A model can be technically capable yet still produce serious harm because the surrounding process encourages over-trust, hides uncertainty, or lets one weak step trigger a real-world action.
A sales team deploys a lead-scoring model trained on messy CRM history. The data under-represents smaller clients and reflects old sales habits. The model starts steering effort toward deals that resemble the past. Revenue forecasts skew, some customer segments receive poorer service, and complaints rise. What began as data quality risk has now become operational, fairness, commercial, and reputational risk.
A customer service agent is allowed to read emails and approve small refunds. A hostile message includes hidden text crafted to redirect the agent's reasoning. The agent follows the malicious instruction, exposes internal policy text, and issues refunds it should not authorise. That sequence moves from security risk to financial loss, confidentiality concerns, incident response, and damaged trust in automation.
Why isolated treatment fails
Isolated treatment fails for another reason. Controls in one area can shift risk elsewhere. More data may improve accuracy but worsen privacy exposure. Tighter guardrails may reduce misuse but make the system less helpful, sending staff back to shadow tools. Heavy human checking may reduce legal risk but erase the economic case for automation. Good governance does not pretend these tensions disappear. It makes them explicit, assigns owners, and sets trade-offs in line with risk appetite and business priority.
Examples
A recruitment firm using AI to summarise CVs
A mid-sized recruitment business wants to use an AI assistant to summarise CVs and draft short candidate notes for consultants. On paper the task looks administrative. In reality it touches fairness, employment, privacy, and reputation. If the assistant overemphasises certain schools, employers, age cues, or gaps in employment, it can nudge consultants toward biased screening even if no automatic ranking is visible.
A proportionate approach is to keep the system in an assistive role, remove direct candidate scoring, test summaries against a representative sample, strip unnecessary personal attributes where possible, and give consultants explicit guidance that the tool is not to be used as the final basis for suitability decisions. Add a DPIA if personal data use is high risk, document the intended use and exclusions, and sample outputs regularly for bias and accuracy.
A retailer deploying a customer service refund agent
An online retailer wants an AI agent to answer delivery questions and issue low-value refunds. The commercial case is strong, but the risk changes the moment the system can act. A prompt injection hidden inside a message or retrieved page could steer the agent off course. The model may also invent policy, disclose internal logic, or apply exceptions inconsistently. That creates financial, security, consumer, and trust risk at the same time.
The safer design is narrow scope. Give the agent read access to order status and a tightly capped refund function, not broad account control. Require human approval above a threshold. Use allow-listed actions, rate limits, strong logging, and daily monitoring for unusual refund patterns or policy drift. Test with adversarial prompts before launch and after major model changes. Keep a kill switch and a fallback route to human support.
A professional services firm adding an internal knowledge assistant
A law, accountancy, or consulting firm introduces a retrieval-based assistant over internal documents and client work. Staff love the speed, but the real risk is confidentiality and over-trust. If permissions are wrong, the assistant may surface material the user should not see. If the retrieved source is weak or out of date, the answer may sound authoritative while being wrong. If staff start pasting client material into external tools outside the approved environment, shadow AI risk appears immediately.
A sensible control set starts with access boundaries that mirror document permissions, plus guidance on what files may and may not be uploaded. Answers should show source references internally so the user can verify them. External-facing advice should require human sign-off. The firm should log usage, review high-risk queries, train staff on confidentiality and checking discipline, and require procurement review before any new AI add-on is connected to client data.
A manufacturer using predictive maintenance
A manufacturer deploys a model to predict equipment failure from sensor data. This is not generative AI, but the risk pattern is still distinctively AI-shaped. If the model drifts as machines age or maintenance patterns change, false negatives can create downtime or safety exposure. If the data pipeline drops or scrambles sensor readings, operators may trust a weak forecast. If the vendor updates the model without clear notice, plant performance can change before anyone realises why.
The right response is to treat the model like an operational control, not a dashboard novelty. Define the human fallback process, track false positives and false negatives, compare the model with maintenance engineer judgement, monitor live drift, and require controlled change management for vendor or feature updates. Where safety is at stake, the model should inform maintenance prioritisation, not quietly replace engineering judgement.
Common misunderstandings
AI risk is mainly about distant existential scenarios. In most businesses, the immediate risks are ordinary and concrete: bad advice, weak controls, privacy failures, unfair treatment, insecure integration, and expensive missteps.
If the tool is only internal, the risk is low. Internal systems can still leak data, mislead staff, distort decisions, and create compliance problems, especially when they touch personal, confidential, or regulated information.
Human-in-the-loop automatically makes AI safe. A human reviewer only helps if they understand the task, can see enough evidence, have time to judge properly, and are empowered to override the system.
Buying from a major vendor transfers the risk. Suppliers can reduce build effort, but the deployer still owns many of the legal, process, and operational effects of use.
If a demo looks impressive, production risk is manageable. Demos rarely show edge cases, hostile prompts, permission problems, or the behavioural shifts that happen once staff start relying on a tool every day.
Compliance and good governance are the same thing. Compliance matters, but a firm can still deploy a legally scoped system badly if it lacks clear ownership, monitoring, change control, or real oversight.
The model is the whole system. In practice, much of the risk sits around the model: data pipelines, prompts, retrieval, tools, permissions, contracts, staff behaviour, and process design.
Risks and boundaries
How AI risk is assessed
Start with the use case and context
Good assessment starts with the task the system will perform, not the model's marketing label. Leaders should ask: What decision or action is this AI influencing? Who could be affected? What data does it need? What would failure look like? How serious would that failure be? Can a person detect and correct it in time? Does the system only advise, or can it act? Is it internal, customer-facing, or part of a regulated process?
This approach prevents two common mistakes. The first is treating all AI as equally risky. The second is assuming the same model has the same risk everywhere. A summarisation model used to draft internal meeting notes is not the same risk as the identical model used to generate medical triage suggestions or rank job candidates.
Use a structured cycle of Govern, Map, Measure, Manage
A practical structure is the NIST AI Risk Management Framework. Govern is the foundation layer. It covers policy, roles, accountability, risk appetite, training, escalation, and links to existing governance. Map is where the business defines context, intended use, stakeholders, data sources, system boundaries, dependencies, and possible harms. Measure is where the organisation tests what it can test, chooses metrics, runs evaluation, and documents what cannot yet be measured reliably. Manage is where leaders decide what to do with the risk, including mitigation, transfer, acceptance, pause, redesign, or decommissioning, and where they keep monitoring live performance.
ISO/IEC 23894 complements this well. It treats AI risk management as something to be integrated into existing organisational risk work rather than run as a separate island. For many firms that is the right mindset. AI should plug into established governance for information security, supplier management, legal review, internal audit, operational resilience, and business continuity.
Rate materiality, not just model quality
Assessment should reflect more than whether the model is "accurate". Leaders need a view of materiality. Useful dimensions include severity of harm, likelihood, number of people affected, speed of impact, detectability, reversibility, and degree of human control. A low-probability error can still be high risk if it affects patient safety, employment rights, access to essential services, or the integrity of financial records.
Quantification helps where it is credible, but AI risk often resists neat probability estimates, especially early in deployment. That is normal. Use a mix of testing evidence, scenario analysis, expert judgement, red-team findings, pilot data, and operational metrics. Where measurement is weak, the answer is not to ignore the risk. It is to recognise that uncertainty itself is part of the risk and act more cautiously.
Connect AI assessment to enterprise risk and impact assessments
AI risk should feed into the organisation's wider enterprise risk management process. Material AI uses should sit on the risk register, with named owners, agreed controls, review dates, and clear thresholds for escalation to senior leadership. Procurement, legal, security, privacy, HR, and operations should all be able to see where AI is being used and which controls apply.
Where personal data is involved, a data protection impact assessment may also be required. In many AI deployments it will be. A DPIA focuses specifically on risks to individuals' rights and freedoms arising from personal data processing. That makes it essential, but not sufficient by itself. It should be treated as one component of the broader AI risk assessment, alongside security review, fairness analysis, supplier due diligence, and any sector-specific or equality impact work that applies. For smaller organisations, the most proportionate method is often one joined-up assessment pack rather than a pile of disconnected forms.
Prioritise with clear decisions
The point of assessment is not paperwork. It is decision-making. Every material AI use should end with one of a small number of decisions: proceed, proceed with conditions, pilot only, redesign, or stop. The conditions might include narrower scope, stronger human review, additional testing, better data controls, or a change in supplier terms. Residual risk should be visible, not hidden. Senior leaders do not need every technical detail, but they do need a clear statement of what could go wrong, what controls are in place, and what would trigger reassessment.
Mitigation and controls
Human oversight and decision rights
Human oversight works when it is designed as an operating control, not a slogan. The reviewer must know what the system is for, what its limits are, when to distrust it, and what authority they have to intervene. In low-risk drafting work, this may mean simple review before external use. In higher-risk settings, it may mean dual approval, mandatory explanation fields, or a rule that the AI can recommend but never decide. The more severe the possible harm, the more meaningful the human control needs to be.
Good oversight also separates roles. The person benefiting from speed is not always the best person to challenge the model. For important decisions, consider whether the same employee should both operate the tool and approve its result. When agentic systems are involved, define who authorises access, who monitors activity, who handles incidents, and who can shut the system down immediately.
Testing and evaluation before release
Before deployment, test the system against the real job it will do. Use representative examples, edge cases, and failure cases, not only ideal prompts and polished data. Compare the system with human performance, rule-based methods, or simpler analytics where relevant. Record acceptable error limits, confidence thresholds, and the conditions under which the tool should not be used.
Evaluation should include more than task accuracy. Check for fairness issues across relevant groups, stability under minor input variations, behaviour under stress, data handling, and whether the system remains helpful when guardrails are applied. If a use case depends on retrieval, evaluate retrieval quality separately from generation quality. If it depends on tool use, test permissions and side effects separately from language quality.
Red-teaming and adversarial testing
Red-teaming is structured challenge testing. It looks for ways the system can be manipulated, bypassed, misused, or pushed beyond its intended boundaries. For a chatbot this may mean jailbreak attempts, prompt injection, abusive content, impersonation, or data exfiltration tests. For a workflow agent it may mean testing whether hostile inputs can trigger unauthorised actions, whether limits can be bypassed, and how the system behaves when tools fail or return conflicting data.
Red-teaming should be proportionate and repeated. The need is greater when the system is customer-facing, high-volume, safety-relevant, or connected to live tools. It is also greater after significant changes, such as a model upgrade, a new connector, broader permissions, or expansion into a new user group.
Guardrails, access control and secure architecture
Many AI harms can be reduced by changing architecture rather than trying to make the model itself perfect. Place rules-based checks around the model. Limit what data it can retrieve. Restrict which tools it can call. Use allow-lists instead of open-ended actions, enforce thresholds for payments or record changes, and keep high-impact tools behind human approval gates. For sensitive environments, isolate the model from critical systems unless there is a clear business need.
Access control matters especially with agentic AI. Use least privilege, short-lived credentials, and strong separation between read, write, and admin capabilities. Treat third-party models, plugins, packages, and public data sources as supply-chain dependencies that require scrutiny. A model with broad access and poor containment is not a productivity feature. It is an incident waiting for a trigger.
Monitoring, observability and drift management
AI control does not stop at launch. Systems need live monitoring for quality, reliability, misuse, drift, unusual behaviour, and user feedback. That usually means logging prompts, key context, outputs, tool calls, safety events, overrides, and incidents, with privacy and security controls around those logs. Sample outputs regularly. Watch for changing error patterns, not just average scores. Track near misses, because they are often the earliest signal of a larger failure.
Model drift deserves explicit attention. Changes in customer behaviour, product mix, language, regulation, or source data can quietly erode performance. Hosted foundation models can also change because the provider updates them. If a change is material, the system may need re-testing, re-approval, narrower use, or retirement.
Documentation, traceability and assurance evidence
Documentation is a control because it makes limits visible and decisions auditable. A strong documentation pack can include intended use, excluded use, training and evaluation context, key metrics, known limitations, data sources, approval records, change logs, human oversight design, and incident pathways. Model cards and similar artefacts are useful because they force the team to state what the model is good for, how it was tested, and where it should not be trusted.
This is also where AI assurance begins. Assurance turns testing, documentation, monitoring, and review into evidence that other people can rely on, whether that is a board, a customer, an auditor, or a regulator. A mid-sized organisation does not need heavyweight bureaucracy for every pilot, but it does need enough artefact quality that another person, six months later, can understand what was approved, on what basis, and with which safeguards.
Vendor due diligence, incident response and policy
Third-party AI can reduce build risk, but it does not remove deployment risk. Ask suppliers how they handle security, access control, data retention, sub-processors, training data governance, incident reporting, model updates, and regional hosting. Understand what the contract says about confidentiality, liability, intellectual property, audit rights, and service changes. If the tool will be used in a regulated or human-impacting process, ask for testing evidence and clear documentation rather than marketing claims.
Have an AI-specific incident path before something goes wrong. Know who investigates, who can suspend a tool or revoke an agent's permissions, how affected users are informed, when legal or privacy teams are involved, and how lessons are captured. Back this with policy and training. Staff should know which tools are approved, what data they may use, what must never be uploaded, how to report problems, and when AI use requires extra review.
Governance and regulation
AI risk management sits inside AI governance
AI governance is the management system around AI use. Risk management is one of its core functions, but not the whole of it. Governance sets policy, ownership, approval routes, inventories, escalation paths, monitoring responsibilities, reporting lines, and assurance expectations. In practical terms, governance answers questions such as: Who can approve an AI use? What evidence is needed? Which uses must go to legal, privacy, or security? How are incidents reported? How often is a live use reviewed?
Without governance, risk work stays ad hoc. One team tests carefully while another deploys an unreviewed tool through procurement or a browser tab. Good governance creates consistency and proportion. It does not mean every use needs a committee. It means every use has a route that matches its materiality.
How ISO/IEC 42001 fits
ISO/IEC 42001 is useful because it gives organisations a management-system structure for governing AI across the business. Think of it as the organisational skeleton rather than the test for any one model. It helps firms define policy, responsibilities, life-cycle controls, monitoring, and continual improvement. ISO/IEC 23894 then sits closer to the mechanics of AI risk management itself, helping organisations identify, analyse, treat, and monitor AI-specific risks in context.
For a mid-sized organisation, these standards are most helpful as design guides. They can stop governance from being improvised and help align AI with existing ISO-style disciplines such as information security, quality, or privacy management. They do not replace judgement, sector rules, or use-case testing, but they can make all three easier to run consistently.
The EU AI Act
The EU AI Act is the clearest current example of risk-based AI regulation with global commercial reach. It matters not only to EU-headquartered firms, but to any business placing certain AI systems on the EU market or whose AI use touches EU users, customers, or supply chains. The Act prohibits a narrow class of unacceptable practices outright, imposes detailed requirements on high-risk systems, creates transparency duties for certain interactive and synthetic-content systems, and adds separate obligations for general-purpose AI models.
As of now, prohibited practices and AI literacy duties already apply. Transparency duties for certain systems apply from 2 August 2026. The Commission's current implementation timetable places the main Annex III high-risk duties later, with the broad set of high-risk use cases scheduled from 2 December 2027 and some AI embedded in regulated products later still. Businesses should not misread the phasing as permission to wait. Procurement, design, record-keeping, and supplier terms often need to change well before formal applicability dates.
For high-risk systems, the compliance burden is substantial. Providers are expected to run documented risk management, suitable data governance, technical documentation, logging, human oversight, accuracy, robustness and cybersecurity controls, conformity assessment, and post-market monitoring. Deployers have their own duties around use, oversight, monitoring, and in some cases impact assessment and information to affected people. The Act also adds obligations for general-purpose AI models, including documentation and copyright-related compliance measures, with extra duties for models that present systemic risk. Even organisations that never build models will feel these requirements through contracts, documentation requests, and customer due diligence.
The UK's regulator-led approach
The UK is taking a different path. Instead of a single horizontal AI law on the EU model, it has chosen a principles-based, regulator-led approach. The core principles are safety, security and robustness, appropriate transparency and explainability, fairness, accountability and governance, and contestability and redress. These are intended to be applied by existing regulators within their remits rather than through one new statute for all AI use.
For business leaders, the practical effect is that AI risk in the UK is often managed through the laws and regulators you already have, not through an AI-only rulebook. Data protection still matters. Equality and employment duties still matter. Consumer protection, financial conduct, product safety, medical device regulation, professional obligations, contract law, and intellectual property still matter. In practice that means mapping the relevant regulator set for your sector, such as privacy, finance, competition, health, or workplace oversight, before you scale a use case.
Copyright is a good example. The policy debate around training on copyrighted works remains live, and the government has said it is not yet ready to reform copyright law until it is satisfied that any change would meet its objectives. That means legal uncertainty remains a real business risk, especially for providers and for deployers who rely heavily on supplier assurances.
Sector rules, assurance, and the practical global picture
In practice, many organisations will be pulled by several regimes at once. A UK business serving EU customers may have to think about the EU AI Act. A firm selling into healthcare, finance, recruitment, or public services may face tighter sector expectations regardless of where it is based. Enterprise customers increasingly ask vendors for evidence on governance, testing, privacy, security, and incident handling before they buy. That procurement pressure is already acting as a form of market discipline.
This is why assurance matters. Boards, customers, and regulators increasingly want evidence, not aspiration. A credible AI governance framework therefore links policy to documented risk assessment, control design, testing records, monitoring data, incident logs, and periodic review. Regulation raises the stakes, but good governance would still be worth doing without it because it turns AI from a promising but fragile capability into something the organisation can rely on.
What to do next
Make an AI inventory. List every live, piloted, and proposed AI use, including vendor tools, embedded features, and likely shadow use in common departments.
Classify each use by impact and autonomy. Ask what it influences, who it affects, what data it touches, and whether it only advises or can act.
Appoint an owner for every material use. The owner should be a business leader, not only a technical specialist, and should be accountable for review, controls, and operation.
Create a minimum control baseline. For most mid-sized organisations this means approved-tool policy, privacy and security review, basic supplier due diligence, documented intended use, human review rules, and incident escalation.
Use one joined-up assessment pack for material cases. Combine AI risk review with DPIA, security review, and sector-specific checks where needed instead of creating disconnected documents.
Pilot in bounded areas first. Choose tasks with clear value, lower inherent harm, and easy rollback. Keep scope tight until the business has evidence that the controls and the tool both hold up.
Monitor after launch. Set review dates, sample outputs, capture user feedback, track drift or misuse, and define triggers for re-approval when models, permissions, or use cases change.
Set board-level reporting that is simple and useful. Senior leaders should see where AI is used, which systems are high impact, what incidents or near misses have appeared, and where residual risk is being accepted.
Have a question or a suggestion, or want to understand how we research and review these guides? Read about our editorial standards and how to reach us.
FAQs
Is AI risk only a problem if we build our own models?
No. Many of the biggest risks sit in procurement, configuration, permissions, use-case design, and live operation. A company using third-party AI can still cause privacy breaches, bad customer advice, unfair decisions, or security incidents.
Who should own AI risk internally?
Each material AI use should have a named business owner. That owner should work with security, privacy, legal, procurement, HR, and operations as needed. Enterprise coordination often sits with a COO, CIO, CDO, or a governance lead, but use-case ownership should stay close to the business process.
Do we need a formal AI policy before we test anything?
You need a minimum rule set before staff start using tools with real data. Start with acceptable use, prohibited data, approval routes, supplier review, and incident escalation. Expand into a fuller policy set as the use of AI broadens.
When is a DPIA needed for AI?
Often when personal data processing is likely to create high risk for individuals' rights and freedoms. Profiling, significant decisions, large-scale monitoring, sensitive data, and public-facing AI can all raise the likelihood. Many material AI uses should be treated as DPIA candidates until shown otherwise.
What is the difference between AI governance and AI risk management?
Governance is the overall system of policy, ownership, decision rights, review, monitoring, and assurance. Risk management is the discipline of identifying, assessing, prioritising, and treating specific AI risks. Risk management sits inside governance.
How often should we reassess a live AI system?
On a schedule and on change. Reassess after model upgrades, new permissions, new data sources, expansion into a higher-impact use, incidents, drift, supplier changes, or relevant legal updates. A good rule is that any material change should trigger at least a targeted review.
Are open-source models automatically riskier than closed models?
Not automatically. Open-source models can offer more control and flexibility, but they can also create more burden around security, patching, hosting, and supply-chain review. Closed models may reduce some operational effort while increasing dependency and visibility limits. The real question is whether your organisation can govern the option it chooses.
What documentation is enough for a mid-sized organisation?
Enough to explain purpose, data, supplier, limits, testing, approvals, human oversight, monitoring, and incident paths. If a second person cannot understand why the deployment exists, what it may do, and what safeguards apply, the record is too thin.
When should a leader say no to an AI use case?
When harm could be serious and the business cannot explain, monitor, bound, or control the system well enough. Also say no when legal footing is too unclear, when the use relies on unchecked automation in a high-impact decision, or when a simpler method can do the job more safely.
Can a mid-sized business manage AI risk without a large governance function?
Yes. The aim is proportion, not bureaucracy. A joined-up inventory, simple classification model, named owners, minimum control baseline, and careful pilots will take most organisations a long way if they apply those disciplines consistently.
Why does agentic AI need extra caution?
Because the same model error can trigger actions across tools and systems. Once an AI can send emails, change records, move money, or call software on your behalf, text mistakes become operational events. Agentic systems need tighter scope, least-privilege access, approval thresholds, detailed logging, and immediate stop controls.
Sources
AI RMF Core - AIRC - NIST AI Resource Center (National Institute of Standards and Technology). The structure of the NIST AI Risk Management Framework, especially the Govern, Map, Measure, and Manage functions and their use in assessment and prioritisation.
ISO/IEC 23894:2023 Information technology - Artificial intelligence - Guidance on risk management (ISO). The role of ISO/IEC 23894 as guidance on AI-specific risk management integrated into wider organisational risk practice.
ISO/IEC 42001:2023 Information technology - Artificial intelligence - Management system (ISO). The role of ISO/IEC 42001 as an organisation-wide management system standard for AI governance.
AI Act | Shaping Europe's digital future (European Commission). EU AI Act overview, governance structure, current implementation status, and timeline of applicability.
Navigating the AI Act | Shaping Europe's digital future (European Commission). High-risk AI system examples, transparency duties, provider and deployer obligations, conformity assessment, and the role of standards.
AI regulation: a pro-innovation approach (UK Government). The UK's principles-based and regulator-led approach to AI regulation.
Implementing the UK's AI regulatory principles: initial guidance for regulators (UK Government). The five UK AI regulatory principles and how they are meant to be interpreted by existing regulators.
What are the accountability and governance implications of AI? (Information Commissioner's Office). DPIAs, accountability, meaningful risk appetite, controller and processor issues, and the link between AI risk work and data protection law.
Guidelines for secure AI system development (National Cyber Security Centre). Lifecycle security controls across design, development, deployment, monitoring, logging, and incident management.
Thinking about the security of AI systems (National Cyber Security Centre). Prompt injection, data poisoning, system architecture, access controls, and the importance of treating AI as part of a wider security system.
Thinking carefully before adopting agentic AI (National Cyber Security Centre). The heightened risks of agentic AI, including over-privileged access, scope limits, human accountability, monitoring, and incident planning.
Report and impact assessment on Copyright and Artificial Intelligence (UK Government). The current UK policy position on copyright and AI training, ongoing uncertainty, and the lack of settled reform at the time of writing.