What is AI procurement and contracting?

How it works

Start with the use case and the role map

AI procurement starts by defining what is actually being bought and how it will be used. A buyer might be purchasing access to a model through an API, a finished application, an embedded AI feature inside another product, a bespoke build, or an integration layer that combines multiple third-party components. Those options look similar in a sales process, but they create different dependencies and different legal roles.

That role map matters because AI laws and control frameworks often attach duties to the role being performed, not just to the product label. Under the EU AI Act, responsibilities are spread across providers, deployers, importers, distributors and other parties in the value chain. A third party that changes intended purpose, makes a substantial modification, or puts its own name on a high-risk system can shift into provider obligations. For a buyer, that means procurement has to identify early whether the organisation is only a deployer, or whether its integration choices could move it into a heavier compliance position.

At this stage, the organisation should also decide whether the procurement needs enhanced control. Key triggers usually include use in decisions about people, use in public services, use of personal data, reliance on a general-purpose model or multiple sub-processors, difficult explainability, safety implications, cross-border data movement, or a critical dependence on the supplier for monitoring and maintenance.

Ownership usually sits with procurement or commercial leads and legal counsel, but the content must come from a multidisciplinary group. In practice that means technical owners, privacy, security, records, service operations, and the business team that will rely on the system in live use.

Turn AI risk into tender questions

Once the intended use is clear, the organisation turns risk into structured diligence. Official guidance is notably consistent on this point. Buyers should ask not just what the system does, but how it is built, what data it relies on, how it is tested, how it changes over time, and what restrictions apply to inspection, challenge and exit.

For many procurements, the core questions are durable. What is the model type and system architecture. What third-party models, data, tools or plug-ins sit underneath it. What categories of data were used for training, fine-tuning or reference retrieval. What information can be given about provenance, licensing and rights. What performance testing has been done, under what conditions, and with what limitations. What instructions for use, technical documentation, logs and support contacts will be available. What hosting, geographic transfer, retention or secondary-use practices apply. What human oversight assumptions does the system make. How are incidents, vulnerabilities and material changes reported. What fallbacks exist if the service is degraded, withdrawn or no longer suitable.

Good AI procurement also tests the supplier's operational maturity, not just the product description. NIST's playbook and GenAI profile both push buyers toward documentation of third-party components, independent testing and verification processes, monitoring plans, legal compliance checks, release-cycle awareness, and clear handling of vulnerabilities and external dependencies. UK government guidance similarly points buyers toward model and data questions, explainability, independent audit where useful, and early consideration of lock-in, liability, training and end-of-life.

This is the point where the organisation should decide what evidence must exist before contract signature. That usually includes technical and user documentation, risk assessments, privacy materials where personal data is involved, security materials, supplier responses on model limitations, sub-processor information, and a documented escalation path for incidents and changes.

Write clauses that allocate responsibility, not just risk

A strong AI contract does more than shift liability. It makes the system governable. In practice, this means using the contract to define purpose, transparency, control points, evidence creation and cooperation duties.

The first class of clauses covers vendor representations and factual commitments. Depending on the use case, buyers often seek representations about the supplier's authority to provide the system, the rights attached to data and model components it brings into the service, the accuracy of information supplied during procurement, the existence of known material limitations, and the supplier's ability to back up important performance, security or provenance claims. For generative AI, NIST specifically points buyers toward ownership, usage rights, quality standards, security requirements, provenance expectations, due diligence on IP and privacy risk, and contractual risk controls including indemnity language and dispute mechanisms where appropriate.

The second class covers data and privacy. If the supplier processes personal data on the buyer's behalf, ordinary procurement terms are not enough. The contract also needs controller-processor terms. GDPR Article 28 is the clearest mature example of what that means: documented instructions, confidentiality, security measures, sub-processor controls, assistance with rights requests and impact assessments, return or deletion of data at the end of service, and access to information needed for audits and inspections. If data leaves the relevant legal area, international transfer clauses may also be needed. This is where data-residency and data-sovereignty questions come into the contract, even though they are not the same topic as AI procurement itself.

The third class covers AI-specific documentation and operating controls. The AI buyer usually needs technical documentation, instructions for use, logging arrangements, human oversight requirements, accuracy and robustness thresholds, cybersecurity expectations, incident notification, and change control. The EU public buyer model clauses are useful because they lay these topics out plainly: risk management, data and data governance, technical documentation, record-keeping, transparency, human oversight, accuracy, robustness, cybersecurity, dataset rights, handover and indemnities. Even where those clauses are not used directly, they show the current anatomy of a serious AI contract.

The fourth class covers IP, datasets and exit. AI procurement often fails when buyers focus on access during the contract but ignore what happens at the end. Dataset rights, access to exportable records, handover of buyer data, rights to continue using documentation, transition support, knowledge transfer, and decommissioning obligations should be designed up front. That matters not only for continuity but also for reducing lock-in and preserving the evidence needed for later audit or challenge.

Make audit, monitoring and change control real

Audit rights in AI contracts are important, but they are often misunderstood. They are not only about source code access, and they are not useful if left as vague boilerplate. The real question is what the buyer must be able to inspect or verify to operate lawfully and safely in context.

A proportionate audit model usually works best. For lower-risk uses, that may mean rights to review policies, documentation, logs, incident notices, test summaries, system instructions and sub-processor lists. For higher-risk uses, it may extend to deeper technical information, access for independent assessors, real-time or periodic log access, validation of change controls, and evidence that the supplier is maintaining the controls promised at award. NIST's playbook is especially clear that third-party AI should be documented, tested, monitored and subjected to documented risk controls. It also notes that transparency should be supported without forcing unnecessary disclosure of proprietary algorithms.

Monitoring after signature is just as important as pre-award diligence. Many AI systems change through retraining, tuning, model substitution, feature updates, modified thresholds, or shifting external dependencies. Contracts therefore need a practical change-management regime. Buyers usually need notice of material changes, a right to re-assess impact when the system changes, the ability to pause or narrow use when risk rises, and a fallback process if a critical model or service is disabled. For generative AI in particular, NIST points to contingency planning, incident response, management of third-party dependencies, and review of non-standard vendor terms that may amplify or defer liability in unexpected ways.

This monitoring layer is also where record retention becomes a governance asset. Good procurement and contracting should produce an evidence trail, not just a signed agreement. That evidence usually includes tender responses, technical schedules, privacy schedules, risk assessments, approvals, logs, change notices, incident records, review reports, training records and exit materials. Together, those records help the organisation answer regulators, auditors, customers, boards and internal reviewers.

Adapt the control set to public sector and high-impact uses

Public sector and other high-impact environments often need more structure because the buyer must be able to justify the use of AI in administrative or rights-affecting settings, not merely run a procurement process.

The UK government's AI procurement guidance shows one route. It treats AI buying as a multidisciplinary exercise, calls for a data assessment before going to market, expects buyers to remain open to non-AI approaches, asks for transparency about models and training data, encourages thinking about independent audits, pushes against black-box dependence, and brings liability, training, maintenance and end-of-life into the procurement design. This is guidance rather than binding law, but it is influential because it translates broad governance aims into concrete procurement practice.

Canada's federal framework adds a more formal assessment layer for automated administrative decision systems. The Algorithmic Impact Assessment is mandatory for systems in scope, assigns impact levels, and links those levels to scaled requirements. The official guidance expects privacy and legal consultation, documentary evidence for AIA responses, updates when system functionality or scope changes, and appropriate privacy clauses in contracts with outside vendors. The peer review guidance adds another operational layer by expecting publication of peer review for projects at impact level 2 or above and by listing the kinds of documentation reviewers should receive, including audit trails, system documentation, data provenance, risk mitigation, supply chain security materials and procurement details for third-party systems.

In the EU, the legal position is stronger still. The AI Act imposes binding duties on high-risk system providers and deployers, including written arrangements along the value chain, technical and organisational measures for use in line with instructions, human oversight, monitoring, log retention, and in some cases a fundamental rights impact assessment before deployment. The right to explanation for certain individual decisions gives buyers another reason to contract for enough transparency and cooperation to explain how the system influenced a decision. The EU public buyer model clauses are not law and are written for public organisations, but they are a practical bridge between the Act's legal duties and day-to-day purchasing.

Know what this mechanism can and cannot do

AI procurement and contracting is powerful, but it is not magic. A contract cannot make an opaque system explainable if the supplier cannot technically support explanation. It cannot cure weak internal governance. It cannot remove the need for impact assessment, logging, staff training, human review or post-deployment monitoring. It can only secure the rights, duties, evidence and cooperation the organisation needs in order to govern the system properly.

It also needs proportion. Not every AI feature justifies a heavy high-risk clause set. The EU public buyer commentary is helpful here because it explicitly recognises lighter and fuller variants, and it explains that even non-high-risk uses may still call for contractual terms on risk management, data governance, technical documentation, dataset rights and registers. The point is not to copy the harshest drafting into every deal. The point is to match the control set to the use case.

Finally, legal status differs across instruments. NIST's framework and profiles are voluntary. The UK procurement guidance is a government toolkit. Canada's federal framework is a public-sector policy instrument. The EU AI Act and GDPR are binding legal regimes. Good AI procurement recognises that mix and uses the voluntary material to operationalise the binding material, not to replace it.