Saudi AI Vendor Due Diligence Checklist: 12 Questions Before You Buy Why does AI vendor due diligence need workflow-specific evidence? An AI agent can interpret a request, retrieve information, prepare an output, and invoke connected tools. Its behaviour depends on the model, sources, instructions, permissions, and surrounding application.
Conventional software can also create serious operational risks. Agentic systems add evaluation questions because their interpretation and outputs can vary, even when a request appears familiar.
A useful assessment starts with one proposed workflow. Establish its users, data, expected outputs, permitted actions, and consequences of failure. Then request evidence that matches those conditions.
The following twelve questions help procurement, operations, technology, security, and legal teams turn vendor discussions into a documented purchasing decision.
- What workflow are we buying, and who owns it? Ask the vendor to describe the inputs, outputs, users, connected systems, and exclusions.
“Support customer service” is too broad for acceptance testing. “Prepare suggested answers for a defined category of enquiries, using approved sources, for employee review” is more assessable.
Name the business owner and the people responsible for operation, technical maintenance, and risk review. Also consider whether a simpler rule, process change, or existing software feature could meet the need. 2. Where does data go throughout the workflow? Request a data-flow diagram covering source systems, extraction, embeddings, retrieval, model processing, outputs, telemetry, logs, backups, and support access.
Identify providers, subprocessors, processing locations, retention periods, and deletion arrangements. Ask what happens when a provider or processing location changes.
Local storage does not establish that all processing is local. Have the relevant legal and privacy owners assess the proposed arrangement and each party’s responsibilities against applicable requirements. 3. What data does the system need? Ask for the fields and document categories required at each step. Compare the intended design with the information actually transferred during an authorised test.
Check whether prompts, files, and outputs may be reused for training, analytics, or service improvement. Review the relevant settings and contractual terms separately from retention and deletion.
Masking can reduce exposure, but it may also remove context needed for the task. Test both its effectiveness and its effect on output quality. 4. What can the agent access and execute? Request a permission matrix connecting each capability to its required access.
Check the service identity and the requesting user’s permissions. An assistant should not reveal a restricted document because its backend account has broader access.
Separate reading, drafting, and execution permissions. Read-only access can limit modification risk while leaving disclosure risks to address.
Any expansion should be a reviewed decision. A system does not need to progress automatically toward greater autonomy. 5. How are permissions and credentials enforced? Ask the vendor to demonstrate restrictions in the application and tool-execution layer.
Relevant checks include identity, permitted records, recipients, parameter validation, transaction limits, and required approvals. Instructions in a prompt alone are insufficient.
Review credential storage, administrative access, rotation, revocation, and prevention of exposure in prompts or logs. Test that access stops as designed when a credential or role is revoked. 6. What makes human approval specific and enforceable? Define which actions need approval and what the reviewer sees: proposed content, supporting evidence, intended action, and recipient.
Approval should apply to a specific version and have a defined validity period. Material changes should trigger renewed review. Silence should not count as approval.
Test rejection, editing, expiry, unavailable approvers, and failed execution. Reviewer corrections can inform controlled improvements, but they should not automatically become training data. 7. What can audit records reconstruct? Request a sample case showing relevant request identifiers, source references, configuration versions, tool calls, approvals, overrides, and outcomes.
Distinguish an attempted action from a confirmed result. A tool call to send a message is different from confirmation that the receiving service accepted it.
Review event coverage, integrity protection, administrative accountability, access, retention, and export. Avoid unnecessary sensitive content, and define what happens if logging fails.
Audit records should support investigation without claiming to reveal the model’s complete internal reasoning. 8. How will quality and acceptance be measured? Agree the baseline, evaluation method, thresholds, and decision owner before implementation.
Measure correctness, critical errors, review time, escalation quality, latency, and cost per accepted result as relevant. Include representative authorised examples and constructed cases for unusual conditions.
An acceptance pack should document the assessed scope, tested versions, dates, results, limitations, and release recommendation. It may establish that criteria were not met; it should not be written only to justify acceptance. 9. Can the system handle our Arabic and English inputs? Use examples reflecting actual documents, terminology, formatting, and language switching.
Test Arabic and English separately where appropriate. For scanned documents, assess text extraction as well as the model’s interpretation.
Include ambiguous wording, missing information, conflicting sources, and outdated versions. Source citations and confidence scores are useful signals to evaluate, but neither guarantees correctness. 10. Which failure and security scenarios have been tested? Agree scope and authorisation before testing. Use an isolated environment with synthetic or otherwise permitted data. Scenario Evidence to request Restricted or cross-customer request Access isolation across answers, citations, and previews Instructions embedded in documents No unauthorised change to permissions or task Invalid tool parameters Enforcement of recipient, action, and value limits Timeout after an action Duplicate prevention or controlled reconciliation Service or logging failure Defined fallback and escalation Shutdown with queued work Predictable handling of pending and running actions
Ask for methods, dates, tested configurations, findings, and remediation status. A selected demonstration is useful evidence, but it is not a comprehensive assessment. 11. What will operation, support, and change cost? Separate implementation fees from model usage, hosting, licences, human review, monitoring, maintenance, and future changes.
Request cost scenarios with stated volumes and assumptions. Confirm support hours, escalation contacts, service commitments, and responsibilities for integration failures.
Review incident procedures, evidence preservation, and cooperation duties. Distinguish contractual notification commitments from applicable legal reporting requirements.
For delivery, ask for milestones, dependencies, and the process for handling delay or scope changes rather than relying on a headline timeline. 12. What rights and exit arrangements do we receive? Distinguish customer data, project deliverables, supplier components, and third-party services.
Confirm operating and modification rights, licence dependencies, documentation, export formats, migration assistance, deletion arrangements, and associated charges.
A licensed platform may be appropriate if its terms and dependencies suit the organisation. Ownership of selected deliverables does not automatically establish portability or eliminate switching costs.
Test whether exported material is usable and whether the receiving team has sufficient instructions to operate or transition the workflow. How should the answers become a purchasing decision? Use a shared evidence register: Field What to record Requirement The control or outcome being assessed Evidence Document, configuration, test, or demonstration Scope and date Workflow, version, environment, and assessment date Status Evidenced, partially evidenced, untested, or not applicable Gap Limitation and potential consequence Action Remediation owner, deadline, and retest requirement
Do not let a high overall score offset a critical unresolved issue. Define mandatory gates according to the workflow’s risks, including access restrictions, required approvals, and handling of consequential actions.
Possible outcomes include acceptance for a limited scope, remediation and retesting, insufficient evidence, or rejection. Record the rationale and any conditions for expansion.
To discuss a defined workflow, explore LeenAI’s pilot scoping, start here, or contact LeenAI. Bring the systems involved, data requirements, intended actions, and evidence your organisation needs before committing.
Frequently Asked Questions
Is a Saudi AI vendor required to keep data inside the Kingdom?
Not in every case. PDPL allows transfers outside the Kingdom under conditions set in its regulations, but many enterprises and sector regulators require local hosting for sensitive data. Ask the vendor where data is stored and processed, and confirm the requirements that apply to you with your legal team.
How long should a governed AI pilot take before we judge it?
A pilot focused on one decision usually takes six to eight weeks, covering scoping, read-only data access, the build and user acceptance testing. Judge it by comparing results with the baseline and the acceptance criteria agreed at the start.
What is an Acceptance Pack and why should we ask for one?
An Acceptance Pack is the documented proof that a pilot worked: user acceptance test results, the golden set of reference cases, the evaluation plan, audit logs, runbooks and a scale-up plan. It keeps the decision to expand based on evidence rather than impressions.




