Why AI agent SLAs are different from software SLAs
Traditional software SLAs promise that a system is available and responsive. An AI agent SLA must also promise that the agent behaves within agreed boundaries: it answers from approved sources, escalates when uncertain, and leaves an audit trail. Uptime alone is a weak promise if the agent is confidently wrong at 99.9% availability.
For Saudi enterprises, the stakes are higher because the agent may touch customer data, procurement decisions, or candidate information governed by PDPL. An SLA that ignores data handling, human oversight, and output quality is not a governance document; it is a server monitoring report. The right frame is a service-level agreement that covers the workflow outcome, not just the infrastructure.
This guide sets out how decision-makers should define SLA and uptime requirements for AI agents, what to measure during a pilot, and how to write acceptance criteria that a vendor can actually be held to.
What should an AI agent SLA actually cover?
A governed AI agent SLA should cover four layers. The first is availability: the agent is reachable during agreed business hours, with a defined fallback when it is not. The second is quality: accuracy against a test set, escalation rate to humans, and response time for common requests. The third is governance: audit-log completeness, role-based access control, data retention rules, and PDPL compliance. The fourth is support: how quickly the vendor responds to incidents, and who owns fixes.
Each layer needs a measurable target and a named owner. For example, an agent handling WhatsApp customer service might target first-response time, containment rate for approved FAQs, and a maximum escalation delay. An agent drafting quotations might target review time per quote and the percentage of drafts accepted without rework. These are workflow metrics, not server metrics, and they belong in the SLA.
Crucially, the SLA should state what happens when the agent is uncertain. A governed agent escalates to a human with full context rather than guessing. The SLA should define the escalation path, the expected handoff time, and how the human decision is logged.
How do uptime and accuracy targets interact?
Uptime and accuracy are not independent. An agent that is available but frequently wrong is worse than one that is occasionally unavailable but reliable when it runs. Buyers should therefore define a combined service level: the agent must be available during agreed hours and, when available, must meet accuracy and escalation thresholds on a defined test set.
In practice, this means the SLA should specify a measurement method. Who runs the test set? How often? What counts as a failure? For Arabic and English workflows, the test set must include both languages, because an agent that performs well in English may degrade in Arabic. LeenAI's approach is to validate with users during the pilot and to document the test set as part of the acceptance criteria.
Uptime targets should also reflect the business rhythm. A procurement agent may need high availability during working hours but not overnight. A customer-service agent on WhatsApp may need broader coverage. The SLA should match the agent's schedule to the workflow it supports, not default to 24/7 promises that add cost without adding value.
What does a governed pilot prove before you sign an SLA?
A pilot is where SLA targets are tested, not assumed. Before committing to a multi-year SLA, buyers should run a workflow-specific pilot with baseline KPIs, user acceptance testing, and an Acceptance Pack that records what was measured and how. LeenAI's AI Pilot Project is structured this way: one workflow, baseline metrics, UAT, and an Acceptance Pack, typically within a 6–8 week operating-model timeline.
During the pilot, the buyer should collect evidence on availability, accuracy, escalation behavior, and audit-log completeness. If the agent handles documents, the pilot should test retrieval accuracy and citation quality. If it handles quotations, the pilot should test pricing-rule adherence and human review time. The pilot output becomes the evidence base for the SLA.
This is also where PDPL compliance is validated in practice: read-only access first, data minimization, role-based access, and retention rules documented before code. An SLA that promises compliance without a pilot that proves it is a claim, not a commitment. LeenAI's default approach is to read approved sources and produce governed outputs rather than replace ERP, CRM, or core systems, which keeps the compliance surface smaller and the SLA more credible.
How should PDPL and audit requirements appear in the SLA?
PDPL requirements should be written into the SLA as measurable obligations, not vague statements. That means specifying which data the agent may access, whether access is read-only, how long data is retained, who can view logs, and how data residency is handled. LeenAI scopes deployment according to the client environment and location requirements, and that scoping should be reflected in the SLA.
Audit logs deserve their own clause. The SLA should state what is logged (inputs, outputs, escalations, human decisions), how logs are protected, and how quickly they can be produced for an audit. For regulated workflows, log completeness is a service level in its own right.
Human-in-the-loop should also be contractual. The SLA should define which decisions require human approval, the expected approval time, and what happens if no human responds. An agent that can act without approval is a different risk category, and the SLA should say so explicitly.
What acceptance criteria make an AI agent SLA enforceable?
An SLA is only as good as its acceptance criteria. Buyers should require a written Acceptance Pack that lists each service level, the measurement method, the test data, and the pass/fail threshold. The pack should be signed before go-live and reviewed at agreed intervals.
Acceptance criteria should be specific enough that a third party could verify them. "The agent is accurate" is not a criterion. "On a 200-item bilingual test set, the agent answers from approved sources with citations and escalates when confidence is below the agreed threshold" is. LeenAI emphasizes formal proof points and a presentation style suited to real buying teams, and the Acceptance Pack is where that shows up.
Finally, the SLA should include a review cadence. AI agents drift as data and processes change, so the SLA should schedule periodic re-testing and define how targets are adjusted. A governed SLA is a living document, not a signature on a PDF.
If you are evaluating AI agents for a Saudi workflow, talk to us and we will reply within 24 hours with scope, KPIs, timeline, access requirements, and an estimated SAR range. You can also see how we scope pilots or explore LeenAI's OpsRAG for document-grounded agents.
Frequently Asked Questions
What uptime should we require from an AI agent?
Match the uptime target to the workflow. A customer-facing agent usually needs a higher target and a clear fallback to staff, while an internal back-office agent can accept planned maintenance windows. Whatever the number, the SLA should also say what happens when the agent is unavailable.
How do we measure an AI agent's accuracy for an SLA?
Agree on a test set of real cases before go-live, define what counts as a correct, partial or wrong output, and measure against it regularly. Track how often people override or correct the agent, and review samples of live outputs, not just the test set.
Can an SLA cover PDPL compliance?
An SLA can include commitments that support compliance, such as data location, access controls, incident notification times, retention and deletion, and audit logs. Compliance itself remains your company's responsibility, so review the terms with your legal and data protection owners.




