Is inline AI governance the same as runtime AI governance?

In practice yes. Runtime governance emphasises when the decision happens, at execution rather than at design or review time. Inline emphasises where it sits, in the path rather than beside it. Both describe enforcement at the moment of the decision, as opposed to policy documents written beforehand or dashboards read afterwards.

Why does inline governance need to be deterministic?

Three reasons, all practical. Latency: a model call in the hot path costs seconds and turns every governed request into two inference calls. Cost: per-token pricing on every governed decision scales with traffic in a way most budgets do not survive. Repeatability: a judge that samples can return different verdicts for the same input, which makes evidence unusable, because an auditor asking why this output was allowed deserves an answer that does not depend on which sample was drawn.

How much does an inline governance step add to a request?

With a deterministic core and no LLM in the hot path, verdicts land in milliseconds rather than seconds. That difference is what decides whether the control survives contact with production: a governance step measured in seconds gets removed the first time someone complains the product feels slow.

Concept

Inline AI governance

Q: What happens if the governance layer is unreachable?

It must fail closed. A governance layer that allows traffic through when it cannot reach its datastore is worse than no governance, because it produces a confident record of controls that were not actually applied. Turrigan fails closed rather than open, so an unreachable datastore can never silently allow.

Inline AI governance means the governance decision happens inside the request path, before an AI output reaches a user or an agent action executes. The application submits the decision, waits for allow, block or escalate, and honours the verdict. It is also called runtime AI governance. The contrast is with governance that reviews AI behaviour afterwards, which can describe what happened and cannot prevent any of it.

Beside the path is not in the path

Most AI governance tooling observes. It samples traffic, scores it, charts it, and raises an alert when something looks wrong. All of that is downstream of the event. By the time the chart moves, the output has been read and the action has run.

This is not a criticism of monitoring, which is genuinely useful for finding out that a problem exists. It is a statement about what monitoring can be asked to do. "Show us how each decision was controlled", the question the EU AI Act, NIST AI RMF and ISO/IEC 42001 all converge on, is not answered by a system that watched. It is answered by a system that decided.

What inline actually requires

Being in the path is a stronger commitment than it sounds, because the path has requirements the sidelines do not.

It must be fast enough that nobody removes it. A governance step measured in seconds gets taken out the first week someone complains the product feels slow. Milliseconds survive; seconds do not.
It must fail closed. If the layer cannot reach its datastore and lets traffic through anyway, it produces a confident record of controls that were never applied. That is worse than having no governance, because now the evidence is wrong.
It must be repeatable. The same input has to produce the same verdict. Evidence that depends on which sample was drawn is not evidence.
It must be bounded. Inputs have a size limit, tenants have rate isolation, and one caller's traffic cannot degrade another's, because a shared control point is also a shared failure point.

Why the core is deterministic

The obvious way to build a governance check is to ask a model. Send the output to an LLM, ask whether it is acceptable, use the answer. It is quick to build and it does not survive production.

Latency doubles, because every governed request is now two inference calls. Cost scales with traffic at per-token prices on every decision, including the overwhelming majority that were fine. And the judge samples, so the same output can be allowed on Monday and blocked on Tuesday. An auditor asking why a specific output was allowed deserves an answer better than "the model felt differently that time".

A deterministic core removes all three problems at once. No LLM in the hot path means verdicts in milliseconds, no per-token cost per decision, and the same verdict every time. It also means the whole thing can run with no external AI service at all, which is what makes air-gapped and sovereign deployment possible rather than aspirational.

The tradeoff is honest: deterministic detectors do not have a language model's judgement about ambiguous cases. That is what the escalate verdict is for.

Allow, block, escalate

Two verdicts are not enough. A system that must answer either allow or block on every decision is forced to guess on the uncertain ones, and half of those guesses are wrong in the direction that matters.

Escalate is the verdict that makes inline governance workable at the boundary. Critical findings block. Clean outputs pass. Cases the floor cannot settle confidently go to a human with the reason attached, rather than being resolved by a threshold nobody chose deliberately. This is also, in practice, what a human-oversight obligation looks like when implemented rather than described.

One chokepoint, not a migration

Inline sounds invasive and does not have to be. Turrigan is one HTTP call placed in a path you already control: before an output is returned to a user, or before an agent's orchestrator executes a tool call. Your models, vendors and stack stay as they are. What changes is that a decision now happens where previously there was none.

Keeping it to a single chokepoint matters for evidence as much as for engineering. Governance spread across a dozen enforcement points produces a dozen partial records. One chokepoint produces one chain.

Runtime governance and the other kind

Governance frameworks mostly describe organizational work: risk assessments, documentation, review boards, sign-off. That work is real and this does not replace it. What it does is close the gap between the policy and the individual decision, which is precisely where audits go badly. An organization can hold an excellent AI policy and still have no mechanism that applied it to the output sent to a customer on a specific Tuesday in March.

Frequently asked questions

What is inline AI governance?

Governance that happens inside the request path, before the output ships or the action runs. The application submits the decision and honours the verdict it gets back.

Is inline the same as runtime AI governance?

In practice yes. Runtime emphasises when the decision happens; inline emphasises where it sits. Both mean enforcement at the moment of the decision rather than policy written beforehand or dashboards read afterwards.

Why does it need to be deterministic?

Latency, cost and repeatability. A model in the hot path costs seconds and per-token pricing on every decision, and it can return different verdicts for the same input, which makes the resulting evidence unusable.

What happens if the governance layer is unreachable?

It fails closed. A layer that allows traffic when it cannot reach its datastore records controls that were never applied.

Does this replace our AI governance programme?

No. It closes the gap between the policy and the individual decision. The organizational work stays yours, and Turrigan reports which controls belong to you as the deployer rather than claiming them.

Put governance in the path