Skip to content

Compliance & coverage

Norviq ships two compliance surfaces, and they answer different questions. Both live under Security Operations in the console and both are driven by the global header time range.

Console page Route Question it answers
Compliance /compliance For a recognized threat framework, which techniques does the policy loaded for this namespace actually enforce — and where are the gaps?
Policy Compliance /compliance/policies For the policies you wrote, which of your own agent classes are violating them right now?

They are siblings, not tabs. /threats/mitre redirects to /compliance.

flowchart TB
    subgraph FW["/compliance — framework coverage"]
        M["Framework mapping<br/>(technique → rule_ids)"] --> P["Loaded rego for the namespace"]
        P --> S["enforced / gap / out-of-scope"]
        S --> X["Evidence pack (JSON · PDF)"]
    end
    subgraph PC["/compliance/policies — your own policies"]
        R["Your policies' rule_ids"] --> A["Audit rows in range"]
        A --> O["Which agent classes violated them"]
        O --> RM["Remediation table"]
    end

Part 1 — Framework coverage (/compliance)

Section titled “Part 1 — Framework coverage (/compliance)”

Two frameworks are computed live, off the same machinery:

Framework id Display name Techniques Enforceable Out of scope
atlas MITRE ATLAS 15 10 5
owasp OWASP LLM Top 10 (2025) 10 6 4

The mappings are files on disk — policies/mitre_mapping.json and policies/owasp_llm_mapping.json — read once and cached per process.

Four more frameworks render as inert roadmap rows with no coverage numbers, so they can never be mistaken for live scoring: OWASP Agentic Top 10, NIST AI RMF, ISO/IEC 42001, EU AI Act.

Each framework technique maps to a set of Norviq rule IDs. Norviq concatenates the rego actually loaded for the namespace (plus any __cluster__-scoped policy), strips whole-line # comments, and looks for each mapped rule ID as a quoted string literal — either the bare id (blocks["deny_sql_injection"]) or a colon-namespaced id whose last segment is the rule (blocks["remediation:atlas:AML.T0049:deny_sql_injection"], which is what the remediation generator emits). Matching literals rather than raw substrings is what keeps prose out: a rule name inside a reason sentence or a docstring cannot make a technique read enforced.

  • Enforced — the technique is runtime-enforceable and at least one mapped rule is defined in the loaded rego.
  • Gap — enforceable, but no mapped rule is loaded. This is what remediation targets.
  • Out of scope — the technique is not something a runtime tool-call PEP can address (training-data poisoning, model backdoors, embedding weaknesses, misinformation). Out-of-scope techniques are shown but excluded from the coverage denominator — never counted as failures.

coverage_pct = enforced ÷ enforceable_total, per namespace, per framework.

MITRE ATLAS — enforceable techniques:

Technique Name Mapped rules Remediable
AML.T0012 Valid Accounts cross_tenant_access yes
AML.T0049 Exploit Public-Facing Application deny_sql_injection yes
AML.T0050 Command and Scripting Interpreter deny_shell_execution yes
AML.T0051 LLM Prompt Injection llm01_prompt_injection yes
AML.T0053 LLM Plugin Compromise llm05_supply_chain, llm06_excessive_agency yes
AML.T0054 LLM Jailbreak deny_shell_execution, llm01_prompt_injection yes
AML.T0055 Unsecured Credentials llm02_data_leakage yes
AML.T0057 LLM Data Leakage llm02_data_leakage, base64_decoded_threat yes
AML.T0056 LLM Meta Prompt Extraction — (remediation: bespoke) no — escalates
AML.T0061 LLM Prompt Self-Replication — (remediation: bespoke) no — escalates

Out of scope on ATLAS: AML.T0018 Backdoor ML Model, AML.T0020 Poison Training Data, AML.T0024 Exfiltration via ML Inference API, AML.T0031 Erode ML Model Integrity, AML.T0048 External Harms.

OWASP LLM Top 10 (2025) — enforceable controls:

Control Name Mapped rules Remediable
LLM01:2025 Prompt Injection llm01_prompt_injection yes
LLM02:2025 Sensitive Information Disclosure llm02_data_leakage, base64_decoded_threat yes
LLM05:2025 Improper Output Handling deny_sql_injection, base64_decoded_threat yes
LLM06:2025 Excessive Agency llm06_excessive_agency, deny_shell_execution yes
LLM07:2025 System Prompt Leakage — (remediation: bespoke) no — escalates
LLM10:2025 Unbounded Consumption — (no mapped rule) no — escalates

Out of scope on OWASP: LLM03:2025 Supply Chain, LLM04:2025 Data and Model Poisoning, LLM08:2025 Vector and Embedding Weaknesses, LLM09:2025 Misinformation.

Coverage is decided by rule IDs present in loaded rego, and only two shipped modules define the eight mapped rule IDs:

Module Defines the mapped rule IDs?
webhook/presets/strict.rego yes — all eight
comprehensive.rego yes — all eight
webhook/presets/moderate.rego no — it defines only moderate_drop_block and moderate_escalate
webhook/presets/permissive.rego no — it registers no control heads at all

A namespace whose only policy is moderate or permissive will therefore score 0% coverage on both frameworks, correctly: those presets genuinely do not carry the detections the frameworks map to. This is not a display bug, and generating remediation is the intended way to close it.

What the headline numbers actually measure

Section titled “What the headline numbers actually measure”

GET /compliance/{framework}/coverage returns these, and the console tiles render them directly:

Field What it counts Sharp edge
coverage_pct enforced ÷ enforceable_total, rounded Rules present, not efficacy. See basis.
basis Always the literal "rules_present" Stated in the payload so no consumer can render coverage as proof the control works.
enforced / gap / oos Technique counts by status oos is excluded from the denominator.
proven How many enforced techniques have block/escalate evidence in the window In a namespace running audit/monitor, this is 0 no matter how good coverage is — the evaluator softens block to audit, so block evidence is structurally impossible.
blocked block and escalate decisions on this framework’s distinct mapped rules Deliberately different from the Overview’s “Blocked” KPI, which counts block only. The two headline numbers can legitimately disagree for the same namespace at the same moment. The console labels this one “blocked or escalated”.
observed All decisions on this framework’s mapped rules Same real-traffic filter.
agent_classes Distinct real agent classes with a block/escalate on a mapped rule
synthetic_excluded Rows dropped as synthetic/probe/eval identity or framework="redteam" Surfaced on the page as “Real traffic only · N synthetic/simulated events excluded”.
degraded true when the audit aggregation query failed Every observed/blocked count is then 0 because it could not be read, not because nothing happened. Carried into the exported pack too.
last_exported Timestamp of the most recent export event Recorded best-effort by /export.

Because ATLAS and OWASP sum over their own distinct mapped rule IDs, the two framework cards show different observed/blocked numbers on the same namespace. That is correct, not a discrepancy.

Coverage is also driven by the red-team efficacy banner, which overlays the last Red Team run’s proven_blocking_pct so rules-present and caught-in-practice are never confused. GET /redteam/results/latest is admin-only, so a viewer sees an explicit “could not read” state rather than a false “not efficacy-tested”.

flowchart LR
    C["Coverage matrix<br/>(per namespace)"] --> G["Gap<br/>(enforceable, no rule loaded)"]
    G -->|"Generate enforcing policy"| D["Remediation draft<br/>(&lt;class&gt;__remediation__)"]
    D --> R["Policy Catalog<br/>review → dry-run → apply"]
    R --> E["Enforced"]
    E -->|"re-score"| C
  1. Read the matrix for a namespace, filtered to Gaps. The tree’s filter segments are All / Gaps / Enforced / OOS.
  2. Click Generate enforcing policy on one gap, or tick several and use Generate for selected with a class-scope picker: This affected class (class_mode: "affected", the default), All affected classes ("all"), or a specific class.
  3. Norviq builds a tighten-only dry-run remediation draft and deep-links you to it in the Policies → Catalog inbox (/policies/catalog?intent_draft=<id>).
  4. Review & apply runs the draft through the standard gated editor. Nothing enforces until you apply.
  5. The next coverage read re-scores the technique as enforced — the generator’s remediation:<fw>:<control>:<rule> block key is recognized by the same rule-present test.

Generation is admin-only and refuses a write aimed at a remote fleet cluster.

Three outcomes are possible, and the console reports each per item in a batch rather than aborting:

status When What is created
draft The technique has at least one mapped rule with a runtime template, and there is a real affected/active class One draft per (control × class)
escalate The mapping is remediation: bespoke, or no mapped rule has a runtime template (AML.T0056, AML.T0061, LLM07:2025, LLM10:2025) Nothing. The message says this risk does not show up in tool-call traffic and needs a bespoke control.
no_affected_classes No real, non-synthetic agent class in range and no explicit agent_class in the body Nothing — “No affected agent classes in range — nothing to remediate yet.”

Requests naming a synthetic/test class are refused with 422, as are out-of-scope techniques.

generate_remediation_rego emits package norviq.remediation.<framework>.<control> — a default-allow module scoped to one agent class (input.agent.agent_class == "<class>"), with one tighten-only blocks[...] clause per mapped rule that has a runtime template. Eight rules have templates:

Rule The generated clause blocks
deny_sql_injection execute_sql whose params contain OR 1=1, DROP TABLE, UNION SELECT, ; -- or ' OR ' (case-folded)
deny_shell_execution any string param containing "; ", &&, $(, a backtick, "| " or ||
llm01_prompt_injection any string param containing ignore previous, ignore all previous, disregard the above or system prompt
llm02_data_leakage tool name in read_env, getenv, get_secret, read_secret, fetch_secret
llm05_supply_chain tool name in load_plugin, download_script, eval, install_package
llm06_excessive_agency tool name in delete_record, drop_table, truncate
cross_tenant_access a tenant_id or namespace param that differs from the caller’s namespace
base64_decoded_threat a base64-shaped param that decodes to drop table, union select, rm -rf, $( or ; --

These mirror comprehensive.rego’s detections but are independent copies used only for draft generation — the /evaluate hot path is untouched.

The draft is persisted under a dedicated overlay key, "<class>__remediation__", never the class’s own key. Apply is a full-replace upsert, so writing the draft at the real class’s key would destroy that class’s enforcing policy. The evaluator resolves *__remediation__ as a hard tighten-only overlay (alongside __guardrail__), so applying it can only add blocks — see Writing policies §1.

Applying a second control for the same class unions into that one overlay rather than replacing it. Each generated rego carries a machine-readable manifest comment (# nrvq:remediation-manifest …) naming every control it encodes; the gated apply path parses the incoming draft and the existing overlay, unions them, and re-materializes one combined module. Without that, applying control B would silently erase control A and produce false coverage.

Drafts land in the intent_drafts table, which the evaluator never reads — a draft cannot auto-enforce. Drafts expire (config.retention.draftTtlDays, default 14; 24h for synthetic classes) and are capped per namespace (config.retention.draftCapPerNamespace, default 50).

Export audit-evidence pack produces a point-in-time attestation, minted in-cluster with no egress, as machine-readable JSON or a dependency-free single-page PDF. Both are streamed as attachments; the export event is recorded so “Last exported” is real.

Per control the pack carries: technique_id, name, scope, status, mapped_policies, enforcing_policies, observed, blocked, blocked_by_rule (per-rule attribution, not the technique-wide total repeated), and affected_classes.

The headline carries the honesty qualifiers, and both formats state them:

  • basis: "rules_present" plus a line reading “Basis: rules present, not efficacy — N of M enforced techniques have block/escalate evidence in this window.”
  • degraded: true plus “DEGRADED: the audit query did not complete, so every blocked/observed count above is 0 because it could not be read — not because nothing happened.”
  • synthetic_excluded plus “Real traffic only · N synthetic/simulated events excluded.”

The pack was deliberately made to read no stronger than the API it came from: it is the durable artifact that ends up in an audit file months later, long after the live view that qualified it has moved on.

The pack counts real traffic only. Red-team framework events and synthetic/probe/eval identities are excluded from observed/blocked — an evidence pack must not read as enforcement proof when the traffic was your own test harness. Red-team efficacy lives on its own labelled surface (see how red-team results map onto Compliance), never merged into these counts.

Audit rows are retained 30 days by default (config.retention.auditRetentionDays). Raise it for a longer compliance window, or export a pack whenever you need evidence that outlives the live log.

Each coverage read upserts at most one snapshot per (namespace, framework, hour), serialized with a Postgres advisory lock so concurrent replicas do not double-write. There is no scheduler — the trend accumulates because people look at the page. Snapshots are retained 30 days (config.retention.coverageSnapshotRetentionDays), which covers the 30d trend view. The series is empty until the first snapshot; no points are fabricated.


Part 2 — Policy Compliance (/compliance/policies)

Section titled “Part 2 — Policy Compliance (/compliance/policies)”

This page is about your own policies, not a framework. It answers the question you have before promoting anything: what will this break?

The backing endpoint is GET /api/v1/policy-compliance?namespace=&range=, and its unit is the control, not the call. A control with 4,000 hits across one agent class is a very different decision from one with 12 hits across nine.

The endpoint scans audit rows in range and sorts each into one of three buckets:

Bucket Row shape Meaning
count (would-block) rule_id prefixed monitor_would_block: or policy_audit_would_block:, or a bare rule_id with decision="audit" that names a shipped control What promoting this control would newly break
enforced decision is block or escalate on a real policy rule What the control already refused
origin: "custom" a bare audit rule_id that is not a shipped control id Your own hand-written or Visual Builder rule running in monitor mode

The prefixed and bare shapes both exist and both matter. A policy in audit mode softens its block and stamps policy_audit_would_block:; a namespace in monitor posture stamps monitor_would_block:. But a baseline control set to monitor registers its head into audits[], so the rego itself decides audit and nothing prefixes it — that is the normal shape on a stock install, and counting only the prefixed form makes the page blind to its own default configuration.

count and enforced are never summed by the endpoint. They answer opposite questions and the caller chooses.

Infrastructure rules are excluded outright. thin_proxy_fail_closed, thin_proxy_fail_open, engine_rejected_request, evaluator_error, evaluator_timeout, evaluator_fallback, policy_load_pending and rate_limit_exceeded are minted by the engine or the throttle, not by any policy — and monitor mode softens them wearing the same prefix as a real control. Reporting them here would read an availability incident as a policy decision. They have a home on /system-health.

Synthetic identities and red-team rows are excluded, and excluded_synthetic counts only what the exclusion actually suppressed — a synthetic row that would have landed in a control — not every excluded row.

scanned is what makes an empty list readable: zero non-compliant calls out of zero traffic means nothing has happened here yet; zero out of 40,000 means genuinely compliant.

Card Measures Reads (unknown) when
Overall resource compliance Agent classes clean across every one of your policies, ÷ all real agent classes seen. A gauge, not a call ratio. The compliance evidence feed errored, or no agent class has run yet
Resources by compliance state Donut: compliant vs non-compliant agent classes No agent classes have run
Policies by state Donut: your policies as Compliant / Non-compliant / Not evaluated You authored no policies in scope
Calls examined scanned — real, non-synthetic audit rows in the window, with the allow/block volume shape behind it The evidence read failed

Two properties are load-bearing and were both bugs once:

  • The resource unit is the agent class, not the call. A class is non-compliant for a policy if any of that policy’s rules flagged it. The denominator is real, non-synthetic classes only, so a synthetic offender can never push a percentage below 0 out of N.
  • A percentage that cannot be computed reads unknown, never 100%. If the evidence feed is unreadable, no percentage on the page is knowable — a failed fetch otherwise looks exactly like a clean one. The page shows a banner saying the figures are unknown, not “compliant”. Likewise, zero agent classes gives Not evaluated, not “100% compliant out of nothing”.

The page joins rule_id → policy by parsing each policy’s rego for its blocks|escalates|audits["…"] heads. A policy whose source cannot be read is Unknown, distinct from one with no rules. Reserved scopes the product owns are excluded from “your own policies”: __baseline__, __controls__, __pack__, __pack_override__, __pack_weaken__, __guardrail__, and any *__remediation__ overlay.

Modelled on Azure Policy’s remediation columns, with a real referent for each: the rego rule is the definition, the (namespace, agent_class) row is the assignment, the agent classes are the resources.

Column Content
Policy definition Policy name, plus its rule count and rule IDs
Assignment Agent class, version, enforcement mode, priority
Resources to remediate N of M agent classes, named, plus total calls flagged (count + enforced)
Scope <namespace> / <agent_class>
Deep links to the Policy Catalog and to the Audit Log, pre-filtered

Nothing on this page changes enforcement. There is no remediation task and the copy says so. The footer states which case you are in:

  • Any non-compliant policy still in audit“A policy in audit RECORDED these calls and let them through — promote it in Policy Catalog once the counts look right.”
  • All of them enforcing — “These policies are enforcing, so the calls above were refused. Remediate the workload, or add an exception if the traffic is legitimate.”

An empty remediation table is only an all-clear when compliance is fully known. If anything read Unknown, the empty state says “Compliance is not fully known yet — this is not an all-clear.”

The Evidence panel is a set of deep links into the Audit Log — all decisions in the namespace, the selected class only, and one link per rule ID — rather than a second, worse audit table. The Audit Log already does filtering, tailing, export and red-team separation.


The 21 shipped detectors are what Policy Compliance measures blast radius for. Each is independently settable per namespace:

Effect What the compiler does Runtime behaviour
deny registers the head into blocks[...] — or preserves an authored escalates[...] refuses (or escalates) the call
monitor registers the head into audits[...] evaluates, records the call as non-compliant, call proceeds
off omits the head entirely not evaluated

A control’s effect is only which set its head registers into. The detector predicates above the CONTROLS-BEGIN/CONTROLS-END region are left byte-identical, so changing what a control does never changes how it detects.

GET /baseline/controls?namespace=&preset= returns every control with title, description, caveat, current effect, its own default_effect, plane (discovery | call | response), surface (tool | mcp), and enforced_as — what deny actually does for that control, which is not always block: mcp_definition_drift and mcp_definition_never_scanned escalate, and scope_violation_dangerous_tool is observe-only by construction.

Defaults are per control, not one global. Seventeen of the 21 ship at deny on evidence from a measured benign corpus; the remaining four — llm02_data_leakage, base64_decoded_threat, strict_default_block and scope_violation_dangerous_tool — ship at monitor.

PUT /baseline/controls is admin-only, refuses a write aimed at a remote fleet cluster, and is also gated by the namespace’s apply_mode (a namespace set to dry_run_only rejects it). It validates every control id and effect before writing anything, persists only deviations from each control’s own shipped default, recompiles the module, writes it to the reserved scope __controls__ at priority 2 in enforcement_mode="block", and invalidates the whole namespace’s evaluation cache.

The module always runs in block mode on purpose: a monitor control already decides audit on its own via its audits[...] head, so softening the policy on top would make deny unreachable and collapse three effects into two.

__controls__ is a separate key from the chart’s __baseline__ because the chart owns __baseline__ — sharing it meant a helm upgrade silently reverted every control a customer had tuned.


All paths are under /api/v1 with bearer-token auth and are namespace-scoped by the caller’s RBAC.

Framework coverage{framework} is atlas or owasp; an unknown value returns 404.

Method & path Auth Purpose
GET /compliance/{framework}/coverage?namespace=&range=24h user the coverage matrix
GET /compliance/{framework}/trend?namespace=&range=30d user coverage % + blocks over time
GET /compliance/{framework}/export?namespace=&range=&format=json|pdf user the audit-evidence pack
POST /compliance/{framework}/generate admin one gap → {technique_id, namespace, agent_class?, range}
POST /compliance/{framework}/generate-batch admin {technique_ids[], namespace, class_mode, range}

The original ATLAS-default routes remain as back-compat aliases and take the framework as a query parameter instead: GET /mitre/coverage, /mitre/coverage/trend, /mitre/coverage/export, POST /mitre/coverage/generate, /mitre/coverage/generate-batch. The /compliance/{framework}/* routes delegate to them, and the path framework wins over any framework in the body.

Policy compliance and controls

Method & path Auth Purpose
GET /policy-compliance?namespace=&range= user non-compliant traffic grouped by control
GET /baseline/controls?namespace=&preset=strict user the control set with current effects
PUT /baseline/controls admin set effects and recompile __controls__

range accepts 1h, 6h, 24h, 7d, 30d on both /policy-compliance and the coverage routes. The write routes additionally reject a request aimed at a remote fleet cluster.

Terminal window
# where does ATLAS coverage stand for a namespace?
curl -s "$NRVQ_API_URL/api/v1/compliance/atlas/coverage?namespace=chatbot-prod&range=24h" \
-H "Authorization: Bearer $NRVQ_API_TOKEN" \
| jq '{coverage_pct, basis, enforced, enforceable_total, proven, blocked, degraded, synthetic_excluded}'
{
"coverage_pct": 80,
"basis": "rules_present",
"enforced": 8,
"enforceable_total": 10,
"proven": 0,
"blocked": 0,
"degraded": false,
"synthetic_excluded": 14
}

proven: 0 alongside coverage_pct: 80 is the expected reading for a namespace whose baseline is in audit mode. The rules are loaded; nothing has been refused.

Terminal window
# what would I break by promoting a control in this namespace?
curl -s "$NRVQ_API_URL/api/v1/policy-compliance?namespace=chatbot-prod&range=7d" \
-H "Authorization: Bearer $NRVQ_API_TOKEN" \
| jq '{scanned, excluded_synthetic,
controls: [.controls[] | {control_id, origin, count, enforced,
classes: [.agent_classes[].name]}]}'
{
"scanned": 41207,
"excluded_synthetic": 18,
"controls": [
{
"control_id": "deny_shell_execution",
"origin": "baseline",
"count": 3184,
"enforced": 0,
"classes": ["order-lookup", "customer-support"]
},
{
"control_id": "llm01_prompt_injection",
"origin": "baseline",
"count": 12,
"enforced": 0,
"classes": ["customer-support"]
}
]
}

Read that the way it is meant: 3,184 would-blocks from deny_shell_execution concentrated in an order-lookup class is almost certainly the documented base64 fan-out false positive, and promoting it would break real traffic. 12 hits from llm01_prompt_injection in one class is probably real.

Terminal window
# promote just the second one to enforcing (admin)
curl -s -X PUT "$NRVQ_API_URL/api/v1/baseline/controls" \
-H "Authorization: Bearer $NRVQ_API_TOKEN" -H "Content-Type: application/json" \
-d '{"namespace":"chatbot-prod","preset":"strict",
"effects":{"llm01_prompt_injection":"deny","deny_shell_execution":"monitor"}}'
{
"namespace": "chatbot-prod",
"preset": "strict",
"enforcing": ["llm01_prompt_injection"],
"disabled": [],
"rego_lines": 1183
}
Terminal window
# export the OWASP evidence pack as JSON
curl -s "$NRVQ_API_URL/api/v1/compliance/owasp/export?namespace=chatbot-prod&format=json" \
-H "Authorization: Bearer $NRVQ_API_TOKEN" -o owasp-evidence.json

The asset & attack graphs tell you what an agent can reach. Framework coverage tells you how the loaded policy maps to a recognized framework. Policy Compliance tells you what your own policies are catching and what promoting them would cost. All three feed the same gated draft → review → apply loop in the Policy Catalog, and all three keep a rule being present separate from a threat being proven blocked — which you close with the red-team suite.

A practical order:

  1. Install with the strict baseline in its default audit posture.
  2. Read Policy Compliance for a week to see the real blast radius per control.
  3. Promote the controls whose counts look right, via PUT /baseline/controls.
  4. Read the Compliance matrix for residual framework gaps and generate remediation for the rule-backed ones.
  5. Run the red-team suite to turn coverage into efficacy.
  6. Export the evidence pack.