Evaluate policies server-side

Proposed beta design for server-side policy evaluation in Kosli: author and ship a versioned policy, then evaluate it to record a trustworthy, replicable decision.

Proposed design — not yet available. This page describes planned beta behavior for server-side policy evaluation and is published to gather feedback before the feature is built. The commands, flags, and output shown are illustrative and will change as the design firms up. Nothing here can be run against Kosli today. Share comments with the Kosli team.

Today, kosli evaluate runs client-side: the policy lives in your repository, the check runs on your CI runner, and Kosli only learns the outcome if you explicitly attest it back — a self-reported result Kosli takes on trust. Server-side policy evaluation moves the Policy Decision Point (PDP) into Kosli. You ship a versioned policy to Kosli, Kosli executes it against facts it already holds, and Kosli records the decision itself — one it can vouch for, connected to the exact policy version and inputs used.

This tutorial walks a governance platform engineer through the beta flow. Each part below links to its section:

1. Author, ship, and evaluate a policy — the core loop: write Rego locally, publish a versioned policy into Kosli, evaluate it server-side to record a decision against a control, then view and trace that decision in Kosli.
2. Validate and test a policy — check a policy holds up before you ship it or gate a pipeline on it.
3. Manage policies — update (creating a new version), tag, and archive policies over time.
4. View the decisions for a policy — review and export the decisions a policy has produced.
5. Inspect a failing evaluation — understand why an evaluation was denied.

This is distinct from the existing client-side Evaluate trails with OPA policies tutorial, where the CLI runs the policy locally. Here, Kosli stores the policy and performs the evaluation.

Why server-side

Running evaluation inside Kosli changes what a decision means — and where the data is processed:

	Client-side kosli evaluate trail (today)	Server-side kosli evaluate policy (proposed)
Where the policy lives	Your repository	Stored and versioned in Kosli
Where it runs	Your CI runner	Kosli (isolated from the API)
What Kosli stores	Whatever you attest back (self-reported)	A decision Kosli computed and can vouch for
Data movement	Full trail evidence is fetched to the runner	Evaluated in place — nothing shipped to the client
Replicability	Depends on your pipeline	Decision cites its exact policy version and inputs

Because Kosli both stores the policy and runs it, a decision points at exactly what ran — the policy version, its parameters, and the trail facts that were in scope — which is what makes the decision trustworthy and replicable for audit.

Running evaluation next to the data is also faster and scales further: Kosli doesn’t have to ship a trail’s full evidence out to your runner before evaluating, so evaluations can cover much larger volumes of facts in the cloud.

Prerequisites

These prerequisites describe the proposed beta experience — they are not runnable today.

• Install Kosli CLI.
• Get a Kosli API token.
• Have at least one Control defined, and a Flow with a Trail carrying the attestations you want to evaluate.

export KOSLI_ORG=<your-org>
export KOSLI_API_TOKEN=<your-api-token>

Author, ship, and evaluate a policy

The core loop takes a policy from local Rego to a recorded decision. The later sections build on it — validating and managing policies, and viewing and inspecting the decisions they produce.

1. Author a policy locally

   Author your policy as code in your own repository, which stays the source of truth. A policy is its policy-as-code (Rego), a short description of its purpose, and an input-context schema declaring whether it evaluates a single trail or multiple trails. You give it a stable identifier when you publish it to Kosli (next step).

   The output schema requires an allow rule; you can return richer output alongside it. Follow the same fail-safe design rules as client-side policies — see Evaluate trails with OPA policies for the reasoning behind them.

   pr-approved.rego

   package policy
   
   import rego.v1
   
   pr_attestation_name := data.params.pr_attestation_name
   
   default allow = false
   
   allow if {
       every pr in input.trail.compliance_status.attestations_statuses[pr_attestation_name].pull_requests {
           count(pr.approvers) > 0
       }
   }
   
   violations contains msg if {
       some pr in input.trail.compliance_status.attestations_statuses[pr_attestation_name].pull_requests
       count(pr.approvers) == 0
       msg := sprintf("pull-request %v has no approvers", [pr.url])
   }
   

   Before you ship it, you can validate and test the policy against real trail data.

   In the beta, a policy evaluates the facts on the trail(s) you point it at — attestations on trail / trails. Evaluating other targets (snapshots, artifacts, generic input) and composing multiple policies into one are not included (see What’s out of scope).

2. Ship a versioned policy into Kosli

   Publish the policy into Kosli via the CLI (or API). You give the policy a stable identifier (pr-approved here) — this is the identifier decisions and evaluations use to refer to the policy, so it is immutable once created; choose one that matches how your organization names its policies. Kosli records the first version on publish. Every later change creates a new version and records who made it, so a decision can always point back to the exact policy that produced it — see Manage policies for updating, tagging, and archiving over time.

   kosli create policy pr-approved pr-approved.rego \
       --type rego \
       --context-schema trail \
       --description "Every pull request on the trail has at least one approver."
   

   Policy 'pr-approved' created
     identifier:  pr-approved
     version:     1
   

   kosli create policy is the same command used for environment policies: --type rego selects an evaluation policy, while the default --type env creates an environment policy. In the beta, authoring is CLI/API only — the Kosli UI is read/tag-only.

3. Evaluate and record a decision

   kosli evaluate policy is the core command. Kosli fetches the in-scope facts, runs the policy server-side, and records a decision against a control. Evaluation is asynchronous by default — the command returns once the evaluation is queued. Add --sync to wait for and return the outcome, as shown here.

   kosli evaluate policy \
       --flow "$FLOW_NAME" \
       --policy pr-approved \
       --control RCTL-043 \
       --evaluation-context trail="$TRAIL_NAME" \
       --fingerprint "$ARTIFACT_FINGERPRINT" \
       --name pr-approval-decision \
       --params '{"pr_attestation_name": "pull-request"}' \
       --sync
   

   RESULT:    ALLOWED
   DECISION:  recorded against control RCTL-043
   POLICY:    pr-approved @ version 1
   

   Flag	Description
   --flow	The flow the decision is recorded on (the destination flow).
   --trail	The trail the decision is recorded on. Defaults to the evaluation trail when a single one is evaluated; required otherwise.
   --policy	The policy to evaluate — a Kosli policy identifier (uses its current version) or a local .rego file (see Evaluate a local policy file).
   --control	The control identifier the resulting decision is recorded against.
   --evaluation-context	One or more typed references to the facts to evaluate (the source), repeatable. In the beta the only type is a trail: trail=<trail_id> (in --flow), or trail=<flow_id>/<trail_id> to name a trail in another flow. Defaults to the decision trail. The typed form leaves room for other context types (snapshots, artifacts, …) later without new flags.
   --fingerprint / --artifact-type	The artifact the decision applies to. Pass --fingerprint directly, or --artifact-type with the artifact name/path so Kosli calculates the fingerprint.
   --name	The attestation slot name the decision is recorded under on the trail.
   --params	Parameters passed to the policy as data.params. Every evaluation records the params it used.
   --description	Optional human-readable context for the decision.
   --annotate	Optional key=value annotations stored with the decision.
   --sync	Wait for the evaluation to finish and return the outcome. Off by default — evaluation runs asynchronously.
   --assert	Exit non-zero when the policy denies, for use as a pipeline gate. Implies --sync, since asserting requires the outcome.
   --dry-run	Evaluate and return the outcome without recording a decision — for testing (see Run an ad-hoc evaluation).

   By default the command queues the evaluation and returns without waiting; the decision is recorded when the evaluation completes. Add --sync to wait for the outcome, or --assert to both wait and fail the step when the policy denies — use --assert as a pipeline gate. Once recorded, a decision appears against the policy — see View the decisions for a policy.

   Evaluating facts from another flow. By default the decision is recorded on the same trail whose facts you evaluate. Because each --evaluation-context reference carries its own flow (trail=<flow_id>/<trail_id>), you can evaluate a trail in a different flow — for example, evaluating a build flow’s trail but recording the decision on a release flow’s trail:

   kosli evaluate policy \
       --policy pr-approved \
       --control RCTL-043 \
       --evaluation-context trail=build-ci/build-42 \
       --flow release \
       --trail release-99 \
       --fingerprint "$ARTIFACT_FINGERPRINT" \
       --name pr-approval-decision \
       --sync
   

   The server captures the evidence for you. Because Kosli runs the evaluation, the decision automatically records the policy (and its version), the evaluation report, and any violations — you don’t attach them by hand with --attachments / --user-data as you do today when you attest a client-side result yourself. Likewise, --compliant is derived from the policy’s allow result rather than passed in. This is the core difference from recording a decision yourself.

   The decision this records is the same kind of decision described in Working with controls — the difference is that Kosli computed it server-side rather than you attesting the outcome yourself. Enforcement is unchanged: require the control via for_control in an environment policy and gate promotion with kosli assert artifact --environment, exactly as before. Recording the decision is the PDP; asserting it is the PEP.

4. View and trace the decision

   As soon as the evaluation completes, the decision appears in Kosli against the control and the policy that produced it. Open it in the Kosli app to see the outcome and its evaluation context — the policy version that ran, the parameters used, the trail moments that were in scope (and, through them, the attestations that were evaluated), and a timestamp.

   Because those inputs are pinned, the decision is fully traceable, and re-running from the stored context yields the same decision — this is what makes a server-side decision replicable for audit. To review and filter every decision a policy has produced, see View the decisions for a policy.

Validate and test a policy

Before you ship a policy — or wire it into a pipeline gate — check that it holds up.

Your policy repository is the source of truth for your policies, so treat it like code: keep Rego tests alongside each policy that assert it allows and denies the cases you expect. Rego’s built-in test framework runs these locally and in CI, catching regressions before you publish a new version.

Validate

kosli validate policy checks that a policy is well-formed without recording anything. It confirms that:

• the Rego compiles;
• it matches its declared input-context schema (trail / trails);
• it satisfies the output schema — an allow rule is mandatory.

kosli validate policy \
    --policy pr-approved.rego \
    --context-schema trail

RESULT:  VALID

Run an ad-hoc evaluation

To see how a policy behaves against real trail data without recording a decision, add --dry-run to kosli evaluate policy. Kosli runs the policy server-side and returns the outcome, but no decision is stored — so --control and --name aren’t needed:

kosli evaluate policy \
    --flow "$FLOW_NAME" \
    --policy pr-approved.rego \
    --evaluation-context trail="$TRAIL_NAME" \
    --params '{"pr_attestation_name": "pull-request"}' \
    --sync \
    --dry-run

RESULT:  ALLOWED  (dry-run — no decision recorded)

This is the fastest way to iterate: point a candidate policy at a trail you know should pass (or fail) and confirm the outcome before publishing a version or gating a pipeline on it.

Evaluate a local policy file

You don’t have to publish a policy before it can record a decision. Point --policy at a local, unpublished .rego file and Kosli evaluates it server-side and records the decision, just as it would for a stored policy — useful while you’re still iterating on a policy you haven’t shipped:

kosli evaluate policy \
    --flow "$FLOW_NAME" \
    --policy ./pr-approved.rego \
    --control RCTL-043 \
    --evaluation-context trail="$TRAIL_NAME" \
    --fingerprint "$ARTIFACT_FINGERPRINT" \
    --name pr-approval-decision \
    --params '{"pr_attestation_name": "pull-request"}'

Because there is no stored policy version to cite, the decision records the exact policy source that was evaluated instead. Unlike the ad-hoc evaluation above, this records a decision — add --dry-run if you only want the outcome without recording one.

Manage policies

Update a policy

Publishing a change is the same command as the initial publish: re-run kosli create policy with the updated Rego or description. Each publish creates a new version and records who made it — add --comment to note what changed — so a decision always points back to the exact version that produced it.

kosli create policy pr-approved pr-approved.rego \
    --type rego \
    --context-schema trail \
    --description "Every pull request on the trail has an approver from a code owner." \
    --comment "Require approval from a code owner"

An evaluation that references a policy by identifier uses its current version. Decisions recorded earlier keep pointing at the version that was current when they were made.

Tag a policy

Tags are versionless labels for organizing and filtering policies — grouping by team, framework, or control area. Tagging does not create a new version.

kosli tag policy pr-approved --set team=platform --set framework=soc2

Archive and unarchive

When a policy is retired, archive it rather than deleting it. Archiving is light-touch: the policy is removed from the main list in the UI but remains evaluable, and its versions and past decisions are preserved. Archiving records who archived it and when, and does not create a new version. Unarchive it if it comes back into use.

kosli archive policy pr-approved
kosli unarchive policy pr-approved

List and filter policies

Browse your policies in the Kosli app — select Policies in the sidebar to see the full catalog. Search by name or identifier, and filter by tag to narrow the list. Each policy shows its identifier, current version, and tags.

You can also list policies from the CLI. Filter to evaluation policies with --type rego, and by name or tag — each entry shows the identifier and current version:

kosli list policies --type rego --tag team=platform

View the decisions for a policy

This is where auditors and control owners review what a policy has decided. Every decision a policy produces is listed against that policy, across all versions of the policy. In the Kosli app, select Policies, choose a policy, and open its Decisions tab. Each row shows the timestamp, the trail and artifact the decision applies to, the policy version that produced it, and the outcome (compliant or non-compliant). Filter by outcome, version, or time range, and page through the results to sample them.

Select a decision to open its detail, showing the trail and artifact it applies to, the policy version and parameters that were used, the evaluation report, and any violations.

For an audit or an offline review, export the filtered set as CSV — from the Decisions view in the UI, or from the CLI:

kosli list decisions \
    --policy pr-approved \
    --compliant=false \
    --since 30d \
    --output csv > decisions.csv

--compliant=false surfaces every non-compliant evaluation for follow-up; combine it with --since or a version filter to scope the export.

Inspect a failing evaluation

When a policy denies, an application team needs to know why — and the decision records it through its violations. Whether the policy’s rules weren’t met (for example, a pull request with no approver) or an expected attestation was missing from the trail, both surface as violations on the decision.

A policy can return more than allow and violations — any additional fields in its output are captured on the decision, so you can surface richer, human-readable context beyond a bare pass/fail.

When you gate a pipeline with --assert, Kosli prints the violations as markdown so failures are readable directly in CI logs:

kosli evaluate policy \
    --flow "$FLOW_NAME" \
    --policy pr-approved \
    --control RCTL-043 \
    --evaluation-context trail="$TRAIL_NAME" \
    --fingerprint "$ARTIFACT_FINGERPRINT" \
    --name pr-approval-decision \
    --params '{"pr_attestation_name": "pull-request"}' \
    --assert

RESULT:      DENIED
VIOLATIONS:  pull-request https://github.com/acme/app/pull/42 has no approvers

In the Kosli UI, the violations are parsed out of the decision and rendered on the decision detail (shown above under View the decisions for a policy), so an application team can see exactly what to fix.

What’s out of scope for the beta

The following are explicitly not included in the beta and may be addressed later:

• Evidence-attachment content evaluation — evaluating the contents of attached files.
• Composable policies — composing many authored policies into one versioned policy.
• Non-trail evaluate targets — snapshots, artifacts, and generic input.
• Scheduled evaluations and risk measurement.
• Associating policies to controls — a stored policy-to-control link that lets Kosli infer which policy closes which control. In the beta you name the control per evaluation with --control; there is no stored mapping.

Next steps

• Working with controls — define controls and enforce them with for_control and kosli assert.
• Environment policies — require controls before an artifact runs in an environment.
• Enforce policies — gate promotion on compliance in your pipeline.
• Evaluate trails with OPA policies — the client-side kosli evaluate this builds on.

Feedback

This is a design preview. If you are reviewing the proposed beta flow, the most useful feedback is on the command surface (kosli evaluate policy and how policies are published), the decision and audit model, and anything in What’s out of scope that you would expect in the beta. Share comments with the Kosli team.