# Semantic decisions in agent pipelines, with Jev

> T25 asks Jev, from TypeSafe, and lets a deterministic policy decide what the probability means.

- By: Djan Magno
- Published: 2026-09-26
- Updated: 2026-09-26
- URL: https://t25.io/en/blog/semantic-decisions-in-t25-with-jev/
- Engineering · semantic decisions, System One, Jev, shadow mode, coding agents, deterministic policy

> **TL;DR:** > - Semantic decisions in agent pipelines, for us, are typed questions to Jev. The answer is a Choice, a Score, or a Noul: a probability, not a paragraph.
> - The default is off, `provider: fake`, and `mode: shadow`. The opinion lands in `semantic-decisions.jsonl` next to the heuristic, with `policyAction: observe`.
> - A missing key, a blown 1500 ms deadline, or JSON that fails the parser: the heuristic stands and the error is stored. A missing fact is not stored as a no.
> - The review verdict still comes from `evaluateReview()`. The merge to main stays with a person.

When QA closes a criterion, T25 does not ask for a paragraph about the proof. It asks `evidence.relation_to_criterion.v1`: does the evidence support the criterion, contradict it, or say nothing. The answer is a Choice with three probabilities. That contract is what we mean by semantic decisions in agent pipelines.

## Semantic decisions in agent pipelines: what we ask

The catalog lives in `src/decisions/questions.ts`. Every id has a version (`v1`) and a type. Jev, from TypeSafe, is a System One model: one POST to `https://api.typesafe.ai/v1/systemone` sends a `state` and several questions, and the typed answers come back together. The [public introduction](https://docs.typesafe.ai/introduction) names three primitives. A Noul is the probability the answer is yes, from 0 to 1. A Choice picks from the list we sent and returns the distribution. A Score places the state on a rubric and also returns `confidence`.

Five families cover the judgment a conditional does not close:

| Family | Question | Type | What is at stake |
|---|---|---|---|
| Context | `context.candidate_helps.v1` | Noul | Does this file help the criteria, or does it only pad the prompt? |
| Progress | `progress.looks_like_workaround.v1` | Noul | Does the change fix the product, or does it weaken the test? |
| Evidence | `evidence.relation_to_criterion.v1` | Choice | Does the proof support the criterion, contradict it, or say nothing? |
| Attention | `attention.finding_lacks_context.v1` | Noul | Does the finding name a criterion or a file, or does someone have to open the rest? |
| Routing | `routing.adapter.v1` | Choice | Which CLI on the allowlist should try this role? |

QA fires `evidence.relation_to_criterion.v1` once per criterion (`supports`, `contradicts`, `says_nothing`). On a stage change, progress sends four Nouls: whether the diff still attacks the open criterion, whether the change looks like a test workaround, whether the error fingerprint repeats, and whether QA describes the same gap. Context looks at the first eight paths in `relatedFiles` at most. With more than three skills, `context.skill_candidate_helps.v1` is included.

`quality.action.v1` and `quality.scope.v1` run after the fact. `decideQualityAction` has already chosen rework, a QA rerun, or a person. The code's choice is stored in `heuristic` with probability 1 on the action taken and 0 on the others, and that Choice is not part of the `state`.

## Why a typed probability audits better than free text

Free text carries the evidence and the conclusion in the same block. Review already showed us the cost: the model writes `APPROVE` and, a few lines later, lists an in-scope finding. `evaluateReview()` decides, by reading the findings again. That case is written up in [The model's APPROVE is not the merge](https://t25.io/en/blog/the-models-approve-is-not-the-merge/).

A System One answer has no second channel for a verdict to hide in. `parseProviderAnswer()`, in `src/decisions/answers.ts`, rejects anything outside the contract:

- A Noul outside `[0, 1]` is dropped.
- A Choice outside the enum we sent is dropped. So is a probability key that was not in the enum.
- The enum probabilities have to sum to 1, with a tolerance of `0.02`.
- A Score without a valid distribution is dropped. The stored `level` is the option with the highest probability, computed here.

Two rows with the same `questionId` and the same `questionVersion` sit on one axis. Changing the wording is a new version, not a silent edit of the history.

## Shadow, active, and the fallback

The `semantic_decisions` section in the [configuration](https://t25.io/en/docs/configuration.html) starts off. Provider `fake`, `mode: shadow`, routing `manual`, model pinned to `jev-1.13.0`. The fake answers inside the process. The test suite does not call TypeSafe.

`createSemanticDecisionEngine()` computes the heuristic on every observation. With `provider: jev` and the required facts present, the call fits in `timeout_ms` (1500 ms). The clock covers the whole call, including up to two retries on HTTP 429 and 529. No key, a timeout, an unexpected status, or JSON the parser refuses: the row keeps a `providerError`, `jev: null`, and the heuristic. `observeSemantic()` catches the exception and the task continues.

A missing required fact is never sent to Jev. `projectObservationState()` writes `{ absent: true }`, sets `factsStatus: insufficient`, and skips Jev. In `composeAttention()`, a null answer or insufficient facts becomes `unknown`, and `unknown` asks for a person.

The YAML accepts `mode: active`. The stored row is still `mode: "shadow"` and `policyAction: "observe"`. `policyActionFor()` returns nothing else, and `assertNoControlFlow()` rejects the rest. Promoting an action out of shadow requires `assertPromotionAllowed()`: a paired comparison, a holdout replay, human adjudication, incremental value, and a test that the path still runs without Jev.

The opt-in that can already change an execution choice is `routing: automatic`, with the layer on and provider `jev`. `selectExperimentAdapter()` switches CLI when `routing.adapter.v1` reports `confidence` of at least `0.7` and the option is on the role's allowlist. Otherwise the first configured adapter stays, with `fallbackReason` of `manual`, `incomplete_answers`, `low_confidence`, `not_authorized`, or `kept_default`. The Score `routing.role_difficulty.v1` only steps down inside the same CLI (two steps at `trivial`, one at `easy`, zero at `moderate` or `hard`), on the same floor. `SKILL_SUGGESTION_PROTOCOL.ready` is `false`, so the skill question can be logged while the prompt still gets the full list.

## What each row stores

Each line of `semantic-decisions.jsonl` is a `semantic-decision.v1`: task, versioned `questionId`, provider, SHA-256 digest of the state, heuristic, answer or `null`, `policyAction`, error, `responseModel`, `factsStatus`, and absent keys. The log keeps the latest 4000 records. A line that fails to parse goes to `semantic-decisions.invalid.jsonl`.

On `DONE`, `FAILED`, or `CANCELLED`, `stampLaterOutcome()` fills an empty `laterOutcome` with `accepted`, `failed`, or `cancelled`. The original answer stays.

The state we send is a projection. `send_code: true` fails YAML loading; the only accepted value is `false`. The allowlist drops `diff`, `patch`, `content`, `source`, `body`, and `prompt`. A line that looks like source becomes `[CODE_OMITTED]`. Secrets go through the redactor. A payload that is still blocked is not asked. The key stays out of the YAML, in `T25_JEV_API_KEY` or in `jev-secret.json` mode `0600`. `t25 jev show` and `GET /api/v1/jev` say whether a key exists. They never return the value.

The default model is `jev-1.13.0`. The [public models page](https://docs.typesafe.ai/models) warns that the `jev-latest` alias starts pointing at another id when a release ships, and that the answers can move with it. A threshold tuned on one version stays pinned to it. `responseModel` stores the id that answered.

## Where the threshold lives, and why we stop there

The public Noul docs put the threshold in the program and send the middle value to a person. `noulBand()` does that: `0.7` or above is positive, `0.3` or below is negative, and the middle is uncertain. `composeAttention()` reads how the evidence relates to the criterion, whether the finding has context, and whether the fingerprint repeats a project incident. One uncertain answer returns `unknown`. A contradiction, an opaque finding, or a repeated incident returns `needs_human`. `skip_human` comes out only when the whole set is decisive and none of those signals appear. It does not merge.

The merge to main stays with the person who answers for the repository. The review verdict stays in `evaluateReview()`. A spec becomes a plan only after a person approves it. The factory runs on the machine of the person who installed it and calls the agent CLIs that person already pays for. While a promotion proof is missing, the heuristic is the decision.

## How to use the same contract in another pipeline

1. One question per judgment, with the enum your code already knows how to handle.
2. Send only the keys that question lists. The repository stays where it is.
3. Fail closed on parse. An invalid body is an error, stored apart from the valid log.
4. Persist the heuristic and the probability. Do not put the heuristic back into the `state`.
5. Write the threshold in code, with a middle band that calls a person. Blow the deadline and continue on the heuristic.

## FAQ

### Does Jev approve the pull request or perform the merge?

The answer comes in as a signal. Review goes through `evaluateReview()`, which recomputes the verdict from the findings. The merge to main stays a human action, after the repository's required checks.

### What happens if Jev expires in the middle of a task?

The heuristic remains. The row gets a `providerError` and `jev: null`. The task continues. On the attention path, a missing answer counts as `unknown` and asks for a person.

### Why not ask for a verdict in prose and parse it?

Prose bundles the proof and the conclusion, and the parser becomes another place for the conclusion to hide. Choice, Score, and Noul return a value in the enum we defined, plus the distribution. We can reject the row and move the threshold without rereading a paragraph.

### Can T25 run without a Jev key?

Yes. The default is the layer off and `provider: fake`. `t25 jev show` reports the provider, the mode, and whether a key exists. Pointing the provider at `jev` is opt-in, through the CLI, through `PUT /api/v1/jev` as an admin, or through the YAML, and the starting mode is still shadow.