# Decisions with Jev

Route with Choice, review coverage with Score, and assess evidence with Noul.

Use the standalone decision SDK to route a request, classify a document, or review an answer. It wraps TypeSafe Jev's **Choice**, **Score**, and **Noul** primitives in typed HarnessX requests and results.

Import it from `harnessx.decisions`. Decisions have their own async provider and usage accounting. An application calls the provider before selecting an agent or after receiving an answer; automatic checks inside the agent loop remain planned.

## Choose a question type

| HarnessX question | Jev primitive | Result | Used in the examples |
| --- | --- | --- | --- |
| `ChoiceQuestion` | `Choice` | Selected label, probabilities for every option, and confidence | Agent routing and document classification |
| `ScoreQuestion` | `Score` | Unrounded expected score, ordered probabilities, rubric levels, and confidence | Answer coverage |
| `BooleanQuestion` | `Noul` | Probability that the proposition is true | Support from supplied evidence |

Choice requires at least two named options with descriptions. Score accepts two to ten ordered descriptions, indexed from zero. A four-level rubric produces a score between 0 and 3, which may be fractional.

`BooleanCriteria` optionally describes the true and false outcomes. Either description can be omitted. Noul returns a probability, without a separate confidence value. Compare `answer.probability` with a threshold chosen for your application; `bool(answer)` raises to prevent implicit conversion.

## Try the offline examples

From a clone of the repository (`uv sync --all-extras`), run:

~~~bash
uv run python -m examples.jev_routing --min-confidence 0.8
uv run python -m examples.jev_classification
uv run python -m examples.jev_answer_review
~~~

These use fixed decisions and scripted agent responses. They need no Jev extra, API keys, or external services. Outputs demonstrate integration behavior; they do not classify changed inputs or measure model quality. The examples are source-checkout entry points and are not bundled in the SDK wheel.

The same three examples appear in the [harness-web example library](../web-demo/#use-the-example-library) as offline demos.

## Install and enable live decisions

Install the optional extra and configure a TypeSafe key:

~~~bash
uv add "harnessx[jev]"
export TYPESAFE_API_KEY='your-key'
~~~

The `jev` extra installs `typesafe-sdk>=0.7.1,<0.8`; it is also included in `all`. Importing `harnessx` or `harnessx.decisions` does not load the vendor SDK. Constructing `JevDecisionProvider` requires the extra.

Add `--live` explicitly to call Jev:

~~~bash
uv run python -m examples.jev_classification --live --text 'Document text here'
uv run python -m examples.jev_answer_review --live
uv run python -m examples.jev_routing --live --min-confidence 0.8 \
  --provider anthropic --model YOUR_MODEL_ID
~~~

Live classification and review require the Jev extra and `TYPESAFE_API_KEY`. Live answer review still uses a scripted agent answer, then sends that answer and its evidence to Jev. Live routing also calls the selected generative model: supply an explicit model ID, its [provider credentials](../providers/), and its optional extra where required. Live calls may incur service charges.

## Evaluate typed questions

This complete example sends a query, answer, and evidence in one request. Each question independently assesses that supplied state. It makes a live Jev call when run.

~~~python
import asyncio
from harnessx.decisions import (
    BooleanCriteria,
    BooleanQuestion,
    ChoiceQuestion,
    DecisionRequest,
    JevDecisionProvider,
    ScoreQuestion,
)

async def main():
    request = DecisionRequest(
        check_id="answer_review",
        version="v1",
        state={
            "query": "What were sales in January and February?",
            "answer": "Sales were $100 in January and $120 in February.",
            "evidence": "Sales report: January $100; February $120.",
        },
        questions={
            "route": ChoiceQuestion(
                instructions="Choose the best route for the query.",
                options={
                    "sql": "Internal structured data or SQL.",
                    "research": "External research and source synthesis.",
                    "general": "General writing and explanations.",
                    "unknown": "Insufficient context or no clear route.",
                },
            ),
            "coverage": ScoreQuestion(
                instructions="How completely does the answer address the query?",
                levels=(
                    "Does not answer the request.",
                    "Addresses a small part, with major omissions.",
                    "Addresses most of the request, with minor omissions.",
                    "Fully addresses every part of the request.",
                ),
            ),
            "supported": BooleanQuestion(
                instructions="Does the evidence support every factual claim?",
                criteria=BooleanCriteria(
                    true="Every claim follows from the supplied evidence.",
                    false="A claim is unsupported or contradicts the evidence.",
                ),
            ),
        },
    )

    async with JevDecisionProvider() as decisions:
        batch = await decisions.evaluate(request)

    if batch.status == "unavailable":
        assert batch.error is not None
        print(batch.error.code, batch.error.message)
        return

    print("Route:", batch.choices["route"].choice)
    print("Coverage:", batch.scores["coverage"].score)
    print("Evidence support:", batch.booleans["supported"].probability)

asyncio.run(main())
~~~

The adapter translates those question objects to the official SDK's `Choice`, `Score`, and `Noul`, then calls `AsyncTypeSafeClient.system_one()`. It preserves the resolved model and available service request ID in the result. The default endpoint is the direct TypeSafe API; OpenRouter compatibility for this adapter has not been verified.

## Route to one agent

The [routing example source](https://github.com/datagol/agent-harness-x/blob/main/examples/jev_routing.py) uses Choice to select from an application-defined mapping of agent factories:

| Decision | Application behavior |
| --- | --- |
| `sql`, `research`, or `general`, with sufficient confidence | Construct and run that one agent |
| `unknown` or an unexpected label | Use the general agent |
| Missing confidence or confidence below `min_confidence` | Use the general agent |
| Unavailable decision | Use the general agent |
| Selected agent fails | Return its failed result; do not start a second agent |

`min_confidence` is required and must be a finite number between zero and one. The example command's `0.8` is illustrative, not a calibrated default. The model selects a label; application code owns the factories and configuration. The minimal specialists draft answers; the example does not connect a database or browse sources.

## Classify documents and review answers

The [classification example source](https://github.com/datagol/agent-harness-x/blob/main/examples/jev_classification.py) asks a Choice question with five labels: **W2**, **Deposit**, **Insurance**, **Payroll**, and **unknown**. It returns the typed decision and its full distribution. The offline fixture always returns W2; pass `--live --text ...` to assess supplied document text.

The [answer-review example source](https://github.com/datagol/agent-harness-x/blob/main/examples/jev_answer_review.py) makes a separate decision request after a successful `Agent.run()` result:

- **Score** evaluates coverage against four ordered levels, from unanswered to fully addressed.
- **Noul** evaluates whether every factual claim follows from the supplied evidence, using explicit true and false criteria.

The review skips unsuccessful runs and leaves the original run's status, answer, and generative-model usage unchanged. It reports probabilities and scores; it does not turn them into an automatic pass/fail gate. Jev assesses the evidence you supply and does not retrieve or independently verify sources.

## Handle results and failures

`DecisionBatch.status` is either `completed` or `unavailable`. A completed batch requires every requested answer to validate. An uncertain but valid answer remains completed; confidence thresholds belong to the application.

Missing or incompatible answers, mismatched option labels or rubric levels, nonfinite probabilities, invalid distributions, and inconsistent scores produce an unavailable batch with **empty answer maps**. The adapter ignores extra answer IDs after SDK decoding. Probability distributions must sum to one within an absolute tolerance of `1e-3`; they are never silently renormalized. Scores must agree with their distribution's weighted mean within `1e-3 * max(1, top_level)`.

| Condition | Outcome |
| --- | --- |
| Missing optional dependency | `ImportError` with installation guidance |
| Invalid local configuration, question, or oversized state | Local exception before I/O |
| Authentication, authorization, or request rejection | Unavailable, without retry |
| Connection failure, transport timeout, HTTP 429 or 500/502/503/504 | Bounded retry, then unavailable if unresolved |
| Invalid or missing requested answers | Unavailable, without retry |
| Caller cancellation | `asyncio.CancelledError` propagates |
| Programming error | Exception propagates |

`DecisionError` exposes a stable code, sanitized message, optional HTTP status, and whether the cause is retryable. Codes are `authentication`, `forbidden`, `invalid_request`, `rate_limited`, `timeout`, `connection`, `server`, `invalid_response`, and `unexpected_http`. Raw server bodies are excluded from these diagnostics. The upstream SDK has its own logging configuration; debug logging can include bodies.

## Configure limits and ownership

| Provider option | Default | Meaning |
| --- | --- | --- |
| `model` | `jev-1.13.0` | Explicit model sent on every request |
| `timeout_seconds` | `2.0` | Total evaluation deadline, including retry delays |
| `max_retries` | `1` | At most two HTTP attempts by default |
| `max_state_bytes` | `16384` | Limit on compact UTF-8 JSON state |
| `api_key` | `TYPESAFE_API_KEY` | Credential used when constructing a client |
| `base_url` | `https://api.typesafe.ai` | Explicit API root; does not inherit `TYPESAFE_BASE_URL` |
| `client` | `None` | Optional borrowed `AsyncTypeSafeClient` |

These are configurable engineering limits, not measured Jev latency claims. `evaluate(request, timeout_seconds=..., max_attempts=...)` can shorten the configured deadline and cap attempts. SDK retries are disabled so they cannot multiply the adapter's retry budget. A `Retry-After` delay is honored when it fits the remaining deadline; otherwise the evaluation returns unavailable.

Use `async with JevDecisionProvider(...)` to close resources reliably. Clients created by the adapter are owned and closed by it. Injected clients remain caller-owned; combining `client` with `api_key` or `base_url` raises. Per-call overrides do not change a borrowed client's defaults.

The provider supports concurrent evaluations. `aclose()` is idempotent, cancels outstanding evaluations, and closes owned resources. Evaluating after close raises `RuntimeError`.

## Preserve state, usage, and results

Request state must be text, a JSON object, or a JSON array containing only finite JSON values and string object keys. State is copied at construction and again before dispatch. Arbitrary objects are never converted with `str()`.

Requests and results expose `to_dict()` and `from_dict()` with `schema_version=1`. Decode validates the data, rejects unknown schema versions and question/answer kinds, and accepts additive metadata fields. Use those codecs before `json.dumps`; score distributions become arrays, and grouped answer maps are read-only.

A batch preserves `check_id`, `version`, requested and resolved model, available request ID, elapsed time, and attempt count. Token counts describe the successful response. After retries, earlier attempts may have consumed unreported tokens, so `usage.incomplete` remains true. Absent measurements stay `None`, including cost: the adapter does not estimate a dollar figure. Decision usage is separate from agent model usage.

## Implementation status

The standalone SDK, all three examples, and offline harness-web entries are implemented. The adapter has tests through the real TypeSafe SDK with a mocked HTTP transport, plus cancellation, lifecycle, validation, packaging, and offline example checks. Those checks do not establish live service access or decision quality.

Automatic agent-loop observation, persisted runtime assessments, approval enforcement, and automatic interventions remain planned. See [Implementation status](../status/) for the broader SDK boundaries and [API reference](../api/#decision-sdk) for the public contracts.