# Knowledge (OKF)

Load Open Knowledge Format bundles from a folder or git URL and retrieve concepts lazily.

The [Open Knowledge Format](https://github.com/GoogleCloudPlatform/open-knowledge-format)
(OKF v0.2) is a vendor-neutral way to give agents curated context: a folder of
markdown files with YAML frontmatter. Every non-reserved `.md` file is a
**concept** with at least a `type`; `index.md` files are directory listings for
progressive disclosure, `log.md` files record change history, and ordinary
markdown links between concepts form the graph. Files are read and indexed when
the bundle loads. Model context is filled **lazily**: the root index goes into
the system prompt, and the model pulls concepts into context with three
read-only tools. Where [skills](../skills/) carry instructions, knowledge
bundles carry facts with provenance.

## Authoring a concept

`knowledge/metrics/active-users.md`:

~~~markdown
---
type: Metric
title: Active Users
description: Daily and monthly active users, counted from qualifying product events.
tags: [metric, engagement]
status: stable
stale_after: 2027-09-27T00:00:00Z
verified:
  - by: human:analytics-lead
    at: 2026-09-27T00:00:00Z
sources:
  - id: dau-model
    resource: /tables/events.md
---

A user is active on a day when they emit one qualifying event.[^dau-model]
Computed from the [events table](/tables/events.md).
~~~

`/tables/events.md` is bundle-root-relative; `./events.md` is relative to the
file, and a directory link such as `tables/` resolves to `tables/index.md`.
Inline and reference-style links become graph edges; links inside code
examples, images, HTML comments, and external URLs do not. Broken links,
unknown `type` values, and missing optional fields are all legal per the spec.
Unknown frontmatter keys are kept on `concept.frontmatter`.

Loading never raises for non-conformant content. Files without frontmatter,
with an empty `type`, with malformed YAML, or with an invalid YAML timestamp are
skipped and reported in `bundle.warnings`. Invalid metadata on a root index
produces a warning while its body stays readable. Hidden files and directories
are ignored. `index.md` and `log.md` can be read with `read_concept` but are
excluded from search.

## Loading knowledge into an agent

~~~python
from harnessx import Agent, AgentConfig

agent = Agent(
    config=AgentConfig(
        system_prompt="Answer from the knowledge bundle and cite concept paths and sources.",
    ),
    knowledge=[
        "./knowledge",                                                          # local folder
        "https://github.com/GoogleCloudPlatform/open-knowledge-format/tree/main/bundles/ga4",
    ],
)
~~~

Loading happens when the agent is constructed, like skill loading. The
`knowledge=` argument also accepts a `KnowledgeManager` or `GitSource` entries:

~~~python
from harnessx import GitSource, KnowledgeManager, OKFBundle

manager = KnowledgeManager.from_paths(
    ["./knowledge", GitSource("git@github.com:acme/kb.git", ref="v2", subdir="bundles/sales")],
    refresh=True,             # re-fetch cached clones
    max_body_chars=12_000,    # read_concept body budget (default 8,000)
    index_char_budget=4_000,  # root index characters per bundle in the prompt
)
agent = Agent(config=AgentConfig(system_prompt="..."), knowledge=manager)

bundle = OKFBundle.load("./knowledge")
concept = bundle.get("/metrics/active-users.md")
print(concept.type, concept.trust_tier, concept.is_stale(), bundle.warnings)
print(bundle.search("active users", limit=3, type="Metric", tags=["engagement"]))
print(bundle.neighbors("/metrics/active-users.md")["backlinks"])
~~~

Bundle names default to the folder name, the git subdirectory, or the repository
name; two bundles with the same name raise `ValueError`, so pass
`OKFBundle.load(root, name=...)` when they collide.

### Loading from git

Git URLs are shallow-cloned into `.agent_knowledge/` on first use and reused
until `refresh=True`. GitHub and GitLab `.../tree/<ref>/<subdir>` links select a
branch or tag and a subdirectory. The `git` binary must be on the path; a
missing binary or a failed clone raises `RuntimeError` with the reason, and a
failed clone is cleaned up so the next load can retry.

A clone is built in a temporary directory and published to the cache only once
it is complete. One lock per repository and ref serializes threads and
processes, and `OKFBundle.from_git` holds it through parsing, so a refresh can
never change files halfway through a load. Subdirectories must resolve inside
the repository, symlinks included: an escaping symlink raises `ValueError`, a
missing subdirectory `FileNotFoundError`. `OKFBundle.from_git(..., timeout=30)`
bounds both the lock wait (`TimeoutError`) and each git command. Archives and
other remote formats are not fetched; extract them and pass the folder.

## What the model sees

The system prompt gains a `<knowledge>` block with the workflow and each
bundle's root `index.md`, truncated to `index_char_budget` characters. The
tools, all `ALLOW` by default:

| Tool | Arguments | Returns |
| --- | --- | --- |
| `search_concepts` | `query`, `limit`, `type`, `tags` | Ranked hits: path, type, title, description, status, score |
| `read_concept` | `path`, `max_chars` | Metadata header (type, status, trust tier, staleness, tags, sources) and the body; also reads indexes and logs |
| `get_neighbors` | `path` | Outbound links, backlinks, and unresolved links, with titles |

With several bundles loaded the tools take a `bundle` argument and the prompt
says so. Paths work with or without `.md`, and a directory path reads its
index. Lookups happen in an in-memory table, never on disk, so a path such as
`/../etc/passwd` returns "Concept not found" rather than a file. A body longer
than the budget ends with a marker that says how much was cut and how to ask
for more.

## How search ranks

`search_concepts` uses BM25F, BM25 with weighted fields:

| Field | Weight |
| --- | ---: |
| Title | 3 |
| Tags | 2.5 |
| Description | 2 |
| Body | 0.5 |

The score rewards rare terms, saturates repeated matches, and normalizes each
field's length (`k1=1.2`, `b=0.75`). Draft concepts are multiplied by 0.8 and
deprecated ones by 0.5; trust and staleness are shown when a concept is read
and do not change the score. Tokens are Unicode-normalized and case-folded, and
common English singular and plural forms match in both directions with a 0.95
multiplier for the expanded form. There is no general stemming, synonym
expansion, or embedding. `type` and `tags` filters are case-insensitive, all
requested tags must match, and `limit` is clamped to 1 to 50.

Scores are relative to the corpus that was searched. `manager.search()` across
every bundle uses one combined index with shared statistics; a search scoped to
one bundle, which is what the tool does, uses that bundle's own. Override the
synchronous `KnowledgeManager.search` to plug in embeddings or hybrid
retrieval: the tool calls it with the query, bundle, limit, type, and tags, and
expects hits with the same `path`, `type`, `title`, `description`, `status`,
`score`, and `bundle` keys.

## Trust, freshness, and computation contracts

Trust tiers follow the spec and come from valid `verified[].by` actors:

| Verification | Trust tier |
| --- | --- |
| No valid actor | `unverified` |
| Only `process:<id>` or `<producer>/<version>` actors | `machine-confirmed` |
| Any `human:<id>` actor | `human-reviewed` |

An actor needs a non-empty id with no whitespace. Malformed entries such as
`{}` or `{by: "human:"}` produce warnings and are left out of the normalized
metadata; the raw values stay on `concept.frontmatter`. Trust is an
author-supplied claim: the harness does not authenticate identities.

A concept is stale on or after `stale_after`. `status` defaults to `stable`;
`draft` and `deprecated` are the other values. `read_concept` reports both in
its header, and the prompt asks the model to prefer stable, human-reviewed,
current concepts and to say when it relies on something else.

For `Attested Computation` concepts, `read_concept` also exposes the contract:
`runtime`, `parameters`, `computation`, the `executor` and its receipt fields,
and the `attester`, plus source credibility metadata. The harness retrieves the
contract and its references; running the computation and attesting the result
are the application's job.

## Reacting to knowledge access

A `KNOWLEDGE_ACCESSED` hook fires on every tool call with `tool`, `bundle`, and
`found`, plus `query` and `hits` for searches, `path` and `body_chars` for reads,
and `path` and `hits` (outbound plus backlinks) for neighbors. A missing bundle
or concept still fires, with `found=False`.

~~~python
from harnessx import HookContext, HookEvent

async def on_knowledge(ctx: HookContext):
    print(ctx.data["tool"], ctx.data.get("query") or ctx.data.get("path"), ctx.data.get("hits"))

agent.hooks.on(HookEvent.KNOWLEDGE_ACCESSED, on_knowledge)
~~~

## Limits in this release

Search is lexical; embeddings are a `KnowledgeManager.search` override away.
Attested computations are surfaced, not executed. Bundles are loaded at
construction; adding one to a running agent is not supported. Archives and
non-git remote sources must be extracted first.

Full demo: `examples/knowledge_agent.py`, with a sample bundle under
`examples/knowledge/`. Set `AGENT_KNOWLEDGE` to a folder or git URL to point
the demo at another bundle, or launch it from [harness-web](../web-demo/). See
[Examples](../examples/).