Agents, knowledge & workspaces

The infrastructure for the first three operational roles of the alignment economy: reason · communicate · negotiate. This is where the substrate is built (KGs), where the conversation channel runs (working groups, threads), and where the non-commutative wanting of the participants gets reshaped into a definite joint prompt (intents). Read the alignment-economy guide first if you haven't — this page is the deep dive on the primitives those three stages depend on.

The thesis: humans align on new agreements through AI-augmented conversation. Each participant has a characteristic intelligence — a specific way of writing prompts over shared knowledge. Two such participants exchange messages; each writes their next prompt in the presence of the other's prompts; the wanting is reshaped by the order of exchange; eventually a measurement event collapses the entangled joint state into a definite joint prompt. YF's agents/workspaces stack is the engineering implementation of that picture.

A reader of this page should leave knowing:

• What a working group is and why it's the bounded channel in which entangled joint wanting builds up.
• What a thread is, the nine types, and how the privacy model supports both group reasoning and per-user private thinking.
• What an agent is in YieldFabric — an edge translator carrying knowledge and intelligence on behalf of a workspace, not a node carrying wanting.
• What a knowledge graph is — the shared substrate every participant draws interpretation from (documents, conversation analyses, deal projections, reasoning artifacts all live in the same frame store).
• How pipelines populate the substrate, how workflows coordinate multi-party action over it, and how the Intent → Confirm boundary is the structural collapse-point that bridges the agentic stages (reason/communicate/negotiate) into the on-chain stages (structure/execute).

The workspace (working group)

The working group is the unit of collaboration. Every persistent artifact in the agentic layer — threads, agents, workflows, KGs, documents, federation policies — lives inside a working group.

Working group
 │
 ├── Members          (humans + agents — both first-class)
 ├── Threads          (conversation containers)
 ├── Workflows        (typed DAGs of human + agent steps)
 ├── Knowledge graphs (typed frame substrate — multiple per group)
 ├── Documents        (live as frames inside KGs)
 └── Federation policies (outbound document-read grants to other groups)

A working group is a row in working_groups with a category — the shape discriminator that says how to interpret the group:

The role model

Membership lives in working_group_members. Each member has a role on a four-step ladder:

Roles are totally ordered. Every action a caller attempts is checked through authorize(pool, group_id, entity_id, GroupAction) against a GroupAction enum (25 variants — ViewGroup, PostMessages, StartWorkflow, ApproveAssignedGate, …). The check is "role ≥ required", so an admin satisfies a member-required action.

Agents are members

A working-group agent member has a row in working_group_members like any human, but member_type = "agent" and an agent_definition JSON column that overrides the registry spec for this WG instance. The agent participates in the same auth funnel — an agent without Member role can read but not post, just like a human Viewer.

Agents don't carry their own bearer tokens. Their tools execute under the caller's JWT, or a delegated JWT minted with acting_as = group_id (the agent acts on behalf of the workspace, not on behalf of any one user). This is how agents inherit human permissions without sprouting a parallel identity surface.

Threads — conversation containers

A thread lives in working_group_threads and is keyed by (group_id, thread_type, parent_thread_id?, private_to_entity_id?, reference_id?). Nine thread types exist, six of them auto-created by other system flows:

The privacy model

Every event published into a thread flows through a ChatEnvelope with an optional recipient_entity_id. The SSE forwarder filters before emitting:

recipient_entity_id == None              → fan to every subscriber in the group
recipient_entity_id == subscriber.entity → emit to this subscriber
recipient_entity_id == other entity      → drop, not for this subscriber

That's how a Weaver's reasoning tokens, final message, and any tool calls land in the user's SSE channel without leaking to other group members. The wire format that reaches the client is the inner ChatEvent only — the envelope is server-side bookkeeping.

Reference IDs anchor threads to external state

reference_id is a free-form string giving the thread an external anchor. Three canonical formats:

weaver_subthread_reference(parent, entity)        → "weaver:{parent}:{entity}"
pipeline_main_reference(workflow_id)              → "workflow:{workflow}:main"
pipeline_step_reference(workflow_id, step_key)    → "workflow:{workflow}:step:{step_key}"

The notebook substrate adds entity:{kind}:{id} for per-entity subthreads. New thread kinds should follow the same kind:scope:id pattern so reverse-lookup-by-reference stays parseable.

Agents

Three different layers make up an agent. Keep them separate:

The model is spec → resolve → runtime. The registry holds specs; the resolver applies user scope + context KGs + workspace overrides; the executor consumes the runtime agent.

The registry — four scope tiers

AgentSpec carries (economy_id, working_group_id, user_id) as optional fields. The resolution rule is User > WG > Economy > System (most-specific wins):

A user can override a System-tier weaver spec with their own personal version without the registry sprouting a parallel namespace.

Tools and skills

An agent has tools (callable capabilities — HTTP, MCP, internal helpers) and skills (named, reusable behaviors registered in the service's skill catalog). Both are gated by the agent spec's tools.allow / skills.allow lists. Tool execution runs under the caller's JWT or a delegated acting_as = group_id JWT — the agent inherits the user's authorisation surface without holding its own.

Memory — three tiers

Agent memory is partitioned across three scopes:

Two KG concepts on every invocation

An invocation has two orthogonal KG sources:

• Context KGs (plural, unioned, read-only) — caller supplies a subset (workspace thread KG, pipeline target KG, Studio picker), unioned with the agent spec's pinned_kg_ids, permission-filtered against the user's group memberships.
• Scope KG (singular, optional) — what the agent is "operating on" for memory scoping and mutations. Pipeline runs set this; chat handlers leave it None so memory stays agent-global.

The split lets one agent carry pinned KGs into any context without fragmenting its memory per-conversation.

The Weaver — the personal agent

The Weaver is each user's persistent personal agent. One Weaver per user, lazy-minted on first invocation via AgentRegistry::get_or_mint_personal_agent. Per-user state lives at the agent-global memory tier keyed on (agent_id = "weaver", user_id), so the Weaver remembers you across every conversation.

The Weaver works inside Weaver subthreads — private 1:1 chats between you and your Weaver, rooted at any source thread. The subthread is created idempotently per (parent_thread_id, entity_id) and behaves like a side conversation: the Weaver reads the parent thread context via read_thread_messages, helps you think it through, but cannot post back to the parent thread.

Agent Studio — the meta-agent

Studio is a conversational agent that proposes changes to other agents' specs. The user chats with Studio inside a regular working group thread; Studio emits proposals that the user accepts (applyProposal_Agents) or rejects. This is how non-engineers build and refine agents inside their own workspace.

The deal_structurer agent — bridging agentic to financial

The platform ships a seeded deal_structurer agent persona whose specific role is to compose DealPlans from confirmed Intents. It reads the conversation history (via the analyzer's observations), the relevant KG (asset frames, party history, prior deals), and the economy manifest (available Concepts, descriptor defaults). It produces a candidate DealPlan with the action DAG, cashflow wiring, and a draft AgreementMetadata block (recitals, governing law, dispute resolution, boilerplate pack).

The Structurer's output is itself a Pipeline Draft that materialises as an Intent — the same safety boundary applies. The human proposer reviews + confirms; the DMS then drives propose → sign → activate. The seeded persona is a starting point; domain authors fork it (see ) to specialise the Structurer for their deal types.

This is the agentic-to-financial bridge in concrete form: an agent composes the structured artifact, a human ratifies it via Intent + sign, the DMS compiles it down to a workflow that produces on-chain effects. See the DMS guide for the lifecycle the Structurer feeds into.

Knowledge graphs

A knowledge graph is a durable substrate of typed frames + typed slot edges. Every artifact — documents, chunks, conversation analyses, deal projections, reasoning rounds, observations — lives in the same shared substrate:

• Knowledge-graph metadata — name, lexicon snapshot, vocabulary.
• Frames — every typed node: its kind (Event, State, Relation, SpeechAct, Mental, Action, or a bare entity), its concept type (document, chunk, utterance, or any domain concept the lexicon defines), properties, raw text, embedding, label.
• Typed edges — every relationship: which slot it occupies on the source frame, the target frame, and a weight.

There's no separate document store, no chunk store, no epistemics store — everything is a frame and every relationship is a typed edge.

The lexicon

Every KG has a lexicon — the controlled vocabulary of predicates and slot types it uses. During the first pass of a pipeline run over a new domain, a vocab discoverer agent proposes predicate types from the source material; the lexicon evolves as the KG learns. Approved entries become part of the KG's permanent vocabulary and constrain future extractions.

Documents and chunks as frames

Documents and chunks are just frames with a particular concept type:

• A document frame (concept_type = "document") carries the title in its label, the body in its raw text, a title+summary embedding, and properties for content hash, tags, summary, url, author, …
• A chunk frame (concept_type = "chunk") carries the first line or heading anchor as its label, the chunk text in raw text, the chunk's own embedding, and properties for position, parent section, source document id.

Typed edges between them describe structure and provenance:

• (chunk) is_part_of (document) — chunk belongs to document.
• (chunk) precedes (next_chunk) — chunk-to-chunk ordering.
• (frame) source_chunk (chunk) — extraction provenance.
• (document) linked_to_kg_node (frame) — explicit doc-to-frame link.

A single retrieval primitive (frames joined by typed edges) answers "documents in this KG", "chunks of this document", "frames extracted from this chunk", AND "documents linking to this entity" — all four are edge walks. One substrate, three retrieval paths (vector / keyword / structural) wired through the same query layer.

Why this matters

Composition is first-class. A file-extracted KG can be the starting state for a reasoning pipeline because reasoning agents read the chunks the extraction emitted. RAG retrieves the frames a previous reasoning run wrote. The agreement projector reads frames written by both. No per-source dispatch, no schema-crossing.

Pipelines — how knowledge moves

A pipeline is a chain of typed steps that read and write a KG. Every document ingest, every reasoning round, every Box 1 + Box 2 of /api/conversation/analyze, every agreement projection is the same engine.

Step types

PipelineStep::Transform              ← runs a transform (Extraction or Reasoning) for N rounds
PipelineStep::Checkpoint             ← user-pause point; waits for explicit `resume`
PipelineStep::Deterministic          ← pure mutator (no LLM call)
PipelineStep::LazyTransform          ← transform whose config depends on prior step's output
PipelineStep::Loop                   ← repeated rounds until a convergence predicate fires
PipelineStep::ConversationalReview   ← human-in-the-loop review gate
PipelineStep::DocumentExtraction     ← per-chunk Phase A extraction loop

Pipelines are resumable across process restarts. Run status is persisted in a durable run record — the single source of truth for any pipeline invocation; SSE events and in-memory transform state mirror it.

The canonical ingestion pipeline

POST /pipelines/ingest-document runs:

upload → persist document frame → chunk → embed → discover vocab →
   extract → structural complete → calibrate

End result: a KG populated with the document + chunks + extracted frames, all queryable via the unified retriever. Re-uploading the same content (same SHA-256 of body) deduplicates — the partial unique index on properties->>'content_hash' makes ingestion idempotent on body identity.

Drafts → Intents → Execution

The agent-proposed action lifecycle is three-stage:

1. Draft — an agent (or a user) composes a PipelineDraft describing what should happen (e.g., "ingest this PDF and extract payment terms into KG deals-2027-Q1").
2. Intent — the draft is submitted, materialising an Intent. The user is notified via the IntentProposed SSE event and asked to confirm.
3. Execution — on confirmation, a pipeline run is created and executed. SSE events report IntentExecuting, then either IntentCompleted or IntentFailed.

This is the safety pattern: an agent never directly mutates KG state; it proposes an intent that the user accepts.

Conversation analyzer

POST /api/conversation/analyze takes a free-form transcript and runs a pipeline that:

1. Extracts entities (people, organisations, concepts) into the notebook KG.
2. Builds hypotheses and observations attached to those entities.
3. Optionally generates RFIs (Request For Information) when the analyser needs more data to confirm a hypothesis.

The output materialises in the personal workspace's notebook KG; the user can promote provisional observations to Accepted or archive them.

Workflows — coordinated human + agent processes

A workflow is a typed DAG of steps anchored to a working group, stored durably, and advanced by an executor that emits SSE events. Workflows are how multi-party financial processes (deal approval, KYC refresh, treasury payment, securitisation close) get done.

Templates

Templates live as YAML under configs/workflows/templates/, loaded into workflow_templates on service boot. A template carries name, display_name, description, version, category, typed variables (inputs bound at instantiation), and a list of steps.

Five templates ship in-tree:

• abs_issuance — asset-backed securities issuance
• compliance_kyc_refresh — periodic KYC re-verification
• multisig_transaction_execution — multi-sig transaction with signature collection
• structured_finance_waterfall_close — period-end waterfall distribution
• treasury_payment_approval — multi-party treasury payment review

Users save their own templates via saveTemplate — they join the same registry.

Step types

Each step's step_type tells the executor how to advance it. The catalogue:

{{ variable_name }} substitutions in step config are expanded at instantiateTemplate time. Substitution is textual; downstream parsing happens against the post-substitution value.

Execution

instantiateTemplate(template_id, variables)
    ↓
GroupWorkflow row in working_group_workflows (status: 'pending')
    ↓
startWorkflow
    ↓
WorkflowExecutor advances ready steps (those whose deps satisfy)
    ├─ SSE: WorkflowEvent::StepReady
    ├─ assignee acts: completeHumanTask | approveGate | submitSignature
    ├─ SSE: WorkflowEvent::StepCompleted
    └─ next ready step(s) evaluated
    ↓
Final step completes
    ↓
GroupWorkflow.status = 'complete'
SSE: WorkflowEvent::WorkflowComplete

The executor uses row-level locking for resume-across-restarts safety. The DB row is the single source of truth; in-memory state and SSE events mirror it. If the process crashes mid-step, the next executor picks up the row and continues.

Workflow → financial primitive

The connection to the financial layer is direct: an automated step can submit an message (instant send, swap creation, obligation creation) and a signature_gate step can collect the JWTs needed for a manual-signature payment. A typical treasury_payment_approval template ends with a signature_gate that gathers 3-of-5 signatures, then an automated step that submits the instant mutation. The deal lifecycle is just a workflow whose terminal step is a federated GraphQL mutation.

RAG — querying the substrate

The unified retriever reads the same frame + typed-edge substrate the pipelines write. Three retrieval paths:

ragQuery_Knowledge composes the three: retrieves candidates, reranks with a configurable rerank model, packs the top-k into a context window, and asks an LLM to synthesise an answer with inline citations to the source chunks.

Federation — cross-workspace RAG

A federation policy grants outbound document-read access from group A to group B. Stored in knowledge_federation_policies with (source_group_id, target_group_id, policy_type, allowed_document_types, allowed_tags, status). Three policy types:

Federation is directional (A → B does not imply B → A) and owner/admin-gated on the source group. The query side (federatedRagQuery_WorkingGroups) consults the policy table at query time to expand the candidate document set; without a permissive policy, group A cannot see group B's documents.

Federation is purely RAG-layer — it doesn't share threads, workflows, members, or KG mutations. It's a side-channel the retriever joins against; the rest of the visibility model is unchanged.

The personal notebook

Every user has a single personal workspace — a category = "notebook", settings.kind = "personal" working group with the user as sole owner. Lazy-minted on first access by the notebook substrate.

Why this is built on working-group machinery: membership auth, group-scoped queries, RAG, threads, agent memory all already exist for working groups and apply uniformly. The alternative (a parallel notebook schema with its own permission axis) would duplicate every primitive.

Personal workspaces have special rules: filtered out of listGroups, addMember rejects, archiveGroup rejects through the normal flow. Their threads can include notebook_entity subthreads — one per (observer × notebook entity) — used by the conversation analyzer to compartmentalise per-entity discussion.

The notebook holds entities, observations, hypotheses, RFIs, and memos — all as frames in the notebook KG. The analyzer extracts these from conversation; the user promotes or archives. CRM contacts are a structured subset (people + organisations) with their own resolver; notebook entities are the flexible-shape catch-all for anything not CRM-modeled.

How it all composes — an end-to-end example

Here's a realistic flow that touches every concept above:

1. Alice uploads a 30-page bond prospectus to Working Group 'Deals 2027 Q1'.
   POST /working-groups/{id}/documents
       → ingestion pipeline runs
       → document frame written, 47 chunks, embeddings generated
       → vocab discoverer proposes predicates ("paysCoupon", "matures",
         "issuedBy", "ratedAt", …)
       → extractor agent reads each chunk, produces ~120 frames
       → structural-complete pass infers missing slot edges
       → SSE: DocumentProcessing(done) lands in the group's chat stream

2. Alice's Weaver subthread (private to her) opens.
   GET /working-groups/{id}/threads/{tid}/weaver
       → idempotent Weaver subthread for thread {tid}, entity Alice
   "Summarise the coupon schedule and identify any conditional clauses."
       → Weaver runs RAG against the bond's KG, returns a 4-bullet summary
         with inline chunk citations
       → SSE: AgentToken stream, AgentDone — both wear recipient_entity_id=Alice
         so only Alice's stream receives them

3. Alice posts in the team chat: "Should we propose this to investors?"
   The workspace's proactive chat agent ('Deals 2027 Q1' agent member)
   reads the conversation, queries the bond KG, and emits an
   IntentProposed SSE event:
       Intent: 'Securitise bond and offer Tranche A to investors X, Y, Z'
       Draft includes: createObligation params, swap structures

4. Alice clicks 'Confirm' in the UI.
   POST /intents/{id}/confirm
       → IntentExecuting fires
       → workflow instantiated from 'abs_issuance' template
       → workflow_main thread auto-created in the group
       → pipeline_task threads spawn for each ready step

5. Workflow step 1: human_task (assigned to Alice as originator).
   'Confirm tranche structure'. SSE: StepReady.
   Alice submits.

6. Workflow step 2: agent_task (Deal Structurer agent).
   'Compose contract obligations'. Agent reads bond KG + workflow
   context, produces a structured composed contract proposal.
   Writes 'proposed_contract' frame to the deal KG.

7. Workflow step 3: signature_gate (3-of-5 approvers).
   SSE: StepReady. Approvers sign one by one.
   SSE: SignatureCollected each time.
   On third signature: SSE: StepCompleted.

8. Workflow step 4: automated.
   Submits createSwap mutation through GraphQL gateway
   for an atomic exchange against the investor's cash.
   The MQ pipeline runs the swap on-chain.
   SSE: DealCommitted lands in the chat stream.
   The deal is on-chain. The Cont_X composed contract appears
   in everyone's wallet.

Eight steps, three layers (agents, workflow, financial settlement) — but one unified surface. The agent doesn't know it's part of a workflow; the workflow doesn't know it's running for a deal; the deal doesn't know the proposal came from an LLM. They compose because the substrate (workspace + KG + SSE event stream) is the same.

SSE event taxonomy

Every event fanned through a working-group's chat stream is a ChatEvent variant. Nineteen variants in five families — what clients subscribe to via GET /working-groups/{id}/chat/stream:

The serde(tag = "type") on the Rust enum is the authoritative wire shape. Per-variant payload schemas live in .

Workflow events have their own broadcaster (workflow_broadcasters) and SSE route (GET /working-groups/{id}/workflows/{wid}/stream) — separate from chat so workflow consumers don't subscribe to every chat message.

Pipeline events ride GET /pipelines/{run_id}/events — used by clients tailing a long-running ingest or reasoning run.

Cross-server event relay

When the service runs behind a load balancer, the platform relays chat and workflow events across instances so subscribers receive updates regardless of which instance accepted the write.

SSE remains best-effort by design: reconnect on disconnect and re-query the relevant thread, workflow, or pipeline run to catch up on any state you missed while offline.

Putting it together — what to build first

If you're an LLM-driven application starting from zero, the natural ordering:

1. Get a JWT (see /docs/guides/building-with-yf).
2. Create or join a working group: POST /working-groups or accept an invite.
3. Open the chat stream: GET /working-groups/{id}/chat/stream — that's where everything fans out.
4. Upload your first document: POST /working-groups/{id}/documents — the ingestion pipeline runs automatically. Wait for the DocumentProcessing(done) SSE event.
5. Query via RAG: POST /knowledge/rag-query — your first knowledge-grounded answer.
6. Open a Weaver subthread on a thread: POST /working-groups/{id}/threads/{tid}/weaver — your private reasoning companion.
7. Instantiate a workflow from a template: POST /workflows/instantiate — your first multi-step human + agent process.
8. When a workflow needs settlement, its automated step calls the federated GraphQL gateway and you're back in the financial layer.

Every primitive above has a REST endpoint; every state change emits an SSE event; every artifact lives in the KG substrate. Build on top.

Reference

See also

• /docs/swaps — atomic swap primitive (the financial counterpart this guide composes with).
• /docs/guides/collateralisation — repo / collateral / rehypothecation.
• /docs/guides/cross-service-walkthrough — an end-to-end flow including a workflow → settlement bridge.
• /docs/guides/mcp — MCP integration for connecting Claude Code / Cursor as agent runtimes.