# Get Agent
Source: https://docs.one.fim.ai/api-reference/agents/get-agent
/openapi.json get /api/agents/{agent_id}
# List Agents
Source: https://docs.one.fim.ai/api-reference/agents/list-agents
/openapi.json get /api/agents
# Create Conversation
Source: https://docs.one.fim.ai/api-reference/conversations/create-conversation
/openapi.json post /api/conversations
# Delete Conversation
Source: https://docs.one.fim.ai/api-reference/conversations/delete-conversation
/openapi.json delete /api/conversations/{conversation_id}
# Get Conversation
Source: https://docs.one.fim.ai/api-reference/conversations/get-conversation
/openapi.json get /api/conversations/{conversation_id}
# List Conversations
Source: https://docs.one.fim.ai/api-reference/conversations/list-conversations
/openapi.json get /api/conversations
# Update Conversation
Source: https://docs.one.fim.ai/api-reference/conversations/update-conversation
/openapi.json patch /api/conversations/{conversation_id}
# Dag Endpoint
Source: https://docs.one.fim.ai/api-reference/dag-endpoint
/openapi.json post /api/dag
Run a DAG planner pipeline with SSE progress updates.
Parameters
----------
request : Request
The incoming HTTP request; used to detect client disconnects.
body : ChatStreamRequest
JSON request body containing the query and optional parameters.
# Inject Message
Source: https://docs.one.fim.ai/api-reference/inject-message
/openapi.json post /api/chat/inject
Inject a user message into a running agent execution.
The message is queued for the agent to absorb at its next natural
breakpoint (between iterations). It is also persisted immediately
to the conversation history.
Returns 409 if no active execution exists for the conversation.
# List Kbs
Source: https://docs.one.fim.ai/api-reference/knowledge-bases/list-kbs
/openapi.json get /api/knowledge-bases
# Retrieve From Kb
Source: https://docs.one.fim.ai/api-reference/knowledge-bases/retrieve-from-kb
/openapi.json post /api/knowledge-bases/{kb_id}/retrieve
# React Endpoint
Source: https://docs.one.fim.ai/api-reference/react-endpoint
/openapi.json post /api/react
Run a ReAct agent query with SSE progress updates.
Parameters
----------
request : Request
The incoming HTTP request; used to detect client disconnects.
body : ChatStreamRequest
JSON request body containing the query and optional parameters.
# Create My Api Key
Source: https://docs.one.fim.ai/api-reference/user-settings/create-my-api-key
/openapi.json post /api/me/api-keys
Create a new personal API key. The full key is returned only once.
# Delete My Api Key
Source: https://docs.one.fim.ai/api-reference/user-settings/delete-my-api-key
/openapi.json delete /api/me/api-keys/{key_id}
Permanently delete one of the user's own API keys.
# List My Api Keys
Source: https://docs.one.fim.ai/api-reference/user-settings/list-my-api-keys
/openapi.json get /api/me/api-keys
List the authenticated user's own API keys (paginated).
# Toggle My Api Key Active
Source: https://docs.one.fim.ai/api-reference/user-settings/toggle-my-api-key-active
/openapi.json patch /api/me/api-keys/{key_id}/active
Enable or disable one of the user's own API keys.
# Authentication
Source: https://docs.one.fim.ai/api/authentication
How to authenticate with the FIM One API
## Overview
FIM One API supports two authentication methods:
1. **API Keys** — Simple, long-lived tokens for service-to-service integration
2. **JWT Tokens** — Short-lived tokens from user login (for SSE endpoints)
Most integrations should use **API Keys**.
## API Keys
API keys are long-lived credentials tied to your user account. They're ideal for:
* Server-to-server integrations
* Scheduled scripts and automation
* External applications accessing FIM One
### Creating an API Key
1. Log in to your FIM One portal
2. Go to **Settings → API Keys**
3. Click **Create API Key**
4. Enter a name (e.g., "Production Integration")
5. (Optional) Set scopes to limit access
6. (Optional) Set an expiration date
7. Click **Create**
8. Copy the key immediately — it's only shown once
The full key looks like: `fim_your44characterkeystringhere` (starts with `fim_` prefix)
### Using API Keys
Include the key in the `Authorization` header as a Bearer token:
```bash theme={null}
curl -H "Authorization: Bearer fim_your_api_key_here" \
https://your-domain.com/api/conversations
```
Or in Python:
```python theme={null}
import requests
headers = {
"Authorization": "Bearer fim_your_api_key_here"
}
response = requests.get(
"https://your-domain.com/api/conversations",
headers=headers
)
print(response.json())
```
### API Key Features
**Visibility**: Each key shows:
* Key prefix (first 8 chars for identification)
* Creation date
* Last used timestamp
* Total request count
* Active status
* Expiration date (if set)
**Management**: You can:
* Enable/disable keys without deleting them
* Set automatic expiration dates
* Delete keys permanently
* Track usage patterns
### Scopes
Scopes limit what an API key can access. If no scopes are set, the key has full access.
Available scopes:
| Scope | Allows |
| - | - |
| `chat` | POST /api/react, POST /api/dag, POST /api/chat/inject |
| `agents` | GET /api/agents, GET /api/agents/`{id}` |
| `kb` | GET /api/knowledge-bases, POST /api/knowledge-bases/`{id}`/retrieve |
| `connectors` | Connector CRUD (connector\_specific endpoints) |
| `admin` | Administrative endpoints |
Create a key with scopes:
```bash theme={null}
curl -X POST https://your-domain.com/api/me/api-keys \
-H "Authorization: Bearer your_current_api_key_or_jwt" \
-H "Content-Type: application/json" \
-d '{
"name": "Chat-only Integration",
"scopes": "chat"
}'
```
## JWT Tokens
JWT tokens are short-lived and issued at login. They're used for:
* **SSE Streaming Endpoints**: Pass tokens in the request body for `/api/react` and `/api/dag`
* **Portal Session**: Frontend authentication
### Getting a JWT Token
JWT tokens are automatically issued when you log in through the web portal or call the authentication endpoint:
```bash theme={null}
curl -X POST https://your-domain.com/api/auth/login \
-H "Content-Type: application/json" \
-d '{
"email": "user@example.com",
"password": "your_password"
}'
```
Response:
```json theme={null}
{
"access_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
"token_type": "bearer",
"expires_in": 7200,
"user": { ... }
}
```
### Using JWT for Streaming
For SSE endpoints, include the token in the request body:
```bash theme={null}
curl -X POST https://your-domain.com/api/react \
-H "Content-Type: application/json" \
-d '{
"q": "What are the top 3 Python frameworks?",
"token": "your_jwt_token_here"
}'
```
Or from JavaScript using `fetch` with a readable stream (the endpoint is POST-only, so native `EventSource` which only supports GET cannot be used):
```javascript theme={null}
const response = await fetch("https://your-domain.com/api/react", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
q: "Hello",
token: "fim_your_api_key_here"
})
});
const reader = response.body.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value);
// Parse SSE events from chunk
console.log(chunk);
}
```
### Token Refresh
Access tokens expire after 2 hours. Use the refresh token to get a new access token without re-logging in:
```bash theme={null}
curl -X POST https://your-domain.com/api/auth/refresh \
-H "Content-Type: application/json" \
-d '{
"refresh_token": "your_refresh_token_here"
}'
```
## Security Best Practices
### API Keys
**Never commit API keys to version control.** They provide full access to your data.
1. **Store in environment variables**:
```bash theme={null}
export FIM_API_KEY="fim_your_api_key_here"
```
Then reference in code:
```python theme={null}
api_key = os.environ.get("FIM_API_KEY")
```
2. **Use key rotation**:
* Create a new key
* Update your application
* Delete the old key
* Repeat quarterly
3. **Set expiration dates**:
* Use short-lived keys for temporary integrations
* Require re-authentication periodically
4. **Use scoped keys**:
* Don't use "full access" keys unless necessary
* Create separate keys for different services
* Limits damage if a key is compromised
5. **Monitor usage**:
* Check "last used" timestamps
* Review request counts
* Delete unused keys
### JWT Tokens
1. **Keep short-lived**: Access tokens expire after 2 hours
2. **Secure storage**:
* In browsers: use HttpOnly cookies (safer than localStorage)
* In servers: use secure session storage
3. **Never expose in logs**: Log only the token prefix, not the full token
4. **Use HTTPS only**: Never send tokens over unencrypted connections
## Rate Limits
Per-key rate limiting is planned for a future release. Currently there are no enforced per-key request limits.
## Troubleshooting
### Invalid Token Error
```json theme={null}
{
"error": "invalid_token",
"message": "The provided token is invalid or expired"
}
```
**Causes**:
* API key prefix is wrong (must start with `fim_`)
* JWT token has expired (refresh it)
* Token is malformed or corrupted
* API key has been deleted
**Solution**: Verify your token format and re-generate if necessary
### Unauthorized Error
```json theme={null}
{
"error": "unauthorized",
"message": "Authentication required"
}
```
**Causes**:
* No Authorization header provided
* Token is missing from request body (for SSE endpoints)
* API key is disabled
**Solution**: Include a valid token in every request
### Forbidden Error
```json theme={null}
{
"error": "forbidden",
"message": "Your API key does not have permission to access this resource"
}
```
**Causes**:
* API key has scoped access and lacks the required scope
* User account has restricted permissions
**Solution**: Use a key with the appropriate scope or increase key privileges
### Expired Token
```json theme={null}
{
"error": "token_expired",
"message": "Your token has expired. Please refresh your authentication."
}
```
**Solution**: Use the refresh token to get a new access token:
```bash theme={null}
curl -X POST https://your-domain.com/api/auth/refresh \
-d '{"refresh_token": "your_refresh_token"}'
```
## Environment Variable Setup
### Python
```python theme={null}
import os
from dotenv import load_dotenv
import requests
# Load from .env file
load_dotenv()
api_key = os.getenv("FIM_API_KEY")
base_url = os.getenv("FIM_API_BASE_URL", "https://your-domain.com/api")
headers = {"Authorization": f"Bearer {api_key}"}
# Now use in requests
response = requests.get(f"{base_url}/agents", headers=headers)
```
### Node.js
```javascript theme={null}
const api_key = process.env.FIM_API_KEY;
const base_url = process.env.FIM_API_BASE_URL || "https://your-domain.com/api";
const headers = {
"Authorization": `Bearer ${api_key}`,
"Content-Type": "application/json"
};
// Use in fetch
const response = await fetch(`${base_url}/agents`, {
headers: headers
});
```
### Bash
```bash theme={null}
#!/bin/bash
FIM_API_KEY="${FIM_API_KEY:-$(cat ~/.fim_api_key)}"
FIM_API_BASE_URL="${FIM_API_BASE_URL:-https://your-domain.com/api}"
curl -H "Authorization: Bearer $FIM_API_KEY" \
"$FIM_API_BASE_URL/agents"
```
## Support
For authentication issues:
* Check the [API Overview](/api/overview) for response format details
* Review token expiration with `GET /api/auth/verify`
* Contact support if you believe a key has been compromised
# API Overview
Source: https://docs.one.fim.ai/api/overview
Getting started with the FIM One REST API
API reference documentation is currently available in **English only**. If you are viewing this page in another language, please switch to the English version — the full API endpoint list will appear in the left sidebar.
FIM One provides a REST API for programmatic access to AI agents, conversations, knowledge bases, and more.
## Base URL
```
https://your-domain.com/api
```
## Authentication
All API requests require authentication via an API key or JWT token.
### Using API Key
Include your API key in the `Authorization` header:
```bash theme={null}
curl -H "Authorization: Bearer fim_your_api_key_here" \
https://your-domain.com/api/agents
```
### Using JWT Token
For SSE streaming endpoints (`/api/react` and `/api/dag`), you can pass the token in the request body instead:
```json theme={null}
{
"q": "What's the weather today?",
"token": "your_jwt_token_here"
}
```
See [Authentication](/api/authentication) for detailed setup instructions.
## Response Format
### Success Response
All successful API responses follow this structure:
```json theme={null}
{
"success": true,
"data": {
"id": "conv_123abc",
"title": "My Conversation",
"created_at": "2024-01-15T10:30:00Z"
}
}
```
### Error Response
Error responses include an error code and human-readable message:
```json theme={null}
{
"success": false,
"error": "conversation_not_found",
"message": "The conversation does not exist or has been deleted"
}
```
### Paginated Responses
List endpoints return paginated results:
```json theme={null}
{
"items": [
{ "id": "item_1", "name": "First Item" },
{ "id": "item_2", "name": "Second Item" }
],
"total": 42,
"page": 1,
"size": 20,
"pages": 3
}
```
## Streaming (Server-Sent Events)
Chat endpoints (`POST /api/react` and `POST /api/dag`) return Server-Sent Events for real-time streaming. Each event has a `type` field indicating its content:
| Event Type | Description |
| - | - |
| `step` | ReAct reasoning step (tool calls, thinking). For models with extended thinking (Claude, o-series, DeepSeek-R1), includes real-time `thinking_delta` content during tool-decision phases |
| `step_progress` | DAG step progress (started/iteration/completed). May include `thinking_delta` content when the underlying agent streams reasoning tokens |
| `phase` | Pipeline phase transition (planning/executing/analyzing) |
| `compact` | Context compaction occurred |
| `answer` | Streamed answer text (start/delta/done) |
| `done` | Final result payload |
| `suggestions` | Suggested follow-up questions |
| `title` | Auto-generated conversation title |
| `end` | Stream terminator (always last) |
### Example SSE Stream
```
event: step
data: {"type": "tool_call", "tool": "search", "input": "Paris weather"}
event: answer
data: {"type": "delta", "text": "Based on current data, "}
event: answer
data: {"type": "delta", "text": "Paris has pleasant weather."}
event: done
data: {"answer": "Based on current data, Paris has pleasant weather.", "tokens": 128}
event: suggestions
data: ["What about tomorrow's forecast?", "Compare with other European cities"]
event: end
data: {}
```
### Cursor-based resume
Every SSE event carries a monotonic `cursor` integer scoped to the conversation. Clients should track the highest cursor seen. When the connection drops mid-turn (network error, proxy timeout, server restart), the client can re-subscribe without re-running the agent:
```http theme={null}
POST /api/chat/resume
Content-Type: application/json
Authorization: Bearer
{
"conversation_id": "",
"cursor": 42
}
```
The response is an SSE stream that replays cached events with `cursor > 42` and ends with a terminal `event: resume_done` frame. Pass `cursor: -1` to request a full replay from the beginning.
Error responses:
* `404 conversation_not_found` — conversation id is invalid or not owned by the authenticated user
* `404 no_recent_assistant_message` — conversation has no persisted assistant turn to resume from (e.g., a brand-new conversation with no user messages yet)
* `400 invalid_cursor` — cursor is not a non-negative integer or `-1`
The reference frontend implementation (`useSseResume` hook in the playground) auto-retries network disconnects with exponential backoff (300ms → 1s → 3s, max 3 attempts) and distinguishes user-initiated abort from network error to avoid unintended reconnects. See the `playground-page.tsx` integration for the full pattern.
Resume is stateless — it replays from the assistant message's stored `sse_events` array. If the agent's turn was interrupted before any assistant message was persisted (e.g., a crash before `done`), the dangling `tool_use` block is auto-repaired by `DbMemory._repair_dangling_tool_calls` on the next turn. Synthetic `tool_result` rows are persisted with `metadata_.synthetic=true` and `metadata_.reason="interrupted"` for audit.
## Rate Limiting
Per-key rate limiting is planned for a future release. Currently there are no enforced per-key request limits, but please use the API responsibly.
## Pagination
List endpoints support pagination via query parameters:
```bash theme={null}
curl "https://your-domain.com/api/conversations?page=2&size=50" \
-H "Authorization: Bearer fim_your_api_key_here"
```
Parameters:
* `page` (default: 1) — page number, starting from 1
* `size` (default: 20) — items per page, max 100
## Timestamps
All timestamps are in ISO 8601 format with UTC timezone:
```json theme={null}
{
"created_at": "2024-01-15T10:30:45.123456Z",
"updated_at": "2024-01-15T14:22:10.654321Z"
}
```
## Error Codes
Common error codes and their meanings:
| Error Code | HTTP Status | Meaning |
| - | - | - |
| `unauthorized` | 401 | Invalid or missing authentication |
| `forbidden` | 403 | User lacks permission for this resource |
| `not_found` | 404 | Resource does not exist |
| `bad_request` | 400 | Invalid request parameters |
| `rate_limit_exceeded` | 429 | Too many requests (planned) |
| `internal_error` | 500 | Server error |
## API Playground
Each endpoint page includes an interactive API Playground where you can:
* Test requests directly from your browser
* View real responses
* Copy request code in multiple languages
* Authenticate with your API key
The API Playground runs requests from your browser, so ensure your API key has appropriate CORS permissions (usually enabled by default for same-origin requests).
## SDK Availability
While FIM One API is fully REST-based, SDKs are available (or coming soon) in popular languages:
* **Python**: Coming soon
* **JavaScript/TypeScript**: Coming soon
* **Go**: Coming soon
## Versioning
The current API version is `v0.8.0`. Version is not included in the URL path; breaking changes will be signaled through major version bumps and a migration period.
## Support
For API issues or questions:
* **Documentation**: See detailed guides on each endpoint
* **GitHub Issues**: Report bugs at [https://github.com/fim-ai/fim-one/issues](https://github.com/fim-ai/fim-one/issues)
* **Discord Community**: Join our community for discussion
# Agent & Resource Discovery
Source: https://docs.one.fim.ai/architecture/agent-discovery
How FIM One discovers and assembles resources — the dual-mode architecture of agent-constrained and global auto-discovery.
## The two modes
Every chat request in FIM One starts with a question: **is an Agent selected?** The answer determines how resources — Connectors, Knowledge Bases, Skills, and MCP servers — are discovered and assembled into the tool set the LLM can use.
**Agent-Constrained Mode** activates when the user picks a specific Agent. The system loads only the resources that Agent has been explicitly configured with:
* **Connectors**: only the Agent's bound `connector_ids` are loaded as tools.
* **Knowledge Bases**: only the Agent's bound `kb_ids` are injected as retrieval tools.
* **Skills**: globally available — all active Skills visible to the user are injected, because Skills are organizational SOPs, not Agent-specific knowledge. (See [Skills as Global SOPs](#skills-as-global-sops) below.)
* **MCP Servers**: always user-scoped — all active MCP servers visible to the user are loaded in both modes.
* **Instructions**: the Agent's `instructions` field defines the persona and behavioral guidelines injected into the system prompt.
**Global Auto-Discovery Mode** activates when no Agent is selected (e.g., a new chat). The system auto-discovers everything accessible to the user:
* **Connectors**: all Connectors visible to the user (own + org-shared + Market-subscribed) are loaded.
* **Knowledge Bases**: all accessible KBs are available for retrieval via `kb_retrieve`.
* **Skills**: all active Skills visible to the user are injected as SOP stubs.
* **MCP Servers**: same as agent-constrained — all active servers visible to the user.
* **Instructions**: a generic assistant persona is used.
The branching happens inside `_resolve_tools()`, which is called on every chat request:
```mermaid theme={null}
flowchart TD
U["User Request"] --> R["_resolve_agent_config()"]
R -->|Agent selected| AC["Agent-Constrained Load agent's bound Connectors + KB"]
R -->|No agent| AD["Auto-Discovery Load ALL visible Connectors + KB"]
AC --> SK["Inject Global Skills (both modes)"]
AD --> SK
SK --> MCP["Load User MCP Servers (both modes)"]
MCP --> TR["ToolRegistry"]
TR --> EE["Execution Engine (ReAct or DAG)"]
```
The practical effect: users can start chatting immediately without configuring an Agent. The system discovers available resources and exposes them as tools. Selecting an Agent narrows the scope — it does not unlock new capabilities, it focuses existing ones.
### What each mode discovers
The two modes differ in **scope**, not in kind. Both produce a `ToolRegistry` — they just fill it differently.
**Auto-Discovery Mode (no Agent selected):**
| Resource | Discovery | Tool Form |
| - | - | - |
| Connectors (API) | `resolve_visibility()` — all visible to user | `ConnectorMetaTool` (progressive) |
| Connectors (DB) | `resolve_visibility()` — all visible to user | `DatabaseMetaTool` (progressive) |
| Knowledge Bases | All accessible KBs | `kb_retrieve` |
| Skills | `resolve_visibility()` — all active | `read_skill` (progressive stubs) |
| MCP Servers | `resolve_visibility()` — all user-visible | `MCPServerMetaTool` (progressive) |
| Agents | `resolve_visibility()` — all active, non-builder | `call_agent` (delegation catalog) |
| Built-in Tools | `discover_builtin_tools()` — full set | No category filter applied |
**Agent-Constrained Mode (Agent selected):**
| Resource | Discovery | Tool Form |
| - | - | - |
| Connectors | Only `agent.connector_ids` | `ConnectorMetaTool` or legacy per-action |
| Knowledge Bases | Only `agent.kb_ids` | `GroundedRetrieveTool` / `KBRetrieveTool` |
| Skills | Global — **not constrained by Agent** | `read_skill` |
| MCP Servers | User-scoped — **not constrained by Agent** | `MCPServerMetaTool` (progressive) |
| Agent Delegation | Not available — agents are specialized | *(disabled)* |
| Built-in Tools | `agent.tool_categories` filter | Subset by category |
The key asymmetry: Connectors and Knowledge Bases are scoped by the Agent, but Skills and MCP Servers remain global in both modes. `CallAgentTool` (agent delegation) is only available in Auto-Discovery mode — it is **not** registered when a specific Agent is selected. This is a security measure: a marketplace agent could otherwise use `call_agent` to invoke other agents and access their private prompts. Skills are organizational rules (everyone follows the same SOPs), while Connectors and KBs are capability bindings (different agents connect to different systems).
## Everything is a tool
At the LLM level, all resource types converge into a flat list of callable tools. The LLM has no structural awareness of whether it is calling a Connector, an MCP server, or a Knowledge Base. It sees a `ToolRegistry` — a set of functions with names, descriptions, and parameter schemas.
| Resource Type | Becomes at LLM Level | Tool Name Pattern |
| - | - | - |
| Connector (progressive) | Single meta-tool | `connector` |
| Connector (legacy) | N tools per action | `{connector}__{action}` |
| Database Connector (progressive) | Single meta-tool | `database` |
| Database Connector (legacy) | 3 tools per database | `{db}__list_tables`, `{db}__describe_table`, `{db}__query` |
| MCP Server (progressive) | Single meta-tool | `mcp` |
| MCP Server (legacy) | N tools per server | `{server}__{tool}` |
| Knowledge Base | Retrieval tool | `kb_retrieve` or `grounded_retrieve` |
| Skill (progressive) | Read tool + system prompt stubs | `read_skill` |
| Skill (inline) | System prompt text only | *(no tool)* |
| Agent itself | Not visible as a tool | *(instructions + tool assembly)* |
The key insight: **an Agent is not a tool — it is the entity that uses tools.** The Agent contributes its instructions to the system prompt and determines which tools are available. But from the LLM's perspective, there is no "agent" concept — only a system prompt and a set of callable functions.
This uniformity is what makes the system extensible. Adding a new resource type means implementing the `Tool` protocol (`name`, `description`, `parameters_schema`, `run()`). The execution engines, context management, and LLM interaction layer remain unchanged.
## Skills as global SOPs
Skills occupy a layer above Agents. They are organizational policies and procedures that every Agent must follow, regardless of which Agent is selected.
### Why Skills are not bound to Agents
A Skill like "Customer Complaint Handling SOP" applies to every agent that interacts with customers. Binding Skills to Agents creates a bidirectional ownership problem: if a Skill orchestrates Agents, and Agents own Skills, who controls whom?
Skills are global by design — they are company rules, not agent-specific knowledge. The `_resolve_tools()` function loads all active Skills visible to the user regardless of Agent selection, using the same `resolve_visibility()` filter used for other resources.
### Two injection modes
Skills support two injection modes -- **progressive** (default) and **inline** -- controlled by `SKILL_TOOL_MODE` or the Agent's `model_config_json.skill_tool_mode`. In progressive mode, only compact stubs appear in the system prompt; the LLM calls `read_skill(name)` on demand to load the full content. This is part of FIM One's broader [Progressive Disclosure](/architecture/progressive-disclosure) architecture that minimizes context consumption across all resource types.
## Agent as persona, not container
FIM One's architecture reflects a deliberate shift from an Agent-centric model to a Resource-centric model.
**Previous model:** the Agent was a container that gated access to all resources. No Agent selected meant no Connectors, no Skills, no specialized KB. The Agent was the mandatory entry point for any capability.
**Current model:** the Agent is a persona — a set of instructions and behavioral guidelines — combined with an optional resource constraint. Resources exist independently of Agents. Selecting an Agent narrows the scope; not selecting one opens it fully.
```mermaid theme={null}
flowchart LR
subgraph OLD["Previous: Agent-Centric"]
direction TB
A1["Agent"] --> C1["Connectors"]
A1 --> K1["KB"]
A1 --> S1["Skills"]
A1 --> M1["MCP"]
N1["No Agent = No Tools"]
end
subgraph NEW["Current: Resource-Centric"]
direction TB
R2["Resources (Connectors, KB, Skills, MCP)"]
A2["Agent = Persona + scope filter"]
R2 -.->|"constrains"| A2
N2["No Agent = All Resources"]
end
```
This means:
* **Users can start chatting immediately** without configuring an Agent.
* **The system auto-discovers available resources** and exposes them as tools.
* **Agents become lightweight personas** that can be created quickly — just write instructions and optionally bind specific Connectors and KBs.
* **Resource management is decoupled** from Agent management. Publishing a Connector to an organization makes it available everywhere — in auto-discovery mode, in Agent binding dropdowns, and in agent delegation resolution.
## Agent Delegation
FIM One supports delegating tasks to specialist agents via `CallAgentTool` — but only in **Auto mode** (no agent selected). When a user selects a specific Agent, delegation is disabled and the Agent focuses exclusively on its own tools.
### Two modes: Auto vs Agent-selected
| Aspect | Auto Mode (no agent selected) | Agent-Selected Mode |
| - | - | - |
| `call_agent` | Enabled — delegates to any visible agent | **Disabled** — not registered |
| Tool scope | All visible Connectors, KB, Skills, MCP | Only the Agent's bound resources + global Skills/MCP |
| Orchestration | System LLM dynamically picks the best agent per iteration | Agent uses its own tools directly |
| Use case | General queries, cross-domain tasks | Focused specialist tasks |
**Why delegation is disabled in Agent-selected mode:** Security. A marketplace agent could use `call_agent` to invoke other agents and read their private system prompts. By restricting delegation to Auto mode — where the system LLM (not any individual agent's prompt) controls the flow — private agent prompts are never exposed to untrusted agent configurations.
### Auto mode as the orchestration layer
Auto mode is a first-class concept in the UI. The agent selector shows "Auto" as the default option. When Auto is active, the system LLM acts as an orchestrator: it sees the full catalog of visible agents and can delegate tasks to the best-fit specialist on each iteration. This replaces the need for a dedicated "parent agent" — the system itself is the orchestrator.
### Agent catalog
At runtime, all active, non-builder Agents visible to the user are assembled into a catalog. Each Agent's name and description are listed in the `call_agent` tool's parameter schema, allowing the LLM to choose the right specialist semantically — no hardcoded routing.
### Full tool inheritance
When a delegated agent is invoked via `call_agent(agent_id, task)`, it receives a complete `ToolRegistry` built from its own configuration — including its bound Connectors, KB, and built-in tools. Delegated agents are full execution units, not text-only advisors.
### One-level delegation
To prevent infinite recursion, delegated agents do not receive the `call_agent` tool. Delegation is always one level deep: Auto mode calls a specialist, the specialist executes and returns a result. The system synthesizes results from multiple delegated agents.
### Parallel execution
In native function-calling mode, the LLM can invoke multiple `call_agent` calls in a single turn. These execute concurrently via `asyncio.gather`, enabling patterns like "search three sources simultaneously."
```mermaid theme={null}
flowchart TD
U["User Query"] --> AM["Auto Mode (system orchestrator)"]
AM -->|"call_agent('Contract Expert', 'find this week's contracts')"| DA1["Delegate: Contract Expert"]
AM -->|"call_agent('Data Analyst', 'analyze refund trends')"| DA2["Delegate: Data Analyst"]
DA1 --> T1["Contract Connector tools"]
T1 --> R1["Result"]
DA2 --> T2["Analytics KB + Python tools"]
T2 --> R2["Result"]
R1 --> SYN["Auto Mode synthesizes both results"]
R2 --> SYN
SYN --> ANS["Final Answer"]
```
## Visibility model
All resource discovery — in both modes — is governed by a unified visibility model with three tiers:
| Tier | Description | Example |
| - | - | - |
| **Own** | Created by the user. Always visible. | A Connector you built for your team's API |
| **Organization-shared** | Resources with `visibility: "org"` from the user's organization(s). Visible to all approved org members. | A company-wide ERP Connector published by IT |
| **Market-subscribed** | Resources installed from the FIM One Market. Visible to the subscriber. | A community-built Slack Connector you installed |
The `resolve_visibility()` function in `web/visibility.py` builds a SQL filter that includes all three tiers in a single query:
```python theme={null}
conditions = [
model.user_id == user_id, # own resources
and_(model.visibility == "org", # org-shared
model.org_id.in_(user_org_ids),
or_(model.publish_status == None,
model.publish_status == "approved")),
model.id.in_(subscribed_ids), # Market-subscribed
]
```
This same filter is used everywhere:
* Auto-discovering Connectors in no-agent mode
* Building the Agent catalog for `CallAgentTool`
* Loading visible Skills for system prompt injection
* MCP server resolution
* Agent configuration lookup (ensuring a user can only select Agents visible to them)
The uniformity means that **publishing a Connector to an organization automatically makes it available** in auto-discovery mode, in Agent binding dropdowns, and in agent delegation resolution — no special wiring required. The visibility model is the single source of truth for "what can this user access."
## Relationship map
FIM One has two parallel execution paradigms — **Chat (Agent-driven)** and **Workflow (DAG-driven)** — that share the same underlying resources but orchestrate them differently.
```mermaid theme={null}
flowchart TD
subgraph CHAT["Chat Execution (Agent-driven)"]
USER["User"] --> AGENT["Agent (ReAct / DAG)"]
AGENT --> TR["ToolRegistry"]
TR --> CONN["Connectors"]
TR --> KB["Knowledge Bases"]
TR --> MCP_T["MCP Servers"]
TR --> BUILT["Built-in Tools"]
TR --> CALL["call_agent → Delegate Agent"]
TR --> READ["read_skill → SOP text"]
end
subgraph WF_EXEC["Workflow Execution (fixed DAG)"]
TRIGGER["Trigger (Manual / Cron / API)"] --> WF["Workflow Engine"]
WF --> WF_AGENT["AGENT Node"]
WF --> WF_LLM["LLM Node"]
WF --> WF_CONN["CONNECTOR Node"]
WF --> WF_KB["KB_RETRIEVAL Node"]
WF --> WF_MCP["MCP Node"]
WF --> WF_HTTP["HTTP Node"]
WF --> WF_CODE["CODE Node"]
WF --> WF_HUMAN["HUMAN Node"]
WF --> WF_SUB["SUB_WORKFLOW Node"]
end
SKILLS["Skills (Global SOPs)"] -.->|"injected into system prompt"| AGENT
WF_AGENT -->|"invokes"| AGENT
```
Key takeaways from the diagram:
* **Agent and Workflow are parallel paradigms.** Both can use Connectors, Knowledge Bases, and MCP Servers — but through different mechanisms. Agents use them as tools in a `ToolRegistry`; Workflows use them as typed DAG nodes.
* **Workflow can orchestrate Agents** via the `AGENT` node — a Workflow step can invoke a full Agent with its own ReAct/DAG loop. The reverse is not true: Agents cannot directly invoke Workflows (only indirectly via API/webhook triggers).
* **Skills are injected into Agents only.** Skills are system prompt text — they guide Agent behavior. Workflows don't consume Skills because Workflow nodes execute deterministic logic, not LLM-guided reasoning.
* **Shared resources, different access patterns.** A Connector can be called by an Agent (via `ConnectorToolAdapter`), by a Workflow (via `CONNECTOR` node), or by both in the same business process — e.g., a Workflow triggers an Agent that queries the same Connector the Workflow also uses in a later step.
## Workflow Engine — the other execution paradigm
While this document focuses on Agent-driven chat execution, FIM One includes a full **Workflow Engine** — a visual DAG editor with 26 node types for fixed-process automation.
| Aspect | Agent (Chat) | Workflow |
| - | - | - |
| Orchestration | LLM decides next step dynamically | Fixed DAG defined at design time |
| Best for | Exploratory tasks, conversations, flexible reasoning | Approval chains, scheduled ETL, multi-step automations |
| Can call | Connectors, KB, MCP, Built-in Tools, Delegated Agents, Skills | Agents, Connectors, KB, MCP, LLM, HTTP, Code, Human approval, Sub-Workflows |
| Trigger | User message in chat | Manual, cron schedule, or API/webhook |
| Nesting | One-level delegation (Auto mode → delegated Agent) | Arbitrary DAG depth via SUB\_WORKFLOW nodes |
The two paradigms are complementary. Use Agents when the task is open-ended ("analyze this quarter's sales data and recommend actions"). Use Workflows when the process is known ("every Monday, pull new invoices from ERP, run compliance checks, and route exceptions to a human reviewer"). A Workflow can invoke an Agent for any step that needs flexible reasoning within an otherwise fixed pipeline.
For details on Agent execution modes and Workflow node types, see [Execution Modes](/concepts/execution-modes).
# Connector Architecture
Source: https://docs.one.fim.ai/architecture/connector-architecture
How FIM One connects legacy systems through AI — from Copilot to Hub.
## Copilot vs Hub
The architecture supports two integration scales:
```mermaid theme={null}
flowchart TD
subgraph Copilot["Copilot (embedded)"]
subgraph Host["Host System (ERP)"]
C["FIM One Copilot (iframe / widget)"]
end
C --> DB1["DB"]
C --> API1["API"]
end
subgraph Hub["Hub (central portal)"]
H["FIM One Hub (Portal / API)"]
H --> ERP["ERP"]
H --> CRM["CRM"]
H --> OA["OA"]
H --> Lark["Lark"]
H --> DB2["DB"]
H --> CustomAPI["Custom API"]
end
```
**Copilot** embeds into a host system's UI. Users interact with AI without leaving their familiar interface. It can use multiple connectors (host DB + notification service, etc.).
**Hub** is a standalone portal that connects all systems. It's not embedded in any single system -- it's the central intelligence layer where systems meet AI.
Same connector architecture, different delivery. A Copilot uses the same `ConnectorToolAdapter` as a Hub.
## Core Principle
**The client changes zero code.** FIM One bridges into their systems proactively -- reading their databases, calling their APIs, pushing to their message bus. The client provides only credentials and network access.
## Three-Layer Architecture
```mermaid theme={null}
flowchart TB
A["FIM One Platform ReAct / DAG / Portal UI Multi-tenant, scheduling, conversation persistence"]
B["Connector Governance Layer audit log, auth passthrough, operation classification read_only enforcement, confirmation gate, circuit breaker"]
C["MCP Protocol (Transport) stdio / HTTP transport Standard tool description, process isolation"]
D["Legacy System DB / API / MQ Client's existing systems"]
A --> B --> C --> D
```
Each layer has a distinct responsibility:
| Layer | Owns | Changes when... |
| - | - | - |
| **Platform** | Orchestration, multi-tenant, UI | New platform features ship |
| **Connector Governance Layer** | Enterprise governance policies | Security/compliance requirements change |
| **MCP Protocol** | Transport, tool interface standard | Never (open standard) |
| **Legacy System** | Business data and logic | Never (that's the whole point) |
## Why MCP as the Transport Layer
Adapters are implemented as **MCP Servers**. This is a deliberate architectural choice:
* **Reuse**: FIM One already ships an MCP Client (v0.3). Adding a legacy system adapter reuses the same infrastructure as adding any MCP tool.
* **Standard protocol**: MCP is an open standard. No proprietary protocol to invent or maintain.
* **Ecosystem**: Third-party MCP Servers (databases, APIs, SaaS tools) work out of the box.
* **Process isolation**: Each MCP Server runs as a separate process. A misbehaving adapter cannot crash the platform.
### What MCP alone does not provide
The **Connector Governance Layer** adds enterprise governance that raw MCP lacks:
| Concern | MCP | Connector Governance Layer |
| - | - | - |
| Read-only enforcement | No | `read_only` flag on operations; write blocked by default |
| Audit logging | No | Every tool call recorded (timestamp, user, tool, params, result) |
| Auth passthrough | No | Proxy host-system auth; agent acts on behalf of logged-in user |
| Confirmation gate | No | Write operations require human approval (SSE `confirmation_required`) |
| Circuit breaker | No | Connection failure triggers graceful degradation |
| Operation classification | No | Operations tagged as read/write/admin with per-level policies |
| Column-level masking | No | Database columns marked as personal data return a mask instead of their values |
### Database column masking
A database connector is often configured with one privileged account, so
every caller reaches the same tables. Marking a column as **PII** in the
connector's schema manager narrows what the agent can see from it:
* the column stays listed in the schema, tagged, so the model knows it
exists and does not invent a substitute for it;
* any value returned under that column name is replaced with `***` before
the result reaches the model, including through `SELECT *`;
* the call log records which columns were masked, alongside whether the
connector was in read-only mode.
Matching is by column name, which is the only identifier a result set
carries. Two limits follow, and both are deliberate: the database still
*uses* the column, so `WHERE salary > 5000` filters normally, and a query
that renames the column (`SELECT salary AS s`) returns unmasked values.
When the requirement is that the query account cannot read a column at
all, grant that in the database and point the connector at the restricted
account.
The owner's own query playground is not masked. Masking governs what
reaches an agent and the conversation transcript, not what the person who
configured the connection can inspect.
### Why not invent a custom protocol
Protocol is commodity. The technical value is in the adapters themselves (domain knowledge, schema mapping, edge-case handling) and the governance layer (audit, auth, safety). Inventing a transport protocol would add maintenance cost without adding capability. Stripe uses HTTPS; Docker uses cgroups; FIM One uses MCP.
## Deployment Model
Everything runs in a single Docker Compose deployment. The client installs nothing.
```mermaid theme={null}
flowchart TD
subgraph Docker["FIM One Deployment (Docker Compose)"]
subgraph Core["FIM One Core"]
R["ReAct / DAG"]
P["Platform API"]
U["Portal UI"]
G["Connector Governance"]
end
subgraph MCP1["MCP Server: Finance DB"]
M1["SQL queries (read-only)"]
end
subgraph MCP2["MCP Server: OA System"]
M2["REST API calls"]
end
subgraph MCP3["MCP Server: Lark"]
M3["Webhook notifications"]
end
Core --> MCP1
Core --> MCP2
Core --> MCP3
end
MCP1 --> Oracle["Client's Oracle DB"]
MCP2 --> Seeyon["Client's Seeyon OA"]
MCP3 --> Lark["Lark API"]
```
All provided by FIM One. Client provides only:
* Database credentials (read-only account recommended)
* API endpoints and keys (if available)
* Network whitelist access
**Access hierarchy**: FIM One adapts to whatever access the client can provide:
| What client has | How FIM One connects |
| - | - |
| API with documentation | HTTP API adapter (best case) |
| API without documentation | HTTP API adapter + manual schema mapping |
| Database access only | Database adapter (direct SQL, read-only by default) |
| Database + message bus | Database adapter + message push adapter |
## Agent-Connector Decoupling
The agent sees connectors as ordinary tools. It does not know or care whether a tool is built-in, a third-party MCP Server, or a legacy system connector.
```mermaid theme={null}
flowchart TB
Agent["Agent's Tool List"]
subgraph BuiltIn["Built-in Tools"]
T1["web_search"]
T2["calculator"]
end
subgraph Connector["Connector Tools"]
T3["contract_query"]
T4["finance_report"]
T5["lark_push"]
end
Agent --- BuiltIn
Agent --- Connector
```
This means:
* **Adding** a new system = adding a connector config. Agent code does not change.
* **Removing** a connector = removing the config. No code changes.
* The same agent can use built-in tools and connectors in a single task.
## Hot-Plug Evolution
| Version | How to add a new connector | Restart required? |
| - | - | - |
| **v0.6** | Write a Python MCP Server with Connector Governance Layer, add to docker-compose | Redeploy |
| **v0.8** | Write a YAML/JSON config, platform generates MCP Server | Restart |
| **v1.0** | Upload OpenAPI spec, AI generates config automatically | **No restart (hot-plug)** |
Enterprise deployments are "implement once, run for months" -- hot-plug is a v1.0 convenience, not a v0.6 requirement.
## Data Flow Example
User: "Check all overdue contracts from the finance system and push a summary to Lark."
```
1. User sends message via Portal / API
2. FIM One (ReAct mode):
Think: I need to query the finance DB for overdue contracts, then push to Lark.
3. Act: contract_query(status="overdue", days_past_due=">30")
→ Connector Governance: audit log, read_only check (pass)
→ MCP Server: translates to SQL
→ Client DB: SELECT * FROM contracts WHERE status='overdue' AND ...
← Returns 7 overdue contracts
4. Think: Found 7 overdue contracts. I'll summarize and push.
5. Act: lark_push(message="7 overdue contracts found: ...")
→ Connector Governance: audit log, write operation → confirmation gate
→ User approves via Portal
→ MCP Server: POST to Lark webhook
← Push successful
6. Answer: "Found 7 overdue contracts. Summary pushed to Lark group."
```
## Connector Standardization Levels
| Level | Version | Approach | Who builds it |
| - | - | - | - |
| **Level 1** | v0.6 | Python MCP Server with Connector Governance | FIM One developer |
| **Level 2** | v0.8 | YAML/JSON config, platform auto-generates MCP Server | Implementation engineer (no Python needed) |
| **Level 3** | v1.0 | Upload OpenAPI/Swagger spec, AI generates config | AI (with human review) |
## Relationship to Existing MCP Ecosystem
FIM One's MCP Client (shipped in v0.3) already supports third-party MCP Servers. Legacy system adapters are simply **domain-specific MCP Servers** built with the Connector Governance Layer for enterprise governance.
```mermaid theme={null}
flowchart TB
subgraph MCP["MCP Ecosystem (third-party)"]
M1["filesystem server"]
M2["GitHub server"]
M3["Slack server"]
M4["any community MCP server"]
end
subgraph CG["Connector Governance (FIM One)"]
C1["read_only enforcement"]
C2["audit logging"]
C3["auth passthrough"]
C4["confirmation gate"]
end
MCP --> TR["ToolRegistry"]
CG --> TR
TR --> Agent["Agent sees all as tools"]
```
The Connector Governance Layer does not replace MCP -- it extends MCP with the governance layer that enterprise legacy system integration requires.
# Context Management
Source: https://docs.one.fim.ai/architecture/context-management
How FIM One manages conversation context — a five-layer defense-in-depth system that prevents token overflow while preserving conversation quality.
## The problem
LLMs have finite context windows. A 128K-token model sounds generous until you subtract the output budget, the system prompt, tool descriptions, and the accumulated history of a multi-turn conversation. Long conversations, large tool results, and multi-step agent loops all push against this limit — often within a single session.
The naive solution is truncation: drop old messages when the window fills up. This is fast and predictable, but it destroys context indiscriminately. The user's original intent, key decisions from earlier turns, and critical data points all vanish when a blunt character cutoff hits them. The opposite extreme — LLM-powered summarization on every turn — preserves semantic content but is expensive, slow, and introduces its own failure modes (hallucinated summaries, lost numerical precision).
The real challenge is not "fit into the window." It is: **degrade gracefully without losing critical information, without burning tokens on unnecessary compaction, and without adding latency the user can feel.**
FIM One solves this with a five-layer defense-in-depth architecture. Each layer addresses a different scale of the problem, and they compose cleanly — no single layer needs to be perfect because the next one catches what it misses.
## Five layers of defense
Context management is not a single mechanism. It is a stack, where each layer handles a specific concern at a specific granularity:
| Layer | Component | What it does | When it acts |
| - | - | - | - |
| **5** | Budget Configuration | Computes usable input token budget from model specs | At startup / per-request |
| **4** | DbMemory | Loads persisted history, compacts on load | Once per request |
| **3** | ContextGuard | Per-iteration budget enforcement | Every ReAct iteration |
| **2** | CompactUtils | Token estimation, smart truncation, LLM compaction | Called by layers 3 and 4 |
| **1** | Memory Implementations | Abstract interface + concrete strategies | Framework-level |
The layers are numbered bottom-up because higher layers depend on lower ones. Layer 5 sets the budget. Layer 4 does the initial load-time compaction. Layer 3 enforces the budget on every iteration. Layers 2 and 1 provide the primitives that layers 3 and 4 use.
```mermaid theme={null}
flowchart TD
L5["Layer 5: Budget Configuration context_size - max_output"]
L4["Layer 4: DbMemory Load history, compact on load"]
L3["Layer 3: ContextGuard Per-iteration enforcement"]
L2["Layer 2: CompactUtils Estimation, truncation, LLM compact"]
L1["Layer 1: Memory Implementations BaseMemory, WindowMemory, SummaryMemory, DbMemory"]
L5 -->|"budget"| L4
L5 -->|"budget"| L3
L4 -->|"calls"| L2
L3 -->|"calls"| L2
L4 -.->|"implements"| L1
```
### Layer 5 — Budget Configuration
The budget is computed from two values:
```
usable_input_tokens = context_size - max_output_tokens
```
With defaults: `128,000 - 64,000 = 64,000 tokens`.
No static system prompt reserve is subtracted. The system prompt (agent instructions, tool descriptions, formatting overhead) is included in the message list that ContextGuard estimates dynamically — subtracting a fixed reserve on top would double-count it and waste budget. A typical system prompt consumes 3K-5K tokens; agents with many tools or long instructions may use more. ContextGuard accounts for the actual size automatically.
Budget values can come from three sources, resolved in priority order:
1. **Database ModelConfig** — per-model `context_size` and `max_output_tokens` set by the admin.
2. **Environment variables** — `LLM_CONTEXT_SIZE` and `LLM_MAX_OUTPUT_TOKENS`.
3. **Hardcoded defaults** — 128K context, 64K output.
The main LLM and fast LLM have independent budgets. DAG step execution uses the fast LLM's budget; ReAct mode uses the main LLM's budget. This matters because operators often pair a large-context model for ReAct (where history accumulates) with a smaller, faster model for DAG steps (where each step starts fresh).
A floor of 4,000 tokens is enforced — if misconfigured values would produce a smaller budget, the system clamps to 4K rather than failing silently.
### Layer 4 — DbMemory
`DbMemory` is the production memory implementation. It loads persisted conversation history from the database and compacts it to fit the token budget before the agent sees it.
The design is **read-only by intention**. Persistence is handled by `chat.py` — the API layer that owns the full message lifecycle (including metadata, usage tracking, and image attachments). `DbMemory` only reads. Its `add_message()` and `clear()` methods are no-ops. This separation prevents double-writes and keeps the persistence logic in one place.
On load, `DbMemory`:
1. Queries all `user` and `assistant` messages for the conversation, ordered by creation time.
2. Drops the trailing user message (the current query, which the agent will re-append).
3. Reconstructs image attachments — user messages that included images store metadata (`file_id`, `mime_type`) in the database, and `DbMemory` rebuilds the base64 data-URLs from disk so the LLM can "see" images from earlier turns.
4. Compacts: if a `compact_llm` was provided, uses `CompactUtils.llm_compact()`. Otherwise, falls back to `CompactUtils.smart_truncate()`.
After compaction, `DbMemory` sets tracking flags (`was_compacted`, `_original_count`, `_compacted_count`) that the SSE layer uses to emit a `compact` event to the frontend.
### Layer 3 — ContextGuard
`ContextGuard` is the per-iteration budget enforcer. It is called at the top of every ReAct iteration — both in standalone ReAct mode and inside each DAG step's agent. This is the last line of defense before the messages hit the LLM API.
The enforcement follows a three-step process:
1. **Truncate oversized individual messages.** Any single message exceeding 50K characters is hard-truncated with a `[Truncated]` suffix. This catches runaway tool outputs — a web scrape that returns an entire webpage, a file read that dumps a large dataset.
2. **Estimate total tokens.** If the total fits within budget, return immediately. Most iterations pass here — compaction is the exception, not the norm.
3. **Compact if over budget.** If a `compact_llm` is available, use LLM-powered compaction with a hint-specific prompt. Otherwise, fall back to `smart_truncate`.
The **hint system** is what makes ContextGuard context-aware rather than one-size-fits-all. Different situations need different compaction strategies:
| Hint | Used by | Preserves | Drops |
| - | - | - | - |
| `react_iteration` | ReAct agent loop | Recent reasoning chain, current goal, critical data | Old redundant steps, failed retries, verbose tool outputs |
| `planner_input` | DAG enriched query | User intent evolution, key decisions, constraints | Dialogue details, greetings, tool call mechanics |
| `step_dependency` | DAG step context | Key data, numbers, conclusions | Reasoning process, failed attempts, verbose formatting |
| `general` | Default fallback | Key facts, decisions, tool results | Greetings, filler, redundant back-and-forth |
Each hint maps to a carefully worded system prompt that tells the compaction LLM what to keep and what to discard. The prompts end with "Write in the same language as the conversation" — a detail that matters for CJK users whose summaries would otherwise default to English.
If LLM compaction fails (network error, empty response, any exception), ContextGuard falls back to `smart_truncate` silently. The agent never sees the failure. This is a deliberate reliability choice: better to lose some context via heuristic truncation than to crash the iteration.
### Layer 2 — CompactUtils
`CompactUtils` is a stateless utility class — no instances, no state, just pure functions. It provides three capabilities that layers 3 and 4 build on.
**Token estimation** converts text to an approximate token count without importing a tokenizer library. The heuristic:
* ASCII characters: \~4 characters per token
* CJK / non-ASCII characters: \~1.5 characters per token
* Images: 765 tokens per image (flat cost)
* Per-message overhead: 4 tokens (role marker, delimiters)
**`smart_truncate`** is the heuristic fallback. It keeps pinned messages unconditionally, then walks backward through non-pinned messages, accumulating until the budget is exhausted. The result is a suffix of the conversation that fits. It also ensures the result never starts with an assistant message — an orphaned assistant turn with no preceding user message confuses LLMs.
**`llm_compact`** is the LLM-powered path. It splits messages into three groups — system messages (always kept), pinned messages (always kept), and compactable messages. The oldest compactable messages are summarized into a single `[Conversation summary]` system message; the most recent 4 messages are kept verbatim. If the compacted result is still too long, it falls back to `smart_truncate` on the compacted output — belt and suspenders.
### Layer 1 — Memory Implementations
The memory layer defines the `BaseMemory` interface: `add_message()`, `get_messages()`, `clear()`. Three implementations exist:
* **WindowMemory** — a count-based sliding window. Keeps the last N non-system messages. Simple, predictable, no LLM calls. Not used in production; useful for testing and stateless scenarios.
* **SummaryMemory** — triggers LLM summarization when the message count exceeds a threshold. Compresses old messages into a `[Conversation summary]` system message. Not used in production; predates the more sophisticated ContextGuard approach.
* **DbMemory** — the production implementation (described in Layer 4). Database-backed, read-only, with LLM or heuristic compaction on load.
WindowMemory and SummaryMemory remain in the codebase because they serve as useful primitives for testing and for users who embed FIM One's core library without the web layer. They are not dead code — they are the simple cases that the architecture grew out of.
## How context flows through ReAct
The ReAct agent uses context management at two distinct phases: load time and iteration time.
```mermaid theme={null}
sequenceDiagram
participant Chat as chat.py
participant Db as DbMemory
participant Agent as ReActAgent
participant CG as ContextGuard
participant LLM as LLM API
Chat->>Db: get_messages(budget)
Db->>Db: Load from DB, compact
Db-->>Chat: history (within budget)
Chat->>Agent: run(query)
Agent->>Agent: Assemble: system + history + pinned query
loop Each iteration (non-streaming)
Agent->>CG: check_and_compact(messages, hint="react_iteration")
CG->>CG: Truncate oversized, estimate, compact if needed
CG-->>Agent: messages (within budget)
Agent->>LLM: chat(messages)
LLM-->>Agent: response
alt tool_call
Agent->>Agent: Execute tool, append observation
opt cycle detected
Agent->>Agent: Inject warning message into context
end
else final_answer
opt tools ≥ threshold, not yet checked
Agent->>Agent: Inject checklist prompt into context
Agent->>Agent: Continue loop (one more iteration)
end
end
end
Agent-->>Chat: AgentResult (tool trace + brief answer)
Chat->>Agent: stream_answer(query, result)
Agent->>LLM: stream_chat(synthesis prompt)
LLM-->>Agent: token stream
Agent-->>Chat: streamed answer → SSE to frontend
Chat->>Chat: Persist, emit compact SSE if compacted
```
Tool iterations use non-streaming `chat()` for speed; answer synthesis uses streaming `stream_chat()` via `stream_answer()`. This two-phase split — fast tool loop followed by streaming synthesis — optimizes both latency and user experience. For full ReAct engine architecture including dual-mode execution and tool selection, see [ReAct Engine](/architecture/react-engine).
The key insight: **DbMemory handles the historical context problem (turns from previous requests), while ContextGuard handles the within-request growth problem (tool results accumulating during an agent loop).** They operate at different timescales and catch different failure modes.
The user's current query is always marked as `pinned=True`. This ensures it survives all compaction — both `smart_truncate` and `llm_compact` preserve pinned messages unconditionally. No matter how aggressively the history is compressed, the user's actual question is never lost.
## How context flows through DAG
DAG mode has a fundamentally different context shape than ReAct. Instead of one long conversation thread, it has a tree: a planning phase, multiple parallel execution steps, and an analysis phase. Each phase has its own context management strategy.
```mermaid theme={null}
flowchart TD
A["1. Load history via DbMemory"] --> B["2. Build enriched_query"]
B --> C{"enriched_query > 16K tokens?"}
C -->|Yes| D["LLM summarize with planner_input prompt"]
C -->|No| E["Use as-is"]
D --> F["3. DAGPlanner (structured_llm_call)"]
E --> F
F --> G["4. Execution: parallel steps"]
G --> H["Step 1: independent ReAct"]
G --> I["Step 2: independent ReAct"]
G --> J["Step N: independent ReAct"]
H --> K["5. PlanAnalyzer (structured_llm_call)"]
I --> K
J --> K
K --> L{"Achieved?"}
L -->|No, low confidence| M["6. Replan: 500-char step summaries"]
M --> F
L -->|Yes| N["stream_synthesize → Final answer"]
```
**Phase 1 — History loading.** DbMemory loads and compacts conversation history, same as ReAct. The compacted history is formatted into a text block prefixed with `"Previous conversation:"`.
**Phase 2 — Enriched query construction.** The history text and current query are combined into an `enriched_query`. If this exceeds 16K tokens, it is LLM-summarized using the `planner_input` hint prompt. The 16K threshold is chosen because the planner needs to read the entire query in a single pass — unlike ReAct, there is no iterative compaction during planning.
**Phase 3 — Planning.** The planner receives a 2-message prompt: system prompt plus enriched query. No ContextGuard here — the enriched query is already size-controlled by the 16K check.
**Phase 4 — Step execution.** Each DAG step runs as an independent ReAct agent with its own ContextGuard. Critically, these step agents have **no memory** — they start fresh with only their task description and dependency context. This is by design: DAG steps should be self-contained units of work. Dependency results are injected via `_build_step_context`, which character-truncates at 50K (the ContextGuard's `max_message_chars` limit).
**Phase 5 — Analysis.** Step results are formatted for the analyzer LLM with per-step truncation at 10K characters. This prevents a single step's verbose output from dominating the analysis context.
**Phase 6 — Replanning.** When the analyzer determines the goal was not achieved and confidence is below the threshold, step results are truncated to just 500 characters each for the replanning context. Replanning needs to know *what happened* and *what went wrong*, not the full detail of every step's output. This aggressive truncation keeps the replan prompt compact enough for the planner to process efficiently.
For the full DAG pipeline architecture including the LLM Call Map and re-planning logic, see [DAG Engine](/architecture/dag-engine).
## Pinned messages
The pinning mechanism prevents compaction from destroying messages that must survive. Two categories of messages are pinned:
1. **The current user query** — always pinned. If the user asks a question and the history is too long, the system compresses the history, not the question.
2. **Injected mid-stream messages** — when a user sends a follow-up while the agent is still running, the injected message is marked as pinned so the agent sees it in the next iteration.
The risk with pinning is accumulation. In a long session with many injected messages, pinned content can grow to consume most of the budget, leaving no room for the actual conversation history. ContextGuard addresses this with a hard cap: **when pinned tokens exceed 50% of the budget, the oldest injected messages are unpinned and moved to the compactable pool.** Only the most recent pinned message (the current query) is preserved.
This is a trade-off. Unpinning old injected messages means they might be summarized or truncated. But the alternative — letting pinned messages crowd out all other context — is worse. The system biases toward preserving the most recent context, which is almost always more relevant than older injections.
## Token estimation
FIM One uses heuristic token estimation rather than a real tokenizer. This is a deliberate choice with clear trade-offs.
**Why not a real tokenizer?** Three reasons:
1. **Dependency cost.** `tiktoken` (OpenAI's tokenizer) is 15MB of compiled Rust bindings. `sentencepiece` (used by some open-source models) has its own build requirements. For a framework that targets multiple LLM providers, there is no single correct tokenizer — each model family uses a different one.
2. **Speed.** Heuristic estimation is a single pass over the string. Real tokenization involves vocabulary lookup, BPE merge operations, and special token handling. ContextGuard calls estimation on every iteration, sometimes multiple times — the speed difference matters.
3. **Good enough.** The heuristic is tuned for mixed-language text (the ASCII/CJK split covers the two major cases). It can be 1.5-2x off for edge cases (heavily punctuated code, unusual Unicode), but context management is inherently approximate. Being 30% off on a 64K budget still leaves a comfortable margin.
The concrete heuristics:
| Content type | Ratio | Rationale |
| - | - | - |
| ASCII text | \~4 chars/token | English prose and code average 3.5-4.5 chars/token across GPT/Claude tokenizers |
| CJK / non-ASCII | \~1.5 chars/token | Each CJK character is typically 1-2 tokens; 1.5 is the geometric mean |
| Images | 765 tokens/image | Approximate cost of a base64-encoded image in the vision API |
| Per-message overhead | 4 tokens | Role marker, delimiters, formatting |
The estimation always returns at least 1 token for non-empty content. This prevents division-by-zero edge cases in budget arithmetic.
## What the user sees
Context management is designed to be invisible in the common case and minimally intrusive when it activates. The user-facing signals are:
**CompactDivider.** When `DbMemory` compacts history on load, the frontend renders a dashed divider with the text "Earlier context (N messages) was summarized by AI." This appears between the summary and the preserved recent messages, giving the user a visual cue that older context has been compressed without interrupting the conversation flow.
**Token usage display.** The `done` card at the end of each response shows "X.Xk in / X.Xk out" — the total input and output tokens consumed. This includes tokens spent on compaction (the fast LLM calls for summarization). Users who monitor token consumption can see when compaction is adding overhead.
**Graceful error handling.** If context management fails entirely — a scenario that should not happen given the fallback chain, but could in theory — the error surfaces as agent error text in the response, not as a system crash. The conversation continues; the user can retry or rephrase.
The goal is that most users never think about context management. They have long conversations, the system handles the budget transparently, and the only visible artifact is an occasional compact divider. For power users and operators who care about token efficiency, the usage display and the configurable budget parameters provide the control they need.
# DAG Engine
Source: https://docs.one.fim.ai/architecture/dag-engine
How FIM One's DAG engine works — dynamic planning, parallel execution, goal-aware analysis, and autonomous re-planning.
For how this engine relates to industry **graph engineering**, classic visual workflows, and the ReAct loop, see [Planning Landscape](/architecture/planning-landscape).
## The pipeline
DAG mode decomposes a complex goal into a directed acyclic graph of steps, executes them with maximum parallelism, then reflects on whether the goal was actually achieved. If not, it re-plans and tries again — autonomously, up to a configurable budget.
The pipeline has four phases that form a loop:
```mermaid theme={null}
flowchart TD
A["1. History + Query -> enriched_query"] --> B["2. DAGPlanner"]
B --> C["3. DAGExecutor (parallel steps)"]
C --> D["4. PlanAnalyzer"]
D --> E{Achieved?}
E -->|"Yes"| F["5. stream_synthesize"]
E -->|"No + confidence < threshold"| G["6. Replan"]
G --> B
E -->|"No + confident enough"| H["Fallback answer"]
```
**Plan.** The smart LLM decomposes the enriched query into 2-6 steps with explicit dependency edges. Each step gets a task description, optional tool hint, and a model hint that controls whether it runs on the fast or smart LLM.
**Execute.** The DAGExecutor launches independent steps in parallel (up to 5 concurrent), respecting the dependency graph. Each step runs as a self-contained ReAct agent with no memory — it receives only its task description and the results of its completed dependencies.
**Analyze.** The PlanAnalyzer evaluates whether the executed plan achieved the original goal, producing a structured verdict: `achieved` (boolean), `confidence` (0.0-1.0), `reasoning`, and an optional `final_answer`.
**Re-plan.** If the goal was not achieved and confidence is below the stop threshold, the pipeline loops back to planning with a replan context that summarizes what happened and what went wrong. This loop runs up to `DAG_MAX_REPLAN_ROUNDS` times autonomously.
Two LLMs collaborate throughout: a **smart LLM** handles planning, analysis, and answer synthesis (tasks requiring high reasoning capability), while the **general LLM** handles step execution by default (with `model_hint="fast"` steps delegated to the fast LLM for cost savings). Context compaction and history summary use the fast LLM. Every structured output call uses `structured_llm_call`, which provides a 3-level degradation chain (Native FC, JSON Mode, plain text with regex fallback) to handle model-specific output quirks.
## LLM call map
The full DAG pipeline makes seven distinct categories of LLM calls. Understanding where each call happens, which model handles it, and what happens on failure is essential for debugging and cost optimization.
| # | Call Site | Module | LLM Role | Format | Fallback |
| - | - | - | - | - | - |
| 1 | History summary | chat.py | fast LLM | plain text | truncate last 20K chars |
| 2 | DAGPlanner | planner.py | smart LLM | structured\_llm\_call | 3-level degradation |
| 3 | Tool selection | react.py | step LLM | structured\_llm\_call | return all tools |
| 4 | ReAct loop (per step) | react.py | general LLM (default) / fast LLM (`model_hint="fast"`) / reasoning LLM (`model_hint="reasoning"`) | chat() | retry/fallback |
| 5 | ContextGuard compact | context\_guard.py | fast LLM | plain text | smart\_truncate |
| 6 | PlanAnalyzer | analyzer.py | smart LLM | structured\_llm\_call | regex + default |
| 7 | stream\_synthesize | analyzer.py | smart LLM | stream\_chat() | analysis.final\_answer |
```mermaid theme={null}
flowchart TD
subgraph pre ["Pre-execution"]
H["① History summary"] --> EQ["enriched_query"]
end
subgraph plan ["Planning"]
P["② DAGPlanner structured_llm_call"]
end
subgraph exec ["Execution (per step)"]
TS["③ Tool Selection structured_llm_call"] --> RL["④ ReAct Loop chat()"]
CG["⑤ ContextGuard compact"]
end
subgraph post ["Post-execution"]
PA["⑥ PlanAnalyzer structured_llm_call"] --> SS["⑦ stream_synthesize stream_chat()"]
end
EQ --> P
P --> TS
PA -->|"replan"| P
```
Calls 1 and 5 are **invisible to the user** — they are infrastructure calls that manage context size. Calls 2, 6, and 7 use the **smart LLM** because they require high reasoning capability (decomposing goals, judging achievement, synthesizing coherent answers). Call 4 uses the **general LLM** by default — only steps explicitly marked `model_hint="fast"` are downgraded to the fast LLM, and steps marked `model_hint="reasoning"` are promoted to the reasoning LLM. Call 3 uses the same LLM resolved for that step.
## DAGPlanner
The planner's job is to turn a high-level goal into a valid DAG of concrete, actionable steps. It does this with a single `structured_llm_call` to the smart LLM.
**Prompt design.** The planning prompt injects the current datetime and year (so the LLM can plan time-aware searches), enforces language matching (task descriptions must use the same language as the goal), and constrains step count to 2-6. Each step has five fields: `id`, `task`, `dependencies`, `tool_hint`, and `model_hint`. The prompt explicitly discourages splitting trivially related sub-tasks — "if several checks can be done in a single script, combine them into ONE step."
**Structured extraction.** The planner uses `structured_llm_call` with a `_PLAN_SCHEMA` that defines the `steps` array schema and a `parse_fn` that converts raw dicts to `PlanStep` objects. If the LLM returns a single step object instead of a `{"steps": [...]}` wrapper, the parser auto-recovers. The 3-level degradation chain documented in [ReAct Engine — structured\_llm\_call](/architecture/react-engine#structured_llm_call--unified-output-extraction) handles model output quirks across providers.
**DAG validation.** After extraction, the planner validates the graph structure using Kahn's algorithm for topological sorting. Two invariants are checked:
1. **No dangling references.** If a step references a dependency ID that does not exist in the plan, the reference is silently pruned with a warning log. This is a recovery mechanism — LLMs sometimes omit steps they referenced, and hard-failing would waste the entire planning call.
2. **No cycles.** If Kahn's algorithm cannot visit all nodes (meaning at least one cycle exists), the planner raises a `ValueError`. Cycles are unrecoverable — a cyclic plan cannot be executed.
**model\_hint.** The planner assigns `"fast"` to steps it considers simple and deterministic (data lookup, format conversion, straightforward retrieval), `null` to steps requiring standard reasoning (resolved to the general model), and `"reasoning"` to steps requiring deep analysis. The executor uses this hint to select the appropriate LLM per step via the `ModelRegistry`. When in doubt, the prompt instructs the LLM to use `null` — it is always safer to use the more capable model. For domain-specific tasks (legal, medical, financial), the planner receives domain context from the router and is guided to assign `model_hint="reasoning"` to steps requiring specialist accuracy.
**Input construction.** The enriched query combines conversation history with the current request. If the conversation is long, history is loaded via `DbMemory` and formatted as `"Previous conversation: ..."`. When the resulting enriched query exceeds 16K tokens (estimated via `CompactUtils.estimate_tokens`), it is LLM-summarized using the `planner_input` hint prompt from ContextGuard before being passed to the planner. The fallback when no fast LLM is available: hard-truncate to the last 20K characters.
## DAGExecutor
The executor takes a validated `ExecutionPlan` and runs its steps concurrently, respecting dependency edges and enforcing resource limits.
**Concurrency model.** An `asyncio.Semaphore` limits parallel step execution to `max_concurrency` (default 5, configurable via `MAX_CONCURRENCY` env var). The dispatch loop identifies all steps whose dependencies have completed, launches them as `asyncio.Task` instances, and waits for at least one to finish before checking again. Steps are launched in sorted ID order for deterministic behavior.
**Per-step ReAct agent.** Each step runs as an independent ReAct agent created by `_resolve_agent()`. If the step has a `model_hint` that matches a role in the `ModelRegistry`, a temporary agent is created with the corresponding LLM. Otherwise, the registry's default (general) model is used. These per-step agents have **no memory** — they start fresh with only their task description, the original goal, any tool hints, and the results of completed dependencies. This isolation is deliberate: DAG steps should be self-contained units of work that do not leak state across the graph. Because each step is a standard `ReActAgent` instance, it automatically inherits all harness features: [cycle detection](/architecture/react-engine#cycle-detection) prevents a step from looping on the same failing tool, and the [completion checklist](/architecture/react-engine#completion-checklist) verifies conclusions before a step finalizes its result.
**Dependency context injection.** `_build_step_context()` formats the results of all completed dependency steps into a text block: each dependency's ID, status, task description, and result. If a `ContextGuard` is configured and the combined context exceeds `max_message_chars`, it is hard-truncated with a `[Dependency context truncated]` suffix. This prevents a step that depends on multiple verbose predecessors from blowing out its own context window.
**Structured content multiplier.** When dependency results contain structured content — legal citations, markdown tables, or code blocks — `_build_step_context()` applies a multiplier to the truncation budget (default `3.0`, configurable via `DAG_STRUCTURED_CONTEXT_MULTIPLIER`). This ensures that citations, tabular data, and other structured artifacts are preserved across step boundaries rather than being truncated mid-reference.
**Step timeout.** Each step is wrapped in `asyncio.wait_for` with a default timeout of 600 seconds (10 minutes). If a step exceeds this, it is cancelled and marked as `"failed"` with a timeout message. The timeout is per-step, not per-plan — a 5-step plan can theoretically run for 50 minutes if steps execute sequentially.
**Interruption and cancellation.** The executor has two distinct cancellation paths, each triggered by a different event:
```mermaid theme={null}
flowchart LR
A["User follow-up message"] -->|"exec_stop_event.set()"| B["Pending steps → skipped\nRunning steps → complete normally"]
C["HTTP client disconnects"] -->|"run_task.cancel()"| D["Running step tasks cancelled\nasyncio.gather awaits cleanup"]
```
*Graceful skip — stop event.* When a user sends a follow-up message during execution, the orchestrator in `chat.py` sets `exec_stop_event`. The executor checks this flag at the top of each dispatch cycle: if set, all remaining `pending` steps are immediately marked `"skipped"` with the reason `"Skipped — user changed requirements"`, and the loop exits. Steps already running are allowed to complete — only unstarted steps are abandoned. This fast-exit lets the pipeline re-plan around the user's updated intent without waiting for the full original plan to finish.
*Immediate abort — asyncio cancellation.* When the HTTP client disconnects, `chat.py` cancels the top-level `run_task` via `asyncio.Task.cancel()`. The executor catches `asyncio.CancelledError`, cancels all currently-running step tasks, waits for them to acknowledge via `asyncio.gather(..., return_exceptions=True)`, then re-raises. Client disconnect is detected by polling `await request.is_disconnected()` every 0.5 seconds inside the SSE event loop.
The semantic difference matters: **stop event** means "skip what hasn't started yet, but preserve what's already running" — completed step results remain available to inform re-planning. **CancelledError** means "abort everything immediately" — all in-flight work is dropped with no result recovery.
**Deadlock detection.** If the dispatch loop finds no tasks running and no steps ready to launch (because their dependencies failed), all remaining pending steps are marked `"failed"` with a message explaining that their dependencies never completed. This prevents the executor from hanging indefinitely.
**Progress callbacks.** The executor fires `(step_id, event, data)` callbacks for three event types: `"started"` (step launched), `"iteration"` (tool call within a step), and `"completed"` (step finished). The SSE layer in `chat.py` bridges these callbacks to `step_progress` events that the frontend uses to render real-time DAG visualization.
## Citation verification
After each step completes, the executor optionally runs a **citation verifier** that checks factual claims in the step's output. This is gated by the `DAG_CITATION_VERIFICATION` environment variable (default: `true`) and targets domains where incorrect citations carry high risk — legal statutes, medical references, and financial regulations.
The verifier operates in three stages:
1. **Extraction.** Regex patterns identify citation-like strings in the step result (e.g., case numbers, statute references, regulation codes).
2. **Verification.** Each extracted citation is evaluated by an LLM judgment call that checks plausibility and internal consistency.
3. **Retry on failure.** If verification fails, the step is retried with correction feedback appended to the task description, giving the agent a chance to fix inaccurate references.
Citation verification adds latency to steps that contain citations but does not affect steps without them. Disable it by setting `DAG_CITATION_VERIFICATION=false` if your use case does not involve citation-sensitive domains.
## Domain-aware routing
The auto-router now classifies queries with a **`domain_hint`** alongside the existing mode selection. Recognized domains are `legal`, `medical`, and `financial`; queries outside these domains receive `null`.
Domain classification influences the pipeline in two ways:
**DAG mode.** When the router selects DAG and provides a non-null `domain_hint`, the domain context is injected into the planner prompt. This guides the planner toward assigning `model_hint="reasoning"` for steps that require specialist accuracy and suggesting `tool_hint="read_skill"` for steps that match available domain skills.
**ReAct mode.** When the router selects ReAct for a domain-specific query, the system escalates from the general model to the **reasoning model** via `registry.get_by_role("reasoning")`. Additionally, mandatory instructions are injected requiring the agent to use `web_search` before writing any domain-specific content and to verify citations via search. The `read_skill` tool is pinned in tool selection (not filtered out), ensuring domain knowledge is always accessible.
The routing bias also shifts: tightly-coupled domain analysis (where multiple sub-tasks share context and citations) prefers ReAct mode to avoid losing context across DAG step boundaries.
## PlanAnalyzer
The analyzer evaluates whether the executed plan achieved the original goal. It produces a structured `AnalysisResult` with four fields:
* **`achieved`** (boolean) — `true` only if the goal was fully accomplished.
* **`confidence`** (float, 0.0-1.0) — how certain the analyzer is in its assessment. Sources that contradict each other lower this score.
* **`final_answer`** (string or null) — a synthesized answer when achieved, `null` otherwise.
* **`reasoning`** (string) — the LLM's chain-of-thought justification.
**Structured extraction.** The analyzer uses `structured_llm_call` with `_ANALYSIS_SCHEMA`, a `parse_fn` that handles type coercion and confidence clamping, and a `regex_fallback` for malformed JSON. The regex fallback (`_regex_extract_analysis`) extracts `achieved`, `confidence`, `final_answer`, and `reasoning` fields from partially valid JSON using pattern matching. This matters because analysis responses tend to be longer and more complex than planning responses, making JSON formatting errors more likely.
**Safe default.** If all extraction levels fail (native FC, JSON mode, plain text, regex), the analyzer returns `AnalysisResult(achieved=False, confidence=0.0, reasoning="Could not parse analysis response")`. This ensures the pipeline always gets a usable result — a parse failure becomes a "not achieved" verdict, which triggers re-planning rather than crashing.
**Step result formatting.** Each step's result is truncated to 10K characters in the analysis prompt. This prevents a single step's verbose output (e.g., a large web scrape or file dump) from dominating the analyzer's context window and crowding out other steps' results.
**Multi-source comparison.** The analysis prompt includes a directive to explicitly compare results from different sources. When web search results, knowledge base retrieval, and file operations all contribute data, the analyzer must flag contradictions (different numbers, dates, claims) and indicate which source is likely more authoritative. Contradictions lower the confidence score, which in turn influences the re-planning decision.
## Re-planning
The re-planning loop is the DAG engine's most distinctive feature: it can autonomously recover from partial failures by reflecting on what went wrong and trying a different approach.
**Decision logic.** After each round of plan-execute-analyze, the orchestrator in `chat.py` evaluates the analysis result:
1. **`achieved == True`** — exit the loop, proceed to streaming synthesis.
2. **User inject occurred during this round** — always re-plan, regardless of confidence or budget. User follow-up messages are treated as requirement changes that demand a fresh attempt. This does not consume the autonomous replan budget.
3. **Autonomous replan budget exhausted** — exit the loop. The budget is `max_replan_rounds - 1` autonomous replans (default: 2 autonomous replans from a budget of 3 total rounds).
4. **`confidence >= replan_stop_confidence`** — exit the loop. Even if the goal was not fully achieved, a high confidence score (default threshold: 0.8, configurable via `DAG_REPLAN_STOP_CONFIDENCE`) indicates the analyzer is fairly certain about what happened — re-planning is unlikely to help.
5. **Otherwise** — re-plan. The goal was not achieved, confidence is low, and budget remains.
**Replan context.** When re-planning, the orchestrator calls `_format_replan_context()` to build a summary of the previous round. This includes the analyzer's reasoning and a truncated preview of each step's result (500 characters max per step). The aggressive truncation is deliberate: the planner needs to know *what happened* and *what went wrong*, not the full detail of every step's output. This context is passed to `DAGPlanner.plan()` as the `context` parameter, alongside the original enriched query.
**Max rounds.** The `DAG_MAX_REPLAN_ROUNDS` environment variable (default 3) controls the total number of planning rounds. With default settings, the first round is the initial plan, leaving up to 2 autonomous re-plans. User-triggered re-plans (via message injection) do not count against this budget — a user can steer the pipeline indefinitely.
**SSE event.** When the pipeline decides to re-plan, it emits a `replanning` phase event containing the analyzer's reasoning. The frontend uses this to show the user why the pipeline is retrying.
**enriched\_query accumulation.** User follow-up messages are appended to the enriched query across rounds: `enriched_query += "\n\n[User follow-up]: {content}"`. This means the planner sees the full evolution of the user's intent — the original request plus all subsequent clarifications — when building a revised plan.
## Streaming synthesis
When the analyzer confirms the goal was achieved (`analysis.achieved == True`), the pipeline streams a synthesized final answer to the user via `PlanAnalyzer.stream_synthesize()`.
**Input.** The synthesis call receives three inputs: the original goal, the formatted step results (10K characters max per step), and the analyzer's reasoning from the non-streaming analysis call. The reasoning provides a "roadmap" for what the synthesis should cover.
**System prompt.** The synthesis prompt instructs the LLM to answer directly without meta-commentary ("do NOT include phrases like 'based on the results'"), match the language of the original goal, and compare results from different sources when applicable. A language directive from user preferences is appended if available.
**Streaming.** The method uses `stream_chat()` to yield tokens incrementally. The SSE layer wraps each chunk in an `answer` event with `status: "delta"`, giving the frontend real-time rendering of the final answer.
**Fallback chain.** Two fallback paths handle failure:
1. **stream\_synthesize raises an exception** — fall back to `analysis.final_answer` from the non-streaming `analyze()` call. This answer was already generated during analysis, so it is available even if the streaming call fails.
2. **Goal not achieved (no synthesis attempted)** — concatenate all completed step results, separated by horizontal rules. Each result is prefixed with its step ID. If no steps completed at all, return `"(goal not achieved)"`.
The fallback design ensures the user always gets an answer — degraded but never empty.
## Multi-LLM architecture
The DAG engine's cost and latency profile is shaped by its multi-model design. The division of labor is:
| Role | Used for | Optimized for |
| - | - | - |
| **Smart LLM** | Planning, analysis, answer synthesis | Reasoning capability |
| **General LLM** | Step execution (default), ReAct agent | Balance of capability and cost |
| **Fast LLM** | `model_hint="fast"` steps, context compaction, history summary | Cost and latency |
| **Reasoning LLM** | `model_hint="reasoning"` steps, domain-escalated ReAct | Deep analysis capability |
The smart LLM handles the three calls that require the deepest reasoning: decomposing a goal into a coherent plan, judging whether the plan achieved the goal, and synthesizing a final answer that coherently integrates multiple step results. These calls happen once per round (or once total for synthesis), so their higher per-token cost is amortized.
The general LLM handles step execution by default — each DAG step's ReAct loop runs on the general model unless the planner explicitly assigns a different `model_hint`. The fast LLM is reserved for steps tagged `model_hint="fast"` (simple lookups, format conversions) and for infrastructure calls (context compaction, history summarization). The reasoning LLM is used for steps tagged `model_hint="reasoning"` and for domain-escalated ReAct tasks (see [Domain-aware routing](#domain-aware-routing)).
**Per-step override.** The `model_hint` field on each `PlanStep` controls which LLM runs that step. When `model_hint` is `null`, the executor uses the general model. When it is `"fast"`, the executor uses the fast LLM via the model registry. When it is `"reasoning"`, the executor uses the reasoning LLM. The planner is instructed to set `"fast"` for deterministic tasks, `null` for standard reasoning, and `"reasoning"` for deep analysis — but it can also be set to any custom role registered in the `ModelRegistry`. Model resolution happens **once per step** via `_resolve_agent()` immediately before that step's ReAct loop begins — all iterations within the step (tool selection, the ReAct loop, ContextGuard compaction) use the same resolved LLM. The model never changes mid-step.
**Budget independence.** Each LLM role has an independent context budget, computed from its model configuration. DAG step execution uses the resolved step model's budget (general by default); planning and analysis calls use the smart LLM's budget. This is important because operators often pair a large-context model (128K+) for planning with different models for step execution. For details on how budgets are computed, see [Context Management — Budget Configuration](/architecture/context-management#layer-5--budget-configuration).
# Design Philosophy
Source: https://docs.one.fim.ai/architecture/design-philosophy
Architectural decisions made during DAG execution design — what we chose NOT to build, and why.
FIM One's DAG execution engine was designed with deliberate restraint. Several patterns common in multi-agent frameworks were evaluated and intentionally excluded. This page documents those decisions and the reasoning behind them.
## Handoff — Not Needed
**Pattern**: Explicit task handoff between agents, where one agent completes its work and passes control to the next.
**Why excluded**: DAG linear chains already cover sequential task handoff naturally. When step B depends on step A, the executor waits for A to complete, then passes A's result into B's context via the dependency mechanism. This is structurally equivalent to a handoff — but without requiring a separate abstraction layer.
Adding an explicit handoff protocol would duplicate what step dependencies already provide, with no additional expressiveness.
## Backflow — Not Needed
**Pattern**: Downstream steps feeding results back to upstream steps for re-evaluation (also called backpropagation of results).
**Why excluded**: The PlanAnalyzer re-plan loop already covers this use case. After all steps complete, the analyzer evaluates whether the original goal was achieved. If not (and confidence is below the stop threshold), it triggers a full re-plan with context from the previous round's results — including what went wrong and what was learned.
This is a coarser-grained but more robust approach than per-step backflow. Per-step backflow creates complex feedback loops that are hard to debug and can lead to infinite cycles. The re-plan loop achieves the same corrective effect with a cleaner execution model: plan, execute, evaluate, re-plan if needed.
## Teams (Agent Inter-Communication) — Not Needed
**Pattern**: Multiple specialized agents communicating with each other during execution, negotiating subtask ownership, or collaborating on shared state.
**Why excluded**: DAG dependency chains are sufficient for data-dependency tasks. When step C needs results from both step A and step B, the dependency edges express this directly. Step C receives the outputs of A and B in its context — no inter-agent messaging protocol required.
The DAG model is simpler and more predictable than agent-to-agent communication. Agents don't need to negotiate; the planner determines the execution structure upfront, and the executor enforces data flow through dependency resolution. This avoids the coordination overhead and non-determinism that multi-agent messaging introduces.
## agent\_hint / role\_hint — Not Needed
**Pattern**: An explicit hint telling the executor which type of agent or persona should handle each step (e.g., "use the researcher agent" or "use the coder agent").
**Why excluded**: The combination of three existing fields already provides sufficient guidance:
* **`task`** — The natural language description of what the step should accomplish. A well-written task implicitly signals the required expertise.
* **`tool_hint`** — Suggests which tools the step should use (e.g., `web_search`, `python_exec`). This constrains the execution approach more precisely than a vague role label.
* **`model_hint`** — Controls whether the step uses the fast or reasoning model tier, which is the most impactful execution decision.
Adding `agent_hint` or `role_hint` would be redundant with these three fields and would introduce another axis of configuration that the planner LLM must get right. Keeping the hint surface small reduces planning errors.
## Deep Multi-Agent Orchestration — Strategically Deferred
The "Teams" exclusion above is an engineering argument: DAG edges already express data dependencies. This section adds a **strategic** argument: the industry trend of model capability absorbing orchestration complexity.
### Why models are eating the orchestration layer
Multi-agent hierarchies emerged as a workaround for model limitations — if one model couldn't handle a complex task, you decomposed it across specialized agents. Each generation of frontier models erodes this rationale:
| Model capability | What it replaces |
| - | - |
| Expanding context windows (200K → 1M tokens) | Splitting context across agents to fit within limits |
| Extended thinking / chain-of-thought | Explicit Planner → Executor agent pipelines |
| Native agentic loops (Claude Code, Codex) | Framework-orchestrated multi-step execution |
| Native multi-agent protocols (Google A2A, OpenAI Swarm) | Framework-level agent coordination |
### The cost of depth
Even when models lack capability, deep agent hierarchies carry compounding costs:
* **Token multiplication** — each nesting level reinjects system prompts and context. Three levels deep, total token spend can be 5-10x a single-agent approach.
* **Latency stacking** — each level is a full LLM round-trip. Users wait for the slowest chain.
* **Information lossy compression** — delegated agents return summaries, not raw data. Each level loses fidelity.
* **Debug opacity** — "which level misunderstood the task?" is nearly impossible to diagnose at depth > 2.
### FIM One's position
One-level delegation via `CallAgentTool` is the sweet spot: "I'm a generalist, but this sub-task needs the SQL expert." This covers the vast majority of real-world delegation needs without the compounding costs of deeper hierarchies.
Deep multi-agent orchestration is deferred indefinitely — not because it's hard to build, but because frontier model trajectory suggests it will become unnecessary. FIM One's value lies in the **tool supply layer** (Connectors, MCP, Governance), not in orchestration depth that models are rapidly internalizing.
# Hook System
Source: https://docs.one.fim.ai/architecture/hook-system
Deterministic enforcement that sits below the LLM loop and cannot be bypassed by agent instructions.
## Why hooks exist
Instructions in a system prompt are **suggestions**. A sufficiently stubborn or confused LLM can ignore them. For most agent behavior that is exactly what you want — instructions give the model room to adapt.
But some requirements are not suggestions. "Every sensitive tool call must be logged." "Write operations are blocked when the org is in read-only mode." "Payments above ¥50k require a human tap before firing." These are **invariants** — facts about the system that must hold regardless of what the model decides in any given turn.
A hook is code that runs **outside the LLM loop** at a well-defined point in the agent's execution lifecycle. The LLM cannot see the hook. The LLM cannot argue with the hook. The LLM cannot talk the hook into skipping a step. If a `PreToolUse` hook returns `allow=False`, the tool call does not happen — no matter how insistent the reasoning trace was.
This is the critical architectural distinction:
| Mechanism | Where it runs | Who controls it | Guarantee |
| - | - | - | - |
| **System prompt instruction** | Inside LLM inference | Model | "Probably follows it" |
| **Tool description / schema** | Inside LLM inference | Model | "Probably follows it" |
| **Hook** | Around LLM inference | Platform code | **Always runs** |
Hooks are how FIM One turns "the agent is supposed to..." into "the agent cannot bypass...".
## Where hooks plug in
Three hook points are defined today. Each marks a boundary the agent crosses during one loop iteration:
```mermaid theme={null}
flowchart TD
S(["Session start"]) --> SS["SessionStart hooks inject dynamic context"]
SS --> LLM["LLM inference (reasoning / tool pick)"]
LLM --> DECIDE{"Tool call?"}
DECIDE -->|no| ANSWER(["Final answer"])
DECIDE -->|yes| PRE["PreToolUse hooks validate / block / rewrite args"]
PRE --> GATE{"allow?"}
GATE -->|blocked| BLOCK["Observation: 'tool call blocked by X'"]
GATE -->|allowed| EXEC["Tool executes"]
EXEC --> POST["PostToolUse hooks side effects / rewrite result"]
POST --> OBS["Observation returned to LLM"]
BLOCK --> OBS
OBS --> LLM
```
| Hook point | Fires when | Can it block? | Can it mutate data? |
| - | - | - | - |
| `SessionStart` | Before the first LLM call of a session | No | Yes — injects context into the initial prompt |
| `PreToolUse` | After the LLM picks a tool, before the tool runs | **Yes** (via `allow=False`) | Yes — can rewrite `tool_args` before execution |
| `PostToolUse` | After the tool returns, before the observation goes to the LLM | No | Yes — can rewrite the observation |
Multiple hooks at the same point run in priority order. An earlier `PreToolUse` hook's rewritten args are carried forward to later hooks, so middleware composes.
## When a hook vs. an instruction
Deciding whether to solve a requirement with a prompt instruction or a hook is the same calculation as "runtime assert vs. code comment":
| Symptom | Solution |
| - | - |
| "The agent should prefer X when Y" | Instruction — soft guidance, model has latitude |
| "The agent must log every call to connector Z" | **PostToolUse hook** — cannot rely on the model to remember |
| "Payments over ¥50k need human approval" | **PreToolUse hook** — cannot rely on the model to ask |
| "The agent should introduce itself in Chinese" | Instruction — stylistic, low cost if missed |
| "The agent cannot write to the production database in read-only mode" | **PreToolUse hook** — safety invariant, zero tolerance |
| "The agent should summarize long DB query results" | Could be either, but a hook is robust — see PostToolUse truncate |
Rule of thumb: **if the wrong behavior is an incident, use a hook. If the wrong behavior is a minor annoyance, an instruction is fine.**
## The hook contract
A hook is a subclass of `PreToolUseHook`, `PostToolUseHook`, or `SessionStartHook` with one required method:
```python theme={null}
class ReadOnlyGuard(PreToolUseHook):
name = "readonly_guard"
priority = 5 # lower runs earlier
def should_trigger(self, ctx: HookContext) -> bool:
return ctx.tool_name.startswith("sql_")
async def execute(self, ctx: HookContext) -> HookResult:
if org_is_readonly(ctx.metadata["org_id"]):
return HookResult(
allow=False,
error="Org is in read-only mode — write blocked.",
side_effects=["readonly_guard: blocked sql write"],
)
return HookResult() # default: allow=True, no mutation
```
The `HookContext` passed in carries `tool_name`, `tool_args`, `agent_id`, `user_id`, and a flexible `metadata` dict the engine populates with per-request facts (org id, conversation id, the connector action's `requires_confirmation` flag, …).
The `HookResult` returned controls the outcome:
* `allow: bool = True` — whether the tool call proceeds (ignored for `PostToolUse` / `SessionStart`)
* `error: str | None` — human-readable reason, surfaced to the LLM as the observation when blocked
* `modified_args: dict | None` — if set, replaces the tool args before execution
* `modified_result: Any | None` — if set (PostToolUse), replaces the observation before it returns to the LLM
* `side_effects: list[str]` — audit trail of what the hook did, merged into the agent's trace
## Case study: `FeishuGateHook`
The first hook shipped on top of this system is `FeishuGateHook` — a `PreToolUse` hook that turns any tool flagged `requires_confirmation=True` into a human-in-the-loop approval card posted to the org's Feishu group.
This hook exercises the full lifecycle:
```mermaid theme={null}
sequenceDiagram
autonumber
participant ReAct as ReAct loop
participant Hook as FeishuGateHook
participant DB as ConfirmationRequest
participant Feishu as Feishu group
participant Op as Operator
ReAct->>Hook: PreToolUse(tool=oa__purchase_pay, args={...})
Hook->>Hook: should_trigger? metadata.requires_confirmation==True
Hook->>DB: INSERT ConfirmationRequest (status=pending, id=uuid)
Hook->>Feishu: POST interactive card (Approve / Reject buttons)
Feishu-->>Op: Card rendered in group
loop poll every 1.5s, up to 120s
Hook->>DB: SELECT status WHERE id=uuid
end
Op->>Feishu: Tap Approve
Feishu->>DB: callback → status = approved
Hook->>DB: status==approved → return allow=True
Hook-->>ReAct: HookResult(allow=True)
ReAct->>ReAct: Execute oa__purchase_pay
```
What this design is buying:
* **The tool call is genuinely suspended.** The agent's SSE stream pauses between "I will call `oa__purchase_pay`" and the observation. The user sees the agent waiting, which matches what is happening under the hood.
* **Approval survives a process restart.** The pending row is in the database, not in memory. If the backend restarts while a card is outstanding, the next poll picks up where it left off.
* **The decision is audited.** `ConfirmationRequest` keeps `payload`, `responded_at`, `responded_by_open_id`, and the final status — an auditable record of who approved what and when.
* **No LLM in the decision loop.** The model produces the tool call. Humans produce the verdict. The hook is the deterministic bridge.
`FeishuGateHook` depends on a configured [Feishu Channel](/configuration/channels/feishu) — the hook sends the card through the channel's `send_interactive_card()` method and listens for callback events the channel parsed. The separation is deliberate: the hook owns "approval state machine", the channel owns "IM platform mechanics". The same hook could target Slack or WeCom tomorrow without changing its logic — only the channel implementation.
## Candidate hooks (not planned)
Four hook patterns fit the same lifecycle. None is on the current roadmap; they are parked until a deployment asks for one:
| Hook | Point | Purpose |
| - | - | - |
| `AuditLogHook` | PostToolUse | Auto-write `ConnectorCallLog` on every connector call. Today this is manual; making it a hook ensures coverage. |
| `ReadOnlyGuard` | PreToolUse | Block writes when the org is in read-only mode. |
| `ResultTruncateHook` | PostToolUse | Truncate oversized tool observations (>8k chars) before they reach the LLM context. |
| `ConnectorRateLimitHook` | PreToolUse | Per-connector per-user call-frequency cap, independent of LLM rate limits. |
A user-defined hook layer is parked on the same terms: per-agent YAML configuration (`hooks: [...]`) declaring shell commands or Python callables to run on matching tool events. This follows the same pattern modern agent frameworks (Claude Code, OpenDevin) have converged on — hook-based enforcement keeps the "must always happen" logic out of prompts.
## Hooks vs. Channels
The two abstractions solve orthogonal problems:
| Concept | What it models | Lifetime | Example |
| - | - | - | - |
| **Hook** | A point in the agent's execution where platform code runs | Per tool call | `FeishuGateHook`, `AuditLogHook` |
| **Channel** | A pluggable adapter to an external messaging platform | Long-lived per org | `FeishuChannel`, planned `SlackChannel` |
Hooks consume Channels — a hook that needs to talk to the outside world (send a card, post an alert, escalate to a group) calls into the org's Channel. A channel without any hook using it is still useful (e.g. agents can proactively send notifications via a tool), but the approval-gate pattern specifically requires both halves to be in place.
Put differently: **Channels are the "where do I talk to humans" plumbing, Hooks are the "when do I have to talk to humans" policy**. Production human-in-the-loop workflows need both.
## Current state (v0.8.4)
Snapshot of what shipped and what is still ahead:
* ✅ `HookRegistry`, `HookContext`, `HookResult` primitives wired into both ReAct and DAG
* ✅ `PreToolUseHook` / `PostToolUseHook` / `SessionStartHook` abstract bases
* ✅ `FeishuGateHook` — complete, including `ConfirmationRequest` table, polling loop, timeout/expire, and callback-driven state flips
* ✅ Feishu channel callback endpoint that decodes `card.action.trigger` and updates the pending row
* ✅ Agent-level hook declarations: `agent.model_config_json.hooks.class_hooks` resolves to an instantiated `HookRegistry` on every ReAct/DAG session
* ✅ **Hooks fire on every execution surface**: the main chat path (Portal, API, DAG), delegated sub-agents (`CallAgentTool`), and Workflow `AGENT` nodes. Eval Center runs intentionally **bypass** hooks (automated evaluation must not block on human approval).
A delegated agent does not inherit its caller's registry — it builds **its own** from its own `model_config_json`. Because the confirmation gate is auto-attached to every agent, rebuilding is a superset of inheriting: the delegate gets the gate *plus* whatever it declares for itself, and the gate resolves the delegate's `require_confirmation_for_all` rather than its caller's. Both surfaces fail **closed**: an agent whose hooks cannot be built is not run.
* ❌ `AuditLogHook`, `ReadOnlyGuard`, `ResultTruncateHook`, `ConnectorRateLimitHook` (not planned)
* ❌ User-defined YAML hook declarations (not planned)
The Hook System is a **load-bearing foundation** for production hardening. Its first user (`FeishuGateHook`) is also a production feature in its own right, which is why the skeleton shipped early for the 2026-04-24 roadshow rather than waiting for the full hook catalog.
# LLM Provider Compatibility
Source: https://docs.one.fim.ai/architecture/llm-provider-guide
How FIM One routes LLM calls, the tool_choice architecture, and provider-specific pitfalls — especially Anthropic thinking + AWS Bedrock.
## Provider detection
FIM One uses LiteLLM as a universal adapter. The `_resolve_litellm_model()` function in `core/model/openai_compatible.py` maps the user's `LLM_BASE_URL` + `LLM_MODEL` to a LiteLLM model identifier with a provider prefix. The prefix determines how LiteLLM routes the request — native API protocol (Anthropic Messages API, Gemini, etc.) or generic OpenAI-compatible `/v1/chat/completions`.
Resolution order:
1. **Explicit provider** (from DB `ModelConfig.provider` field) — highest priority. If the provider matches a known domain in the URL, no `api_base` is returned (LiteLLM routes natively). Otherwise, `api_base` is set to the relay URL.
2. **Domain match** against `KNOWN_DOMAINS` — official API endpoints are recognized by hostname.
3. **URL path hint** against `PATH_PROVIDER_HINTS` — common on relay platforms like UniAPI where `/claude` or `/anthropic` in the path indicates the upstream protocol.
4. **Fallback** — `openai/` prefix (generic OpenAI-compatible).
| Domain / Path | Provider prefix | Protocol |
| - | - | - |
| `api.openai.com` | `openai/` | OpenAI Chat Completions |
| `anthropic.com` | `anthropic/` | Anthropic Messages API |
| `generativelanguage.googleapis.com` | `gemini/` | Google Gemini |
| `api.deepseek.com` | `deepseek/` | DeepSeek (OpenAI-compatible) |
| `api.mistral.ai` | `mistral/` | Mistral |
| Path contains `/claude` or `/anthropic` | `anthropic/` | Anthropic Messages API (via relay) |
| Path contains `/gemini` | `gemini/` | Google Gemini (via relay) |
| Anything else | `openai/` | Generic OpenAI-compatible |
When the provider prefix is a native protocol (anthropic, gemini, etc.) and the URL is not the official endpoint, LiteLLM uses the native protocol but sends requests to the relay's `api_base`. This means provider-specific behaviors — including the Bedrock prefill issue described below — apply regardless of whether the request goes to the official API or through a relay.
If your relay URL contains `/claude` in the path, FIM One automatically routes via Anthropic's native protocol. This is usually correct (better streaming, thinking support), but means provider-specific behaviors apply — including the Bedrock prefill issue described below.
## tool\_choice — the four modes
The `tool_choice` parameter is standardized via the OpenAI format. LiteLLM translates it to each provider's native protocol before sending the request.
| Mode | Meaning | Provider support |
| - | - | - |
| `"auto"` | Model decides whether to call a tool or respond with text | All providers |
| `"required"` | Must call a tool, but model chooses which | Most providers |
| `{"type":"function","function":{"name":"X"}}` | Must call function X specifically | Most providers — **incompatible with Anthropic thinking** |
| `"none"` | Cannot use tools, text only | All providers |
The distinction between `"auto"` and forced (`{"type":"function",...}`) is the crux of every compatibility issue in FIM One. These two modes are used by completely different subsystems with different requirements.
## Where tool\_choice is used
Two subsystems use `tool_choice`, and they use it in fundamentally different ways.
### ReAct engine — tool\_choice="auto"
The ReAct loop needs the model to decide each iteration: call a tool, or give a final answer. Only `"auto"` makes sense here — the model freely chooses between producing `tool_calls` or text content. This is compatible with all providers, all models, and all modes including extended thinking.
```mermaid theme={null}
flowchart TD
A["ReAct iteration"] --> B{"Model decides (tool_choice=auto)"}
B -->|"tool_calls present"| C["Execute tool → next iteration"]
B -->|"text content only"| D["Final answer"]
```
The ReAct engine uses native function calling (`_run_native`) when `abilities["tool_call"] = True`, falling back to JSON-in-content mode (`_run_json`) otherwise. Both modes use `"auto"` — the difference is whether tools are passed via the `tools` parameter or described in the system prompt. See [ReAct Engine — Dual-mode execution](/architecture/react-engine#dual-mode-execution) for details.
### structured\_llm\_call — tool\_choice=forced
One-shot structured extraction (schema annotation, DAG planning, plan analysis). Forces the model to call a specific virtual function, guaranteeing structured JSON output. This is the call site that triggers provider-specific errors.
`structured_llm_call` implements a 3-level degradation chain. The three rungs are named after the code, and their numbering is not the same as the four guarantee tiers described in [the section below](#structured-output-the-four-guarantee-tiers):
```mermaid theme={null}
flowchart TD
START["structured_llm_call()"] --> L1{"abilities.tool_call AND tool_choice?"}
L1 -->|Both true| FC["Level 1: native_fc forced tool_choice"]
L1 -->|Either false| L2
FC -->|"Success"| DONE["Return StructuredCallResult"]
FC -->|"Fail (catch Exception)"| L2{"abilities.json_mode?"}
L2 -->|Yes| JM["Level 2: json_mode response_format=json_object"]
L2 -->|No| PT["Level 3: plain_text extract JSON from free text"]
JM -->|"Success"| DONE
JM -->|"Fail → retry once"| PT
PT -->|"Success"| DONE
PT -->|"Fail → retry once"| ERR["StructuredOutputError or default_value"]
```
The critical design difference: `structured_llm_call`'s fallback is **runtime** — it dynamically tries each level and catches exceptions to fall through. The ReAct engine's mode selection is **build-time** — it checks `_native_mode_active` once at the start and commits to one mode for the entire loop. This means `structured_llm_call` can recover from provider-specific 400 errors transparently, while ReAct relies on the mode being correctly chosen upfront.
## Structured output: the four guarantee tiers
"Structured output" is one phrase covering four different mechanisms. They all produce JSON and they do not offer the same guarantee. What separates them is where the constraint lives: inside the decoder, which cannot emit a token the schema forbids, or inside the prompt, which can only ask.
| Tier | What the request carries | Where the payload arrives | What is guaranteed | Enforced by |
| - | - | - | - | - |
| **T1 — native function calling, `strict: true`** | `tools=[…]` with `strict` set on the function definition | A JSON string in `tool_calls[0].function.arguments` | Schema-level: every required field present, no undeclared keys, types and enums correct | Decoder state machine |
| **T2 — `response_format: {"type": "json_schema"}`** | The schema itself, inside `response_format`, with `strict: true` | A JSON string in `message.content` | Schema-level, same as T1, but on the assistant's own reply rather than on a tool call | Decoder state machine |
| **T3 — `response_format: {"type": "json_object"}`** | The type alone, no schema | A JSON string in `message.content` | Syntactic only: the text parses as JSON. Which keys appear is unconstrained | Decoder (syntax) |
| **T4 — free text plus instructions** | Nothing. The schema is described in the prompt | `message.content`, possibly wrapped in prose or code fences | Nothing | The prompt |
The mechanism behind T1 and T2 is constrained decoding. The provider compiles the schema into a grammar and, at each decoding step, masks out every token that would make the output unparseable against it. A required field cannot be skipped because the token that would close the object is not in the allowed set until that field has been emitted. This is a different kind of claim from "the model was asked nicely and complied," and it is why T1/T2 hold at temperature 1 and on small models, where prompt compliance does not.
### T1 and T2 are not interchangeable
They give the same guarantee through different channels, and the channel matters in three ways.
**The turn shape differs.** T1 returns a tool-call turn; the model has decided to *act*. T2 returns an ordinary assistant message; the model has *answered* in a shape. When the caller wants one extraction and not an agent loop, T2 says so directly instead of inventing a virtual function for the model to "call".
**Forcing T1 collides with thinking.** Getting a schema-gated payload out of T1 usually means forcing the choice, either `required` or a named function. Several providers reject a forced tool choice while extended thinking is active. Table B below records which ones. T2 has no such conflict: it is a response constraint, not a tool constraint, so it is the only schema-gated path that survives thinking being on. For a deployment standardised on reasoning models, that is the practical argument for T2 rather than a stylistic one.
**T1 costs a round of schema plumbing that T2 does not.** A virtual function needs a name, a description, and a decision about whether the model may decline to call it.
### What T2 costs
Constrained decoding is not free and the schema you already have is usually not the schema it accepts.
* **A schema subset.** OpenAI's strict mode wants `additionalProperties: false` on every object and every property listed in `required`; optionality is expressed as a union with `null` rather than by omission from `required`. Root must be an object. There are caps on total properties and nesting depth. Most hand-written schemas need editing before they are eligible.
* **Grammar compilation.** The first request carrying a new schema pays a one-off compile latency on the provider side. Reusing a stable schema amortises it; generating a fresh schema per request does not.
* **A new failure surface.** A model that will not answer returns a refusal rather than a schema-shaped object, so the caller needs a branch for it.
### What no tier guarantees
Every tier constrains form. None constrains truth. A T2 response can be perfectly schema-valid and factually wrong, and an enum-constrained field will return one of the allowed values even when none of them is correct, because the decoder's job is to keep the output inside the grammar rather than to know the answer. Schema gating removes parse failures and field-shape bugs. It does not remove the need to check what the value says.
**Two numbering schemes, and they do not line up.** This page uses **T1–T4** for the guarantee tiers above. It uses **Level 1/2/3** for the three rungs of `structured_llm_call`'s degradation chain, which are named after the code (`native_fc`, `json_mode`, `plain_text`). FIM One's Level 2 is `json_object`, which is **T3**, not T2. The chain has three rungs because it skips T2 entirely, not because there are only three tiers.
### Where FIM One sits
| `structured_llm_call` level | Tier | What the code actually does |
| - | - | - |
| Level 1 — `native_fc` | **T1 without `strict`** | `_build_tool_def` emits `name` / `description` / `parameters` and nothing else. No call site sets `strict`, so `convert_tools` in `responses_adapter.py` defaults it to `False`. The schema reaches the model as a description of what to produce, not as a constraint on what it can produce |
| *(absent)* | **T2** | Not implemented. `json_schema` appears nowhere in the codebase, and `abilities` has no flag that could express per-model support for it |
| Level 2 — `json_mode` | **T3** | `{"type": "json_object"}`, hardcoded in `structured.py::_call_llm` and `react.py::_json_response_format` |
| Level 3 — `plain_text` | **T4** | `extract_json()` over free-form content, with an optional per-call-site regex fallback |
Five consequences follow, and they are the honest limitations of the current design.
1. **The chain has no schema-gated rung to fall back to.** When Level 1 fails, the next rung down is `json_object`, which guarantees only that the text parses. There is no intermediate step that still enforces fields.
2. **Level 1's guarantee is weaker than its name suggests.** Without `strict`, native function calling is the pre-2024 non-strict variety: the model usually honours the schema and is not prevented from omitting a required field, inventing a key, or returning a string where a number was declared.
3. **On `anthropic/` routes, Level 2 is not really T3.** The Anthropic Messages API has no `response_format`, so LiteLLM emulates JSON mode by injecting an assistant prefill. That is a prompt-layer device, which puts the effective guarantee between T4 and T3 rather than at T3. The Bedrock prefill trap documented below is this emulation failing loudly; the quieter cost is that the guarantee was never the one the parameter name implies.
4. **Nothing validates the result against the schema afterwards.** `jsonschema` is not a dependency. Whatever checking happens is per-call-site, inside the optional `parse_fn`, and its rigour varies by call site.
5. **Failure is absorbed rather than surfaced.** Nearly every caller passes `default_value`, so an exhausted chain returns a plausible-looking object instead of raising. `StructuredCallResult.level_used` records which rung produced the value, but no call site reads it, and on the `default_value` path it reports `plain_text` for a chain that in fact produced nothing.
**How to see which rung you are actually on.** `structured_llm_call` logs one line per completed call at `INFO`, emitted only after the data has survived `parse_fn`, so it names a rung that genuinely succeeded:
```
structured_llm_call: fn=classify_mode level=native_fc attempt=first calls=1
structured_llm_call: fn=annotate_tables level=json_mode attempt=reformat_retry calls=3
```
Total failure logs at `WARNING` with `level=none outcome=default_value`. Grepping these two lines over a day of traffic is the only way to tell whether the missing T2 rung costs anything in a given deployment, since the `default_value` design means the symptom is a mediocre answer rather than an error.
### Provider support for each tier
**This table documents upstream APIs, not FIM One code paths.** Every other table on this page names the function that implements the behaviour. This one cannot, because FIM One uses no T2 feature from any provider and sets `strict` on none of them. It is here so that a decision to implement T2 starts from what is actually available. Vendor capability moves quickly and support often lands on some models in a family before others; re-check the vendor's own documentation before relying on a row.
| Provider | T1 `strict` | T2 `json_schema` | T3 `json_object` | Notes |
| - | - | - | - | - |
| **OpenAI** | ✅ | ✅ | ✅ | The reference implementation of both strict tiers. Strict function calling and `response_format: json_schema` share one grammar engine and one schema subset. `json_object` is the older JSON mode, kept for compatibility |
| **Anthropic** | ⚠️ | ⚠️ | ❌ | No `response_format` on the Messages API at all; tool `input_schema` is validated rather than decoder-enforced. Strict tool use and a schema-constrained output format exist behind a beta header on recent Claude models. FIM One reaches Anthropic through LiteLLM, which emulates JSON mode by prefill and does not use the beta, so none of this is live here |
| **Gemini** | ✅ | ✅ | ✅ | `responseSchema` (an OpenAPI 3.0 subset) and the later `responseJsonSchema` are decoder-enforced, and the OpenAI-compatible endpoint accepts `response_format: json_schema`. Forced calling is `ANY` with `allowed_function_names` |
| **xAI (Grok)** | ✅ | ✅ | ✅ | OpenAI-compatible surface including strict structured outputs |
| **Mistral** | ✅ | ✅ | ✅ | Custom structured outputs accept a user-supplied JSON Schema |
| **DeepSeek** | ❌ | ❌ | ✅ | JSON output is `json_object` only. The documented requirement is that the prompt contain the word "json" and show an example, which is a T4 device attached to a T3 parameter. `deepseek-reasoner` additionally rejects a forced tool choice, so its schema-gated path is closed at both ends |
| **Qwen** (DashScope) | ⚠️ | ⚠️ | ✅ | `json_object` is available on the compatibility endpoint. Schema-constrained decoding is reliable on self-hosted Qwen through the serving stack rather than the API; DashScope's own `json_schema` support varies by model generation. Verify per model |
| **GLM** (Zhipu / Z.AI) | ❌ | ⚠️ | ✅ | `json_object` on the OpenAI-compatible endpoint. Forced tool choice is unsupported outright, which is why `tool_choice_enabled=false` is the recommended setting in Table B. Treat `json_schema` as unavailable unless the platform documents it for your model |
| **Kimi** (Moonshot) | ⚠️ | ⚠️ | ✅ | `json_object` is documented. Thinking variants accept only `auto` tool choice, so on those the T1 path cannot be forced regardless of `strict` support |
| **MiniMax** | ⚠️ | ⚠️ | ✅ | Function calling works with a forced choice even with thinking always on, which makes it the most usable T1 route among the always-thinking models. Schema-level enforcement is not documented; assume T1 is non-strict |
| **Doubao** (Volcengine Ark) | ⚠️ | ⚠️ | ✅ | Ark documents structured output for parts of the Doubao line. Coverage is per model, so confirm before assuming the account's model has it |
| **Ollama / vLLM / llama.cpp** | ✅ | ✅ | ✅ | The strongest tier support of anything on this list, and the least advertised. Grammar-constrained decoding (GBNF, XGrammar, Outlines) runs locally, so a self-hosted 14B can be given a guarantee that several hosted APIs above cannot offer. Exposed as Ollama's `format`, vLLM's guided decoding parameters, or an OpenAI-compatible `response_format` depending on the server |
Three patterns are worth naming, because they explain the shape of the table rather than just restating it.
**T3 is universal, T2 is not.** Every provider FIM One supports offers `json_object`, and roughly half offer a schema tier. A portable structured-output path therefore has to be built on T3 or below, which is the reason FIM One's chain looks the way it does. Adding T2 means adding a per-model capability flag, not a global switch.
**Always-on thinking tends to close the T1 door.** GLM, Kimi's thinking models, and `deepseek-reasoner` all restrict or reject a forced tool choice, and Anthropic rejects it while thinking is active. MiniMax is the counterexample. Where the door is closed, T2 is the only remaining schema-gated option, which is why the missing rung matters more on a Chinese-model deployment than on an OpenAI one.
**Self-hosting inverts the usual ordering.** Constrained decoding is a property of the serving stack, so it is available on a local checkpoint whose instruction-following is far weaker than a frontier model's. The models that need schema gating most are the ones most able to get it.
## The Bedrock prefill trap
When `response_format={"type":"json_object"}` is passed for a model resolved with the `anthropic/` prefix, LiteLLM internally injects an assistant prefill message to simulate JSON mode. The Anthropic Messages API has no native `response_format` parameter, so LiteLLM approximates it by prepending an opening brace as assistant content:
```json theme={null}
{"role": "assistant", "content": "{"}
```
This works on Anthropic's direct API. However, newer AWS Bedrock model versions reject any conversation whose last message has `role: "assistant"` — they call this "assistant message prefill" and throw:
```
ValidationException: This model does not support assistant message prefill.
The conversation must end with a user message.
```
This error occurs only when **all three conditions** are met simultaneously:
1. The model is resolved with the `anthropic/` prefix (via domain match or URL path hint).
2. `response_format={"type":"json_object"}` is passed (the json\_mode code path in `structured_llm_call`).
3. The actual backend is AWS Bedrock (which rejects prefill).
**Bedrock via OpenAI-compatible endpoint?** If your Bedrock relay exposes an OpenAI-compatible `/v1/chat/completions` endpoint (either AWS's own OpenAI-compatible gateway or a third-party proxy), and the URL path does **not** contain `/claude` or `/anthropic`, FIM One resolves it with the `openai/` prefix. LiteLLM then treats the backend as a standard OpenAI-compatible server, passes `response_format` directly without injecting any prefill, and the server handles JSON constraining natively. **The prefill trap does not apply** — you do not need to set `json_mode_enabled=false`.
This does NOT affect native tool calling (`tool_choice="auto"` with `tools=` parameter). The prefill injection only happens for `response_format`. ReAct agent execution is completely unaffected.
If both Level 1 (native\_fc) and Level 2 (json\_mode) fail on Bedrock, the system recovers at Level 3 (plain\_text). The `json_mode_enabled` flag described below eliminates the wasted Level 2 call.
### The fix: json\_mode\_enabled
A per-model `json_mode_enabled` flag controls whether Level 2 (json\_mode) is ever attempted:
* **DB-configured models**: toggle in Admin → Models → Advanced settings. The flag is stored on `ModelProviderModel.json_mode_enabled` (default `TRUE`).
* **ENV-configured models**: set `LLM_JSON_MODE_ENABLED=false` in your environment.
* **Effect**: when disabled, `abilities["json_mode"]` returns `False` → `response_format` is never passed → no prefill → Bedrock works. The degradation chain becomes `native_fc → plain_text`, skipping the doomed json\_mode call entirely.
* **Cost of the skip**: the model still returns valid JSON in practice, because the system prompt asks for it and `extract_json()` parses free-form content reliably on modern models. What is lost is the guarantee rather than the output: the chain now ends at T4, where nothing but the prompt constrains the result. On an `anthropic/` route that loss is smaller than it looks, since the emulated JSON mode was never a decoder-level guarantee either.
## Thinking models + forced tool\_choice
Several providers reject a forced `tool_choice` while extended thinking is active, on the grounds that pinning a specific function call contradicts the model's freedom to reason first:
```
tool_choice 'specified' is incompatible with thinking enabled
```
**This is a per-provider rule, not a law of thinking models.** Anthropic enforces it at the protocol level and Moonshot (Kimi) behaves the same way, but MiniMax thinks on every call and still accepts a forced tool choice. Table B in the [Provider Capability Matrix](#provider-capability-matrix) records the verdict provider by provider; do not generalise from one row to another.
For Anthropic models, `structured_llm_call` resolves the conflict on its own by passing `reasoning_effort=None` on the native-FC level, which turns thinking off for that one call (`structured.py::_call_llm`). Structured output needs **schema compliance**, not deep reasoning, so disabling thinking there is both correct and cheaper.
Where thinking cannot be switched off through the API, native\_fc fails with a 400 on every structured call and costs roughly ten seconds before the chain falls through to json\_mode. Kimi is the common case: with thinking on only `auto` is supported, and a forced tool choice requires turning thinking off, which Moonshot exposes only through the model id (`kimi-k2` has it off, `kimi-k2.5` and `kimi-k2-thinking` have it on). FIM One has no parameter that flips it, so the remedy is the `tool_choice_enabled` flag below.
### The fix: tool\_choice\_enabled
A per-model `tool_choice_enabled` flag controls whether Level 1 (native\_fc) is ever attempted:
* **DB-configured models**: toggle in Admin → Models → Advanced → "Native Function Calling". The flag is stored on `ModelProviderModel.tool_choice_enabled` (default `TRUE`).
* **ENV-configured models**: set `LLM_TOOL_CHOICE_ENABLED=false` in your environment.
* **Effect**: when disabled, `abilities["tool_choice"]` returns `False` → the degradation chain starts from Level 2 (json\_mode) or Level 3 (plain\_text), skipping native\_fc entirely. This eliminates the \~10s penalty per structured call for incompatible models.
* **ReAct agent unaffected**: `tool_choice_enabled` only controls forced tool selection in `structured_llm_call`. The ReAct engine uses `tool_choice="auto"` (model freely decides), which works with all models regardless of this setting.
`tool_choice_enabled` and `tool_call` are separate ability flags. `tool_call` (always `True` for `OpenAICompatibleLLM`) gates whether tools are passed to the model at all — disabling it would break the ReAct agent. `tool_choice` only gates whether **forced** tool selection is attempted for structured output extraction.
`tool_choice="auto"` is unaffected by thinking mode. The ReAct engine uses `"auto"` exclusively, so agent execution works with thinking enabled.
Do NOT set `abilities["tool_call"] = False` to avoid this constraint. That would disable ReAct's `_run_native` mode (which uses `tool_choice="auto"` and works fine with thinking), forcing it into the less reliable `_run_json` mode.
**Provider migration note:** Some third-party relays silently drop unsupported parameters like `reasoning_effort` (`drop_params=True`), so thinking is never activated even when configured. When migrating to a provider that properly supports thinking (Bedrock, direct Anthropic API), the `reasoning_effort=None` in native\_fc ensures consistent behavior. No user action is needed — structured output works identically across all providers.
## Provider Capability Matrix
This section is the authoritative record of what each provider supports and what FIM One does about it. Every row names the function that implements the behaviour, so any claim here can be checked against the code. Other pages link here instead of repeating the data; when the code changes, this section changes with it.
A row describes a provider's protocol, not a single model. Where models inside one family differ (DeepSeek chat against reasoner, Kimi with thinking on against off), the cell says so.
### Table A: Protocol routing
How a configured `base_url` plus `model` becomes a LiteLLM call, and what happens when the first choice of interface is unavailable.
| Provider | Detected by | LiteLLM prefix | Interface surface | Downgrade chain | Code anchor |
| - | - | - | - | - | - |
| **OpenAI** | Domain `api.openai.com`, or an explicit `provider` of `openai` on the model config | `openai/` | `completions`. GPT-5.x uses `responses-native` (`litellm.aresponses`), with `responses-bridge` (`openai/responses/`) retained as a rollback | GPT-5.x talks Responses natively, falls back to `completions` on a 404 (cached per endpoint and model in `_RESPONSES_NATIVE_SUPPORT`) or on a 400 (not cached). `gpt-6.1-*` and `gpt-6-astra` keep a 400 on a request with tools instead of falling back, since their `completions` rejects tools. Every other model goes straight to `completions` | `_resolve_litellm_model`, `_should_use_native_responses`, `_dispatch_acompletion` |
| **Anthropic** (Bedrock note below) | Domain `anthropic.com`, path segment `/claude` or `/anthropic`, or an explicit `provider` of `anthropic` | `anthropic/` | `anthropic messages` | None. Native routes never enter the Responses bridge; their protocol is built in `_build_request_kwargs` | `_resolve_litellm_model`, `_dispatch_acompletion` |
| **Gemini** | Domain `generativelanguage.googleapis.com`, path segment `/gemini`, or an explicit `provider` of `gemini` | `gemini/` | `gemini` | None. A domain match also drops `api_base`, so an OpenAI-compat suffix such as `/v1beta/openai/` is ignored and the call goes to the native Gemini API | `_resolve_litellm_model` |
| **xAI (Grok)** | No domain or path entry. Resolves as generic OpenAI-compatible unless `provider` is set explicitly on the model config | `openai/` with `api_base`, or the explicit provider prefix | `completions` | None | `_resolve_litellm_model` |
| **DeepSeek** | Domain `api.deepseek.com` | `deepseek/` | `completions` | None | `_resolve_litellm_model` |
| **Qwen** (DashScope) | Generic fallback | `openai/` with `api_base` | `completions` | None | `_resolve_litellm_model` |
| **GLM** (Zhipu, Z.AI) | Generic fallback | `openai/` with `api_base` | `completions` | None | `_resolve_litellm_model` |
| **MiniMax** | Generic fallback | `openai/` with `api_base` | `completions` | None | `_resolve_litellm_model` |
| **Kimi** (Moonshot) | Generic fallback | `openai/` with `api_base` | `completions` | None | `_resolve_litellm_model` |
| **Doubao** (Volcengine) | Generic fallback | `openai/` with `api_base` | `completions` | None | `_resolve_litellm_model` |
| **Mistral** | Domain `api.mistral.ai` | `mistral/` | `completions` | None | `_resolve_litellm_model` |
| **Ollama / local** | Generic fallback, typically `http://localhost:11434/v1` | `openai/` with `api_base` | `completions` | None | `_resolve_litellm_model` |
| **Relay / proxy** | Path hint first, then the generic fallback. An explicit `provider` on the model config outranks both | The hinted provider prefix, else `openai/`, always with `api_base` pointing at the relay | Whatever the resolved prefix implies | None beyond the resolved prefix. See the relay gotchas below | `_resolve_litellm_model` |
**How GPT-5.x picks a protocol.** `FIM_GPT5_RESPONSES_MODE` selects it: `native` (the default) talks `/v1/responses` directly through `litellm.aresponses`, `bridge` uses LiteLLM's chat-completions translation, and `off` forces plain chat completions. The native path exists because the bridge is lossy in the one place that matters: it discards the reasoning items, so a GPT-5.x agent re-derives its chain of thought on every tool round. Talking the protocol directly lets those items be replayed. A call that explicitly passes `reasoning_effort=None`, which is what `structured_llm_call` and the finish-signal probes do, stays on chat completions, because a call that wants no thinking has no reasoning state to preserve. The exception is models that cannot switch reasoning off (`gpt-6.1-*`, `gpt-6-astra`): their chat completions reject function tools at any `reasoning_effort`, so those calls stay on `/v1/responses` and run at the lowest effort, `low`. For the same reason `off`, or an endpoint without `/v1/responses`, leaves these models without tool calls.
Two properties of that native request are load-bearing and easy to get wrong:
* `store=false` keeps the conversation stateless upstream, and `include=["reasoning.encrypted_content"]` asks for the encrypted payload to be returned. Without the include, the reasoning items arrive empty and replay silently becomes a no-op.
* A replayed reasoning item must have its server-side `id` stripped. With `store=false` nothing is persisted upstream, so echoing the id back earns `Item with id 'rs_...' not found. Items are not persisted when 'store' is set to false`. The `encrypted_content` blob carries the state on its own, so dropping the id costs nothing (`sanitize_reasoning_item`).
**Bedrock.** Bedrock-hosted Claude follows the row it resolves to, not the fact that Bedrock is hosting it. Reached through an `anthropic/`-routed relay it inherits Anthropic protocol behaviour, including LiteLLM's json-mode assistant prefill, which newer Bedrock versions reject. Reached through an OpenAI-compatible gateway it resolves as `openai/`, no prefill is injected, and `json_mode_enabled` can stay on.
### Table B: Conflicts and workarounds
FIM One emits three of the four `tool_choice` states: `auto` from the ReAct loop (`react.py::_run_native`), a named function from structured output (`structured.py::_call_llm`), and `none` when the finish-signal answer replays the tools payload. No call site emits `required`; that column records the provider's constraint on non-`auto` tool choice, which applies to `required` and to a named function alike.
| Provider | `auto` | `required` | Named function | `none` | With thinking on | FIM One's workaround | Code anchor |
| - | - | - | - | - | - | - | - |
| **OpenAI** | ✅ | ✅ | ✅ | ✅ | GPT-5.x on chat completions rejects tools combined with reasoning | Runs GPT-5.x on Responses, where both are allowed. On the chat-completions fallback it sends an explicit `reasoning_effort` of `none` whenever tools are present | `_should_use_native_responses`, `_build_request_kwargs` |
| **Anthropic** | ✅ | ⚠️ | ⚠️ | ✅ | Accepted while thinking is off, rejected with a 400 while it is on. `auto` is unaffected either way | `structured_llm_call` passes `reasoning_effort=None` on the native-FC level, so thinking is off for that one call. ReAct keeps `auto` and keeps thinking | `structured.py::_call_llm` |
| **Gemini** | ✅ | ✅ | ✅ | ✅ | No conflict | Defaults: `tool_choice_enabled` and `json_mode_enabled` both on | `OpenAICompatibleLLM.abilities` |
| **xAI (Grok)** | ✅ | ✅ | ✅ | ✅ | Reasoning variants accept tools | Defaults, both on | `OpenAICompatibleLLM.abilities` |
| **DeepSeek** | ✅ | ⚠️ | ⚠️ | ✅ | `deepseek-chat` (V3.2, non-thinking) accepts a forced tool choice. `deepseek-reasoner` (V3.2 thinking mode) rejects it | Set `tool_choice_enabled=false` on `deepseek-reasoner` only; leave it on for `deepseek-chat` | `OpenAICompatibleLLM.abilities` |
| **Qwen** | ✅ | ✅ | ✅ | ✅ | `enable_thinking` is a provider-side switch that FIM One never sends, so thinking follows the model default | Defaults, both on | `_build_request_kwargs` |
| **GLM** | ✅ | ❌ | ❌ | ✅ | Forced tool choice is unsupported whether or not the model thinks | Set `tool_choice_enabled=false` | `OpenAICompatibleLLM.abilities` |
| **MiniMax** | ✅ | ✅ | ✅ | ✅ | Thinking is always on and a forced tool choice still works. This is the counterexample to the "always-on thinking rejects forced tools" rule | Defaults, both on. Thinking arrives as `` tags and is rerouted to the reasoning stream | `_ThinkTagStreamParser` |
| **Kimi** (Moonshot) | ✅ | ⚠️ | ⚠️ | ✅ | With thinking on only `auto` is supported; a forced tool choice requires turning thinking off. `kimi-k2` has it off, `kimi-k2.5` and `kimi-k2-thinking` have it on | No API parameter flips Moonshot thinking, so set `tool_choice_enabled=false` on the thinking models | `OpenAICompatibleLLM.abilities` |
| **Doubao** | ✅ | ✅ | ✅ | ✅ | Accepts `reasoning_effort` alongside tools | Defaults, both on | `_build_request_kwargs` |
| **Mistral** | ✅ | ✅ | ✅ | ✅ | No thinking mode | Defaults, both on | `OpenAICompatibleLLM.abilities` |
| **Ollama / local** | ⚠️ varies | ⚠️ varies | ⚠️ varies | ⚠️ varies | Depends entirely on the checkpoint | 14B parameters is the floor for usable tool calls and 32B is the practical target. Turn both flags off for smaller models so structured output goes straight to plain text | `OpenAICompatibleLLM.abilities` |
| **Relay / proxy** | Inherits upstream | Inherits upstream | Inherits upstream | Inherits upstream | Inherits upstream, and an unsupported parameter is dropped rather than rejected (`litellm.drop_params=True`) | Per-model flags, plus the relay gotchas below | `_build_request_kwargs` |
### Table C: Thinking protocol
`LLM_REASONING_EFFORT` accepts `low`, `medium` and `high`; any other value is read as unset (`deps.py::_reasoning_effort`). What FIM One then puts on the wire is per-provider, and that is what this table records. The replay column is the return value of `reasoning_replay_policy`, which is a small closed set of four states rather than a per-provider list.
| Provider | Thinking enabled by | Accepted `effort` values | Replay policy | Where the output lands | Code anchor |
| - | - | - | - | - | - |
| **OpenAI GPT-5.x** | `reasoning` of `{effort, summary: "auto"}` on Responses; `reasoning_effort` on chat completions | `low`, `medium`, `high` from config, plus a forced `none` when tools are present on chat completions | `openai_responses`: the opaque reasoning items are replayed verbatim on Responses, and the readable text is dropped on chat completions exactly as `informational_only` would | Encrypted items plus a readable summary, rendered in the Reasoning panel. Chat completions returns no reasoning text | `_build_responses_kwargs`, `reasoning_replay_policy` |
| **OpenAI o-series** | Always on | `reasoning_effort` passed through unchanged | `informational_only`, matched on the `o1`, `o3` and `o4` fragments | Internal; the tokens are billed but not returned | `reasoning_replay_policy` |
| **Anthropic, adaptive** (Opus 4.6 / 4.7 / 4.8, Sonnet 4.6, Fable 5, Mythos 5) | `thinking` of type `adaptive` plus `output_config.effort` | `low`, `medium`, `high` | `anthropic_thinking`: the block and its `signature` are replayed verbatim or the API rejects the turn | `reasoning_content` plus `signature`, rendered in the Reasoning panel | `_uses_adaptive_thinking`, `_extract_thinking_signature` |
| **Anthropic, legacy** (4.5 and older) | `thinking` of type `enabled` with `budget_tokens` when `LLM_REASONING_BUDGET_TOKENS` is set, otherwise `reasoning_effort` handed to LiteLLM | `low`, `medium`, `high`, or an explicit token budget with a floor of 1024 | `anthropic_thinking` | Same as adaptive | `_build_request_kwargs` |
| **Gemini** | `reasoning_effort` on the compatibility endpoint | `low`, `medium`, `high` | `informational_only` for ids carrying a `flash-thinking` fragment. Other Gemini ids resolve to `unsupported`, which drops the field just the same | Internal | `reasoning_replay_policy` |
| **xAI (Grok)** | `reasoning_effort` through LiteLLM | Provider-defined | `informational_only`, because the generic `reasoning` fragment matches any id containing the word | Internal | `reasoning_replay_policy` |
| **DeepSeek** | Model id: `deepseek-reasoner` for V3.2 thinking mode, `deepseek-chat` for non-thinking | None. There is no effort parameter | `informational_only` | `reasoning_content` field, rendered in the Reasoning panel | `_parse_choice_message` |
| **Qwen** | `enable_thinking`, provider-side. FIM One does not send it | Not applicable | `informational_only` for `qwq` ids, `unsupported` otherwise | `` tags inside content, rerouted to the reasoning stream | `_ThinkTagStreamParser` |
| **GLM** | Built into `glm-5`; no API toggle | Not applicable | `unsupported` | Not externalised | `reasoning_replay_policy` |
| **MiniMax** | Always on; no toggle | Not applicable | `unsupported` | `` tags inside content, rerouted | `_ThinkTagStreamParser`, `_THINK_RE` |
| **Kimi** (Moonshot) | Model id: `kimi-k2-thinking`, and `kimi-k2.5` thinks by default | Not applicable | `unsupported` | An API reasoning field, read as `reasoning_content` or `reasoning` | `_parse_choice_message` |
| **Doubao** | `reasoning_effort` | The provider documents `minimal`, `low`, `medium` and `high`; FIM One emits only the middle three | `unsupported` | Internal | `_build_request_kwargs` |
| **Mistral** | No thinking mode | Not applicable | `unsupported` | Not applicable | `reasoning_replay_policy` |
| **Ollama / local** | Model-dependent | Not applicable | `informational_only` for `deepseek-r1` distills and `qwq` builds, `unsupported` otherwise | `` tags where the checkpoint emits them | `_ThinkTagStreamParser` |
| **Relay / proxy** | Whatever the upstream accepts | Whatever the upstream accepts | Resolved from the model id exactly as on a direct route | Depends on the upstream. A Claude adaptive-thinking model behind a generic `openai/` relay never gets thinking at all, and the constructor logs a warning saying so | `OpenAICompatibleLLM.__init__` |
`unsupported` and `informational_only` produce the same bytes on the wire: both strip `reasoning_content` and `signature` from outgoing history. They differ in intent, so a model that clearly does reason but lands in `unsupported` is a gap in the fragment table rather than a live bug.
### Relay/proxy gotchas
Third-party gateways fail in ways a direct provider does not, and most of those failures are silent. Each row below pairs the symptom with its mechanism and with what FIM One already does about it.
**Support boundary.** FIM One guarantees the behaviour documented on this page for first-party endpoints: OpenAI's own API, Anthropic, Google, and any vendor serving its own models directly. Third-party relays are supported on a best-effort basis and are not covered by that guarantee, because what a relay does to a request is outside our control and frequently outside its own documentation. A relay can drop a parameter, rewrite history, strip a cache breakpoint, or answer a protocol it only partially implements, and in most of those cases it returns a `200` rather than an error.
This is a statement about what we promise, not a restriction on what runs. FIM One does not maintain an allowlist of approved hosts, and nothing here is gated on a domain. Capability is decided by what an endpoint actually does: a missing route answers `404` and is remembered, an ignored `include` yields empty reasoning items and the replay becomes a no-op, and a rejected request falls back for that call. Probing the endpoint is more accurate than inferring its capabilities from its hostname, and it is the only approach that keeps working for Azure OpenAI, enterprise gateways, and self-hosted proxies that implement the protocol correctly.
If a relay misbehaves in a way the fallbacks do not catch, pin the protocol yourself with `FIM_GPT5_RESPONSES_MODE` (`bridge` or `off`) or the per-model `tool_choice_enabled` and `json_mode_enabled` toggles, and reproduce against the first-party endpoint before filing it as a FIM One bug.
| Symptom | Mechanism | What FIM One does |
| - | - | - |
| Thinking is configured but never appears | The relay path has no `/claude` hint, so the model resolves as `openai/`. The Chat Completions schema has no thinking concept, so the parameter is dropped before the request leaves the process | Logs a warning at construction naming the model and the resolved prefix, and telling you to set `provider` or use an Anthropic base URL (`OpenAICompatibleLLM.__init__`) |
| A parameter appears to be accepted and has no effect | `litellm.drop_params=True` removes anything the resolved provider does not declare, silently and per-parameter | Deliberate. It keeps one request builder working across every provider. The cost is that "no error" is not evidence the parameter arrived |
| First token takes minutes on a non-OpenAI model, or the agent returns text and makes no tool calls | The relay advertises `/v1/responses` for a model that is not OpenAI's, accepts the request, then buffers the whole answer before replaying it. Observed on Uniapi with Claude at roughly four minutes to first token, and in a second reproduction the call returned normally but the agent then made zero tool calls. Nothing errors, so an error-triggered fallback never fires | The bridge is benefit-gated to GPT-5.x, the one family that gains capability from Responses. Everything else goes straight to chat completions and never probes (`_dispatch_acompletion`, commit `137ede4c`) |
| `ValidationException: This model does not support assistant message prefill` | json\_mode on an `anthropic/`-routed Bedrock relay. LiteLLM simulates `response_format` by prefilling an opening brace as an assistant message, and newer Bedrock versions reject a conversation ending in an assistant turn | Set `json_mode_enabled=false` for that model, or route through an OpenAI-compatible gateway where no prefill is injected |
| `404` on every call to a Zhipu endpoint | The client appends an OpenAI-style `/v1` to a base URL that already ends in `/v4` | Configure the base URL exactly as the provider documents it. FIM One passes `api_base` through unchanged |
| `APIConnectionError: Connection error` after a quiet period | An intermediary reaped an idle pooled connection without sending FIN or RST, and httpx handed back the half-dead socket on the next write | Keep-alive expiry defaults to 5 seconds so cross-turn idle connections are discarded rather than reused. Set `LLM_HTTP_MAX_KEEPALIVE=0` to disable reuse entirely (`_get_shared_http_client`) |
| `Cannot send a request, as the client has been closed` | LiteLLM evicted a cached SDK client on its idle TTL, and the OpenAI SDK closed the shared httpx session that client was holding | The pool is re-validated before every attempt and rebuilt when closed, and LiteLLM's stale client cache is flushed alongside it (`_get_shared_http_client`, `_flush_litellm_client_cache`) |
| Billed input tokens do not match the reported cache reads | The relay strips `cache_control` before forwarding, so you pay full price while the response still reports cache counters | `TurnProfiler` logs `read_tokens` and `create_tokens` per turn, which doubles as a relay honesty probe. Compare it against the invoice |
| `Function tools with reasoning_effort are not supported ... Please use /v1/responses instead` | The relay guards chat completions on the presence of the `reasoning_effort` field, not on its value, so the explicit `none` FIM One sends to disable reasoning trips the guard too. Observed on Uniapi with `gpt-5.6-luna` | Nothing, and nothing is needed while the Responses path is working: that model only reaches chat completions after a Responses request has already failed. Read it as a sign the relay wants Responses, not as a reason to set `FIM_GPT5_RESPONSES_MODE=off` |
| GPT-5.x stays on chat completions on an endpoint that does support Responses | A `404` was cached as a negative verdict for that endpoint and model | Only a `404` is cached, because a missing route is structural. A `400` falls back for that one call and is deliberately not cached, so a single stale reasoning item cannot blacklist the endpoint permanently (`_remember_native_failure`). The cache is per process, so a restart re-probes either way |
| Thinking blocks are rejected or the prefix cache never hits | The relay rewrites or reorders history, so the replayed `signature` no longer matches | Replay is decided centrally by `reasoning_replay_policy`, and only Anthropic-family ids replay at all. If a Claude model behind a relay carries an unrecognisable id, add its fragment to the policy table |
## Recommended per-model configuration
Both `tool_choice_enabled` and `json_mode_enabled` can be toggled per model in Admin → Models → Advanced settings. The defaults, both `TRUE`, are correct for most providers; only adjust when you see errors or wasted latency. Which providers need an adjustment is recorded in Table B above, and the per-model view an operator fills in lives in [Model Management](/configuration/model-management#per-provider-configuration-matrix).
**When to change:** if you see `structured_llm_call: native_fc call raised` warnings in your logs followed by successful json\_mode extraction, the model does not benefit from native\_fc. Disable "Native Function Calling" for that model to eliminate the wasted API call (\~10s per structured output request).
**ENV-level overrides** apply to all models configured via environment variables (not admin UI):
```bash theme={null}
# Disable native_fc globally (for thinking-model-only deployments)
LLM_TOOL_CHOICE_ENABLED=false
# Disable json_mode globally (for Bedrock relay deployments)
LLM_JSON_MODE_ENABLED=false
```
## Reasoning effort and thinking configuration
FIM One exposes two env vars for controlling extended thinking / reasoning:
| Variable | Values | Effect |
| - | - | - |
| `LLM_REASONING_EFFORT` | `low`, `medium`, `high` | Turns thinking on. Anything outside this set is read as unset (`deps.py::_reasoning_effort`). How the level is translated, and which values the provider itself accepts, is per-provider: see Table C in the [Provider Capability Matrix](#provider-capability-matrix). |
| `LLM_REASONING_BUDGET_TOKENS` | integer (e.g. `10000`) | Anthropic legacy path only: sets an explicit `thinking.budget_tokens` cap on models that still take the `enabled` form, bypassing LiteLLM's auto-mapping. Adaptive-thinking models ignore it in favour of `output_config.effort`. |
Two behaviours follow automatically once thinking is on, and neither needs user configuration:
1. **Temperature is handled for you.** On an `anthropic/` route with thinking active, `_build_request_kwargs` pins `temperature` to 1.0, which is what Bedrock demands. Models that reject sampling parameters outright (Opus 4.7 and 4.8, Fable 5, Mythos 5) have `temperature` removed from the request entirely, thinking or not. Do not set `LLM_TEMPERATURE=1` by hand for this.
2. **GPT-5.x keeps tools and reasoning together where it can.** FIM One probes the Responses bridge first for GPT-5.x, because that is the only surface where the two combine. An endpoint with no usable `/v1/responses` route falls back to chat completions, the verdict is cached per endpoint, and on that path a request carrying `tools` sends an explicit `reasoning_effort` of `none`. Omitting the field is not equivalent, since the server default is not `none`.
## Defensive parsing for structured output
Even with native\_fc working correctly, the structured output pipeline includes a defensive parsing layer to handle edge cases from any provider or compatibility layer.
The DAG planner's `_dict_to_steps` parser handles three common edge cases:
1. **Single object instead of array.** Some models return `{"steps": {"id": "1", "task": "..."}}` (a single step object) instead of `{"steps": [{"id": "1", "task": "..."}]}` (an array). The parser detects this by checking for `id` or `task` keys and wraps the object in a list.
2. **Double-encoded JSON string.** When structured output falls through to json\_mode (which lacks schema enforcement), some providers return the `steps` value as a JSON string rather than a native array — e.g., `{"steps": "[{\"id\": \"1\", ...}]"}`. This string may also contain literal newlines (from the model's formatting) that break standard `json.loads`. The parser uses `extract_json_value()` (which includes `_repair_json_strings`) to handle:
* Literal newlines inside JSON string values
* Invalid escape sequences (common with LaTeX or code content)
* Other serialization quirks from compatibility layers
3. **Missing `steps` wrapper.** The model may return a single step as the top-level object without the `steps` wrapper key. The parser detects `id` and `task` at the root level and wraps accordingly.
Under normal operation, native\_fc returns properly structured tool call arguments and these edge cases do not arise. The defensive parsers exist as a safety net for custom `BaseLLM` subclasses, unusual provider behaviors, or fallback scenarios where structured output degrades to json\_mode or plain\_text.
## Prompt caching (cross-provider)
FIM One implements Anthropic's explicit prompt caching via `cache_control` breakpoints and simultaneously benefits every other provider's automatic prefix caching through the **Prompt Section Registry**. The goal is a single prompt-assembly path that works across all providers without per-call prompt shape divergence.
### Architecture
The `fim_one.core.prompt` module exposes three primitives:
* **`PromptSection`** — a named fragment with either a static `content: str` or a dynamic `content: Callable`
* **`PromptRegistry`** — a memoized store (static sections render once, dynamic sections re-render per call)
* **`DYNAMIC_BOUNDARY`** — a sentinel marker the registry inserts between the last static section and the first dynamic one, so callers can split the rendered prompt at the cache breakpoint
System prompts for ReAct (JSON mode, native function-calling mode, synthesis) are split into:
* **Static prefix** (\~95% of the prompt) — identity, core guidelines, tool descriptions
* **Dynamic suffix** — current datetime, per-request language directive, handoff context
### Capability detection
`fim_one.core.prompt.caching.is_cache_capable(model_id)` returns `True` when the model id contains any of: `claude`, `anthropic`, `bedrock/anthropic`, `vertex_ai/claude`. These providers receive **two** `role="system"` messages with `cache_control: {"type": "ephemeral"}` on the first (static) message.
Every other provider receives a **single** concatenated system message with no `cache_control` field — necessary because non-Anthropic endpoints either reject the field or silently drop it, and sending it through some relays causes `400 unknown parameter` errors.
### Cross-provider coverage
| Provider | Mechanism | Read discount | Our handling |
| - | - | - | - |
| **Anthropic Claude** (3, 3.5, 4) | Explicit `cache_control` | 0.10× | Two system messages with ephemeral breakpoint |
| **AWS Bedrock Anthropic** | Passes through Anthropic cache | 0.10× | Same as Anthropic |
| **GCP Vertex AI Claude** | Passes through Anthropic cache | 0.10× | Same as Anthropic |
| **OpenAI GPT / o-series** | Auto prefix hash (≥1024 tokens) | 0.50× | Byte-stable prefix via Section Registry → automatic hit |
| **DeepSeek (v3 / R1)** | Auto disk-backed prefix cache | 0.10× | Same as OpenAI |
| **Moonshot Kimi (K1/K2)** | Auto prefix cache | 0.10×/0.50× | Same |
| **ZhipuAI GLM-4.5+** | Auto long-context cache | 0.20× | Same |
| **Grok (xAI)** | Auto prefix cache | 0.25× | Same |
| **Google Gemini** | Separate `createCachedContent` API | 0.25× | **Not implemented** — a `GeminiCacheAdapter` is parked, not planned |
| **Mistral / Cohere** | No native cache | N/A | N/A |
The `PromptRegistry` benefits every provider with auto prefix caching "for free" — by keeping the static portion byte-identical across calls (current datetime lives in the dynamic suffix, not prefix), every auto-caching provider's hash matches and hits their cache. This is why the Registry is a foundational modelless win even before considering Anthropic-specific `cache_control`.
### Observability
Every `chat/*` response's `done_payload` now includes:
```json theme={null}
"cache": {
"read_tokens": 1067,
"creation_tokens": 0
}
```
`TurnProfiler` emits a structured log line per turn: `turn_cache summary | model=claude-sonnet-4-6 | read_tokens=1067 | create_tokens=0 | saved_input_tokens=961 (~90%)`. This also functions as a **relay honesty probe** — if you route through an API relay, compare actual billed input vs `read_tokens` to detect whether the relay strips `cache_control` or keeps the 0.10× discount.
No dollar estimate is returned at the LLM layer — pricing and relay markup are applied above, so the LLM layer only returns objective token counts.
### Multi-turn cache ROI
Measured on Claude 4 ReAct turns with the default agent prompt:
| Mode | Static prefix tokens | Dynamic suffix tokens | Cache ratio |
| - | - | - | - |
| JSON mode, no tools | \~753 | \~46 | 94.2% |
| JSON mode with \~10 tools | \~1067 | \~46 | 95.9% |
| Native function-calling | \~523 | \~46 | 91.9% |
A 10-iteration ReAct run with 10 tools saves \~8,640 input tokens per turn after the first (9 cache hits × 1067 tokens × 90%). Anthropic charges 1.25× for cache write on the first call, so the breakeven is at the **second** call — single-shot queries do not benefit.
## Reasoning replay policy (modelless correctness)
Extended thinking / reasoning blocks behave differently across providers. A uniform serialization policy breaks both protocol contracts and automatic prefix caches. `fim_one.core.prompt.reasoning.reasoning_replay_policy(model_id)` returns one of four values and gates `ChatMessage.to_openai_dict(replay_policy=...)` in `OpenAICompatibleLLM._build_request_kwargs()`.
### Four policies
* **`anthropic_thinking`** — Claude family (including `anthropic/`, `bedrock/anthropic`, `vertex_ai/claude`). Thinking blocks MUST be replayed with `signature` attached; Anthropic rejects subsequent turns if the signature is missing or altered.
* **`informational_only`** — models that emit CoT but do NOT expect replay: DeepSeek reasoning mode (`deepseek-reasoner` on V3.2, and the older `deepseek-r1` and R1-Distill ids the fragment table still matches), Qwen QwQ, Gemini flash-thinking, OpenAI o1 / o3 / o4. Their documentation explicitly says "do not send `reasoning_content` back in message history". Sending it anyway:
* Violates the provider contract (may start rejecting in future versions)
* **Silently invalidates their automatic prefix cache** — message bytes mutate on every turn, breaking the hash
* **`openai_responses`** — GPT-5.x, matched on the `gpt-5` fragment. Its reasoning state is not text but a sequence of opaque items carrying encrypted payloads, and only `/v1/responses` has a slot for them. On that protocol the items are replayed verbatim, which is what keeps the model's chain of thought alive across tool rounds. The readable summary is still dropped from outgoing requests, so on the chat-completions fallback this behaves exactly like `informational_only`. Checked before the informational fragments, whose generic `reasoning` entry would otherwise swallow proxy-tagged GPT-5 ids.
* **`unsupported`** — the catch-all: models with no reasoning capability (GPT-4o, Gemini 1.5, Mistral, Llama), and reasoning models whose id matches no fragment (GLM, MiniMax, Kimi, Doubao). No field should be replayed either way, so this policy puts the same bytes on the wire as `informational_only`. It is also the safe default for unknown model ids.
The readable `reasoning_content` and the opaque `reasoning_items` are independent fields on `ChatMessage`. `to_openai_dict()` never serialises the items at all, so they are structurally incapable of leaking onto a chat-completions request, whatever the policy says.
### Enforcement
All policy evaluation happens in one place (`_build_request_kwargs`). `ChatMessage.to_openai_dict(replay_policy=None)` preserves the A3 permissive default so uncoordinated callers don't regress. The cross-provider test matrix lives in `tests/test_reasoning_replay_policy.py` with **reverse assertions** proving that non-Anthropic requests do NOT leak `reasoning_content`.
### For users
Both feature and bug behavior is automatic — you don't need to configure anything. Workflow implications:
* **If you switch agents between Claude and DeepSeek in the same conversation**, history is stored with thinking blocks intact; on the next turn, the outgoing message shape adapts per the current model.
* **If you use a proxy / custom `BaseLLM` subclass**, make sure its model id is recognizable (contains one of the fragments) or the default `unsupported` policy will apply — which is safe but means Claude behind an unusual proxy might lose thinking replay. Add the model-id fragment to `_CACHE_CAPABLE_MODEL_FRAGMENTS` (in `core/prompt/caching.py`) and/or the reasoning policy lookup.
## Troubleshooting
**"This model does not support assistant message prefill"**
Bedrock + json\_mode. Two fixes: (1) set `LLM_JSON_MODE_ENABLED=false` or disable JSON Mode in the admin model settings; or (2) if your Bedrock provider offers an OpenAI-compatible `/v1/chat/completions` endpoint, switch to that — FIM One resolves it as `openai/` and the prefill injection never occurs.
**"Thinking may not be enabled when tool\_choice forces tool use"** / **"tool\_choice 'specified' is incompatible with thinking enabled"**
For Anthropic models, `structured_llm_call` disables thinking for native\_fc calls automatically. Where thinking cannot be turned off through the API, such as `kimi-k2.5` and `kimi-k2-thinking` or `deepseek-reasoner`, disable "Native Function Calling" in the model's Advanced settings, or set `LLM_TOOL_CHOICE_ENABLED=false` globally. The degradation chain will skip native\_fc and extract structured output via json\_mode or plain\_text instead. Check Table B of the [Provider Capability Matrix](#provider-capability-matrix) before assuming a thinking model has this problem; MiniMax does not.
**"DAG pipeline failed: LLM 'steps' is not an array"**
The LLM returned the `steps` field as a string or single object instead of an array. This typically means structured output fell through to json\_mode (which lacks schema enforcement). Check the log for `structured_llm_call: level=xxx` — if it shows `json_mode` instead of `native_fc`, native\_fc is failing silently. If using a custom `BaseLLM` subclass, verify it accepts the `reasoning_effort` kwarg.
**ReAct falls back to JSON mode unexpectedly**
Check that the model's `abilities["tool_call"]` is `True`. This is always `True` for `OpenAICompatibleLLM`, but a custom `BaseLLM` subclass might override it. Verify with the model detail endpoint in the admin API.
**structured\_llm\_call exhausts all levels and raises StructuredOutputError**
The model failed to produce parseable JSON at any level. This is rare with modern models. Check: (1) the schema is valid JSON Schema, (2) the model has enough `max_tokens` to produce the full response, (3) the system prompt is not contradicting the schema instructions. The DAG planner and analyzer both provide `default_value` fallbacks, so this error only propagates from call sites that explicitly omit defaults.
# Market
Source: https://docs.one.fim.ai/architecture/market
How the Market organizes and distributes resources — from two-tier classification to unified subscription.
## Overview
The Market is FIM One's resource marketplace. Users publish resources they have built, others discover and subscribe to them, and subscribed resources appear in the subscriber's workspace as if they were their own. The entire system is built on a single architectural insight: **the Market is an organization** — a system-managed shadow org with special trust rules.
This page explains the Market's internal architecture. For a user-facing overview of publishing and subscribing, see [Market (Features)](/concepts/market). For how subscribed resources are loaded into tool sets, see [Agent & Resource Discovery](/architecture/agent-discovery).
## Two-tier classification
The Market organizes resources into two categories based on what the resource does, not how it is implemented.
### Solutions
Solutions are things that **do work for you**. A user subscribes to a Solution and gets a ready-to-use capability.
| Resource Type | What It Does |
| - | - |
| **Agent** | A conversational AI assistant with bound tools, knowledge, and instructions |
| **Skill** | A global SOP (Standard Operating Procedure) that can orchestrate multiple agents via `call_agent` |
| **Workflow** | A DAG-based automation flow with visual editing and deterministic execution |
Solutions may depend on other resources. An Agent might require a specific Connector for its API calls and a Knowledge Base for its retrieval pipeline. The Market handles these dependencies automatically during subscription (see [Dependency resolution](#dependency-resolution) below).
### Components
Components are **reusable building blocks** for developers. They provide capabilities that Solutions consume.
| Resource Type | What It Does |
| - | - |
| **Connector** | An API or database integration adapter definition |
| **MCP Server** | A tool service configuration using the Model Context Protocol |
Components are simpler to subscribe to — they have no internal dependencies, only credential requirements.
### Why Knowledge Bases are not listed independently
Knowledge Bases are not shareable at all — not in the Market, not to an organization. A KB reaches other users only as an internal dependency of a shared Agent: at run time the Agent delegates the owner's content, so the subscriber never holds a KB subscription and never needs to find, evaluate, or manage KBs separately.
The two-tier classification (Solutions vs. Components) is a **display-layer concept**. It is derived from `resource_type` at query time, not stored as a separate field. The underlying subscription mechanism, visibility filter, and review process are identical for all resource types.
## Unified architecture
### Market as a shadow organization
The Market's most important architectural decision is that it is not a separate subsystem. It is an **organization** — a system-managed org with a fixed ID (`MARKET_ORG_ID`), created automatically during platform initialization.
```mermaid theme={null}
flowchart TD
subgraph Platform["FIM One Platform"]
subgraph Orgs["Organizations"]
O1["Acme Corp (user-created org)"]
O2["DevTeam (user-created org)"]
MK["Market (system-managed org)"]
end
VF["build_visibility_filter()"]
RS["ResourceSubscription"]
O1 --> VF
O2 --> VF
MK --> RS
RS --> VF
end
VF --> TR["User's visible resources"]
```
This means:
* **The same visibility filter** (`build_visibility_filter()`) handles personal, org, and Market resources in a single query. No special-case code for Market lookups.
* **The same subscription mechanism** (`ResourceSubscription`) applies to both org and Market resources. Subscribing to an org resource and subscribing to a Market resource create the same record.
* **The same credential handling** (fallback, per-user override) works in both contexts. The `allow_fallback` flag on Connectors and MCP Servers behaves identically regardless of source.
* **The same review process** (`apply_publish_status()`) handles both org-level and Market-level review. The only difference is that the Market org has all review flags locked to `true`.
The key distinction between a regular organization and the Market organization:
| Aspect | Organization | Market |
| - | - | - |
| **Trust model** | High trust (team membership) | No trust assumed (global community) |
| **Review** | Optional per resource type | Always mandatory for all types |
| **Access** | Automatic for all members | Requires explicit subscription |
| **Scope** | Team or company | Global |
Because the Market is just an organization with special rules, any feature built for organizations — review workflows, credential management, resource lifecycle — automatically works for the Market with zero additional implementation.
### How the visibility filter handles it
Nobody holds membership in the Market org. Users do not "join" the Market — they subscribe to individual resources. This means `MARKET_ORG_ID` is never present in a user's `user_org_ids` list, and the org-membership visibility condition is naturally skipped for Market resources.
Instead, subscribed Market resources flow through the `subscribed_ids` path in `build_visibility_filter()`:
```mermaid theme={null}
flowchart LR
subgraph Filter["build_visibility_filter()"]
C1["user_id == me (own resources)"]
C2["visibility == 'org' AND org_id IN my_orgs AND approved"]
C3["id IN subscribed_ids"]
end
C1 -->|OR| Result["Visible"]
C2 -->|OR| Result
C3 -->|OR| Result
Note1["Market resources skip C2 (not a member) match C3 (subscribed)"]
Note1 -.-> C3
```
This three-condition OR clause is the entire visibility model. Personal resources, org-shared resources, and Market-subscribed resources are resolved in one query, with no branching logic for different resource origins.
### Scope-based browsing
The Market page provides a **scope selector** that switches between two browsing contexts:
| Scope | What It Shows | Who Reviews |
| - | - | - |
| **Global Market** | Resources published by anyone to the Market org | Platform administrators |
| **Organization: \[name]** | Resources published by members of a specific org | Org administrators |
The same UI, the same tabs (Solutions / Components), and the same subscription flow apply in both scopes. Switching scope only changes the `org_id` filter in the browse query. From the user's perspective, the experience is identical — they are browsing a catalog and choosing what to install.
## Subscription flow
### Browsing and discovery
Users browse the Market through a paginated catalog. Each resource displays its name, description, icon, publisher username, and a subscribe button. Resources the user has already subscribed to are marked accordingly. The browse API (`GET /api/market`) excludes the user's own resources — you cannot subscribe to something you published.
### Subscribing to a Solution
Subscribing to a Solution (Agent, Skill, or Workflow) involves dependency analysis:
```mermaid theme={null}
sequenceDiagram
participant U as User
participant M as Market API
participant DA as Dependency Analyzer
U->>M: Subscribe to Agent "Contract Analyzer"
M->>DA: Analyze dependencies
DA-->>M: Requires: ERP Connector, Legal KB
alt Content dependency (Skill)
M->>M: Auto-subscribe to Skill (silent)
end
alt Content dependency (KB)
M->>M: No subscription — Agent delegates the owner's KB at run time
end
alt Connection dependency (Connector)
M-->>U: ERP Connector requires credentials
U->>M: Configure API key (or skip)
end
M->>M: Create ResourceSubscription record
M-->>U: Agent appears in workspace
```
1. The system analyzes the Solution's dependencies — which Connectors, Knowledge Bases, MCP Servers, and Skills it requires.
2. **Content-type dependencies** are handled without asking the user. A Skill is auto-subscribed silently. A Knowledge Base is never subscribed at all — the Agent delegates its owner's KB at run time, so the subscriber has nothing to provision.
3. **Connection-type dependencies** (Connector, MCP Server) are listed as requirements. An onboarding wizard collects credentials.
4. The `ResourceSubscription` record is created, and the resource appears in the user's visibility filter.
### Subscribing to a Component
Components (Connectors and MCP Servers) have a simpler flow — no dependency analysis is needed. The user subscribes, optionally configures credentials, and the component is ready to use.
### Credential configuration
Credentials follow a **hybrid model** that balances convenience with flexibility:
* **Offered during subscription.** When a connection-type dependency requires credentials, the onboarding wizard presents the credential form immediately.
* **Skippable.** The user can choose "Skip, configure later." The resource is subscribed but tools requiring those credentials return a "please configure your credentials" message when invoked.
* **Deferred configuration.** Users can configure or update credentials at any time from their settings page.
This is the same `allow_fallback` mechanism used in organizations. If the publisher has enabled fallback and set a default credential, subscribers can use the resource immediately without providing their own key. If fallback is disabled, each subscriber must bring their own.
When using a Market resource with credential fallback enabled, your API requests flow through the publisher's credentials. For sensitive operations, consider providing your own credentials or verifying the publisher's trustworthiness.
### Unsubscribing
Unsubscribing removes the `ResourceSubscription` record. The resource disappears from the user's visibility filter and is no longer loaded into tool sets. For Solutions with auto-subscribed dependencies, the dependent resources (KBs, Skills) are cleaned up as well. User-configured credentials for the resource are removed.
## Dependency resolution
When a Solution is published or subscribed to, the system analyzes its dependency tree. Dependencies fall into two categories with different handling strategies.
### Content-type dependencies
**Knowledge Bases** and **Skills** referenced by a Solution are content-type dependencies. They provide read-only data — retrieval documents, SOP procedures — that the Solution consumes.
* **Skills — on subscription:** auto-subscribed silently. The user does not see a separate subscription step.
* **Knowledge Bases:** never subscribed. A bound KB is delegated by the owning Agent at run time (owner-delegation), so there is nothing to provision or clean up on the subscriber's side.
* **Access model:** read-only reference to the original author's resource. The subscriber cannot modify the content.
* **On unsubscription:** auto-subscribed dependencies are cleaned up automatically when the parent Solution is unsubscribed.
### Connection-type dependencies
**Connectors** and **MCP Servers** referenced by a Solution are connection-type dependencies. They require credentials to function.
* **On subscription:** listed as requirements in the onboarding wizard. The user is prompted to configure credentials (or skip).
* **Smart matching:** if the user already has a compatible Connector (same type, same base URL), the system offers to reuse it instead of creating a new subscription.
* **On unsubscription:** the subscription is removed, but user-created credentials are preserved (the user may use the same Connector elsewhere).
```mermaid theme={null}
flowchart TD
S["Subscribe to Solution"]
S --> A["Analyze dependencies"]
A --> CT["Content-type (KB, Skill)"]
A --> CN["Connection-type (Connector, MCP)"]
CT --> AS["Auto-subscribe (silent, read-only)"]
CN --> SM{"User already has compatible resource?"}
SM -->|Yes| RE["Offer reuse"]
SM -->|No| OB["Onboarding wizard (credential collection)"]
OB --> SK{"Skip?"}
SK -->|Yes| DEF["Deferred config (tools show 'please configure')"]
SK -->|No| CONF["Credentials saved (ready to use)"]
```
## Publishing
### Publishing a Solution
When an author publishes an Agent, Skill, or Workflow to the Market:
1. The system sets `visibility: "org"` and `org_id: MARKET_ORG_ID` on the resource.
2. The system analyzes the Solution's dependencies and builds a manifest — listing required Connectors, KBs, and MCP Servers.
3. The manifest is shown to the author for confirmation.
4. `apply_publish_status()` sets the resource to `pending_review` (the Market org has all review flags locked to `true`).
5. A system administrator reviews and approves or rejects the resource.
### Publishing a Component
Publishing a Connector or MCP Server is simpler:
1. The system sets visibility and org\_id as above.
2. Credential schema is extracted (what fields subscribers need to fill in).
3. The resource enters `pending_review` and awaits admin approval.
### Review process
The review process is the same mechanism used by organizations, with one critical difference:
| Context | Review Required? | Who Reviews |
| - | - | - |
| **Organization** | Configurable per resource type (`review_agents`, `review_connectors`, etc.) | Org admins |
| **Market** | Always required for all resource types | Platform admins (Market org owner) |
The Market org is initialized with all six review flags set to `true`, and this configuration cannot be changed. Every resource published to the Market must pass admin review before it becomes visible in the browse catalog.
Org owners bypass review automatically — their published resources are immediately available. For the Market, only the Market org owner (the system administrator) has this bypass privilege.
When an approved resource is edited by its author, `check_edit_revert()` automatically reverts the `publish_status` to `pending_review`. This ensures that changes to live Market resources are re-reviewed before becoming visible to subscribers.
## Implementation notes
### The shadow org
The Market organization has a well-known fixed ID (`00000000-0000-0000-0000-000000000001`) and slug (`market`). It is created by `ensure_market_org()` during platform initialization — typically on the first admin user's login. The function is idempotent; calling it multiple times is safe.
### ResourceSubscription
The `ResourceSubscription` table is the core data structure for Market access:
| Column | Purpose |
| - | - |
| `user_id` | The subscriber |
| `resource_type` | `agent`, `connector`, `mcp_server`, `skill`, or `workflow` (knowledge bases are never subscribed — they ride on Agent owner-delegation) |
| `resource_id` | The subscribed resource's ID |
| `org_id` | The source org (Market org ID or a regular org ID) |
A unique constraint on `(user_id, resource_type, resource_id)` prevents duplicate subscriptions. The `org_id` column tracks where the subscription came from, enabling scope-aware unsubscription.
### Visibility filter integration
The `resolve_visibility()` function performs two lookups in a single call:
1. Fetches the user's org memberships (`user_org_ids`)
2. Fetches the user's subscriptions (`subscribed_ids`)
These are passed to `build_visibility_filter()`, which produces a single SQL WHERE clause combining all three visibility tiers (own, org-shared, subscribed). This function is used everywhere resources are queried — agent lists, connector dropdowns, skill injection, auto-discovery mode — ensuring consistent visibility across the entire platform.
### Credential encryption
Credentials configured during subscription (or later in settings) are encrypted at rest using the platform's encryption key. The Market API never exposes credential values in browse responses — only metadata (name, description, icon, type) is returned in the `_*_market_info()` helper functions.
## See also
* [Organization & Market](/architecture/organization) -- organization-level sharing and trust model
* [Agent & Resource Discovery](/architecture/agent-discovery) -- how subscribed resources are loaded into tool sets
* [Connector Architecture](/architecture/connector-architecture) -- connector design, auth injection, and audit
* [System Overview](/architecture/system-overview) -- the unified tool abstraction that all resources converge into
# Organizations
Source: https://docs.one.fim.ai/architecture/organization
Team-level resource sharing with role-based access, optional review, and credential fallback.
## Overview
Organizations are FIM One's unit for team collaboration. They let groups of users share resources — Agents, Connectors, MCP Servers, Workflows, and Skills — within a trusted scope. Knowledge Bases are the one exception: they are never shareable, and reach other members only as an internal dependency of a shared Agent (see [Sharing and credentials](/architecture/sharing-and-credentials)).
Every resource in FIM One starts as **personal** (visible only to its creator). When you publish a resource to an organization, it becomes **discoverable** by other org members through the Market's organization scope. Members browse the org's shared resources and subscribe to the ones they need.
```mermaid theme={null}
flowchart TD
R["Personal Resource"]
subgraph OrgPublish["Organization"]
O["Org Scope in Market"]
M1["Member A (subscribes)"]
M2["Member B (subscribes)"]
M3["Member C (browses only)"]
end
R -->|"publish to org"| O
O -->|"subscribe"| M1
O -->|"subscribe"| M2
O -.->|"visible but not subscribed"| M3
```
Organizations and the global Market share the same subscription-based access model. The key difference is trust: organizations represent a team or company where members know and trust each other, so review is optional and credential sharing is straightforward.
## Creating and Managing Organizations
Every user can create **unlimited** organizations and join any number of them. An organization has three roles:
| Role | Permissions |
| - | - |
| **Owner** | Full control — manage members, configure settings, bypass review |
| **Admin** | Manage members and review published resources |
| **Member** | Browse and subscribe to shared resources |
The owner is always the user who created the organization. Ownership can be transferred but not shared.
## Publishing Resources
When you publish a resource to your organization, it does **not** automatically appear in every member's workspace. Instead, the resource becomes discoverable in the Market's organization scope, where members can browse and subscribe to it.
This subscription-based model gives each member control over their workspace. A large organization might share dozens of Connectors, but an individual member only subscribes to the ones relevant to their work.
```mermaid theme={null}
sequenceDiagram
participant U as Publisher
participant P as Platform
participant O as Org Settings
participant M as Other Members
U->>P: Publish Agent to Org
P->>O: Check review_agents flag
alt Review disabled
P->>P: publish_status = NULL (auto-approved)
P-->>U: Resource discoverable by org members
else Review enabled
P->>P: publish_status = "pending_review"
P-->>U: Pending admin approval
Note over P: Org admin approves or rejects
P->>P: publish_status = "approved"
end
M->>P: Browse org resources in Market
M->>P: Subscribe to Agent
P-->>M: Agent appears in workspace
```
### Review System
Review is **optional** and configured per resource type. Each organization has independent toggle flags:
* `review_agents`
* `review_connectors`
* `review_kbs`
* `review_mcp_servers`
* `review_workflows`
* `review_skills`
When review is disabled for a resource type, published resources are immediately discoverable by members — no admin action needed. When review is enabled, resources enter a `pending_review` state and require admin approval before they become visible.
Organization owners bypass review automatically. Their published resources are always immediately discoverable.
This flexibility lets organizations match their governance needs. A small startup might disable all review toggles for frictionless sharing, while a compliance-focused enterprise enables review on Agents and Connectors to maintain oversight.
## Credential Fallback
Connectors and MCP Servers often require credentials (API keys, database passwords, OAuth tokens). FIM One provides a **fallback mechanism** so that members don't have to configure every credential themselves.
```mermaid theme={null}
flowchart TD
E["Member uses a shared Connector"]
C1{"Member has own credential?"}
C2{"allow_fallback enabled?"}
C3{"Owner set default credential?"}
E --> C1
C1 -->|Yes| U["Use member's credential"]
C1 -->|No| C2
C2 -->|Yes| C3
C3 -->|Yes| F["Fallback to owner's default"]
C3 -->|No| ERR["Error: credential required"]
C2 -->|No| ERR2["Error: credential required"]
```
There are two modes:
* **Fallback enabled** (`allow_fallback=true`, the default): Members who don't provide their own credentials automatically use the owner's default credentials. This works well for team-shared API keys or internal services where a single key covers the whole team.
* **Fallback disabled** (`allow_fallback=false`): Every member must configure their own credentials. This is appropriate when each user needs a personal API key (for example, per-seat SaaS licenses).
Resources that don't require credentials — such as a read-only public API connector or an Agent with no authentication — work immediately after subscription. No configuration needed.
Credential fallback only applies after a member subscribes to the resource. The fallback mechanism determines how credentials are resolved at runtime, not whether the resource is accessible.
## Resource Visibility
Every resource in FIM One has a `visibility` that determines its access scope:
| Visibility | Scope | Who Can Discover It |
| - | - | - |
| `personal` | Owner only | The user who created it |
| `org` | Organization | Org members can browse and subscribe (if approved) |
The visibility filter follows a unified query pattern:
```
A resource is available in your workspace if:
1. You own it (any visibility), OR
2. It's published to an org you belong to, approved, AND you've subscribed to it
```
Publishing a resource to an organization does not grant automatic access. Members must subscribe through the Market's org scope to add the resource to their workspace.
## Infrastructure Resources (org-wide, no subscription)
A second class of org-scoped resource does **not** follow the personal → publish → subscribe lifecycle above. These are **configured once by an Org Owner or Admin** and then transparently available to every agent in the org:
| Resource | Purpose | Configured by | Consumed by |
| - | - | - | - |
| **Channels** | Outbound bridge to an IM platform (Feishu today; Slack / WeCom / Teams planned) | Org Owner / Admin | Every agent's [Hook System](/architecture/hook-system) and proactive-notification tools |
| **OAuth Providers** | Social-login credentials (GitHub, Google, Feishu, Discord) | System admin (env vars) | Sign-in page |
| **API Keys** | Programmatic access tokens for headless clients | Each user, org-scoped | External integrations |
```mermaid theme={null}
flowchart LR
Admin["Org Owner / Admin"]
subgraph Org["Organization"]
Ch["Channel (e.g. FIM One ↔ Feishu group)"]
A1["Agent A"]
A2["Agent B"]
A3["Agent C"]
end
Admin -->|"configure once"| Ch
A1 -. "uses via feishu_gate hook" .-> Ch
A2 -. "uses for notifications" .-> Ch
A3 -. "uses for approval cards" .-> Ch
```
The key distinction: **publishable resources** (Agents, Connectors, Skills, …) are per-user artifacts that need explicit member subscription; **infrastructure resources** (Channels, OAuth, API keys) are platform-level wiring that all agents share implicitly. Channels specifically are the IM hotline for the whole org — one well-maintained Feishu channel backs every agent's approval gate, every scheduled report, every escalation event.
See [Channels overview](/configuration/channels/overview) for the channel lifecycle and supported platforms.
## Practical Scenarios
### Team Sharing a Database Connector
1. Alice creates a Connector to the team's PostgreSQL database
2. Alice publishes it to her team's org (review is disabled for connectors)
3. The Connector becomes discoverable in the org scope of the Market
4. Bob browses the org's shared resources, finds the Connector, and subscribes
5. The Connector appears in Bob's workspace, using Alice's database credentials as fallback
6. Carol subscribes too. Dave (an external contractor) subscribes and configures his own read-only credentials instead
### Organization with Strict Review
1. A compliance-focused company enables `review_agents=true` and `review_connectors=true` on their org
2. When an employee publishes a new Agent, it enters `pending_review` state
3. An org admin reviews the Agent configuration and approves it
4. The Agent becomes discoverable — other members can now find and subscribe to it
5. If the publisher later edits the approved Agent, it automatically reverts to `pending_review` for re-approval
### Selective Subscription in a Large Organization
1. An organization publishes 50+ Connectors covering internal APIs, databases, and third-party services
2. The data team subscribes only to database Connectors and analytics APIs
3. The marketing team subscribes only to the CRM and email platform Connectors
4. Each team member's workspace stays focused and uncluttered
## See Also
* [Market Architecture](/concepts/market) — For the global Market and how it relates to organizations. Both use the same subscription model, but the Market serves as the cross-organization discovery channel with mandatory review.
* [Agent & Resource Discovery](/architecture/agent-discovery) — How subscribed resources are assembled into tool sets during chat.
# Philosophy
Source: https://docs.one.fim.ai/architecture/philosophy
Why dynamic planning is the hard middle ground — and where FIM One stands.
## Why dynamic planning is the hard middle ground
The AI agent landscape split into two camps, and both picked the easy path. Traditional workflow engines -- Dify, n8n, Coze -- chose static orchestration: visual drag-and-drop flowcharts with fixed execution paths. This was not ignorance; enterprise customers demand determinism (same input, stable output), and static graphs deliver that. On the other extreme, fully autonomous agents (AutoGPT and its descendants) promised end-to-end autonomy but proved impractical: unreliable task decomposition, runaway token costs, and behavior nobody could predict or debug.
The sweet spot is narrow but real. Simple tasks do not need a planner. Tasks complex enough to require dozens of interdependent steps overwhelm current LLMs. But in between lies a rich class of problems -- tasks with clear parallel subtasks that are tedious to hard-code yet tractable for an LLM to decompose. Dynamic DAG planning targets exactly this zone: the model proposes the execution graph at runtime, the framework validates the structure and runs it with maximum concurrency. No drag-and-drop, no YOLO autonomy.
## The bet on improving models
Every few months the foundation shifts -- GPT-4, function calling, Claude 3, the MCP protocol. Building a rigid abstraction on shifting ground is risky; LangChain's over-abstraction is the cautionary tale everyone in this space has internalized. FIM One takes the opposite approach: **minimal abstraction, maximum extensibility**. The framework owns orchestration, concurrency, and observability. The intelligence comes from the model, and the model keeps getting better.
Today, LLM task decomposition accuracy sits around 70-80% for non-trivial goals. When that reaches 90%+, the "sweet spot" for dynamic planning expands dramatically -- problems that were too complex yesterday become tractable tomorrow. FIM One's DAG framework is designed to capture that expanding value without rewriting the plumbing.
## Will ReAct and DAG planning become obsolete?
ReAct will not disappear -- it will sink into the model. Consider the analogy: you do not hand-write TCP handshakes, but TCP did not go away; it got absorbed into the operating system. When models are strong enough, the think-act-observe loop becomes implicit behavior inside the model, not explicit framework code. This is already happening: Claude Code is essentially a ReAct agent where the loop is driven by the model itself, not by an external harness.
DAG planning's lasting value is not "helping dumb models decompose tasks" -- it is **concurrent scheduling**. Even with infinitely capable models, physics imposes latency: a serial chain of 10 LLM calls takes 10x longer than 3 parallel waves. DAG is an engineering problem (how to run things fast and reliably), not an intelligence problem (how to decide what to run). Retry logic, cost control, timeout management, observability -- these do not go away when models get smarter.
The endgame: **models own the "what" (planning intelligence internalizes into the model), frameworks own the "how" (concurrency, retry, monitoring, cost governance)**. A framework's lasting value is not intelligence -- it is governance.
This is the sentence the rest of FIM One is downstream of. Once the model owns the "what," durable value moves to governing the "how" -- and the highest-value "how" in any enterprise is *reach* into the systems that refuse to talk to each other. The connector hub is one inferential step from here, and the history shows it: the project began as a DAG orchestration framework (`fim-agent`) and became a connector platform within a single week -- not through a rewrite, but because this bet already required it.
## Why not mirror Dify's workflow editor
A natural question: if DAG covers the flexible case, shouldn't we also build a static workflow editor for the deterministic case?
No. The reasoning:
1. **The workflows already exist elsewhere.** Government and enterprise clients' fixed processes live in their OA, ERP, and legacy systems. They don't want to rebuild those flows in yet another platform -- they want AI that can connect to the systems where those flows already run. The Connector Platform (v0.6) solves this directly.
2. **Existing capabilities compose into fixed pipelines.** Scheduled Jobs (v1.0) trigger a DAG agent with a fixed prompt; the DAG dynamically plans the steps; Connectors (v0.6) connect to the target systems. The combination is equivalent to a static pipeline -- but more flexible, because the LLM can adjust its plan based on the data it encounters. No separate pipeline DSL needed.
3. **Investment mismatch.** A production-quality visual workflow editor (canvas, node types, variable passing, debug/replay) is months of dedicated effort. The result would be a low-fidelity copy of what Dify's 120K-star community already maintains. That effort is better spent on the Connector architecture -- a capability no competitor offers.
The strategic bet: **don't compete on workflow visualization; compete on being the agent platform where systems meet AI**. Let Dify own "build AI workflows visually." FIM One owns "the all-in-one agent platform for Global × China enterprises."
## Where FIM One stands
FIM One is not an "AGI task scheduler" and not a static workflow engine. It occupies the middle ground: planning capability with constraints, concurrency with observability.
* Compared to **Dify**: more flexible -- runtime DAG generation vs. design-time flowcharts. You do not need to anticipate every execution path in advance. Does not compete on visual workflow editing; competes on legacy system integration.
* Compared to **AutoGPT**: more controlled -- bounded iterations, re-planning limits, with human-in-the-loop on the roadmap. Autonomy within guardrails.
The strategy is straightforward: build the orchestration framework now, and let improving models fill it with capability over time.
## Where FIM One Sits: Planning x Execution Spectrum
The AI execution landscape can be mapped on two axes -- how plans are created (static vs dynamic) and how they are executed (rigid vs adaptive):
```mermaid theme={null}
quadrantChart
title Planning x Execution Spectrum
x-axis "Static Execution" --> "Dynamic Execution"
y-axis "Static Planning" --> "Dynamic Planning"
quadrant-1 "Autonomous Agent"
quadrant-2 "Transitional (unstable)"
quadrant-3 "BPM / Workflow"
quadrant-4 "ACM"
Camunda: [0.20, 0.25]
Activiti: [0.25, 0.20]
Dify: [0.30, 0.30]
n8n: [0.35, 0.25]
Coze: [0.25, 0.35]
Salesforce Case: [0.75, 0.30]
AutoGPT: [0.85, 0.85]
Manus: [0.80, 0.80]
Claude Code Teams: [0.75, 0.75]
FIM One:::highlight: [0.70, 0.65]
classDef highlight color: #D97706, radius: 12, stroke-color: #B45309, stroke-width: 2px
```
**Why Dify/n8n are Static Planning + Static Execution**: The DAG is drawn by a human on a visual canvas at design time. Each node executes a fixed operation (an LLM call with a fixed prompt, an HTTP request, a code block). There is no runtime replanning -- if a step fails, the workflow fails or follows a pre-wired error branch. This is structurally the same as BPM, just with AI nodes in the graph.
**FIM One's position: Dynamic Planning + Dynamic Execution**
* **DAG topology is generated by LLM at runtime** (dynamic planning) -- no human designs the graph
* **Each DAG node runs a full ReAct loop** (dynamic execution) -- nodes reason, use tools, and adapt
* **Re-planning mechanism** (execute → analyze → re-plan if unsatisfied)
* But bounded: max 3 re-plan rounds, token budgets, human confirmation gates
This places FIM One in the same quadrant as AutoGPT, but with engineering constraints that prevent runaway behavior. More flexible than BPM/Dify, more controlled than AutoGPT.
## Concept Glossary
For readers unfamiliar with the terminology used in this document:
| Term | One-line explanation | Relation to FIM One |
| - | - | - |
| **BPM** (Business Process Management) | Processes fully fixed at design time, executed rigidly. Camunda, Activiti. | FIM One is **not** BPM. No fixed process engine. |
| **FSM** (Finite State Machine) | System is in exactly one state at any time; events trigger transitions. Supports loops (reject → resubmit). | Target systems (ERP, contract systems) use FSM internally. FIM One **doesn't need** its own FSM -- it calls the target system's API. |
| **ACM** (Adaptive Case Management) | Skeleton static, branches dynamic. Main flow pre-defined, each node adapts at runtime. | FIM One's DAG + ReAct naturally falls in this quadrant. |
| **HTN** (Hierarchical Task Network) | Recursive task decomposition: high-level → subtasks → atomic operations. | DAG re-planning covers most scenarios; full HTN not needed yet. |
| **iPaaS** (Integration Platform as a Service) | Cloud integration platform connecting multiple SaaS/on-prem systems. MuleSoft, Zapier. | FIM One's Hub Mode is like **AI-native iPaaS** -- natural language drives cross-system integration. |
| **MDM** (Master Data Management) | Deduplicates and unifies entity records across systems into a "golden record." Reltio, Informatica, Tamr. | FIM One **connects to** MDM systems; it does not replicate entity resolution. |
| **Context Layer / System of Context** | Unified entity + relationship graph that gives AI agents trusted business context. Term popularized by Reltio (2026). | FIM One delegates this to upstream MDM/data platforms. Skills provide lightweight aggregation for common cases. |
## Architecture Boundary: FIM One Does Not Replicate Workflow Logic
Complex business processes (approval chains, transfers, rejections, escalation, co-signing, countersigning) are the **target system's responsibility**. These systems spent years building this logic -- ERP, OA, contract management systems all have mature state machines.
From the Connector's perspective:
| Operation | What the Connector does |
| - | - |
| Transfer | Call one API, pass target person |
| Reject | Call one API, pass rejection reason |
| Escalate | Call one API, pass escalation person list |
| Co-sign | Call one API, pass co-signer list |
Every complex workflow operation collapses to an HTTP request with parameters. FIM One calls the API; the target system manages the state machine.
This is a **deliberate architectural boundary**, not a capability gap. Duplicating workflow logic that already exists in target systems would:
1. Create maintenance burden (two state machines to keep in sync)
2. Add failure modes (what if they disagree?)
3. Provide zero additional value (the target system already does this correctly)
The connector pattern is simple by design: **one operation = one API call**.
## Architecture Boundary: Connector Layer, Not Context Layer
Enterprise AI is converging on a layered architecture for agent context:
| Layer | What it does | Representative players |
| - | - | - |
| **Decision Traces** | Records *why* something happened -- audit trails, lineage | Palantir Decision Lineage, Arize |
| **Entity Context** | Unified golden records + relationship graphs -- the "System of Context" | Reltio/SAP, Informatica/Salesforce, Tamr |
| **Data Connectivity** | Connects agents to source systems via APIs and protocols | **FIM One**, CData, MCP ecosystem |
| **Source Systems** | CRM, ERP, contract management, databases, SaaS apps | SAP, Salesforce, custom systems |
FIM One operates at the **data connectivity layer**. It does not attempt to build the entity context layer above it. This is a deliberate choice, not a gap.
### Industry context: the 2025-2026 consolidation
Two landmark acquisitions reshaped the enterprise AI data landscape and made the context layer a strategic battleground:
* **Salesforce acquired Informatica** (November 2025) -- adding enterprise MDM and data governance to its Data Cloud + Agentforce stack. The goal: make Agentforce agents trustworthy by grounding them in Informatica's golden records.
* **SAP announced acquisition of Reltio** (March 2026) -- adding AI-native entity resolution and relationship graphs to its Business Data Cloud (BDC). The goal: create a "System of Context" that spans SAP and non-SAP environments, serving as the trusted foundation for Joule Agents.
These moves signal that the world's largest enterprise platforms now consider unified entity context a prerequisite for reliable AI agents -- not a nice-to-have.
### Data ownership × Context depth spectrum
```mermaid theme={null}
quadrantChart
title Data Ownership x Context Depth
x-axis "Federated / Pass-through" --> "Centralized / Copy-in"
y-axis "Raw Connectivity" --> "Entity Intelligence"
quadrant-1 "Deep Ontology Platform"
quadrant-2 "Context-Aware Hub"
quadrant-3 "Lightweight Connector"
quadrant-4 "Data Warehouse / Lake"
Dify: [0.15, 0.20]
n8n: [0.20, 0.15]
CData: [0.25, 0.25]
MCP Ecosystem: [0.30, 0.20]
FIM One:::highlight: [0.25, 0.35]
Tamr: [0.55, 0.65]
DataHub: [0.35, 0.45]
Salesforce Data Cloud: [0.40, 0.70]
SAP BDC Reltio: [0.70, 0.80]
Palantir Foundry: [0.90, 0.90]
classDef highlight color: #D97706, radius: 12, stroke-color: #B45309, stroke-width: 2px
```
**Reading the chart:**
* **Bottom-left (Lightweight Connector)**: minimal data ownership, raw API connectivity. Dify, n8n, and basic MCP servers live here -- they connect to systems but don't understand entities. FIM One is in this quadrant but higher on the y-axis due to progressive-disclosure meta-tools, Skill-based context aggregation, and domain-aware escalation.
* **Top-left (Context-Aware Hub)**: federated access with entity-level understanding. Salesforce Data Cloud exemplifies this -- zero-copy federation with on-demand entity resolution. DataHub provides metadata-level context (schema, lineage, ownership) without owning business data.
* **Top-right (Deep Ontology Platform)**: full data ingestion with deep entity intelligence. SAP BDC + Reltio builds persistent multi-domain golden records with relationship graphs. Palantir goes furthest -- three-layer ontology with decision lineage.
* **Bottom-right (Data Warehouse / Lake)**: centralized data storage without entity semantics. Traditional data platforms that ingest everything but lack entity resolution or relationship modeling.
FIM One's position reflects a deliberate choice: stay federated and lightweight, but invest in making upstream entity context (from any MDM) easily accessible to agents.
### Three models for entity context
**SAP: multi-domain golden record (copy-in + govern)**
SAP is building a three-layer enterprise architecture:
| Layer | System | Role |
| - | - | - |
| **Transaction** | S/4HANA | Business execution -- orders, invoices, deliveries |
| **Intelligence** | Business Data Cloud (BDC) + Reltio | Semantic integration, entity resolution, relationship graphs |
| **Agent** | Joule + Joule Agents | Intent understanding, tool orchestration, autonomous execution |
Reltio sits at the heart of the intelligence layer. Its job: ingest entity data from SAP and non-SAP systems, resolve duplicates via AI-based matching, and produce **multi-domain golden records** (customer, supplier, product, patient, asset) with relationship graphs. Reltio's early adoption of the **MCP protocol** is strategically significant -- it positions golden records as directly callable by any MCP-compatible agent, not just SAP's own Joule.
SAP's bet: agents need a persistent, governed, cross-domain "single source of truth" to operate reliably in high-stakes enterprise processes.
**Salesforce: zero-copy federation (federate + resolve on demand)**
Salesforce takes a lighter approach. Data Cloud does not require physical data movement; instead it uses **zero-copy partnerships** (Databricks, Snowflake, BigQuery) to federate data in place. Entity resolution happens on demand within Data Cloud, creating unified customer profiles without a persistent golden record store.
Key numbers (Q3 FY2026): Data Cloud ingested 32 trillion records, with 15 trillion via zero-copy (341% YoY growth). This scale validates the federation model for CRM-centric use cases.
Agentforce agents are grounded in this federated context: they reason across the distributed data estate without requiring all data to land in one system. For front-office scenarios (sales, service, marketing), this is flexible and fast to deploy.
Salesforce's bet: agents don't need a heavyweight golden record store -- they need real-time access to federated, resolved context where it already lives.
**Palantir: deep ontology (the heavyweight reference)**
At the extreme end, Palantir Foundry ingests all data and builds a three-layer ontology (semantic objects + kinetic actions + dynamic rules) with full decision lineage. This is the most complete context system in production, but at \$10M+ contracts and 6-18 month deployments. It serves as a reference architecture, not a realistic model for most organizations.
### Where FIM One fits
| Dimension | SAP + Reltio | Salesforce + Informatica | FIM One |
| - | - | - | - |
| **Data philosophy** | Copy-in: ingest into BDC, build golden records | Federate: zero-copy access, resolve on demand | Pass-through: connect to source, don't store |
| **Entity resolution** | Deep -- AI-native MDM with relationship graphs | Medium -- Data Cloud entity resolution | None -- delegates to upstream MDM |
| **Agent integration** | Joule Agents + MCP | Agentforce + Data Cloud grounding | Any agent + any connector |
| **Deployment cost** | Enterprise platform pricing | Enterprise SaaS pricing | Lightweight, hours to days |
| **Lock-in** | High (BDC + S/4HANA ecosystem) | Medium (Salesforce ecosystem) | Low -- connectors are interchangeable |
| **Best fit** | Manufacturing, supply chain, life sciences (complex multi-domain) | CRM-driven enterprises (front-office focus) | Mid-market, multi-system, provider-agnostic |
FIM One aligns most closely with the **Salesforce philosophy** (don't own the data, connect to it) but without the Salesforce ecosystem dependency. The strategic position: be provider-agnostic at the connectivity layer, and let any MDM -- whether SAP/Reltio, Salesforce/Informatica, Tamr, or a custom solution -- serve as the entity context source.
### Why FIM One stays at the connector layer
Building an entity context layer requires persistent data storage, ML-based entity resolution, survivorship rules, relationship graph engines, and governance frameworks. This is a distinct product category (MDM) with decade-deep competition (Reltio, Informatica, Tamr). Attempting it would:
1. **Transform FIM One from a connector into a data platform** -- fundamentally different product, team, and go-to-market
2. **Duplicate capabilities that upstream systems already provide** -- the same anti-pattern we avoid with workflow logic
3. **Increase lock-in** -- once users store golden records in FIM One, switching costs rise dramatically
The correct strategy: **be the best ingress point for any MDM system**, not a replacement for one. FIM One should make it trivial for Reltio, Informatica, Tamr, or a custom MDM to serve entity context to agents -- via MCP Resources, connector APIs, or knowledge base injection.
### Current approach: Skills as lightweight context aggregation
For scenarios that need cross-connector context (e.g., contract review requiring supplier data + payment history + risk flags), **Skills** already provide a pragmatic solution. A Skill's SOP can orchestrate multiple connector actions in sequence, aggregate results, and present structured context to the agent -- without building a persistent entity layer.
This covers the 80% case. If demand emerges for real-time entity resolution at the connector level, the architecture leaves room for a future `ContextSource` connector type that integrates with external MDM systems -- but that is a future consideration, not a current commitment.
### The analogy
FIM One's relationship to the context layer is like SQLAlchemy's relationship to the database: it does not store or manage data, but it makes any data source accessible to the application layer. Let MDM platforms own entity resolution; let FIM One own the connection.
## Where FIM One Sits in the AI Stack
### The Layered Model
A useful mental model (credit [Phil Schmid](https://www.philschmid.de/)) maps AI systems to a four-layer stack analogous to computing hardware:
| Layer | Analogy | FIM One Component |
| - | - | - |
| **Model** | CPU | ModelRegistry -- any OpenAI-compatible LLM |
| **Context** | RAM | ContextGuard + Memory (Window / Summary / DB) |
| **Harness** | Operating System | ReAct Agent + DAG Planner + Hooks + ToolRegistry |
| **Application** | App | Portal, Copilot, Hub API |
FIM One operates at the **harness layer** -- it does not compete with models; it makes them productive by providing the right tools, constraints, and feedback loops. The model supplies intelligence; the harness supplies governance, concurrency, and reliability.
### Three Eras of AI Engineering
The discipline has evolved through distinct phases: **Prompt Engineering** focused on crafting better instructions to elicit correct behavior from a fixed model. **Context Engineering** shifted attention to assembling the right information -- retrieving documents, injecting tool results, managing memory -- so the model has what it needs to reason well. **Harness Engineering** goes further: designing the full execution environment around the model, including deterministic guardrails, tool orchestration, feedback loops, and cost controls.
FIM One's architecture embodies harness engineering principles. The [Hook System](/architecture/hook-system) runs platform code outside the LLM loop at well-defined lifecycle points — today `FeishuGateHook` intercepts sensitive tool calls and routes approval through an IM channel, and the same abstraction can carry audit logging, read-only-mode enforcement, rate limits, and result truncation if those are ever needed. ContextGuard manages what enters the context window and when. Dynamic tool selection (progressive-disclosure meta-tools) keeps the tool surface tractable as connector count grows. The ReAct loop's self-reflection mechanism and the DAG planner's re-plan cycle provide structured feedback that improves execution quality without requiring a more capable model.
# Planning Landscape
Source: https://docs.one.fim.ai/architecture/planning-landscape
Planning approaches, graph engineering vs FIM One layers, and where ReAct, DAG, and Workflow fit.
## Graph engineering (2026) and FIM One
Industry discussion often frames a shift from **loop engineering** (a single agent repeatedly thinking and acting) to **graph engineering** (making multi-node topology an explicit engineering object). The label is new; the practice is not. LangGraph-style systems and multi-agent orgs have used graphs for years. What matters for FIM One is which **kind** of graph people mean, and how that maps onto our three execution layers.
### What graph engineering usually means
| Layer (common framing) | Concern |
| - | - |
| **Harness** | Tools, sandbox, auth, logging, quotas around the model |
| **Loop** | One agent's think → act → observe cycle |
| **Graph** | How **nodes** connect: branches, joins, cycles, human checkpoints, shared state |
Consensus across that framing: **graph does not replace loop**. Graph designs **relationships between processes**; each agentic node may still run a full loop.
"Graph" is overloaded in practice. Three usages get mixed:
| Usage | Example | Role |
| - | - | - |
| **Control-flow graph** | LangGraph `StateGraph`: conditional edges, retries, interrupt/resume | State machine for who runs next |
| **Task dependency graph** | Claude Code Tasks `blockedBy`; FIM One `depends_on` | Schedule and parallelize work |
| **Knowledge graph** | Entities and typed relations | Retrieval / memory (not orchestration) |
This page is about the first two (orchestration).
### Not "Dify Workflow with a new name"
Dify-style **visual workflows** and graph engineering share a bloodline (**explicit topology**) but they are not the same product.
| | Classic visual Workflow (e.g. Dify) | Graph engineering (2026 usage) |
| - | - | - |
| **Primary artifact** | Canvas / blueprint the implementer draws | Topology as a versioned system design (code or config) |
| **Nodes** | Fixed capability blocks (LLM, HTTP, KB, condition…) | **Heterogeneous**: functions, routers, **full agents**, humans, joins |
| **Where intelligence lives** | Mostly inside isolated LLM nodes; the graph is a pipe | Often **inside nodes** (each may be a loop); the graph organizes roles |
| **Edges** | Pipeline success/fail and branches | Control-flow **and** state transitions, including **controlled cycles** |
| **Human-in-the-loop** | Forms / approvals as add-ons | Interrupt / resume as first-class primitives |
| **Relation to loop** | Often sold as "Workflow **vs** Agent" | Explicitly **Graph over Loop** |
Short form: classic Workflow is a **deterministic (or semi-deterministic) pipe** with occasional LLM steps. Graph engineering is **multi-node system topology**, where critical posts can still be autonomous agents.
```mermaid theme={null}
flowchart LR
subgraph classic["Classic visual Workflow"]
A1[HTTP] --> A2[LLM] --> A3[Condition] --> A4[Notify]
end
subgraph ge["Graph engineering shape"]
B1[Router] --> B2[Agent loop]
B1 --> B3[Agent loop]
B2 --> B4[Join / state]
B3 --> B4
B4 --> B5[Human gate]
B5 -->|retry / rework| B2
end
```
### How the industry map sits on FIM One
FIM One already spans the control spectrum; chat "Planner mode" is only one slice.
| Industry idea | Closest FIM One piece | Fit |
| - | - | - |
| Classic visual Workflow | **Workflow Engine** (design-time blueprint) | Strong |
| Control-flow graph + HITL + durable state | Workflow human steps + confirmation gates; partial on chat DAG | Medium |
| Task dependency graph + parallel schedule | **DAGPlanner** (`depends_on`, topological fan-out) | Strong on scheduling; weaker on mid-graph HITL |
| Single agent loop + checklist memory | **ReAct** (`update_plan`, tools, `ask_user_question`) | Strong |
| Runtime LLM-drawn plan every turn | **DAGPlanner** (not Dify; not LangGraph design-time DSL) | Distinct product |
```mermaid theme={null}
flowchart TB
subgraph industry["Industry labels"]
GE["Graph engineering"]
Loop["Loop engineering"]
WF["Visual Workflow tradition"]
end
subgraph fim["FIM One"]
W["Workflow Engine design-time graph"]
D["DAGPlanner runtime task graph"]
R["ReAct Agent loop + plan board + ASK"]
end
WF --> W
GE --> W
GE --> D
Loop --> R
GE -.->|"nodes may contain loops"| R
D -->|"each step is already a ReAct agent"| R
```
### Graph engineering vs FIM One DAG (chat Planner)
Same family (explicit multi-step topology). Different species.
| Dimension | Graph engineering (typical) | FIM One DAG (chat) |
| - | - | - |
| **Who draws the graph** | Developer / implementer (stable topology) | **LLM at runtime** (new graph per goal) |
| **Edge meaning** | Often **control flow** (where next, including cycles) | Mostly **dependency / data** (`depends_on`, acyclic run) |
| **Correction** | Edge-level retry, human node, resume same graph | **PlanAnalyzer → replan** (often a **new** graph) |
| **Node depth** | Node may be a full agent loop | **Already full ReAct per step**: `DAGExecutor` runs `agent.run()` with a multi-iteration tool loop (default cap from `DAG_STEP_MAX_ITERATIONS`, commonly 15). Default model is registry **general** (not infrastructure fast); `model_hint` can escalate to reasoning or demote to fast. Tool cache, step/citation verification, and source evidence ship with each step. Step agents set `enable_plan_tool=False` (no `update_plan` board inside a step) because the unit of work is already a focused sub-task. |
| **Shared state** | First-class typed state + checkpoint narrative | Step results + memory/compact; step checkpoints exist |
| **Mid-run questions** | Interrupt as a product feature | Confirmation gates exist; **`ask_user_question` is ReAct-only** (chat outer loop), not a first-class DAG wait edge |
| **Product entry** | Framework or backend org chart | User-visible mode next to ReAct |
```mermaid theme={null}
flowchart TB
subgraph ge2["Graph engineering runtime"]
S[Shared state] --> N1[Node]
N1 -->|cond| N2[Node]
N2 -->|interrupt| H[Human]
H -->|resume| N3[Node]
N3 -->|cycle| N1
end
subgraph fimdag["FIM One DAG chat path"]
G[User goal] --> P[LLM Planner]
P --> J[JSON depends_on DAG]
J --> E[Topological executor]
E --> Z[PlanAnalyzer]
Z -->|not achieved / recoverable| P
end
```
**Implication:** industry heat around graph engineering **validates keeping a graph/scheduling engine**. It does **not** require a chat default that forces users to pick "Planner" for every turn. Most tasks still want a strong loop; graphs earn their keep on parallel branches, fixed compliance paths, and multi-role orchestration.
### What FIM One can learn (empower ReAct, DAG, Workflow)
Concrete takeaways, not slogans. Ordered by leverage.
| # | Graph engineering idea | Apply to | Direction |
| - | - | - | - |
| 1 | **Graph over loop, not instead of** | Product + Auto routing | Default UX stays **ReAct** (loop + `update_plan` + `ask_user_question`). Use DAG/Workflow when topology earns cost. |
| 2 | **Heterogeneous deep nodes** | DAG | **Already the status quo.** Each step is a full ReAct agent (`agent.run`, multi-iteration tools, general-or-better model by default), not a one-shot LLM call. Remaining knobs are optional (e.g. plan board on long steps), not a structural gap. |
| 3 | **Interrupt / resume as edges** | DAG + ReAct | Mid-run clarification belongs in the **outer** loop or as a first-class wait state. Avoid fake DAG steps that "ask the user" in text and then replan as "goal not achieved". |
| 4 | **Controlled cycles vs whole-graph replan** | DAG | Prefer step-local retry / partial carryover (already partly true for completed steps) over dumping all step text as a failed final answer. |
| 5 | **Typed shared state along edges** | DAG + Workflow | Stronger contracts for step I/O (schema, deliverables) reduce Analyzer false confidence and answer/source drift. |
| 6 | **Topology as a versioned asset** | Workflow | Keep human-authored graphs for audit; do not expect runtime LLM DAGs to replace compliance blueprints. |
| 7 | **Honest "most tasks never need a graph"** | Auto + docs | Route clarify-and-choose, short Q\&A, and exploratory work to ReAct; reserve DAG for decomposable parallel work. |
**Anti-patterns to avoid when "doing graph":**
* Using a **dependency graph** to simulate a **human state machine** (multi-step "wait for answers" with no wait primitive).
* Equating ReAct **`update_plan`** (checklist memory) with graph engineering (scheduling topology).
* Shipping graph heat as a second chat personality instead of as **engine composition** (loop outside, schedule inside when needed).
* Describing FIM One DAG steps as "shallow LLM calls": that understates the engine and misleads product decisions.
## Five kinds of "planning" in the AI tooling landscape
The word "planning" is overloaded. At least five distinct approaches exist today, and they solve different problems:
| Approach | Plan format | Execution | Approval | Core value |
| - | - | - | - | - |
| **Implicit model planning** | Internal chain-of-thought | Single inference pass | None | The model thinks through steps on its own |
| **Claude Code plan mode** | Markdown document | Serial | Human reviews before execution | Align on approach before touching code |
| **Claude Code Teams** | Task list with dependency edges | **Concurrent** (multi-agent) | Human approves plan, then autonomous | Dynamic agent pool + parallel execution |
| **Kiro spec-driven dev** | Structured spec (requirements + design + tasks) | Serial | Human reviews spec | Traceable requirements, acceptance criteria |
| **FIM One DAG** | JSON dependency graph | **Concurrent** (single orchestrator) | Automatic (PlanAnalyzer) | Parallel execution + runtime scheduling |
The first two are **design-time** planning — they produce a plan *before* work begins, and a human (or the model itself) follows it step by step. The last three introduce **runtime** planning — execution graphs are generated and scheduled programmatically, with independent branches running in parallel. The difference is *who* executes: Claude Code Teams spawns autonomous agents; FIM One DAG dispatches steps within a single orchestrator.
These approaches are not competitors; they are complementary layers. A Kiro-style spec can define *what* to build, while a FIM One DAG can schedule *how* to execute the subtasks concurrently. Claude Code's plan mode ensures a human agrees with the approach; FIM One's PlanAnalyzer verifies the outcome automatically.
## Three-Layer Nesting: The Full-Power Architecture
Both Claude Code Teams and FIM One DAG, at full capacity, exhibit a **three-layer nested architecture**:
```mermaid theme={null}
flowchart TB
subgraph L1["Layer 1: Human-in-the-Loop"]
direction TB
H["User approves plan / confirms direction"]
subgraph L2["Layer 2: DAG Orchestration"]
direction TB
D1["Task graph with dependency edges"]
D2["Parallel dispatch of independent branches"]
D1 --> D2
subgraph L3A["Layer 3: ReAct Loop"]
R1["Perceive → Reason → Act → Observe"]
end
subgraph L3B["Layer 3: ReAct Loop"]
R2["Perceive → Reason → Act → Observe"]
end
subgraph L3C["Layer 3: ReAct Loop"]
R3["Perceive → Reason → Act → Observe"]
end
D2 --> L3A & L3B & L3C
end
H --> L2
end
```
* **Layer 1 — Human gate**: User reviews the plan and approves before execution begins.
* **Layer 2 — DAG orchestration**: The approved plan is decomposed into tasks with dependency edges. Independent tasks run in parallel; downstream tasks wait for their blockers to resolve.
* **Layer 3 — ReAct inner loop**: Each task is executed by an agent running a full ReAct cycle (Perceive → Reason → Act → Observe), capable of multi-step reasoning, tool use, and autonomous retry.
The key insight: **Claude Code Teams and FIM One DAG implement the same three layers, just with different Layer 2 mechanics** — message-passing vs dependency-edge resolution.
## Full-Power Runtime: FIM One vs Claude Code Teams
Both are genuine Agents — the core loop is identical: **Perceive → Reason → Act → Feedback**. The difference lies in how they orchestrate parallel work at full capacity.
| Dimension | Claude Code Teams | FIM One DAG |
| - | - | - |
| **Parallel model** | Leader spawns SubAgents, assigns tasks via messages | Topological sort auto-parallelizes independent steps |
| **Task graph** | TaskList with `blockedBy` / `blocks` edges (dynamic DAG) | Static JSON DAG with `depends_on` edges |
| **Coordination** | Explicit message passing (SendMessage / Broadcast) | Implicit dependency edges — no messages, just data flow |
| **Agent lifecycle** | Dynamic pool — agents spawned on demand, shut down when done | Per-step **ReAct agents** (fresh agent per step via `_resolve_agent`); each step runs a full multi-iteration tool loop |
| **Feedback & correction** | Each SubAgent retries autonomously; Leader re-assigns on failure | PlanAnalyzer evaluates outcomes → Re-Planning loop (up to 3 rounds); completed steps can carry into the next plan |
| **Human involvement** | Plan mode approval, then autonomous execution | Fully automatic — PlanAnalyzer decides pass/replan (chat-level ASK is ReAct-only) |
| **Context management** | Each SubAgent gets isolated context window (no cross-contamination) | Shared DbMemory + LLM Compact across the turn; each step agent still sees its task + dependency results, not a full peer transcript |
| **Token economics** | `N agents × per-agent tokens` — time↓ tokens↑ (multiplicative cost) | Parallel branches cost more than serial ReAct, but shared memory and step caps usually stay below multi-agent team spend |
| **Scaling pattern** | Add more SubAgents (horizontal, message-coupled) | Add more DAG branches (horizontal, dependency-coupled) |
| **Best suited for** | Diverse, loosely-related tasks (research + code + test) | Structured workflows with clear data dependencies |
### Real-World Benchmark: v0.5 RAG System
Claude Code Teams built FIM One's entire v0.5 RAG subsystem in a single session:
* **8 phases**: Embedding → Reranker → Loaders → Chunking → VectorStore → Retrieval → KB Backend → Frontend + Docs
* **46 tests** passing, frontend build clean
* **Wall time**: \~5 minutes
* **Token cost**: \~100k tokens per agent task × 8+ tasks ≈ 800k+ total tokens
* **Dependency edges**: Phase 5 depends on Phase 4 + 1b; Phase 6 depends on Phase 5 + 2 + 3 — a genuine DAG
This demonstrates the core trade-off: **time parallelism at the cost of token multiplication**. Claude Code Teams trades compute dollars for developer hours.
### Converging, Not Competing
The boundary between "team collaboration" and "pipeline scheduling" is blurring:
* **Claude Code Teams' `blockedBy`/`blocks` IS a DAG** — tasks have explicit dependency edges, and the leader dispatches newly-unblocked tasks as predecessors complete. This is topological scheduling with extra steps (messages).
* **FIM One DAG steps are already full ReAct agents** — Layer 3 is not aspirational. Each step runs `agent.run()` with tools and multi-iteration loops; differences vs a top-level ReAct chat turn are deliberate (no per-step plan board, no chat-level `ask_user_question` wait, model chosen by `model_hint` / general default).
**Takeaway:** Same Agent essence, converging parallel philosophies. Claude Code follows a **team collaboration** model — a Leader delegates to Workers who communicate via messages. FIM One follows a **pipeline scheduling** model — a DAG Executor dispatches **ReAct steps** based on dependency resolution. In practice, both implement dependency-driven parallel execution; the difference is coordination overhead (messages vs edges), isolation (peer agents vs task + dependency results), and token economics. The product gap is less "make nodes into agents" and more **when to schedule a graph vs stay in one chat loop**, plus **human wait edges** where the graph cannot pause.
## Structured Output Degradation
All structured LLM call sites in the DAG pipeline (Planner, Analyzer, Tool Selection) use a unified `structured_llm_call()` utility that implements a 3-level degradation chain:
| Level | Condition | How it works |
| - | - | - |
| **Native FC** | `llm.abilities["tool_call"]` | Forces a virtual tool call; extracts from `tool_calls[0].arguments` |
| **JSON Mode** | `llm.abilities["json_mode"]` | Sets `response_format={"type":"json_object"}`; parses with `extract_json()` |
| **Plain text** | always available | Parses free-form content with `extract_json()`, then optional `regex_fallback()` |
Each text-based level retries once with a reformat prompt before falling to the next. The result is a `StructuredCallResult` containing the parsed value, which extraction level succeeded, and accumulated token usage.
This design means the same prompt works reliably across GPT-4 (native FC), Claude (JSON mode), and local models (plain text), with consistent error handling and retry logic in one place instead of scattered across four call sites.
## Three Execution Layers: Control Spectrum
The five planning approaches above describe the landscape. Within FIM One itself, three execution layers offer a **control spectrum** from full human control to full AI autonomy. Relative to graph engineering: **Workflow** is the design-time graph product; **DAGPlanner** is a runtime task graph; **ReAct** is loop engineering (with optional plan-board memory, not a schedule graph).
| Layer | Graph defined by | Graph exists when? | Graph-engineering role | Best for |
| - | - | - | - | - |
| **Workflow Engine** | User (visual canvas / JSON blueprint) | Design time | Closest to classic visual Workflow + durable process topology | Deterministic processes, compliance/audit trails, cron-scheduled automation |
| **DAGPlanner** | LLM (auto-decomposes the goal) | Runtime | Task dependency graph + parallel schedule + outcome analysis | Decomposable multi-step work with clear sub-task boundaries |
| **ReAct Agent** | None (iterative loop) | Never as a schedule graph | Loop layer; nodes in a larger graph can be ReAct agents | Exploratory, conversational, clarify-then-act, iterative refinement |
The design principle: **give users exactly as much control as they want**.
* Need to prove every loan approval passes five specific steps? → Workflow.
* Need to research three topics in parallel then synthesize? → DAG.
* Need to draft, refine, or ask structured questions mid-run? → ReAct.
* Not sure? → `execution_mode: "auto"` classifies the query and routes to DAG or ReAct at runtime (prefer ReAct when the goal is primarily clarification or short exploration).
This layering is what separates FIM One from single-paradigm tools:
* **Dify** emphasizes the Workflow layer (static or semi-static visual graphs).
* **LangGraph** emphasizes a code-level control-flow graph DSL (developer-defined topology, dynamic routing, checkpointing). Structurally related to visual workflows, expressed as software.
* **Manus-class agents** emphasize the Agent/loop layer (autonomous execution, little user-defined structure).
FIM One covers all three layers, plus automatic routing between chat engines. The progression **user-defined graph → LLM-generated task graph → no schedule graph** matches a broader industry honesty: as models improve, **explicit topology becomes optional for most turns**, not mandatory UI. Graph engineering heat is a reason to **invest in engines and composition**, not to force every chat through a planner.
### Related docs
* [ReAct Engine](/architecture/react-engine) — loop, tools, plan board, clarifying questions
* [DAG Engine](/architecture/dag-engine) — planner, executor, analyzer, replan
* [Competitive Landscape](/strategy/competitive-landscape) — Dify / Manus / Coze positioning
# Progressive Disclosure
Source: https://docs.one.fim.ai/architecture/progressive-disclosure
How FIM One minimizes context consumption by loading tool definitions on demand -- stub-first, detail-on-request.
## The problem
LLMs pay for context in two currencies: tokens and attention. Every tool definition injected into the system prompt costs both. A single MCP server can expose 90+ tools. Five API connectors with 20 actions each produce 100 tool definitions. Three database connectors with 30 tables each generate another 90 schema descriptions. Before the user even types a word, the system prompt can consume 50--100KB of context -- half the budget of a 128K model.
The cost is not just tokens. Research and practice consistently show that **LLM accuracy degrades as irrelevant context grows.** An agent with 80 tool definitions in its system prompt performs measurably worse on tool selection than one with 6. The model spends attention on tool schemas it will never use, diluting its focus on the tools and instructions that matter.
The naive solution -- inject everything, let the model sort it out -- does not scale. FIM One takes the opposite approach: **show the LLM the minimum it needs to make a decision, and let it request more when it needs more.**
## The pattern
Progressive disclosure follows a two-tier architecture across all resource types:
1. **Tier 1 -- Stubs in the system prompt.** Lightweight summaries: a name, a short description, and enough metadata (action count, table count, tool count) for the LLM to decide whether it needs more.
2. **Tier 2 -- Full details on demand.** The LLM calls a meta-tool to retrieve complete schemas, parameters, and execution capabilities. The full detail enters the conversation as a tool result message -- scoped to that turn, not permanently occupying the system prompt.
```mermaid theme={null}
sequenceDiagram
participant SP as System Prompt
participant LLM as LLM
participant MT as Meta-Tool
participant T as Actual Tool/API
SP->>LLM: Stubs only (name + description)
LLM->>LLM: Decides which resource to use
LLM->>MT: discover("salesforce")
MT-->>LLM: Full action schemas (JSON)
LLM->>MT: execute("salesforce", "get_contacts", {...})
MT->>T: Proxied execution
T-->>MT: Result
MT-->>LLM: Formatted result
```
The key insight: **full tool schemas are conversation-scoped, not prompt-scoped.** They appear as tool result messages that the context management system can summarize or truncate in later turns. In contrast, system prompt content persists across the entire conversation at full size.
## Five disclosure mechanisms
FIM One applies progressive disclosure uniformly across five resource types. Each uses the same two-tier pattern but with a meta-tool tailored to its semantics.
| Resource | Meta-Tool | Stubs Show | On-Demand Returns | Config Var | Default |
| - | - | - | - | - | - |
| Skills | `read_skill` | Name + description (120 chars) | Full SOP content + embedded script | `SKILL_TOOL_MODE` | `progressive` |
| API Connectors | `connector` | Connector name + action list | Full action schemas with parameters | `CONNECTOR_TOOL_MODE` | `progressive` |
| Database Connectors | `database` | DB name + table names + counts | Column schemas, SQL query execution | `DATABASE_TOOL_MODE` | `progressive` |
| MCP Servers | `mcp` | Server name + tool list | Full tool schemas + invocation | `MCP_TOOL_MODE` | `progressive` |
| Built-in Tools | `request_tools` | Compact catalog (name + 80-char desc) | Full tool schema injected into session | *(auto)* | Auto when >12 tools |
### Skills -- `read_skill`
**What the LLM sees initially:**
```
## Available Skills
Call read_skill(name) to load full content before executing any of these:
- Customer Complaint SOP: Handle escalations per company policy...
- Refund Processing: Step-by-step refund workflow with approval gates...
```
Each stub is roughly 30 tokens -- a name plus a 120-character description truncated from the full Skill content.
**What happens on demand:** The LLM calls `read_skill("Customer Complaint SOP")` and receives the complete SOP text -- potentially thousands of tokens of step-by-step instructions, decision trees, and embedded scripts. This content enters as a tool result, not as system prompt text, so it is subject to normal context management (summarization, truncation) in later turns.
**Legacy mode:** `SKILL_TOOL_MODE=inline` embeds the full Skill content directly in the system prompt. Suitable when you have few, small Skills -- but scales poorly.
**Context savings:** A deployment with 10 Skills averaging 2,000 tokens each consumes \~300 tokens in progressive mode (stubs only) vs. \~20,000 tokens in inline mode. That is a 98% reduction in persistent context cost.
### API Connectors -- `connector`
**What the LLM sees initially:**
```
Interact with external services. Available connectors:
- salesforce: CRM system -- actions: get_contacts, create_lead, update_opportunity
- jira: Project management -- actions: create_issue, get_issue, search_issues
Subcommands:
discover -- list actions with full parameter schemas
execute {"param": "value"} -- run an action
```
Each connector stub lists action names but not parameter schemas. The LLM knows *what* actions exist but not *how* to call them -- which is exactly the right level of detail for deciding whether to use a connector.
**What happens on demand:** `connector("discover", "salesforce")` returns the full action schemas including HTTP methods, URL paths, parameter JSON Schemas, and request body templates. `connector("execute", "salesforce", "get_contacts", {"limit": 10})` proxies execution through `ConnectorToolAdapter` with full auth injection and audit logging.
**Legacy mode:** `CONNECTOR_TOOL_MODE=legacy` registers each action as a separate tool (`salesforce__get_contacts`, `salesforce__create_lead`, etc.). A connector with 20 actions becomes 20 tool definitions in the system prompt.
**Context savings:** A connector with 15 actions generates \~50 tokens of stub vs. \~3,000 tokens of full schemas. Five connectors: \~250 tokens progressive vs. \~15,000 tokens legacy.
### Database Connectors -- `database`
**What the LLM sees initially:**
```
Query connected databases. Available databases:
- hr_postgres: HR system (12 tables: employees, departments, salaries ...)
- analytics_db: Analytics warehouse (45 tables: events, sessions, users ...)
Subcommands:
list_tables -- table names, descriptions, column counts
discover [table] -- full column schemas for one or all tables
query -- execute a SQL query
```
Database stubs include table names (up to 10) and counts, giving the LLM enough to decide which database to query without loading any column schemas.
**What happens on demand:** Three subcommands form a natural discovery flow:
1. `database("list_tables", "hr_postgres")` -- returns all table names with descriptions and column counts.
2. `database("discover", "hr_postgres", table="employees")` -- returns full column schemas (name, type, nullable, primary key, descriptions).
3. `database("query", "hr_postgres", sql="SELECT ...")` -- executes a validated SQL query with safety checks and row limits.
The three-step flow mirrors how a developer explores a new database: browse tables, inspect schema, then query. The LLM follows the same pattern naturally.
**Legacy mode:** `DATABASE_TOOL_MODE=legacy` registers three tools per database (`{db}__list_tables`, `{db}__describe_table`, `{db}__query`). With 5 database connectors, that is 15 tool definitions instead of 1.
**Context savings:** A database with 30 tables and 200 columns generates \~80 tokens of stub vs. \~5,000 tokens of full schema. The savings compound with multiple databases.
### MCP Servers -- `mcp`
**What the LLM sees initially:**
```
Interact with MCP servers. Available servers:
- github: GitHub (35 tools: create_issue, list_repos, get_pull_request ...)
- slack: Slack (12 tools: send_message, list_channels, upload_file ...)
Subcommands:
discover -- list tools with full parameter schemas
call {"param": "value"} -- invoke an MCP tool
```
MCP servers are the most dramatic case for progressive disclosure. A GitHub MCP server exposes 35+ tools. A filesystem server exposes 20+. Without progressive disclosure, connecting 3 MCP servers could inject 70+ tool definitions into the system prompt -- each with full JSON Schema parameters.
**What happens on demand:** `mcp("discover", "github")` returns the complete tool catalog with parameter schemas. `mcp("call", "github", "create_issue", {"title": "Bug report", "body": "..."})` delegates to the stored `MCPToolAdapter`, which communicates with the MCP server process.
**Legacy mode:** `MCP_TOOL_MODE=legacy` registers each MCP tool as a separate tool (`github__create_issue`, `github__list_repos`, etc.). This can easily exceed the tool selection threshold and trigger unnecessary selection phases.
**Context savings:** The savings here are extreme. A GitHub MCP server's 35 tools might consume 10,000+ tokens of schema. In progressive mode, the stub costs \~100 tokens. If the user never needs GitHub in that conversation, those 10,000 tokens are never spent.
### Built-in Tools -- `request_tools`
The fifth mechanism is architecturally different from the other four. It does not consolidate a resource type behind a meta-tool. Instead, it addresses the **tool selection bottleneck** -- what happens when the agent has more than 12 tools available.
**How it works:** When the total tool count exceeds `REACT_TOOL_SELECTION_THRESHOLD` (default: 12), the ReAct engine runs a lightweight LLM call to select the top 6 most relevant tools for the current query. The remaining tools are stored in a full registry. A `request_tools` meta-tool is automatically registered, listing all unloaded tools as a compact catalog (name + 80-character description).
```mermaid theme={null}
flowchart TD
A["30 tools available"] --> B{"Count > 12?"}
B -->|Yes| C["LLM selects top 6"]
C --> D["6 active tools + request_tools meta-tool"]
D --> E["LLM uses active tools"]
E --> F{"Needs a tool not in active set?"}
F -->|Yes| G["request_tools(['web_search', 'email_send'])"]
G --> H["Tools loaded into active set"]
H --> I["System prompt rebuilt with new tools"]
F -->|No| J["Continue normally"]
```
**What the LLM sees initially:**
```
Load additional tools into the current session.
Available tools not yet loaded:
- web_search: Search the web for current information and return relevant results...
- email_send: Send an email to one or more recipients with subject, body, and opt...
- python_exec: Execute Python code in a sandboxed environment and return the output...
```
**What happens on demand:** `request_tools(tool_names=["web_search", "email_send"])` copies those tools from the full registry into the active registry. The system prompt is rebuilt on the next iteration so the LLM sees the full schemas. This is a side-effect -- the tool mutates the active tool set mid-conversation.
**No env var:** This mechanism activates automatically when tool selection filters the set. There is no `REQUEST_TOOLS_MODE` environment variable. If you want to disable tool selection entirely, set `REACT_TOOL_SELECTION_THRESHOLD` to a very high number.
**Context savings:** The savings depend on how many tools are available and how many the selection picks. An agent with 30 tools seeing only 6 active schemas + the `request_tools` catalog saves roughly 60--70% of the tool schema context.
## How it fits into the tool assembly pipeline
The [System Overview](/architecture/system-overview) describes an 8-step per-request tool assembly pipeline. Progressive disclosure acts at multiple points:
| Pipeline Step | Progressive Disclosure Role |
| - | - |
| **1. Base discovery** | No effect -- built-in tools are loaded normally |
| **2. Agent category filter** | No effect -- category filtering applies regardless of mode |
| **3. KB injection** | No effect -- KB tools are naturally lightweight (1--2 tools) |
| **4. Connector loading** | `ConnectorMetaTool` consolidates all API connectors; `DatabaseMetaTool` consolidates all DB connectors |
| **5. MCP loading** | `MCPServerMetaTool` consolidates all MCP servers into one tool |
| **6. Skills injection** | `ReadSkillTool` replaces full content with compact stubs in system prompt |
| **7. CallAgent registration** | No effect -- `call_agent` is already a single tool with a catalog |
| **8. Runtime selection** | `request_tools` meta-tool registered when selection filters the set |
The net effect: steps 4--6 each reduce their tool count to 1 (or a small constant), and step 8 adds a safety net for dynamically loading anything the selection phase missed. A Hub agent that would have 50+ tools in legacy mode might present 8--10 in progressive mode -- well below the selection threshold.
## Configuration
Four environment variables control progressive disclosure, one per resource type:
| Variable | Values | Default | Effect |
| - | - | - | - |
| `SKILL_TOOL_MODE` | `progressive` / `inline` | `progressive` | Skills: stubs + `read_skill` vs. full content in system prompt |
| `CONNECTOR_TOOL_MODE` | `progressive` / `legacy` | `progressive` | API Connectors: single `connector` meta-tool vs. individual action tools |
| `DATABASE_TOOL_MODE` | `progressive` / `legacy` | `progressive` | DB Connectors: single `database` meta-tool vs. 3 tools per database |
| `MCP_TOOL_MODE` | `progressive` / `legacy` | `progressive` | MCP Servers: single `mcp` meta-tool vs. individual server tools |
**Agent-level override.** Each env var can be overridden per-Agent via the `model_config_json` field:
```json theme={null}
{
"model_config_json": {
"skill_tool_mode": "inline",
"connector_tool_mode": "legacy",
"database_tool_mode": "progressive",
"mcp_tool_mode": "progressive"
}
}
```
**Priority order:** Agent config > environment variable > default.
This means you can run `progressive` globally (the default) and selectively override for specific Agents. An Agent with a single small Skill might use `inline` mode. An Agent that needs the LLM to see all connector actions upfront (e.g., for fine-tuned models that do not reliably call meta-tools) might use `legacy` mode.
**`request_tools` has no configuration.** It activates automatically when tool selection produces a filtered subset. The threshold is controlled by `REACT_TOOL_SELECTION_THRESHOLD` (default: 12) and the max selection count by `REACT_TOOL_SELECTION_MAX` (default: 6).
## Design decisions
### Why explicit (LLM-driven) rather than implicit (framework-driven)?
An alternative design would have the framework automatically expand tool schemas based on heuristics -- e.g., detecting which connector the user's query is about and injecting its schemas before the LLM sees the prompt. FIM One deliberately chose the LLM-driven approach for three reasons:
1. **The LLM is better at intent detection than heuristics.** A query like "check if the customer has an open ticket and update their profile" involves two connectors. Heuristic matching on keywords is fragile; the LLM naturally identifies both.
2. **Transparency.** When the LLM calls `connector("discover", "jira")`, the action appears in the tool trace. The user (and the developer debugging) can see exactly which schemas were loaded and when. Implicit expansion is invisible.
3. **Context efficiency.** The framework cannot know which actions within a connector the LLM will need. Expanding all actions for a connector wastes tokens on irrelevant ones. The LLM first sees the action names (via the stub), then requests only the specific action's schema -- two-tier disclosure at its purest.
### Why per-resource meta-tools rather than one universal tool?
A single `discover_resource(type, name)` tool would be simpler to implement but worse for the LLM. Per-resource meta-tools provide:
* **Typed parameters.** `connector` has `subcommand`, `connector`, `action`, `parameters`. `database` has `subcommand`, `database`, `table`, `sql`. The parameter schemas tell the LLM exactly what is expected.
* **Enum constraints.** Each meta-tool lists its valid names (connector names, database names, server names) as enum values in the schema. The LLM cannot hallucinate a connector name.
* **Category semantics.** The `connector` tool has category `connector`, `database` has category `database`, `mcp` has category `mcp`. This feeds into agent category filtering -- an Agent configured with only `connector` category will not see the `database` or `mcp` meta-tools.
### Why both progressive and legacy modes?
Not all LLMs handle meta-tools equally well. Smaller or fine-tuned models may struggle with the two-step discover-then-execute pattern. Legacy mode provides a direct fallback where every action is a standalone tool with its full schema visible -- no meta-tool indirection required.
The dual-mode design also supports migration. Existing deployments can switch to progressive mode incrementally, testing one resource type at a time by changing a single environment variable. Each environment variable acts as a feature flag -- blast radius is limited to a single resource type, and rollback is a one-line config change.
A third, more practical reason: **debugging.** In legacy mode every tool call is explicit and self-contained -- `salesforce__get_contacts(limit=10)` is immediately readable in logs and traces. In progressive mode the same call is `connector("execute", "salesforce", "get_contacts", {"limit": 10})` -- an extra layer of indirection that requires parsing the meta-tool arguments to understand what actually happened. During development and troubleshooting, switching a single resource type back to legacy mode can significantly speed up diagnosis without affecting other resource types.
# ReAct Engine
Source: https://docs.one.fim.ai/architecture/react-engine
How FIM One's ReAct engine works — dual-mode execution, structured LLM calls, and streaming answer synthesis.
For how this loop sits next to DAG scheduling, Workflow blueprints, and industry **graph engineering**, see [Planning Landscape](/architecture/planning-landscape).
## The architecture
The ReAct engine implements a two-phase execution model. The first phase is an iterative tool-use loop: the agent repeatedly asks the LLM for an action, executes any requested tool, appends the observation, and continues until the LLM signals "done." The second phase is answer synthesis: a separate streaming LLM call that reads the full execution trace and produces the user-facing response.
This split is deliberate. Tool iterations are optimized for speed — every LLM call in the loop uses non-streaming `chat()`, because the user does not need to see partial JSON actions or intermediate reasoning tokens. Answer generation is optimized for UX — it uses streaming `stream_chat()` so the user sees tokens appearing in real time. The result is the best of both worlds: fast tool execution with responsive answer delivery.
```mermaid theme={null}
flowchart TD
A["User query"] --> B["Tool loop (non-streaming chat() x N)"]
B -->|"tool_call"| E["Execute tool"]
E --> F{"Cycle detected?"}
F -->|"No"| B
F -->|"Yes"| G["Inject warning: try different approach"]
G --> B
B -->|"done signal"| H{"tools ≥ threshold?"}
H -->|"Yes (once)"| I["Completion checklist: verify answer"]
I --> B
H -->|"No / already checked"| C["stream_answer() (streaming)"]
C --> D["SSE token stream → frontend"]
```
The tool loop produces an `AgentResult` containing the full conversation history — system prompt, user query, every assistant message, every tool result. The `stream_answer()` method distills this trace into a concise, coherent answer. Tool results are truncated to 2,000 characters each in the synthesis context, keeping the prompt lean even after complex multi-tool workflows.
**Model binding.** The LLM is injected into `ReActAgent.__init__()` and stored as `self._llm`. Every call within a single `run()` invocation — all tool loop iterations and the final answer synthesis — uses this same instance. The model does not change between iterations. To use a different model, a new `ReActAgent` must be constructed. In DAG mode, `DAGExecutor._resolve_agent()` exploits this pattern: it creates a fresh agent per step (selecting the model from `ModelRegistry` based on `step.model_hint`) immediately before that step's ReAct loop begins. See [DAG Engine — Per-step override](/architecture/dag-engine#multi-llm-architecture) for details.
## Dual-mode execution
The ReAct engine supports two distinct modes of interacting with the LLM during the tool loop.
**JSON Mode** (`_run_json`) embeds tool descriptions directly in the system prompt and instructs the LLM to respond with a JSON object — either a `tool_call` action with a tool name and arguments, or a `final_answer` signal. The agent parses the JSON from the response content, executes the tool, and appends the observation as a user message.
**Native Function Calling** (`_run_native`) uses the LLM provider's built-in tool-calling API. Tool descriptions are passed via the `tools` parameter, and the LLM returns structured `tool_calls` in the API response rather than emitting JSON in its content. This is the preferred mode for models that support it.
Mode selection is automatic. The property `_native_mode_active` returns `True` only when both conditions hold: the agent was created with `use_native_tools=True` (the default) and the LLM advertises `abilities["tool_call"] = True`. If either condition fails, the engine falls back to JSON mode.
| Aspect | JSON Mode | Native Function Calling |
| - | - | - |
| LLM output | JSON object in message content | `tool_calls` in API response |
| System prompt | Embeds full tool descriptions in text | Tools passed via `tools` parameter |
| Parallel tool calls | One tool per iteration | Multiple via `asyncio.gather` |
| Parse failure handling | Retry with reformat prompt | N/A (structured by API) |
| Loop LLM calls | Non-streaming `chat()` | Non-streaming `chat()` |
| Best for | Models without tool-call support | GPT-4, Claude, and similar |
Both modes share the same answer synthesis phase — `stream_answer()` works identically regardless of how the tool loop ran.
## structured\_llm\_call — unified output extraction
Any call site that needs the LLM to return data conforming to a JSON schema uses `structured_llm_call()`. This is the single entry point for structured output across the entire framework — the DAG planner, plan analyzer, tool selection, and any future component that needs parsed JSON from an LLM.
The function implements a 3-level degradation chain, attempting each level in order based on the LLM's advertised capabilities:
**Level 1: Native Function Calling.** Uses the LLM's `tool_call` / `tool_choice` API to force a structured response. Available when `abilities["tool_call"] = True`. If the LLM returns `tool_calls`, the arguments are extracted directly. If parsing fails, falls through to the next level.
**Level 2: JSON Mode.** Sets `response_format={"type": "json_object"}` to constrain the LLM's output format. Available when `abilities["json_mode"] = True`. If the response cannot be parsed, retries once with a reformat prompt ("Your previous response could not be parsed as valid JSON..."), then falls through.
**Level 3: Plain Text.** Calls the LLM with no format constraints and extracts JSON from free-form text using `extract_json()`. If extraction fails, an optional `regex_fallback` function is tried. Retries once with the reformat prompt before giving up.
```mermaid theme={null}
flowchart TD
START["structured_llm_call()"] --> CHECK1{"abilities.tool_call?"}
CHECK1 -->|Yes| L1["Level 1: Native FC"]
CHECK1 -->|No| CHECK2{"abilities.json_mode?"}
L1 -->|Success| DONE["Return StructuredCallResult"]
L1 -->|Fail| CHECK2
CHECK2 -->|Yes| L2["Level 2: JSON Mode"]
CHECK2 -->|No| L3["Level 3: Plain Text"]
L2 -->|Success| DONE
L2 -->|Fail| L2R["Retry with reformat prompt"]
L2R -->|Success| DONE
L2R -->|Fail| L3
L3 -->|Success| DONE
L3 -->|Fail| L3R["Retry with reformat prompt"]
L3R -->|Success| DONE
L3R -->|Fail| DEFAULT{"default_value?"}
DEFAULT -->|Yes| DVAL["Return default"]
DEFAULT -->|No| ERR["Raise StructuredOutputError"]
```
The degradation chain means every model — from GPT-4 with full tool-call support to a local LLM that can only produce plain text — can participate in structured output scenarios. The worst case is 5 LLM calls (1 native + 1 JSON + 1 JSON retry + 1 plain + 1 plain retry), but in practice most calls resolve at Level 1 in a single attempt.
| Model capability | Path taken | Max LLM calls |
| - | - | - |
| tool\_call + json\_mode | L1 → L2 → L3 | 5 |
| json\_mode only | L2 → L3 | 4 |
| Plain text only | L3 | 2 |
The result is a `StructuredCallResult` containing the parsed value, the raw dict, which level succeeded, and cumulative token usage. Call sites use `parse_fn` to transform the raw dict into a domain object (e.g., a DAG plan) and `default_value` to provide a fallback when total failure is acceptable.
`structured_llm_call` is used by: the DAG planner (plan schema), the plan analyzer (analysis schema), tool selection (tool list schema), and any component that needs reliable structured output. It is also discussed in [Planning Landscape](/architecture/planning-landscape).
## Tool selection
When an agent has access to many tools — common in Hub mode where multiple connectors each expose several actions — injecting every tool's full schema into the conversation context is wasteful. A connector hub with 20 tools consumes roughly 5K tokens in tool descriptions alone, crowding out space for conversation history and tool results.
The engine addresses this with a lightweight selection phase. When the total number of registered tools exceeds `TOOL_SELECTION_THRESHOLD` (12), the agent runs a preliminary LLM call before entering the main loop. This call receives a compact catalog — approximately 80 characters per tool, containing only the name and a one-line description, no parameter schemas — and picks the most relevant tools for the current query, up to `_TOOL_SELECTION_MAX` (6).
The selection uses `structured_llm_call` with a simple schema (`{"tools": ["tool_name_1", "tool_name_2"]}`), so it benefits from the same 3-level degradation. The selected tool names are used to build a filtered `ToolRegistry` that the main loop uses for both system prompt construction and tool execution.
Selection failure is deliberately non-fatal. If the LLM returns unparseable output, if all selected names are invalid, or if any exception occurs, the agent falls back to the full tool set. This ensures that a flawed selection never prevents the agent from functioning — it just uses more context than optimal.
## The iteration loop
The core loop drives both JSON mode and native mode, with minor differences in message handling. Each iteration follows the same high-level pattern: check the context budget, call the LLM, process the response, and either execute a tool or break.
```mermaid theme={null}
sequenceDiagram
participant Agent as ReActAgent
participant CG as ContextGuard
participant LLM as LLM API
participant Hooks as HookRegistry
participant Tool as Tool Registry
loop Each iteration (max 50)
Agent->>CG: check_and_compact(messages)
CG-->>Agent: messages (within budget)
Agent->>LLM: chat(messages) [non-streaming]
LLM-->>Agent: response
alt tool_call action
Agent->>Hooks: run_pre_tool_use_hooks(ctx)
alt hook blocks
Hooks-->>Agent: allow=False, error="..."
Agent->>Agent: append block message, continue
else hook allows (possibly rewrote args)
Hooks-->>Agent: allow=True, modified_args?
Agent->>Tool: execute(tool_name, args)
Tool-->>Agent: observation
Agent->>Hooks: run_post_tool_use_hooks(ctx, result)
Hooks-->>Agent: modified_result?
Agent->>Agent: cycle check (hash args)
opt identical call ≥ threshold
Agent->>Agent: inject "try different approach" warning
end
Agent->>Agent: append observation, continue
end
else final_answer action
opt tools used ≥ threshold AND not yet checked
Agent->>Agent: inject completion checklist
Agent->>Agent: continue (one more LLM call)
end
Agent->>Agent: break loop
end
end
Agent->>LLM: stream_answer() [streaming]
LLM-->>Agent: token stream → SSE to client
```
**JSON mode loop.** The LLM's response is parsed via `_parse_action()`, which uses `extract_json()` to find a JSON object in the content. If parsing fails, the agent appends the raw response and a reformat request, then continues — this counts against `max_iterations`, preventing infinite retry loops. On success, the action is either a `tool_call` (execute the tool, append the observation as a user message) or a `final_answer` (break and proceed to synthesis).
**Native mode loop.** The LLM's response may contain one or more `tool_calls`. All tool calls in a single response are executed in parallel via `asyncio.gather`, and all tool result messages are appended before any other message. This ordering constraint is critical — the OpenAI API (and compatible providers) requires that `tool` messages immediately follow the `assistant` message that produced the `tool_calls`. Inserting any other message (such as a user interrupt) between them would break the protocol. When no `tool_calls` are present, the response is treated as the final answer.
**Max iterations.** The default limit is 50 iterations. If the loop exhausts this limit without producing a `final_answer`, the agent synthesizes a fallback response from the accumulated step results — a summary of which tools were called and whether they succeeded or failed. This is a safety net, not a normal exit path.
[Context Management](/architecture/context-management) explains how ContextGuard enforces the token budget on every iteration, including the hint system that tells the compaction LLM to preserve recent reasoning chains.
## Tool execution hooks
Every tool call in the iteration loop is wrapped by two hook points:
* **`PreToolUse`** — fires after the LLM picks a tool, before the tool runs. A hook can:
* **Block** the call by returning `allow=False` (the LLM sees an observation explaining why, e.g. "waiting for human approval" or "org is in read-only mode")
* **Rewrite** the tool arguments before execution
* **Trigger side effects** (post a Feishu card, create an audit row, emit a metric)
* **`PostToolUse`** — fires after the tool returns, before the observation reaches the LLM. A hook can:
* **Rewrite** the observation (e.g. truncate a 200 KB SQL result to a 4 KB summary)
* **Trigger side effects** (write to `ConnectorCallLog`, update usage counters)
Hooks run **outside the LLM loop** — nothing the LLM says or does can skip them. This is the deterministic enforcement layer for policies that cannot be trusted to a prompt instruction: human-in-the-loop approval (`FeishuGateHook`), audit logging, rate limiting, and read-only-mode guards. See [Hook System](/architecture/hook-system) for the full design.
The same hook points exist in the [DAG engine](/architecture/dag-engine) — each DAG step runs its own ReAct loop, so hooks apply uniformly regardless of which execution engine is driving the tool call.
## Mid-loop self-reflection
Long reasoning chains (10+ tool calls) risk **goal drift** — the agent gradually shifts focus from the original objective to a local sub-problem, repeats similar actions, or enters circular retry loops. Mid-loop self-reflection is a lightweight countermeasure.
Every `_SELF_REFLECTION_INTERVAL` tool-call iterations (default: **6**), the agent injects a user message into the conversation asking the LLM to pause and reflect:
* Is it still on track toward the original goal?
* Has it been repeating similar actions or going in circles?
* What is the most direct next step to finish?
* Should it produce a final answer now?
The counter tracks **actual tool calls only** — JSON parse retries, thinking events, and interrupt injections do not count. In native mode, the reflection message is appended strictly after all `tool_result` messages to preserve the tool\_use/tool\_result pairing constraint.
This costs \~100 tokens per injection (no extra LLM call) and has no effect on short runs (`< 6` tool calls). It complements ContextGuard (which manages token budgets) and per-step verification (which validates individual results) by addressing a different failure mode: the agent losing sight of its goal over many iterations.
## Cycle detection
Self-reflection asks the LLM whether it is going in circles — but the LLM often answers "no" and keeps looping. Cycle detection is a deterministic alternative that cannot be bypassed.
After every tool execution, the agent hashes the `(tool_name, arguments)` pair. When the same hash appears `_CYCLE_DETECTION_THRESHOLD` times (default: **3**, configurable via `REACT_CYCLE_DETECTION_THRESHOLD`), a warning message is injected into the conversation:
> You have called `{tool}` with identical arguments N times and received the same result. Please try a different approach or tool.
The warning is a regular user message in the agent's internal history — it is not surfaced to the end user as a separate UI event. The agent sees the warning on its next LLM call and is forced to change strategy. The tracker resets at the start of each `run()` call.
Because DAG steps execute via ReActAgent, cycle detection also protects individual DAG steps from tool-call loops.
## Completion checklist
When the agent has used at least `_COMPLETION_CHECK_MIN_TOOLS` tools (default: **3**, configurable via `REACT_COMPLETION_CHECK_MIN_TOOLS`) and decides to produce a final answer, a one-time verification prompt is injected before the answer is accepted:
1. Does the answer fully address the original question?
2. Were key facts verified from tool results?
3. Are there contradictions in the gathered information?
The checklist is an internal exchange. The turn that answers it is verification, not the answer — accepting it verbatim is what leaks "Let me verify: 1. ... ✓" into the chat. So that turn is consumed internally and the agent is asked once more for the user-facing answer, which is what `stream_answer()` streams. If the agent instead decides something is missing, it resumes the tool-call loop and its later answer is taken as-is.
The checklist fires at most once per run, does not trigger for simple conversational responses (no tools used), and is skipped when the remaining iteration budget cannot cover both the verification turn and the answer turn.
The threshold prevents unnecessary latency on simple tasks — a single tool call (e.g., running a quick calculation) does not warrant verification, while a multi-tool investigation benefits from a final sanity check. Set `REACT_COMPLETION_CHECK_MIN_TOOLS=1` for always-on verification.
The feature is enabled by default (`completion_check=True` on `ReActAgent`) and can be disabled per agent instance. DAG steps inherit the default — each step gets its own one-shot checklist.
## Answer synthesis (stream\_answer)
The separation between the tool loop and answer synthesis is a core architectural decision. Tool iterations produce raw data — JSON actions, tool observations, error messages. The user needs a coherent, well-formatted answer, not a dump of the agent's internal trace.
`stream_answer()` builds a synthesis prompt from two components. The system prompt instructs the LLM to act as a synthesizer: present results directly, use markdown formatting, avoid meta-commentary ("based on the tool output..."), and match the language of the original query. The user message contains the original question and a formatted execution trace — each tool call and its result, with tool results truncated to 2,000 characters.
The synthesis call uses `stream_chat()`, yielding tokens incrementally. The web layer wraps these tokens in SSE `answer` events with `delta` status so the frontend can render them as they arrive.
If `stream_answer()` fails — network error, LLM timeout, any exception — the web layer falls back to `result.answer`, the brief text from the tool loop's final iteration. This is a degraded experience (no streaming, potentially less polished prose), but it ensures the user always gets a response.
## Interruption handling
Users can send follow-up messages while the agent is still processing. These are delivered via an `interrupt_queue` — an `InterruptQueue` registered per conversation that accumulates messages between iterations.
The drain timing differs between modes because of the tool-call ordering constraint:
* **JSON mode**: the queue is drained immediately after each assistant message, before checking whether the action is a `final_answer`. This is safe because JSON mode uses plain user/assistant messages with no structural pairing requirements.
* **Native FC mode**: the queue is drained only after tool result messages have been appended. The `tool` messages must immediately follow the `assistant` message containing `tool_calls` — inserting a user message between them would violate the API protocol and cause errors.
Injected messages are marked as `pinned=True`, ensuring they survive any subsequent compaction by ContextGuard. See [Pinned Messages](/architecture/context-management#pinned-messages) for how the pinning mechanism prevents compaction from discarding critical messages.
When a `final_answer` is pending but injected messages have arrived, the agent suppresses the final answer and continues looping so it can address the user's follow-up. Multiple injections from the same drain are combined into a single `[USER INTERRUPT]` message — this prevents the LLM from seeing a fragmented sequence of short messages and encourages it to address all follow-ups holistically.
## Error handling and fallbacks
The engine is designed to never crash on LLM or tool failures. Every error path either recovers silently or surfaces a useful message to the user.
```mermaid theme={null}
flowchart TD
A["JSON parse failure"] -->|"retry once"| B["Append raw + reformat request"]
B -->|"still fails"| C["Treat raw content as final_answer"]
D["Tool not found"] --> E["Error observation → continue loop"]
F["Tool execution exception"] --> G["Error observation → continue loop"]
H["Context over budget"] --> I["ContextGuard compacts → continue"]
J["Max iterations reached"] --> K["Summarize partial results"]
L["stream_answer() failure"] --> M["Fall back to result.answer"]
```
**JSON parse failure.** When the LLM returns non-JSON content in JSON mode, `_parse_action()` wraps it as a `final_answer` with the reasoning `"(could not parse LLM output as JSON)"`. The loop detects this sentinel, appends the raw content and a reformat instruction, and continues. If the retry also fails, the raw content becomes the answer — imperfect, but not a crash.
**Tool errors.** Both "tool not found" and "tool execution exception" produce error observations that are appended to the conversation. The LLM sees the error on the next iteration and can decide whether to retry with different arguments or move on. This makes the agent self-healing for transient tool failures.
**Extended thinking.** Models like DeepSeek R1 return reasoning content in a separate `reasoning_content` field rather than in the JSON body. The engine checks for this and uses it as a fallback when the JSON `reasoning` field is empty.
**Rich content.** When a tool produces HTML or markdown artifacts, the observation sent to the LLM is replaced with a short summary (`"[Artifact generated: filename] The content is rendered as a preview in the UI..."`). This prevents the LLM from echoing large HTML blobs in its final answer — a common failure mode where the model helpfully pastes back the entire tool output.
## SSE event protocol
The web layer translates the agent's iteration callbacks into Server-Sent Events for the frontend. Events are emitted on two SSE channels: `step` for the tool loop, and `answer` for the synthesis phase.
```mermaid theme={null}
sequenceDiagram
participant Agent as ReActAgent
participant SSE as SSE Stream
participant UI as Frontend
Agent->>SSE: step {type: "thinking", status: "start"}
Note over Agent: LLM call (non-streaming)
Agent->>SSE: step {type: "thinking", status: "done", reasoning}
Agent->>SSE: step {type: "iteration", status: "start", tool_name, tool_args}
Note over Agent: Tool execution
Agent->>SSE: step {type: "iteration", status: "done", observation, iter_elapsed}
Note over Agent: ...more iterations...
Agent->>SSE: step {type: "answer", status: "start"}
Agent->>SSE: answer {status: "start"}
loop Token stream
Agent->>SSE: answer {status: "delta", content}
end
Agent->>SSE: answer {status: "done"}
Agent->>SSE: done {answer, iterations, usage, elapsed}
```
| Event | Channel | Payload | When |
| - | - | - | - |
| Thinking start | `step` | `{type: "thinking", status: "start", iteration}` | Before each LLM call |
| Thinking done | `step` | `{type: "thinking", status: "done", iteration, reasoning}` | After LLM responds, before tool execution |
| Iteration start | `step` | `{type: "iteration", status: "start", iteration, tool_name, tool_args}` | Tool execution begins |
| Iteration done | `step` | `{type: "iteration", status: "done", iteration, tool_name, observation, error, iter_elapsed}` | Tool execution completes |
| Answer signal | `step` | `{type: "answer", status: "start"}` | Agent signals final\_answer |
| Answer start | `answer` | `{status: "start"}` | Synthesis streaming begins |
| Answer delta | `answer` | `{status: "delta", content}` | Each streamed token |
| Answer done | `answer` | `{status: "done"}` | Synthesis streaming completes |
| Compact | `compact` | `{original_messages, kept_messages}` | Context was compacted on load |
| Phase | `phase` | `{phase: "selecting_tools", total_tools}` | Tool selection phase active |
| Inject | `inject` | `{type: "inject", content}` | User interrupt received |
| Done | `done` | `{answer, iterations, usage, elapsed}` | Final result payload |
The frontend uses `step` events to render the collapsible tool-call cards (showing which tool is running, its arguments, and the observation), the `answer` deltas to stream the response text, and `compact` to display the context-summarization divider. The `done` event carries the complete metadata — total iterations, token usage, and elapsed time — for the response footer.
# Sharing & Credentials
Source: https://docs.one.fim.ai/architecture/sharing-and-credentials
Who can see a shared resource, whose credentials it runs with, and how access flows through bound resources.
This page is the single reference for **sharing permissions** and **credential
flow**. The shareable resource types are **Agents, Connectors, and MCP
servers** (plus **Skills** and **Workflows** when those optional modules are
enabled). Use it to reason about what happens when you share a resource that is
itself built on top of other resources.
**Knowledge bases are not shareable.** A KB reaches other people only by being
bound to a shared Agent, which delegates the owner's content at run time (see
"How access flows through bound resources"). There is no publish-KB action.
## The two questions
Every sharing decision answers two independent questions:
1. **Visibility** — *can the other person see/use this resource at all?*
2. **Credentials** — *whose secret (API key, DB password, server env) does it
run with?*
Visibility is about the resource; credentials are a separate, per-user concern
that only Connectors and MCP servers carry. Keep them separate in your head —
most confusion comes from conflating "I can see it" with "I can use its owner's
key".
## Visibility: own + subscribed
A resource is visible to you only if **you own it** or **you have an explicit
subscription** to it. There is no implicit "everyone in my org can see it"
anymore — org sharing is delivered through subscriptions.
| Share scope | What it means | How others gain access |
| - | - | - |
| **Personal** (default) | Only the owner can see it. | — |
| **Organization** | Published to an org. | Org members **subscribe** from the Market (or are auto-subscribed on join). |
| **Global / Market** | Published to the public Market. | Anyone **subscribes** from the Market. |
The same rule governs everywhere a resource is used — chat tool assembly, binding
to an agent, and a workflow step all resolve **own + subscribed**. So a connector
you subscribed to can be bound to an agent and called from a workflow; one you
can neither see nor subscribe to is refused in all three places.
Subscriptions are scoped: leaving (or being removed from) an organization
revokes the subscriptions you only had through it, and deletes the per-user
credentials you saved for those resources.
## Credentials & "Allow fallback"
Only **Connectors** and **MCP servers** hold credentials. Each one has an
**Allow fallback** switch that decides whether subscribers may borrow the
owner's credential:
| State | A subscriber **with** their own credential | A subscriber **without** their own credential | The owner |
| - | - | - | - |
| **Allow fallback OFF** *(default)* | Uses their own credential. | **No access** — the tool is hidden and the UI prompts them to configure their own credential. | Always uses their own. |
| **Allow fallback ON** | Uses their own credential. | Falls back to the **owner's** credential (and any usage/billing lands on the owner). | Always uses their own. |
Key rules:
* **Off is the default.** Sharing your token is opt-in. A newly created connector
or MCP server does not share the owner's credential, and existing ones were
migrated to off.
* **The owner is always exempt.** "Allow fallback" only governs *other* users;
you can always use your own connector with your own credential.
* **No silent failure.** When fallback is off and a user has no credential, the
tool is **removed from the toolset** rather than offered and then failing at
call time. Every surface (chat, the **Playground**, workflow steps) must show a
"configure your own credentials" prompt instead of a broken tool.
## How access flows through bound resources
Resources compose. Sharing one resource implicitly exposes the resources bound
underneath it — but **credentials do not automatically flow with them**.
```mermaid theme={null}
flowchart TD
A["Agent"] --> KB["Knowledge Base"]
A --> C["Connector"]
A --> M["MCP Server"]
A -. "skill_ids (legacy)" .-> S["Skill"]
S -- "resource_refs / @alias" --> C
S -- "resource_refs / @alias" --> A
W["Workflow"] --> A
W --> KB
W --> C
W --> M
```
| Container | Can reference | How |
| - | - | - |
| **Agent** | Knowledge Bases, Connectors, MCP servers | `kb_ids` / `connector_ids` / `mcp_server_ids` |
| **Agent** | Skills | `skill_ids` *(legacy — prefer Skill→Agent)* |
| **Skill** | Connectors, Agents, … | `resource_refs` (the `@alias` mechanism) |
| **Workflow** | Agents, Knowledge Bases, Connectors, MCP servers | per-node `agent_id` / `kb_id(s)` / `connector_id` / `mcp_server_id` |
When you share the container, the references travel with it as a **graph of ids**
— a shared workflow or agent carries no embedded secrets. At run time, each
referenced resource re-checks the *runner's* access:
* **Knowledge Bases** have no per-user credential, so a KB bound to a shared
agent **delegates its content**: whoever runs the agent reads the KB owner's
data (the agent needs its KB to function). The delegation is gated by the
**agent owner's** visibility — if the agent owner loses access to a bound KB
(for example after leaving the organization it was shared through), that KB
drops out of the agent instead of continuing to be read. Ad-hoc KB search
outside a bound agent is access-checked against the caller (own + subscribed).
* **Connectors / MCP servers** carry credentials, so they delegate **only when
Allow fallback is on** (see below).
## Worked scenarios
You own an Agent that binds a **personal** connector (you have not shared the
connector itself). You share the **Agent** to your org or globally. Someone else
runs it:
| Scenario | Connector's "Allow fallback" | What the other user gets |
| - | - | - |
| **3a** | **ON** | The connector tool is available and runs with **your** credential — the other user can use it. Usage bills to you. |
| **3b** | **OFF** *(default)* | The connector tool is **hidden** for that user. The rest of the agent still works; the UI prompts them to add **their own** credential for that connector, after which the tool reappears and runs with their key. |
In other words: **sharing the agent does not share the connector's secret.** The
connector's own "Allow fallback" setting is the single switch that decides
whether the credential penetrates through the binding. The same holds for a
connector or MCP server referenced from a Skill or a Workflow step.
This is why the recommended pattern for a connector you intend to share is:
leave **Allow fallback off**, and have each user bind their own credential. Turn
it on only when you deliberately want to lend your key (and absorb the usage) —
for example a metered API you're comfortable paying for on others' behalf.
# System Overview
Source: https://docs.one.fim.ai/architecture/system-overview
How Agent, Knowledge Base, Connector, Built-in Tools, MCP, Skills, and Workflows compose into a unified architecture.
## The unified tool abstraction
The central design insight in FIM One is that **everything the agent can do is a tool**. A calculator, a knowledge base query, an ERP API call, and a third-party MCP server all implement the same `Tool` protocol: `name`, `description`, `parameters_schema`, `category`, and `run()`. The agent does not know or care whether it is calling a local Python function, querying a vector database, proxying into a legacy system, or invoking a community MCP server. It sees a flat list of callable tools in a `ToolRegistry`.
This is a deliberate architectural choice, not an accidental simplification. It means adding a new capability source never requires changing the agent, the execution engines, or the context management layer. You register tools; the agent uses them.
```mermaid theme={null}
flowchart TD
U["User Request"] --> API["API Layer (auth, agent lookup)"]
API --> A["Agent ReAct or DAG engine"]
A --> TR["ToolRegistry (unified interface)"]
TR --- BT["Built-in Tools computation, web, filesystem, general"]
TR --- KB["Knowledge Base KBRetrieve or GroundedRetrieve"]
TR --- CT["Connectors HTTP API, Database, auth injection, audit"]
TR --- MCP["MCP Servers user + global, process-isolated"]
TR --- SK["Skills global SOPs (dynamic recipe), read_skill on demand"]
TR --- WF["Workflows static recipe (DAG), run_workflow on demand"]
```
Six capability sources converge into one registry. The agent draws from all of them equally. The last two are a matched pair: a **Skill** is a *dynamic recipe* (an SOP the LLM interprets step by step), and a **Workflow** is a *static recipe* (a deterministic graph that runs the same way every time). Both are pulled through a tool — `read_skill` and `run_workflow` respectively — and neither contains the other; they are peers consumed by the same runtime.
## Six capability sources
### Built-in Tools
Auto-discovered at startup via `discover_builtin_tools()`. Drop a `BaseTool` subclass into `core/tool/builtin/`, and it registers without any configuration. Categories include computation (`calculator`, `python_exec`), web (`web_search`, `web_fetch`), filesystem (`file_ops`), and general (`email_send`, `json_transform`, `template_render`, `text_utils`). These are the agent's native abilities -- always available, zero setup.
### Knowledge Base
Conditional. When an agent has bound `kb_ids`, the generic `kb_retrieve` tool is swapped for a specialized retrieval tool. In **simple mode**, `KBRetrieveTool` performs basic RAG retrieval. In **grounding mode**, `GroundedRetrieveTool` runs a 5-stage pipeline: multi-KB retrieve, citation extraction, alignment scoring, conflict detection, and confidence computation. The Knowledge Base is not a separate subsystem sitting beside the agent -- it enters the agent as a specialized tool, subject to the same `Tool` protocol as everything else.
### Connector
`ConnectorToolAdapter` wraps enterprise system actions as tools. Each action becomes a tool named `{connector}__{action}`, categorized as `connector`. The adapter adds HTTP proxy with auth injection (bearer, API key, basic), operation-level access control (read/write/admin), response truncation, and audit logging. For direct database access, `DatabaseToolAdapter` provides schema-aware SQL execution with optional read-only enforcement. Connectors are the bridge between AI and legacy systems -- the core differentiator. See [Connector Architecture](/architecture/connector-architecture) for the full design.
### MCP
External MCP servers provide third-party tools via the standard protocol. Each server runs in its own process (stdio or HTTP transport), fully isolated from the platform. Tools are adapted into the `Tool` protocol and registered under category `mcp`. Admins can provision **global MCP servers** that load for all users automatically. MCP is the ecosystem play -- any MCP-compatible server works without custom integration.
### Skills
Skills are reusable Standard Operating Procedures (SOPs) -- company policies, handling procedures, step-by-step workflows -- that apply globally regardless of which Agent is selected. Unlike Connectors and Knowledge Bases (which can be scoped to specific Agents), Skills are always loaded for every user based on visibility (personal, org-shared, or Market-subscribed).
Skills support two injection modes -- **progressive** (default) and **inline** -- controlled by `SKILL_TOOL_MODE`. In progressive mode, compact stubs appear in the system prompt and the LLM calls `read_skill(name)` on demand. This is part of the broader [Progressive Disclosure](/architecture/progressive-disclosure) architecture that applies the same stub-first, detail-on-request pattern across Skills, Connectors, Databases, and MCP Servers.
For a deeper look at why Skills are global (not Agent-bound) and how they interact with the dual-mode resource discovery, see [Agent & Resource Discovery](/architecture/agent-discovery).
### Workflows
A Workflow is the **static recipe** to a Skill's dynamic one: a deterministic DAG an agent reaches for when a sub-task must run identically every time and leave an audit trail (scheduled reconciliations, multi-step approval-free pipelines). Like Skills, runnable Workflows are loaded globally per user by visibility — independent of agent selection — and exposed through a single `run_workflow(name, inputs)` tool, with each workflow's input fields advertised as a compact system-prompt stub. The LLM pulls one by name when a task maps onto it.
Only Workflows that are **active** and **free of human-approval (`HUMAN_INTERVENTION`) nodes** are inline-runnable — a gate that blocks for minutes or hours cannot sit inside an agent turn, so those Workflows remain trigger-only from the Workflows page. Inline runs execute as the workflow owner (subscribed Workflows use the publisher's bound credentials), persist a `WorkflowRun` for the audit trail, and are protected by a re-entrancy guard plus a nesting-depth cap so a Workflow's `AGENT` node can never start an unbounded call cycle.
## Per-request tool assembly
Every chat request assembles a fresh tool set through a filtering pipeline in `_resolve_tools()`. This is not a static configuration -- it is computed per request based on the agent's settings, the user's identity, and the available connectors and MCP servers.
```mermaid theme={null}
flowchart LR
A["discover_builtin_tools()"] --> B["filter_by_category()"]
B --> C["KB injection"]
C --> D["Connector + DB loading\n(progressive)"]
D --> E["MCP loading\n(progressive)"]
E --> SK["Skills injection"]
SK --> WF["Workflow injection"]
WF --> CA["CallAgent registration"]
CA --> F{"Tools > 12?"}
F -->|Yes| G["LLM tool selection"]
F -->|No| H["Use full set"]
G --> I["Final ToolRegistry"]
H --> I
I --> HK["HookRegistry PreToolUse / PostToolUse hooks wrap every tool call"]
```
The eight steps:
1. **Base discovery.** `discover_builtin_tools()` loads all built-in tools, scoped to the conversation's sandbox.
2. **Agent category filter.** `filter_by_category(*agent.tool_categories)` restricts to only the categories the agent is allowed to use.
3. **KB injection.** If the agent has `kb_ids`, the generic retrieval tool is replaced with `KBRetrieveTool` or `GroundedRetrieveTool` based on retrieval mode.
4. **Connector loading.** In agent-constrained mode, only the agent's bound connectors are loaded. In auto-discovery mode (no agent selected), all connectors visible to the user are loaded. Both API connectors (`ConnectorMetaTool`) and database connectors (`DatabaseMetaTool`) use [progressive disclosure](/architecture/progressive-disclosure) by default -- lightweight stubs in the system prompt, full schemas loaded on demand.
5. **MCP loading.** The user's personal MCP servers plus admin-provisioned global MCP servers are loaded and connected. In progressive mode (default), a single `MCPServerMetaTool` consolidates all servers; the LLM calls `discover` and `call` subcommands on demand. See [Progressive Disclosure](/architecture/progressive-disclosure).
6. **Skills + Workflows injection.** All active Skills visible to the user are loaded -- regardless of agent selection. In progressive mode, `ReadSkillTool` is registered with compact stubs in the system prompt; in inline mode, full Skill content is embedded directly. The same step loads all active, inline-runnable Workflows and registers `RunWorkflowTool` with one stub per workflow (name, description, input fields). Workflows carrying a human-approval node are skipped here.
7. **CallAgent registration (Auto mode only).** When no specific Agent is selected, all active, visible Agents are assembled into a catalog and exposed via `CallAgentTool`, enabling the LLM to delegate tasks to specialist agents. Delegated agents receive a full `ToolRegistry` built from their own configuration but exclude `call_agent` to prevent infinite recursion. When a specific Agent is selected, `CallAgentTool` is not registered — agents are specialized and do not delegate to other agents. This prevents marketplace agents from accessing other agents' private prompts.
8. **Runtime selection.** If the total tool count exceeds 12, a lightweight LLM call picks the most relevant subset (up to 6) for this specific query. A `request_tools` meta-tool is automatically registered, allowing the LLM to dynamically load additional tools mid-conversation if the initial selection missed a needed tool. Selection failure is non-fatal -- the agent falls back to the full set. See [Progressive Disclosure](/architecture/progressive-disclosure).
9. **Hook registration.** The agent's declared hooks (from `model_config_json.hooks`) are instantiated and attached to a `HookRegistry`. Each chosen tool call will be wrapped: `PreToolUse` hooks can block or rewrite args before execution; `PostToolUse` hooks can rewrite the observation before it returns to the LLM. Hooks run **outside the LLM loop** and cannot be bypassed by agent instructions — see [Hook System](/architecture/hook-system).
The result: the agent sees exactly the tools it needs, no more. A simple agent with no connectors and no KB might see 5 tools. A Hub agent connected to 3 enterprise systems with a grounded knowledge base and 2 MCP servers might see 30 -- but after selection, only the 6 most relevant make it into the context.
## When to use what
| Need | Use | Why |
| - | - | - |
| General computation, code execution, text transforms | Built-in Tool | Always available, no config needed |
| Enterprise system integration (ERP, CRM, OA) | Connector | Auth governance, audit trail, operation-level access control |
| Knowledge retrieval with citations and evidence | Knowledge Base | RAG pipeline, grounded generation, confidence scoring |
| Third-party tool ecosystem | MCP | Standard protocol, process isolation, community servers |
| Organizational policies, SOPs, handling procedures | Skill (dynamic recipe) | Global by default, progressive loading, visibility-scoped |
| A deterministic, repeatable multi-step process invoked mid-conversation | Workflow (static recipe) | Same output every run, audit trail, runs as owner; human-gated ones stay trigger-only |
| Delegating tasks to specialist agents | CallAgent | Semantic agent routing, full tool inheritance, parallel execution |
| Direct database access | Database Connector | Schema-aware SQL, optional read-only enforcement |
| Custom internal tooling | MCP or Built-in | MCP for process isolation; built-in for tight integration |
The categories are not mutually exclusive. A single agent can use all five capability sources in one conversation -- loading a Skill for the complaint handling SOP, querying a knowledge base for policy documents, calling a connector to check the ERP, delegating analysis to a specialist agent (in Auto mode), and using a built-in tool to format the results.
## Execution engines are orthogonal
The tool system and execution engines are independent concerns. The LLM-driven engines (ReAct and DAG) consume tools from the same `ToolRegistry`. The choice of engine affects how tools are orchestrated, not which tools are available.
**ReAct** is an iterative tool loop. The agent reasons, picks a tool, observes the result, and repeats until done. It excels at exploratory, conversational tasks where the next step depends on the previous result. The loop runs up to 50 iterations with per-iteration context management via ContextGuard. See [ReAct Engine](/architecture/react-engine) for implementation details.
**DAG** decomposes a goal into 2-6 parallel steps. Each step runs an independent ReAct agent. A PlanAnalyzer evaluates whether the goal was achieved; if not, the pipeline re-plans autonomously (up to 3 rounds). DAG excels at tasks with clear subtasks that can run concurrently -- "search three sources and compare results" finishes in the time of one search, not three. See [DAG Engine](/architecture/dag-engine) for the full pipeline.
The two engines share infrastructure: `structured_llm_call` for reliable structured output, `ContextGuard` for token budget enforcement, and the `ToolRegistry` for tool resolution. Adding a new tool requires zero changes to either engine. Adding a new engine (should one ever be needed) requires zero changes to the tool system.
Both engines also support **agent delegation** via `CallAgentTool` when in Auto mode (no agent selected). In native function-calling mode, the LLM can invoke multiple `call_agent` calls in a single turn, which execute concurrently via `asyncio.gather`. Each delegated agent receives its own `ToolRegistry` and runs as a full execution unit. For the detailed design of agent discovery, Skills as global SOPs, and agent delegation, see [Agent & Resource Discovery](/architecture/agent-discovery).
### Workflow Engine — the third paradigm
Alongside the LLM-driven ReAct and DAG engines, FIM One includes a **Workflow Engine** — a visual DAG editor with 9 core node types (Start, End, LLM, Condition Branch, Agent, Knowledge Retrieval, Connector, MCP, Human Intervention) for fixed-process automation. Use Agents for flexible, exploratory tasks; use Workflows for deterministic, repeatable processes. See [Execution Modes](/concepts/execution-modes) for details.
The two compose **in both directions**, but each direction routes through the single runtime — the agent loop — never through a second execution engine:
* **Workflow → Agent.** A Workflow's `AGENT` node runs an agent as one deterministic step.
* **Agent → Workflow.** An agent (or a Skill it is following) pulls `run_workflow` to delegate a sub-task that must run deterministically.
This is the resolution to the obvious cycle worry: only the **agent loop is a runtime**. Skills and Workflows are inert recipes it loads; they never execute each other. "A Skill calling a Workflow" physically means the agent, while following the Skill's SOP, calls `run_workflow`. Every invocation passes through the agent, so the call graph is a **star**, not a mesh — and the back-edges are bounded (`read_skill` only returns text; `call_agent` and `run_workflow` carry depth caps and a re-entrancy guard).
```mermaid theme={null}
flowchart TD
U["User message / conversation"] --> RT
subgraph RT["Agent Loop = the only runtime"]
direction TB
note["System prompt carries a stub catalog + a set of tools. The LLM pulls what it needs — nothing is auto-pushed."]
end
RT -.->|pull| BT["Built-in tools (calculator, web_search, file_ops…)"]
RT -.->|pull| RS["read_skill → Skill (dynamic recipe) · returns text only"]
RT -.->|pull| RW["run_workflow → Workflow (static recipe) · depth-capped"]
RT -.->|pull| CA["call_agent → another Agent · excludes itself, depth-capped"]
RT -.->|pull| RES["Connectors / MCP / KB (SQL lives here)"]
RW -. "AGENT node" .-> RT
CA -. delegate .-> RT
```
Two reads of the same picture: **tools are pulled, not pushed** (a Skill or Workflow the LLM never selects simply never fires — which is why dynamic capabilities can feel invisible until a query matches one), and **Skills and Workflows are peers** routed through one runtime rather than nested inside each other.
## Lifecycle overview
**Startup.** `start.sh` runs Alembic migrations, launches the FastAPI server, discovers built-in tools, and establishes MCP server connections for any pre-configured global servers.
**Per-request.** JWT authentication, agent configuration lookup, tool assembly (the 8-step pipeline above), engine selection (ReAct or DAG based on agent config), execution with SSE streaming, and result persistence.
**Cross-cutting concerns.** [Context management](/architecture/context-management) (5-layer token budget) protects every LLM call from overflow. The [Hook System](/architecture/hook-system) wraps every tool call with platform-controlled `PreToolUse` / `PostToolUse` logic — the mechanism behind human-in-the-loop approval (`FeishuGateHook`), audit logging, and read-only-mode enforcement. Audit logging tracks every connector tool invocation. Sandbox isolation contains code execution tools. The two-LLM architecture (smart + fast) optimizes cost across planning, execution, and synthesis.
The architecture is designed so that each concern -- tool registration, execution orchestration, context management, security -- can evolve independently. A new connector type, a new execution engine, or a new context strategy can be added without cascading changes across the system.
# Changelog
Source: https://docs.one.fim.ai/changelog
All notable changes to FIM One.
All notable changes to this project are documented here.
Format follows [Keep a Changelog](https://keepachangelog.com/). Versions correspond to [Roadmap](/roadmap) milestones.
## \[Unreleased]
## \[v0.9.2] - 2026-10-09
### Fixed
* **Outbound requests are checked against private addresses when they are sent, not only when the URL is saved.** Workflow webhooks resolve their host again at delivery and refuse private or internal addresses, so a hostname that changes its DNS after it was saved can no longer reach the internal network. Redirects to a private IP address are refused for every outbound request, and Alibaba Cloud's metadata address (`100.100.100.200`, range `100.64.0.0/10`) is now blocked along with the other internal ranges.
## \[v0.9.1] - 2026-10-07
### Fixed
* **A saved database password is only used for the connection it was saved with.** Testing a connection or editing a database connector reuses the stored password only for the connector's owner and only while driver, host, port, database, username and TLS settings are unchanged; otherwise the password must be entered again. Saving an edit with the password field left blank now keeps the stored password instead of clearing it.
## \[v0.9.0] - 2026-10-01
### Added
* **Structured-output calls report which extraction level answered them.** Each call logs the level that actually produced the result, and logs a warning when every level fails and a default value is returned, so a silent degradation to free-text parsing is now visible in the logs.
* **Database columns can be marked as personal data.** A column marked PII in the connector's schema manager still appears in the schema the agent sees, but its values come back masked in every query result, `SELECT *` included. The connector call log records which columns were masked and whether the connector was read-only.
### Fixed
* **GPT-6 models work.** `gpt-6-luna`, `gpt-6-sol`, `gpt-6.1-sol` and `gpt-6-astra` were sent `max_tokens` and a custom temperature, which they reject, so every call failed. `gpt-6.1-sol` and `gpt-6-astra` cannot switch reasoning off and accept tools only on the Responses API; structured output and tool calls on them now run there at the lowest reasoning effort. With `FIM_GPT5_RESPONSES_MODE=off`, or an endpoint without `/v1/responses`, these two models cannot call tools.
* **Translated docs render bold lead sentences.** CJK translations that dropped the space after a closing `**` left the asterisks visible on the page; the translation pipeline now keeps it.
* **Unpublishing a resource now takes it back.** Access was granted by the subscription alone, so anyone who had subscribed to an agent, connector, MCP server, skill or workflow kept it after the owner reverted it to private, and the only way to actually withdraw one was to delete it. Subscriptions are re-checked against the resource's current sharing state on every use, including inside a running conversation.
* **Removing a resource from the Market removes it.** The admin action only hid it from the listing: existing subscribers were unaffected, and anyone with the resource id could still subscribe to it. It now reverts the resource to private, as the owner-side unpublish does.
* **Unsubscribing from an agent, skill or workflow no longer deletes connectors you set up yourself.** The dependency cleanup could not tell a connector installed on your behalf from one you had subscribed to and entered credentials for, and removed both — discarding the stored credential with it. Only what a solution installed is removed with it.
* **Deleting an organization no longer fails, and no longer leaves access behind.** Deletion errored once any member had published into the org, and members kept the subscriptions they held through it. Resources published to the org return to their owners as private, and org-scoped subscriptions are revoked as they are when a member leaves.
* **Subscriptions are removed with the resource they point at**, instead of remaining after it is deleted.
* **A skill shared by two Market agents survives deleting one of them.**
* **Subscribing to a skill or workflow is refused while the module is disabled**, matching the listing, which already hid them.
## \[v0.8.12] - 2026-09-04
### Added
* **ENV fallback can be synced from the admin panel.** `scripts/env_from_active_group.py` copies the active model group (general / fast / reasoning, optionally image generation) into `.env`, so the i18n hook, evals and other DB-less paths stop drifting onto a stale provider.
### Changed
* **The AI assistant panels use the same composer as the chat.** The knowledge base, agent and connector assistants now get the pill-shaped input with a multi-line textarea and round send button; Enter while an input method is composing no longer sends the message.
### Fixed
* **Image generation works against OpenAI's `gpt-image-*` models.** The request no longer sends `response_format`, and aspect ratios map to the sizes those models accept.
* **The i18n translation script accepts GPT-5.x and o-series models.** It omits `temperature` for reasoning models instead of failing with a 400.
## \[v0.8.11] - 2026-09-03
### Changed
* **The chat composer floats over the message stream.** Attachments, the mode and agent selectors, and the send button now sit in one pill-shaped input that stays over the conversation while messages scroll beneath it; on an empty chat it is centered above the example cards.
## \[v0.8.10] - 2026-08-28
### Added
* **Instance access model for billing.** Admins pick one of three postures: no subscriptions (software default, monthly quota `0` = unlimited), included + paid, or paid only. The included default plan cannot be deleted; paid templates no longer ship with a FIM Stripe Price ID.
* **GPT-5.x agents keep their reasoning across tool calls.** FIM One now talks the OpenAI Responses API directly for GPT-5.x models and carries each turn's encrypted reasoning into the next one, so a multi-step agent builds on what it already worked out instead of re-deriving it at every tool round. Reasoning token counts appear in usage, and the thinking panel shows the model's own summary. Set `FIM_GPT5_RESPONSES_MODE=bridge` (or `off`) to return to the previous behavior; endpoints without a `/v1/responses` route still fall back to chat completions on their own.
* **Pure transformation steps in DAG plans run as a single direct LLM call.** The planner tags merge/rewrite/summarise steps that only consume upstream outputs as direct steps, skipping the tool loop they would never use, which lowers cost and latency; every step result now also records how it ran (direct call or agent loop, iterations, tools used).
* **Agents can ask clarifying multiple-choice questions mid-run.** When a task could go in materially different directions, the agent pauses and presents up to four structured questions in the chat, each with 2-4 options plus a free-text "Other"; answering resumes the run in place, and skipping or a timeout lets it continue on its own judgment.
* **Agent steps carry live one-line titles.** While an agent works, each iteration gets a short generated label ("Verifying pricing data…"), shown in a single folded header that cycles to the newest step; expanding it reveals the full step timeline, and the labels persist into conversation history.
* **A provider's models can be deleted in bulk.** Each model row in Admin → Models carries a checkbox, the header selects the whole list, and holding Shift while clicking extends the selection across a range of rows. Deleting the selection removes every chosen model in one request, after a confirmation naming the count.
* **Small comparison tables in chat replies now render as card-style layouts.** A 2-3 column table whose first column holds attribute labels shows each attribute inside its value cell (matching the comparison-card style popularized by AI chat UIs); larger tables keep the classic grid.
* **Chat replies now render Mermaid diagrams and SVG figures.** A `mermaid` code block becomes a themed diagram and an `svg` block becomes an image with a source toggle and download; both fall back to plain code if the source is incomplete or does not parse, so a half-streamed diagram never flashes an error.
* **Answers, code blocks, and tables can be copied out.** Every reply has a copy-as-Markdown action, code blocks carry a header naming their language, and tables offer copy-as-Markdown plus a CSV download (UTF-8, opens cleanly in Excel).
* **Citations show which document they came from.** A citation marker now renders as a pill carrying the source's document name, with adjacent markers collapsed into one pill and a count; the popover lists every source behind it.
* **`/clear` works in the chat input.** Typing `/clear` starts a fresh conversation, identical to clicking New chat; a running turn is stopped first.
* **Diagrams and code blocks can be downloaded, not just copied.** Mermaid diagrams export the rendered SVG, and every code block downloads with a filename matching its language; a `csv` block downloads as a UTF-8 file that opens cleanly in Excel.
* **The composer warns before an image goes to a text-only model.** Attaching an image while the active model cannot read images shows a note saying the model will only receive the file name, resolved from the model that turn would actually use.
* **Unsent input is kept.** Text, pasted clips and finished attachments that have not been sent are remembered per conversation, and separately for a new chat, so they survive a refresh, a switch to another conversation, and an expired session. A draft is cleared once the message is sent or its conversation is deleted.
* **Chat replies accept LaTeX math delimiters.** Formulas written with `\[...\]` or `\(...\)` render as math, matching the existing dollar-sign syntax; delimiters inside code fences stay as text.
### Fixed
* **Admin billing and access-model settings read in the interface language again.** The billing posture, access-model and token-quota strings under Admin → Settings, and the matching subscription errors and account usage panel, showed English text or a raw key in Chinese, Japanese, Korean, German and French.
* **A Feishu channel's callback URL can now pass Feishu's verification.** Feishu omits its signature headers on the one-time URL verification handshake, and the callback endpoint rejected anything unsigned, so saving the request address always failed with "Challenge code not returned" and the channel's card buttons never worked. An unsigned push is now authenticated by the other shared secrets instead: its encrypted envelope must open with the channel's Encrypt Key, and its token must match the Verification Token when one is configured. An unsigned, unencrypted body is still refused.
* **A social login can no longer be captured by an account holding an unproven email address.** Linking a social login to an existing account now requires the address to have been verified on both sides, not just by the incoming provider. Signing in with a provider that never confirmed the address still creates an account, but that account is no longer something a later verified sign-in can be matched into.
* **A reply cut off by the output limit no longer runs half its tool calls.** When the model hits the token limit mid-message, every tool call in that reply is now discarded and the model is asked to re-issue them smaller. Previously calls whose arguments happened to be complete were executed, so a batch the model planned as a unit could run only partway while the transcript read as if all of it had.
* **Sending while an attachment is still uploading no longer drops it.** The send now waits for the upload to finish and includes the file, with the send button showing a spinner while it waits; previously the message went out without the image or document and without any warning.
* **A message that fails to send is put back in the input box.** When creating the conversation fails (expired session, network down), the typed text returns to the composer with an error toast instead of disappearing.
* **Follow-up suggestion chips stay in the user's voice.** Suggested next questions are framed as something the user would send (requests or questions to the assistant), not reverse preference polls like "which do you prefer…?".
* **Exa docs no longer call Exa the default search provider.** Integration and environment-variable pages now describe Exa as an optional backend you enable with `WEB_SEARCH_PROVIDER=exa`; the default search path stays Jina.
* **Sidebar logo is a real link again.** Middle-click and cmd/ctrl-click open the dashboard in a new tab; left-click still navigates and keeps the short bounce animation.
### Changed
* **The admin panel opens in a new tab.** Choosing Admin Panel from the user menu leaves the current workspace open behind it, matching the other external links in that menu.
* **The sidebar puts navigation first.** Module navigation sits above the conversation list as a compact continuous list (no Build / Resources / Operate section labels), with tighter spacing on nav items, New chat / Search, and conversation rows, instead of being tucked into a dock at the bottom of the sidebar.
* **Streamed replies render steadily instead of flickering.** A growing answer now renders block by block: finished blocks stay stable rather than re-rendering on every frame, and half-arrived formatting (bold, links, math) is completed or held until it finishes streaming in.
* **Final answers stream as they are written.** The agent now signals when it is done gathering and writes the answer as a live streamed turn, so long replies arrive token by token with full detail instead of replaying after a wait. `REACT_FINISH_SIGNAL=false` restores the previous behavior.
* **Architecture docs map graph engineering onto FIM One.** [Planning Landscape](/architecture/planning-landscape) explains loop vs graph engineering, how that differs from classic visual workflows, how it maps to Workflow / DAG / ReAct, and what to borrow for each engine. It also states the current fact that every DAG step is already a full ReAct agent (not a one-shot LLM call). ReAct and DAG engine pages link to it.
* **Plain chats always offer follow-up suggestions.** Conversations without a bound agent get suggested next questions after each answer; agent-bound chats still require the per-agent Follow-up Suggestions toggle (default off). Closure turns such as greetings or thanks may return no chips.
* **A newly sent message starts at the top of the screen.** The transcript keeps just enough room below the current turn for its first line to reach the top, so the question stays in view and the answer grows into the space below instead of the whole turn hugging the bottom edge. The room disappears once the turn outgrows one screen.
* **Reasoning blocks stay folded while a turn runs.** Reasoning in both ReAct and DAG output now shows a one-line preview by default instead of expanding as it streams, keeping the result in view; click any block to read the full trace.
* **Exported PDFs and Word files are properly typeset for Chinese.** PDF export now embeds a real CJK font instead of referencing one, fixing collapsed spacing in English text and bullets that rendered as unrelated Chinese characters, and bold now works. Word exports declare an East Asian font so mixed Chinese/English bold text no longer comes out half-bold, and both formats use a five-tier size scale that keeps headings inside a reply subordinate to the transcript's own structure.
* **Agents stay on plan through long tasks.** The agent is now reminded to change approach when it repeats the same tool fruitlessly, nudged to write a plan when working without one, and can no longer deliver a final answer while plan items are still open without an explicit verification pass.
* **Streamed answers ease in instead of popping.** Newly arrived blocks fade in briefly as they stream, honouring the reader's reduced-motion preference. Long tables also highlight the row under the cursor, and list bullets sit behind the text instead of competing with it.
* **Block actions stay out of the way until you reach for them.** Copy and download on answers, code blocks, diagrams and tables are icon-only with tooltips (download always rightmost), revealed on hover, and remain reachable by keyboard.
* **The sidebar now keeps chat together.** The conversation list sits directly under New chat and Search, and the module navigation moved into a compact tinted dock above the user area, giving the list more room and the sidebar a clear top-to-bottom hierarchy.
* **Reasoning is collapsed by default.** Thinking sections in both live runs and past conversations fold to a one-line preview and expand on click; reasoning still streams open while it is being generated, then folds when the step completes.
* **List pages ease their cards into place.** Agents, connectors, knowledge bases, skills, workflows, tools, conversations, artifacts and the market stagger their cards in on first load instead of appearing all at once. Filtering and paging stay instant, so typing in a search box never replays the entrance.
### Security
* **Raw HTML in rendered markdown is now sanitized.** Model output and uploaded markdown files are scrubbed before display, so injected scripts, event handlers, iframes, forms and style overrides can no longer reach the page. Math, syntax highlighting, tables, task lists and disclosure blocks are unaffected. Raw inline `