Agents & Agent Runtime
The agent is the platform's primary unit of capability: a configured LLM
persona with tools, skills, memory, reasoning depth and guardrail posture,
invocable by chat, channel, workflow, schedule, event or API. This page
specifies the functional surface of agent authoring and execution in
core/agents.
session · agent · tenant · graph"] T --> SK["Skill injection
SKILL.md progressive disclosure"] T --> L["LLM call
reasoning depth FAST/CAREFUL/RIGOROUS/PLANNED"] L --> G["Guarded tool-call pipeline"] G -->|approved| EX["Tool executes"] G -->|parked| AP["Approval inbox"] EX --> L L --> R["Answer + step events
SSE / AG-UI stream"]
FR-AGT-01 · Agent creation & editing workbench
Agents are created and maintained in the web-admin workbench.
An agent record captures identity (name, description, avatar), the model
binding (provider + model, sourced from tenant BYOK keys, the
platform-hosted pool, or the on-device edge tier), system prompt, tool
selection, skills, memory scopes, reasoning depth, and autonomy/jev
settings. A generation assist path
(AgentGenerationService) can draft an agent configuration
from a natural-language brief.
| Requirement | Description | Status |
|---|---|---|
| FR-AGT-01.1 | Create/edit agent configuration through the workbench; changes to reasoning depth apply via PATCH /api/agents/{id}/reasoning-depth without recreating the agent. | live |
| FR-AGT-01.2 | Per-agent provider binding: tenant BYOK key, platform-hosted pool model, or edge/on-device model. | live |
| FR-AGT-01.3 | NL-to-config generation assist produces a draft agent from a brief. | live |
FR-AGT-02 · Versioning & promotion
Agent configuration is versioned. Edits produce new versions; promotion
moves a chosen version into the serving position so channels, share links
and jobs pick it up consistently. The snapshot codec
(AgentConfigSnapshotCodec) serialises a full config —
including reasoning depth — so a version is a complete, restorable
artifact rather than a diff.
FR-AGT-03 · A/B experiments
Agent experiments (migration V265) let a builder split
traffic between configuration arms — e.g. two prompts, two models, or two
reasoning depths — and compare outcomes. Arms are described by
ChatArmConfig; the runtime resolves the arm per session/turn
so results can be attributed.
| Requirement | Description | Status |
|---|---|---|
| FR-AGT-03.1 | Define experiment arms over agent config; deterministic arm assignment per session. | live |
| FR-AGT-03.2 | Arm attribution flows into metering/outcome records so arms can be compared on resolutions, not just volume. | live |
FR-AGT-04 · Skills
Skills are packaged prompt-and-resource bundles that teach an agent a
repeatable procedure. A skill is a SKILL.md document loaded
with progressive disclosure: the model sees the skill's
summary in its system context and pulls the full body (and any bundled
files) only when the skill is invoked. Five built-in skills ship with the
platform; tenants manage theirs on the /skills page in
web-admin.
| Requirement | Description | Status |
|---|---|---|
| FR-AGT-04.1 | SKILL.md format with progressive disclosure (summary in context, body on demand). | live |
| FR-AGT-04.2 | 5 built-in skills shipped with the platform; tenant-authored skills managed on the /skills page. | live |
FR-AGT-05 · Reasoning depths
Every agent carries a ReasoningDepth
(core/agents/.../ReasoningDepth.java) — a per-agent knob that
controls how much verification and deliberation wraps its turn, independent
of model sampling parameters.
| Depth | Behaviour | Intended use |
|---|---|---|
FAST | No extra verification pass; lowest latency. | High-volume, low-risk agents (FAQ bots, lookups). Default. |
CAREFUL | One self-consistency re-check of the final answer (AnswerConsistencyChecker); revised once if a genuine problem is found. | Agents whose answers must be right but take no side effects. |
RIGOROUS | CAREFUL plus a Reflexion self-critique pass (ReflexionService) plus an independent verifier check on every tool call (ToolCallVerifierService) before it executes. | Agents that send messages, spend money, or write/delete data. |
PLANNED | Plan-and-execute: a cheap planning call (TaskPlanningService) decomposes the request into an ordered step list surfaced as context and as a plan entry in the trace. | Multi-step tasks; opt-in and not combined with CAREFUL/RIGOROUS passes in the same turn. |
FR-AGT-06 · Tools
Agents call tools through Spring AI ToolCallbacks resolved by
AgentToolProvider. The platform ships 50+ built-in
tools spanning web search, messaging, knowledge, commerce
(payment links), media (YouTube upload, b-roll, clip generation) and
workflow invocation, plus tenant-connected integration and MCP tools.
Every tool call executes inside the guarded pipeline — see
VD-FN-006 for the
GuardedToolCallback stage list.
| Requirement | Description | Status |
|---|---|---|
| FR-AGT-06.1 | 50+ built-in tools selectable per agent; classification registry (V224__tool_classification.sql) tags tools by risk for the guard pipeline. | live |
| FR-AGT-06.2 | All tool calls — built-in, integration, MCP — route through GuardedToolCallback (step ceiling → RIGOROUS verify → HITL gate → output validation → sanitization). | live |
| FR-AGT-06.3 | Step events emitted per tool call (AgentStepEventEmitter) stream intermediate reasoning/actions to the UI and to AG-UI consumers. | live |
FR-AGT-07 · Memory
Memory is scoped, not global: session memory covers the current conversation, agent memory persists facts the agent should retain across sessions, and tenant memory is shared across a tenant's agents. A graph memory layer stores entities and relationships and supports multi-hop recall — see VD-FN-004 for its retrieval behaviour.
FR-AGT-08 · Share & embed surfaces
An agent can be exposed to end-customers without console access via
public share links and embeddable chat widgets
(AgentShareController). Share surfaces reuse the same
runtime, memory and guardrail stack as console chat; plan features such as
white-labelling the widget are gated by plan_limits when billing
is on (see Functional Overview).
FR-AGT-09 · A2A agent cards
Agents can be published to other agents through Agent-to-Agent (A2A)
agent cards (core/agents/.../a2a), advertising capabilities
so external agents can discover and delegate to a VegaDūta agent without a
human client. See API: Developer
surface.
FR-AGT-10 · AG-UI runs
Agent runs can be driven and streamed over the AG-UI protocol, giving external UIs a structured event stream of a run (messages, tool calls, state deltas). Transport details are specified in API: Streaming (SSE & AG-UI).
FR-AGT-11 · Autonomous jobs
Agents run unattended as jobs (core/agents/.../job): a job
binds an agent to a schedule and/or event triggers, with durable
bookkeeping so a job survives restarts and can be paused for approval.
| Requirement | Description | Status |
|---|---|---|
| FR-AGT-11.1 | Cron-style scheduling of agent jobs. | live |
| FR-AGT-11.2 | Event triggers: platform events (e.g. inbound channel messages, webhook events) dispatch agent runs via AgentEventTriggerService/AgentEventTriggerDispatcher. | live |
| FR-AGT-11.3 | Approval parking: a gated action inside a job run parks the job on the approval inbox rather than failing it; the run resumes when the approval resolves. | live |
| FR-AGT-11.4 | Durability: job executions persist state and are reconciled on boot so a restart neither loses nor double-runs a job. | live |
| FR-AGT-11.5 | Structured output: jobs can demand a schema-bound result, recorded on the execution rather than free text. | live |
FR-AGT-12 · Vision sessions
Vision sessions (core/agents/.../vision) let an agent consume
camera/screen frames. VisionSessionMode distinguishes
SNAPSHOT frames (per-user frame quota) from continuous WATCH/LIVE modes
gated by the live_vision_allowed plan flag —
VisionNotAllowedForPlanException enforces it.
Continuous vision (live_vision_allowed) is FALSE on FREE and TRUE on paid plans (migration V180). SNAPSHOT frames ride the per-user quota and are not gated by the flag.
FR-AGT-13 · Crew handoffs
Agents can be composed into crews: a delegating agent hands a sub-task to a specialist agent and consumes its result, enabling multi-agent collaboration without exposing the internal split to the caller.
FR-AGT-14 · Jev decision controls
"Jev" is the platform's decision-control surface for agent autonomy: a policy layer plus a per-agent threshold that tunes how much independence an agent has before an action escalates to the approval pipeline. It composes with tenant autonomy tiers and allow-lists — see VD-FN-006.
FR-AGT-15 · Edge / on-device agents
The edge tier runs models off-platform: the web console can run a model in
the browser (WebLLM, vendored by web/scripts/vendor-webllm.mjs)
when NEXT_PUBLIC_EDGE_AGENTS=1. The core supports the split via
EdgeToolExecutionController, which executes tool calls on
behalf of an on-device run that cannot reach the tools itself. Other
on-device clients can use the same endpoint; none ship in this repository.
Cross-references
- AI / LLM Layer — Spring AI ChatClient construction and provider resolution.
- API: Streaming — SSE and AG-UI wire formats.
- API: Developer surface — A2A cards and MCP.
- VD-FN-006 — the guarded pipeline, approvals, autonomy tiers.
- Billing and plans — plan gates touching agents (
byok_allowed,live_vision_allowed,max_agents), off by default.