Skip to main content

Architecture overview

Istota is a self-hosted personal AI operating system. It combines a durable agent runtime with native applications for health, location, money, feeds, briefings, chat, and memory. Each module owns its schema and per-user storage, while a common skill interface lets an authorized task query more than one module without merging all personal data into one store.

Each agent task runs through a pluggable Brain. Two brains ship behind the same protocol: ClaudeCodeBrain (the default) wraps Anthropic's Claude Code CLI as a subprocess, and NativeBrain runs Istota's own in-process agent loop against any OpenAI-compatible endpoint (Anthropic, OpenRouter, or a local model). Swapping brains does not change executor orchestration. Messages arrive from Nextcloud Talk, the in-app web chat, WhatsApp, SMS, email, file-based task queues, scheduled jobs, the interactive REPL, or the CLI. Each surface sits behind a uniform Transport seam. Requests flow through a SQLite task queue, are claimed by per-user worker threads, and produce responses for the originating channel. Nextcloud can supply files, calendars, contacts, login, notifications, and Talk, but the default storage is a local directory and needs no Nextcloud server. See Deployment for how an install is put together.

Tool calling, function dispatch, and the agent loop live in the brain, not the executor — and which code runs them depends on the brain. With ClaudeCodeBrain (the default) they are Claude Code's job, so new Claude Code capabilities (tool use, model improvements) come for free. With NativeBrain they are Istota's own: an in-process loop that dispatches tools, compacts context, and retries against any OpenAI-compatible model. Either way the executor's job is the same — it constructs the prompt and hands off a BrainRequest.

Core data flow​

Every interaction follows the same path:

  1. Input arrives from one of several channels (Talk message, web chat, email, TASKS.md edit, CLI/REPL command, cron trigger)
  2. A turn on a room surface is first recorded in the room's transcript. In a room more than one person reads, the speech gate then decides whether the bot answers, and only an answered turn goes on. See rooms and multi-user chat
  3. A task is created in the SQLite tasks table with status pending
  4. The scheduler dispatches a UserWorker thread for the task's user
  5. The worker claims the task (atomic UPDATE...RETURNING, setting status to locked then running)
  6. The executor assembles the prompt: persona + resources + memory + context + skills + guidelines + the actual request
  7. The executor resolves which brain to use for this task's source type (resolve_brain_kind), builds a BrainRequest, and calls brain.execute(req), rerunning through the fallback brain if the primary is unavailable. The default ClaudeCodeBrain invokes claude -p - --output-format stream-json as a subprocess
  8. The brain returns a BrainResult; the executor composes the final text (CM-aware), stores it in the DB, and delivers it to the originating channel
  9. Post-completion: conversation indexed for memory search, deferred DB operations processed, scheduled job counters reset

Task lifecycle: pending -> locked -> running -> completed | failed | pending_confirmation -> cancelled

From schedule to agent: the subtask handoff​

A scheduled job is not always an agent invocation. A CRON.md job carries one of three payloads, and only the first puts a model in the loop:

  • prompt (or prompt_file) — a natural-language request. The task routes straight to the Brain: an LLM task.
  • command — a shell command run in a subprocess (_execute_command_task). Deterministic, no model.
  • skill — a skill CLI invocation such as istota-skill feeds run-scheduled (_execute_skill_task, auto-promoted from a pure istota-skill … command row). Deterministic, no model.

The dispatch fork lives in process_one_task: task.skill and task.command take the deterministic paths; everything else goes to the Brain.

So how does a "dumb" schedule reach the agent? Through the deferred subtask mechanism. All three task paths run with a deferred directory set — the sandboxed Brain path (bwrap masks the database directories out entirely, so the model can neither read nor write the DB directly and defers instead), the skill path, and the raw shell command path, which reuse the same rail — and write JSON op files into ISTOTA_DEFERRED_DIR, which the unsandboxed scheduler applies after the task succeeds. A command row was the odd one out until ISSUE-233: it got no deferred dir, so a skill CLI it invoked wrote directly where it could and silently wrote nothing where it could not. Two consequences of putting it on the rail are worth knowing before you write one. A write is now applied only if the command as a whole exits 0 — _purge_deferred_files_for_retry discards the ops of a failed attempt, so istota-skill kv set … ; something-that-fails no longer persists the value. And a write is not visible to a later read within the same command, because reads still go straight to the database. Subtask creation is the reason to accept both: it is the only route from a deterministic schedule to an agent, and command rows are admin-only. One of those file types is a subtask request:

$ISTOTA_DEFERRED_DIR/task_${ISTOTA_TASK_ID}_subtasks.json

A deterministic skill run — including an auto-promoted istota-skill … cron row — that emits this file hands its follow-up work to an agent. On success, _drain_deferred_ops → _process_deferred_subtasks reads it and calls db.create_task(source_type="subtask", …). A deferred subtask carries a natural-language prompt — a command key is explicitly rejected — so the new task always routes to the Brain. That is the handoff from a deterministic schedule to an LLM task:

Guardrails on this path: subtask creation is admin-only, prompt-only (never a nested command), rate-limited (max_subtasks_per_task, max_subtask_depth, max_subtask_prompt_chars), and the child's conversation_token is pinned to the parent so a subtask can't redirect its own output. The same deferred-writeback rail carries the other post-success writes — knowledge-graph facts, KV entries, health ops, sent-email records — but those are terminal writes to their own stores; only the subtask arm re-enters the task queue.

Module map​

Input channels​

ModulePurpose
transport/Uniform seam over messaging surfaces: IncomingMessage / Transport protocol / TransportRegistry / ingest_message (inbound) + resolve_delivery_plan (outbound). Eight transports ship — Talk, Web, WhatsApp, Sms, Email, Ntfy, IstotaFile, Repl
transport/talk/inbound.pyLong-polls Talk conversations, creates tasks, intercepts !commands, handles confirmations (the TalkTransport inbound body)
transport/email/inbound.pyPolls INBOX via IMAP, creates tasks from known senders, downloads attachments (the EmailTransport inbound body)
transport/whatsapp/One business phone number behind a provider seam: whatsapp_cloud (Meta's Cloud API webhook) or baileys (a paired WhatsApp Web session driven by a Node sidecar). Inbound resolves the sender to a user binding; outbound is one message per logical response
transport/sms/One service number through Twilio or Telnyx. Signed provider webhooks create tasks; outbound is segment-budgeted and tracked through delivery callbacks
webui/app.py (/api/chat/*)In-app web chat: POST → ingest_message creates a source_type="web" task; SSE tails task_events
repl/Interactive terminal loop (istota repl); each line is an inline source_type="repl" task streamed to the terminal
tasks_file_poller.pyWatches TASKS.md files for changes, identifies tasks by SHA-256 content hash
cli.pyDirect task execution (istota task "prompt" -u USER -x), supports --dry-run
cron_loader.pyReads CRON.md (markdown with embedded TOML), syncs jobs to scheduled_jobs DB table

Core processing​

ModulePurpose
scheduler.pyMain loop: daemon mode (long-running with WorkerPool) and single-pass mode
executor.pyBuilds prompts, constructs the per-task environment, orchestrates a Brain, composes results
brain/Pluggable model-invocation backend: Brain Protocol + make_brain factory, BrainRequest/BrainResult types, stream events, ClaudeCodeBrain (subprocess + stream-json + transient-API retry), and NativeBrain (Istota's in-process agent loop). The native loop's machinery lives in llm/ (provider abstraction), agent/ (the loop + tool dispatch), and session/ (turn state + compaction).
context.pySelects relevant conversation history using hybrid recent + LLM-triaged approach
rooms/The multi-user room model: speech gate, host and guest policy, what a task in a shared room may reach, private replies, the veto
skills/_loader.pyLoads skill documentation selectively: always_include, source types, file types, sticky skills, companions. Keyword and resource matching are deliberately not selectors

Storage and state​

ModulePurpose
db.pyAll SQLite operations: task CRUD, resources, conversation history, state tracking
config.pyTOML config loading with nested dataclasses, per-user overrides, secret env vars
storage.pyWorkspace path management, user workspace creation, OCS sharing when the workspace is on Nextcloud

Memory​

ModulePurpose
memory/sleep_cycle.pyNightly orchestration: extracts memories from completed tasks, writes dated files, drives curation and retention
memory/search.pyHybrid BM25 + vector search, indexing, and unified chunk retention
memory/knowledge_graph.pyTemporal entity-relationship triples with validity windows
memory/curation/Op-based USER.md curation (parser, ops, prompt, audit)

See Memory for the layered design (USER.md, CHANNEL.md, dated memories, recall, knowledge graph) and how each layer enters prompts.

Output​

ModulePurpose
nextcloud/talk.pyAsync HTTP client for Nextcloud Talk API (send, poll, download attachments)
async_runtime.pyOne persistent asyncio loop + one pooled httpx client for all Talk I/O (run_coro, get_talk_client singleton); started/stopped by run_daemon
notifications/delivery.pyDelivery. Unified dispatcher for Talk, email and ntfy; per-user purpose-keyed routing table. Distinct from the two below, which are the inbox
notifications/store.pyThe inbox. The notifications table — the durable open set of what is waiting on a user, behind the app-bar bell. The write and the send are two separate calls, so a push that reached nobody leaves the row intact
notifications/sources.pyThe resolver seam behind the inbox: one resolver per source, returning None for an object that is gone. That single value is the whole anti-staleness story, so an item answered over Talk can never render stale
events.pyTask-event-streaming: TaskEvent, EventWriter, EventSubscriber + the task_events log that feeds every output surface
consumers/Event consumers: TalkEventSubscriber, LogChannelSubscriber, PushNotificationSubscriber
commands.pySurface-agnostic !command dispatch (CommandContext + registry), handled synchronously across Talk / web / CLI

Modules (in-tree)​

PackagePurpose
feeds/Native RSS/Atom/Tumblr/Are.na — poller, per-user SQLite, routes, OPML
health/Body stats, lab panels, biomarker trends, Garmin sync, immunizations, medical history
location/ (+ location/logic.py)GPS pings, place detection, visit logging, cluster discovery
money (vendored)Beancount ledger, invoicing, transactions, work log, investment portfolio
briefings/Block/source briefings — per-user SQLite, source resolvers, generation, reader and settings routes

Subsystems​

ModulePurpose
heartbeat.pyEvaluates health checks from HEARTBEAT.md
shared_file_organizer.pyScans for files shared with the bot, auto-organizes by owner
nextcloud/compat.pyShared Nextcloud HTTP plumbing (OCS + WebDAV)
nextcloud/user_metadata.pyEnriches user configs from Nextcloud OCS API at startup
webui/app.pyAuthenticated web interface (FastAPI + Nextcloud OAuth2)
webui/webhook_receiver.pyFastAPI webhook receiver (Overland GPS)
devbox/proxy.pyPer-user host-side credential proxy for the devbox container
logging_setup.pyCentralized logging configuration (console, file, rotation)

Browser container​

The headless browser runs in its own container (docker/browser/), the stack's optional browser service — Google Chrome driven over a Flask API (with VNC for observation):

ModulePurpose
browse_api.pyFlask API endpoints: get, screenshot, extract, interact, close, health
render.pyPage rendering helpers behind the extract/screenshot endpoints
chrome.pyChrome process lifecycle and CDP connection management
browsing.pyHuman simulation: Gaussian mouse movements, Bezier curves, scrolling patterns, captcha detection
xdotool.pyX11 input helpers for CDP-free browser interaction
stealth-extension/Chrome extension (manifest v3): overrides navigator properties, WebGL fingerprints, handles cookie consent

Anti-detection strategy: Chrome launches with the stealth extension natively. Patchright connects via CDP only for content extraction, then disconnects. Navigation uses xdotool keyboard input rather than CDP commands. Human simulation adds 5-10s delays between page actions with realistic mouse movement patterns.

Deployment​

Istota has one deployment: the istota stack on a Debian 13 machine dedicated to it. The machine is a server, a cloud VM or bare metal, or, on an Apple silicon Mac, a Lima VM. Both run the same stack, set up the same way.

  • One stack. Inside the machine the stack is a set of containers run by Docker Compose (docker/docker-compose.yml): nginx, the only published listener; daemon, the scheduler and every task, each in its own bubblewrap sandbox; and web, the web UI and webhook receiver. The optional browser, whatsapp-baileys and signaling services are Compose profiles the setup turns on from your answers, and a devbox per developer user is added from config.toml. The images are built on the machine from the checked-out release; nothing is pulled from a registry.
  • One machine. install.sh clones the repository, checks the release tag's signature, and runs host/provision.sh, which installs Docker Engine, the sandbox's seccomp and AppArmor profiles and the systemd units. The machine runs nothing else, because the daemon container holds what bubblewrap needs to build namespaces.
  • One command. istota-stack manages the machine afterwards: first-run setup (setup --web in a browser, or setup in the terminal), update to a signed release tag or, with update --unsigned latest, to the tip of main, rollback, apply for config changes, and up and down. On a Mac, istota-stack on the Mac creates the VM and forwards every other command into it.
  • Three storage modes. The workspace is a local directory in the state directory (the default), the bot's files on an existing Nextcloud mounted with rclone (full Nextcloud integration), or, on the Mac, a folder in your home shared into the VM (~/Istota by default). Nextcloud is never part of the stack; a Nextcloud you already run can supply Talk, files and login.
  • Three ingress modes. nginx listens on loopback only (local, the Mac's mode), on a private address behind your own reverse proxy (proxied), or on ports 80 and 443 with its own certificate (direct).
  • State, config and credentials are host directories under /srv/app/istota: data/ (the databases, the master key and a local workspace) is bound at /data, config/ is bound read-only into every service, and each credential is a file under secrets/ mounted as a compose secret. The web config editor writes a request that istota-stack on the host applies, so no running service writes the config itself.

Running the stack covers the server, and Local install the Mac. Security describes the sandbox and the local VM's trust model.

Design decisions​

Pluggable execution — delegate, or run the loop in-house. The default brain invokes the existing Claude Code CLI as the execution engine. The native brain instead runs Istota's own in-process agent loop (tool dispatch, context compaction, retries) against any OpenAI-compatible model. Same executor, same skills — the brain is the swappable seam, so Istota isn't bound to one vendor.

Regular Nextcloud user, not bot API. The bot runs as an ordinary user. File sharing, CalDAV, and Talk messaging work through standard protocols. No special server configuration.

File-as-config for user self-service. Users configure briefings, cron jobs, heartbeats, and their own memory through markdown files in their workspace. The persona is the operator's, one file at the file root. No CLI access needed.

Functional over object-oriented. Most code is module-level functions. Classes exist only where shared state across calls is necessary (TalkClient, UserWorker, WorkerPool).

Graceful degradation, except for the sandbox. Memory search falls back to BM25-only without sqlite-vec, and indexing failures never affect core processing. The sandbox is the exception: on Linux a daemon whose bubblewrap sandbox cannot be built refuses to start rather than run tasks unconfined. Only a development checkout on macOS (istota serve) runs without it.

Security by environment, not tool restriction. Rather than limiting the model's tools, credentials are stripped from the execution environment and optionally routed through a credential proxy.

Worker-per-user for fairness. Each user gets their own serial worker thread per queue type (foreground/background). One user's slow task never blocks another.

Deferred writes for sandbox compatibility. With bubblewrap masking the database directories out of the sandbox, skills write JSON files to a writable temp dir. The scheduler processes these after task completion. (A skill CLI invoked through the credential proxy runs host-side and could write directly; it still defers, so the write lands only if the task succeeds.)