maestro/AGENTS.md

AGENTS.md — Codebase orientation for contributors

This is a map of the MAESTRO codebase for contributors (and AI coding agents). For build/test/PR mechanics see CONTRIBUTING.md; for the request lifecycle in depth see docs/architecture.md.

Working norms

• State conclusions and corrections directly; don't pad with reflexive agreement.
• Before editing code, understand the blast radius of a change (callers, tests). If a change is large or risky, describe the approach before implementing.
• Investigate reported bugs by reproducing the actual behavior before concluding.

Execution flow

UI POST
  → bridge/server.ts (Express API)
  → Repository (SQLite: jobs table)
  → Worker.poll() picks up a queued job
  → piece-runner.ts: loads pieces/*.yaml
  → agent-loop.ts: ReAct loop (LLM ↔ tool calls, up to safety.maxIterations)
  → each movement completes → `transition` (intermediate) / `complete` (terminal)
  → job finishes → DB update / progress comment

Main layers (src/)

• engine/piece-runner.ts — loads pieces/*.yaml, runs movements in order; carries verify-movement feedback into the next execute; loop-detection aborts on excessive repeat visits; transition.lessons accumulates cross-movement lessons.
• engine/agent-loop.ts — the ReAct loop for one movement. Intermediate hops use the transition tool; termination (success/aborted/needs_user_input) uses the complete tool. complete.result is the only user-visible final output. A ContextManager tracks token usage from LLM usage responses and fires warn/prompt/force_transition at thresholds.
• engine/context-manager.ts — threshold-based context-usage monitoring; can auto-detect a model's context limit from the provider API.
• engine/piece-classifier.ts — LLM-based piece selection from the task text and all piece descriptions.
• engine/tools/index.ts — dynamically loads and dispatches all tool modules.
• llm/openai-compat.ts — OpenAI-compatible (Ollama/vLLM/…) SSE streaming client; accumulates tool_calls deltas into LLMEvents. Retry via provider.retry.
• worker.ts / worker-manager.ts — Workers poll the DB for jobs matching their profiles/task_classes and run them; multiple workers run concurrently and are rebuilt on config change.
• config.ts / config-manager.ts — single config.yaml; snake_case YAML ↔ camelCase code via transformKeys; runtime read/write with optimistic locking and change events.
• db/repository.ts — SQLite (better-sqlite3). Manages jobs, local_tasks, local_task_comments, audit_log, etc. Schema: db/schema.sql.
• bridge/server.ts — Express API server. Submodules: pieces-api, config-api, tools-api, scheduled-tasks-api, admin-api, share-api, browser-api, subtask-activity-api, and more.
• scheduler.ts — cron-style scheduled tasks (daily/weekly/monthly/cron/once).
• bridge/auth.ts — Passport OAuth2 (Google / Gitea). requireAuth allows active users; requireAdmin allows admins. Auth is optional (unset = no auth).
• gateway/ — optional LLM gateway (a proxy with virtual keys, budgets, and Prometheus metrics). Note: its env vars use the AAO_* prefix and the aao_gateway connection type for historical reasons (AAO = the gateway).

pieces/*.yaml — task definitions

Each piece is an array of movements. Per movement: allowed_tools, edit (Write/Edit permission), and rules (transition conditions). Tools not in allowed_tools are not offered to the LLM.

Movement transition principles: transition is for intermediate hops only (rules[].next lists allowed targets); termination uses complete. default_next is an engine-internal sentinel (context-overflow forced transition, ASK fallback). Progressive pressure warns on repeat visits and aborts past a threshold.

Tool modules

Module	Tools
core.ts	Read / Write / Edit / Bash / Glob / Grep
web.ts	WebSearch / WebFetch / DownloadFile
image.ts	ReadImage / AnnotateImage
office.ts	ReadPdf / ReadExcel / ReadDocx / ReadPPTX / PdfToImages / Split…
data.ts	SQLite
review.ts	BatchReviewTextWithLLM / MergeReviewedResults
browser.ts	BrowseWeb (Playwright)
knowledge.ts	SearchKnowledge / ListDocuments / IngestDocument / …
orchestration.ts	SpawnSubTask
x.ts, maps.ts, youtube.ts, amazon.ts, speech.ts, ms-learn.ts	optional integrations
checklist.ts	CreateChecklist / CheckItem / GetChecklist
pieces.ts	ListPieces / GetPiece / CreatePiece / UpdatePiece
skills.ts	ReadSkill / ListSkills / InstallSkill (META_TOOL)
docs.ts	ReadToolDoc (META_TOOL)
ssh.ts	SSH execution / transfer

raw-save.ts and structured-blocks.ts are helper modules (not registered tools).

Tool descriptions are kept to one sentence (they ride every LLM call); detailed guidance lives in docs/tools/<name>.md and is fetched via ReadToolDoc.

Bash sandbox

The Bash tool runs inside a bwrap sandbox (filesystem confined to the task workspace, env scrubbed, network unshared) when available, with a hardened whitelist fallback otherwise. safety.bash_sandbox selects the mode (auto/always/off). Runtime pip/npm install is rejected; Python packages are pre-baked from runtime/python-requirements.txt. See docs/operations/bash-sandbox-provisioning.md.

Workspace layout (per job)

{worktree_dir}/local/{taskId}/
  input/    uploads & DownloadFile output
  output/   artifacts (the main Write/Edit-allowed area)
  logs/     activity.log, history files
  subtasks/ SpawnSubTask results
  skills/   skill files materialized by ReadSkill

DB migrations

db/schema.sql is the initial schema. New columns are applied idempotently in db/migrate.ts (PRAGMA table_info → existence check → ALTER TABLE ADD COLUMN). Update both schema.sql and migrate.ts.

Tests

Backend tests live next to their source as *.test.ts (vitest auto-discovers).

Adding a tool

1. Export TOOL_DEFS and an executeTool from src/engine/tools/<module>.ts.
2. Register the dynamic import in tools/index.ts.
3. Add the tool name to the using piece's allowed_tools.
4. Add docs/tools/<name>.md. See docs/maintenance-checklist.md for the full list of places that must stay in sync.