135 lines
6.4 KiB
Markdown
135 lines
6.4 KiB
Markdown
# AGENTS.md — Codebase orientation for contributors
|
|
|
|
This is a map of the MAESTRO codebase for contributors (and AI coding agents).
|
|
For build/test/PR mechanics see [CONTRIBUTING.md](CONTRIBUTING.md); for the
|
|
request lifecycle in depth see [docs/architecture.md](docs/architecture.md).
|
|
|
|
## Working norms
|
|
|
|
- State conclusions and corrections directly; don't pad with reflexive agreement.
|
|
- Before editing code, understand the blast radius of a change (callers, tests).
|
|
If a change is large or risky, describe the approach before implementing.
|
|
- Investigate reported bugs by reproducing the actual behavior before concluding.
|
|
|
|
## Execution flow
|
|
|
|
```
|
|
UI POST
|
|
→ bridge/server.ts (Express API)
|
|
→ Repository (SQLite: jobs table)
|
|
→ Worker.poll() picks up a queued job
|
|
→ piece-runner.ts: loads pieces/*.yaml
|
|
→ agent-loop.ts: ReAct loop (LLM ↔ tool calls, up to safety.maxIterations)
|
|
→ each movement completes → `transition` (intermediate) / `complete` (terminal)
|
|
→ job finishes → DB update / progress comment
|
|
```
|
|
|
|
## Main layers (`src/`)
|
|
|
|
- **`engine/piece-runner.ts`** — loads `pieces/*.yaml`, runs movements in order;
|
|
carries verify-movement feedback into the next execute; loop-detection aborts on
|
|
excessive repeat visits; `transition.lessons` accumulates cross-movement lessons.
|
|
- **`engine/agent-loop.ts`** — the ReAct loop for one movement. Intermediate hops
|
|
use the `transition` tool; termination (success/aborted/needs_user_input) uses the
|
|
`complete` tool. `complete.result` is the only user-visible final output. A
|
|
`ContextManager` tracks token usage from LLM `usage` responses and fires
|
|
warn/prompt/force_transition at thresholds.
|
|
- **`engine/context-manager.ts`** — threshold-based context-usage monitoring; can
|
|
auto-detect a model's context limit from the provider API.
|
|
- **`engine/piece-classifier.ts`** — LLM-based piece selection from the task text and
|
|
all piece descriptions.
|
|
- **`engine/tools/index.ts`** — dynamically loads and dispatches all tool modules.
|
|
- **`llm/openai-compat.ts`** — OpenAI-compatible (Ollama/vLLM/…) SSE streaming client;
|
|
accumulates `tool_calls` deltas into `LLMEvent`s. Retry via `provider.retry`.
|
|
- **`worker.ts` / `worker-manager.ts`** — Workers poll the DB for jobs matching their
|
|
`profiles`/`task_classes` and run them; multiple workers run concurrently and are
|
|
rebuilt on config change.
|
|
- **`config.ts` / `config-manager.ts`** — single `config.yaml`; snake_case YAML ↔
|
|
camelCase code via `transformKeys`; runtime read/write with optimistic locking and
|
|
change events.
|
|
- **`db/repository.ts`** — SQLite (better-sqlite3). Manages `jobs`, `local_tasks`,
|
|
`local_task_comments`, `audit_log`, etc. Schema: `db/schema.sql`.
|
|
- **`bridge/server.ts`** — Express API server. Submodules: `pieces-api`, `config-api`,
|
|
`tools-api`, `scheduled-tasks-api`, `admin-api`, `share-api`, `browser-api`,
|
|
`subtask-activity-api`, and more.
|
|
- **`scheduler.ts`** — cron-style scheduled tasks (daily/weekly/monthly/cron/once).
|
|
- **`bridge/auth.ts`** — Passport OAuth2 (Google / Gitea). `requireAuth` allows active
|
|
users; `requireAdmin` allows admins. Auth is optional (unset = no auth).
|
|
- **`gateway/`** — optional LLM gateway (a proxy with virtual keys, budgets, and
|
|
Prometheus metrics). Note: its env vars use the `AAO_*` prefix and the
|
|
`aao_gateway` connection type for historical reasons (AAO = the gateway).
|
|
|
|
## `pieces/*.yaml` — task definitions
|
|
|
|
Each piece is an array of `movements`. Per movement: `allowed_tools`, `edit`
|
|
(Write/Edit permission), and `rules` (transition conditions). Tools not in
|
|
`allowed_tools` are not offered to the LLM.
|
|
|
|
**Movement transition principles:** `transition` is for intermediate hops only
|
|
(`rules[].next` lists allowed targets); termination uses `complete`. `default_next`
|
|
is an engine-internal sentinel (context-overflow forced transition, ASK fallback).
|
|
Progressive pressure warns on repeat visits and aborts past a threshold.
|
|
|
|
## Tool modules
|
|
|
|
| Module | Tools |
|
|
|--------|-------|
|
|
| `core.ts` | Read / Write / Edit / Bash / Glob / Grep |
|
|
| `web.ts` | WebSearch / WebFetch / DownloadFile |
|
|
| `image.ts` | ReadImage / AnnotateImage |
|
|
| `office.ts` | ReadPdf / ReadExcel / ReadDocx / ReadPPTX / PdfToImages / Split… |
|
|
| `data.ts` | SQLite |
|
|
| `review.ts` | BatchReviewTextWithLLM / MergeReviewedResults |
|
|
| `browser.ts` | BrowseWeb (Playwright) |
|
|
| `knowledge.ts` | SearchKnowledge / ListDocuments / IngestDocument / … |
|
|
| `orchestration.ts` | SpawnSubTask |
|
|
| `x.ts`, `maps.ts`, `youtube.ts`, `amazon.ts`, `speech.ts`, `ms-learn.ts` | optional integrations |
|
|
| `checklist.ts` | CreateChecklist / CheckItem / GetChecklist |
|
|
| `pieces.ts` | ListPieces / GetPiece / CreatePiece / UpdatePiece |
|
|
| `skills.ts` | ReadSkill / ListSkills / InstallSkill (META_TOOL) |
|
|
| `docs.ts` | ReadToolDoc (META_TOOL) |
|
|
| `ssh.ts` | SSH execution / transfer |
|
|
|
|
`raw-save.ts` and `structured-blocks.ts` are helper modules (not registered tools).
|
|
|
|
Tool descriptions are kept to one sentence (they ride every LLM call); detailed
|
|
guidance lives in `docs/tools/<name>.md` and is fetched via `ReadToolDoc`.
|
|
|
|
## Bash sandbox
|
|
|
|
The Bash tool runs inside a bwrap sandbox (filesystem confined to the task
|
|
workspace, env scrubbed, network unshared) when available, with a hardened
|
|
whitelist fallback otherwise. `safety.bash_sandbox` selects the mode
|
|
(`auto`/`always`/`off`). Runtime `pip`/`npm install` is rejected; Python packages
|
|
are pre-baked from `runtime/python-requirements.txt`. See
|
|
[docs/operations/bash-sandbox-provisioning.md](docs/operations/bash-sandbox-provisioning.md).
|
|
|
|
## Workspace layout (per job)
|
|
|
|
```
|
|
{worktree_dir}/local/{taskId}/
|
|
input/ uploads & DownloadFile output
|
|
output/ artifacts (the main Write/Edit-allowed area)
|
|
logs/ activity.log, history files
|
|
subtasks/ SpawnSubTask results
|
|
skills/ skill files materialized by ReadSkill
|
|
```
|
|
|
|
## DB migrations
|
|
|
|
`db/schema.sql` is the initial schema. New columns are applied idempotently in
|
|
`db/migrate.ts` (`PRAGMA table_info` → existence check → `ALTER TABLE ADD COLUMN`).
|
|
Update both `schema.sql` and `migrate.ts`.
|
|
|
|
## Tests
|
|
|
|
Backend tests live next to their source as `*.test.ts` (vitest auto-discovers).
|
|
|
|
## Adding a tool
|
|
|
|
1. Export `TOOL_DEFS` and an `executeTool` from `src/engine/tools/<module>.ts`.
|
|
2. Register the dynamic import in `tools/index.ts`.
|
|
3. Add the tool name to the using piece's `allowed_tools`.
|
|
4. Add `docs/tools/<name>.md`. See `docs/maintenance-checklist.md` for the full
|
|
list of places that must stay in sync.
|