Rubrion Platform

Rubrion builds open-source, white-label infrastructure products — publishing engines, job intelligence tools, and marketplace platforms — designed to be self-hosted, customised, and deployed at scale.

Products

EdgePress

EdgePress is a high-performance, white-label publishing engine. It ships as a single Cloudflare Worker (Astro SSR + D1 + R2) that runs a public blog, an admin editor, a media uploader, and a newsletter dispatcher — all in one deployment with zero sidecars.

Each white-label tenant gets one Worker deployment + one D1 database. Brand visuals (name, logo, theme color, email sender) are configured live from an admin panel — no redeploy required.

Email providers:

• Resend — HTTP API, called directly from the Worker.
• Gmail SMTP — TCP from the Worker to smtp.gmail.com:465 via cloudflare:sockets.

Live demo: edgepress.rubrion.ai
Repository: github.com/rubrion/edgepress

SpecFit

SpecFit is an open-source job application tracker with LLM-powered CV matching. Paste a job description and your CV to receive a structured fit score, matched skills, gaps, and a tailored recommendation. Track applications through their full lifecycle.

Key capabilities:

• CV PDF upload with text extraction (pypdf), 5 MB limit.
• LLM match analysis via OpenRouter (default openai/gpt-4o-mini), results cached per (model, job_description, cv_text) hash.
• LinkedIn profile suggestions powered by the Brave Search API.
• Authentication via Auth.js v5 — email + password (Resend-verified) and Google / GitHub OAuth.
• Per-user daily token budget with enforced cap.

Platform:

Repository: github.com/rubrion/specfit

Rubot

Rubot is a multi-tenant, LLM-driven chat-agent scaffold built on Cloudflare Workers (gateway + middleware) and Railway (Python agents). Specialist agents are forked from a common template, registered in an orchestrator, and routed by an LLM planner — all connected through a short-lived, HMAC-signed bearer chain that never exposes long-lived credentials to the model.

Services:

Two deployment modes:

• Bearer mode — full multi-tenant identity: manager accounts, PIN-based sender→tenant binding, HMAC-signed short-lived data bearers, per-tenant agent toggles.
• Open mode — auth chain bypassed; all services share one implicit tenant. Suitable for single-tenant or development deployments.

Repository: github.com/rubrion/docs (under rubot/)

Rubrion Store

Rubrion Store is a white-label marketplace platform — multi-tenant storefronts, vendor onboarding, product catalogue, and a CMS-driven front-end with transparent infrastructure costs. Run it for your community or your customers.

The platform is built on Medusa + Mercur for commerce and marketplace logic, with a Next.js B2C storefront and a Sanity Studio for content management.

Monorepo layout:

CMS — Sanity Studio:

• Hosted Studio (no self-hosting needed); schema lives in apps/storefront/sanity/.
• Supports Home pages, Landing pages, Navigation, and Static pages — each per locale (BR / ES / US).
• Storefront revalidates via signed webhook on publish (~1 s); ISR fallback at 60 s.

Live demo: rubrion.store
Repository: github.com/rubrion/store

Getting started

Each product has its own setup guide in this documentation. Choose a product from the sidebar to find prerequisites, environment variables, deployment instructions, and API references.

For access credentials, onboarding details, or white-label customisation, reach out to the Rubrion team:

• Portal: rubrion.ai
• Email: hello@rubrion.ai
• WhatsApp: Chat with Samuel

EdgePress

EdgePress is a high-performance, white-label publishing engine. It ships as a single Cloudflare Worker (Astro SSR + D1 + R2) that runs a public blog, an admin editor, a media uploader, and a newsletter dispatcher — all in one deployment with zero sidecars.

Each white-label tenant gets one Worker deployment + one D1 database. Brand visuals (name, logo, theme color, email sender) are configured live from an admin panel — no redeploy required.

Email providers:

• Resend — HTTP API, called directly from the Worker.
• Gmail SMTP — TCP to smtp.gmail.com:465 via cloudflare:sockets. No sidecar container.

Live demo: edgepress.rubrion.ai
Repository: github.com/rubrion/edgepress

Architecture

EdgePress runs entirely inside a single Cloudflare Worker. There is no separate container or server to manage.

                    ┌─RESEND─▶ api.resend.com (HTTPS)
[Reader]──HTTPS──┐  │
                 ├─▶[Astro/CF Worker]──┬──▶[D1] (per-tenant SQLite: posts, subscribers, campaigns, settings)
[Admin] ──HTTPS──┘  │                  └──▶[R2] (media uploads, optional shared bucket)
                    └─GMAIL──▶ smtp.gmail.com:465 (TCP+TLS via cloudflare:sockets)

• All public pages, the admin UI, /api/*, and media uploads run inside one Worker.
• D1 holds posts, subscribers, campaigns, settings (schema in src/db/schema.ts).
• R2 holds uploaded images and videos, organized as edgepress/<CLIENT_SLUG>/<yyyy-mm>/<uuid>.<ext>. One bucket can be shared across tenants — slug-prefixed paths keep them isolated.
• Brand visuals (name, tagline, logo, favicon, theme color, email From-address) live in D1 and are editable from /admin/settings without redeploy.
• Provider choice is a config flip (EMAIL_PROVIDER var); no code changes.

Built-in features

• Markdown editor with live preview, drag-drop / paste / button image + video upload (R2-backed, 50 MB cap), per-post Publish + Send to active subscribers.
• Newsletter dispatch with List-Unsubscribe + one-click POST headers (Gmail/Yahoo bulk-sender compliant), plain-text alternative, and per-subscriber unsubscribe links.
• Subscriber unsubscribe — /api/unsubscribe?id=<uuid> (GET for link clicks, POST for one-click). Sets subscribers.status = 'unsubscribed' so future dispatches skip them.
• Post-delete media cleanup — when a post is deleted from the admin, any R2 objects it referenced under your own bucket prefix are removed. Externally-pasted URLs are left alone.
• Dark / light mode toggle. Detects prefers-color-scheme on first visit, persists choice in localStorage.
• i18n — en and pt-BR translations for all public-facing UI. Detects browser Accept-Language on first visit, persists choice in a lang cookie.
• Live brand admin at /admin/settings — change name, tagline, logo, favicon, accent color, email From-address without a deploy.

Prerequisites

Useful Commands

Configuration Reference

EdgePress splits configuration into three layers, each with a different lifecycle.

Admin-managed (D1 settings table) — change anytime, no redeploy

Resolution at request time: DB row → wrangler.jsonc seed → hard-coded default. Saving an empty value in admin removes the override and falls back to the seed.

Wrangler vars — change requires redeploy

CLIENT_FONT is read at build time by astro.config.mjs, so changing it requires a redeploy.

Wrangler bindings

Secrets — wrangler secret put

Local development (.dev.vars)

.dev.vars is the Wrangler equivalent of .env — only loaded when wrangler dev runs. Do not commit it.

MASTER_ADMIN_KEY=local-dev-admin-key
RESEND_API_KEY=re_dev_xxx
GMAIL_USER=your-gmail@gmail.com
GMAIL_APP_PASSWORD=xxxxxxxxxxxxxxxx

Per-Tenant Deployment

Run all commands from the repo root. Replace <tenant> with the tenant's slug (lowercase, hyphen-separated).

1. Install

bun install

2. Create the D1 database

bunx wrangler d1 create <tenant>-edgepress

Copy the returned database_id into wrangler.jsonc:

"d1_databases": [
  {
    "binding": "DB",
    "database_name": "<tenant>-edgepress",
    "database_id": "<paste-here>",
    "migrations_dir": "./drizzle"
  }
]

The binding name must be DB for every tenant — that's the contract the code reads. Only database_name and database_id change per tenant.

3. Create (or pick) an R2 bucket for media

bunx wrangler r2 bucket create <your-bucket>

One bucket can be shared across all tenants — uploads land under edgepress/<CLIENT_SLUG>/... so each tenant has its own folder.

4. Set tenant vars in wrangler.jsonc

"routes": [
  { "pattern": "<tenant>.example.com", "custom_domain": true }
],
"vars": {
  "CLIENT_DOMAIN": "<tenant>.example.com",
  "CLIENT_SLUG": "<tenant>",
  "CLIENT_FONT": "Inter",
  "MEDIA_PUBLIC_BASE": "https://media.example.com",
  "EMAIL_PROVIDER": "RESEND"
},
"r2_buckets": [
  { "binding": "MEDIA", "bucket_name": "<your-bucket>" }
]

5. Apply database migrations

bunx wrangler d1 migrations apply <tenant>-edgepress --local   # local
bunx wrangler d1 migrations apply <tenant>-edgepress --remote  # production

To regenerate the SQL after a schema change: bun run db:generate.

6. Set secrets

bunx wrangler secret put MASTER_ADMIN_KEY        # always

# If EMAIL_PROVIDER=RESEND
bunx wrangler secret put RESEND_API_KEY

# If EMAIL_PROVIDER=GMAIL
bunx wrangler secret put GMAIL_USER
bunx wrangler secret put GMAIL_APP_PASSWORD

7. Regenerate types and deploy

bun run cf-typegen   # refresh worker-configuration.d.ts
bun run deploy       # astro build + wrangler deploy

8. Configure brand visuals via admin

Open https://<tenant>.example.com/admin/login, paste your MASTER_ADMIN_KEY, then navigate to /admin/settings and fill in brand name, tagline, logo URL, favicon URL, theme primary color, and email From-address.

Saved values take effect immediately on the next page render.

Email Providers

EdgePress supports two email providers, selected per-tenant. Switching providers is a config-only change — no code edits, no infra moves.

Switching providers

The from address differs between providers:

• Resend uses emailFromAddress from /admin/settings if set, otherwise falls back to noreply@$CLIENT_DOMAIN. The domain must be verified in Resend.
• Gmail uses $GMAIL_USER directly (Gmail rejects mismatched senders).

Email deliverability (avoiding spam)

For Resend, three DNS records on $CLIENT_DOMAIN are required for emails to land in inboxes:

Add them in your Resend dashboard → Domains → Add domain.

EdgePress already sends the headers Gmail's bulk-sender policy requires (List-Unsubscribe, List-Unsubscribe-Post: List-Unsubscribe=One-Click, plain-text alternative).

Operational notes

• Free-tier Worker CPU is 10 ms per invocation. Each Gmail send opens a TCP+TLS handshake to smtp.gmail.com:465. Realistic ceiling on the free tier is a few hundred recipients per publish. The paid Workers tier ($5/mo) lifts the cap to 30 s CPU.
• Gmail send quota — Gmail SMTP caps at ~500 messages/day per account. Tenants with growing lists must move to Resend.
• The Gmail SMTP path uses cloudflare:sockets, which only runs on the Cloudflare runtime (production or wrangler dev). It cannot run in plain Node/Bun.

Local Development

bunx wrangler d1 migrations apply <tenant>-edgepress --local   # one-time
bun run dev                                                    # http://localhost:4321

bun run dev runs astro dev, which has a Vite server but no Cloudflare bindings. For features that depend on cloudflare:workers env (D1, R2, env vars), use bunx wrangler dev instead.

Smoke test

curl -X POST http://localhost:4321/api/subscribe \
  -H 'Content-Type: application/json' \
  -d '{"email":"me@test.com"}'
# → {"ok":true}

bunx wrangler d1 execute <tenant>-edgepress --local \
  --command "SELECT email, status FROM subscribers"

Then open /admin/login, paste your MASTER_ADMIN_KEY, configure brand visuals at /admin/settings, write a markdown post (drag-drop images!), and hit Publish + Send.

.dev.vars

Create .dev.vars at the repo root (do not commit it):

MASTER_ADMIN_KEY=local-dev-admin-key
RESEND_API_KEY=re_dev_xxx
GMAIL_USER=your-gmail@gmail.com
GMAIL_APP_PASSWORD=xxxxxxxxxxxxxxxx

SpecFit

Open-source job application tracker with LLM-powered CV matching.

Paste a job description and your CV to receive a structured fit score, matched skills, gaps, and a tailored recommendation. Track applications through their full lifecycle. Sign in with email + password (with verification) or Google / GitHub OAuth. Every user gets an enforced daily LLM token budget.

Platform:

Repository: github.com/rubrion/specfit

API Reference

All /applications/* and /auth/oauth-upsert require the relevant credential. /auth/* (except oauth-upsert) and /healthz are public.

Data Model

Application status enum: saved → applied → interviewing → offer / rejected / withdrawn.

CV text is never persisted. Only the SHA256 hash and the structured analysis are stored.

Deployment

Backend — Railway

Push backend/ to Railway. Attach the Postgres plugin (auto-injects DATABASE_URL). Set the following env vars:

• OPENROUTER_API_KEY
• BACKEND_JWT_SECRET
• AUTH_SHARED_SECRET
• RESEND_API_KEY
• EMAIL_FROM
• FRONTEND_BASE_URL
• CORS_ORIGINS

Railway builds the Dockerfile and runs boot.sh (alembic upgrade head then uvicorn).

Frontend — Cloudflare Workers

See web-client/DEPLOY.md for the full guide. Set build-time vars in .env.production and runtime secrets via wrangler secret put:

wrangler secret put AUTH_SECRET
wrangler secret put BACKEND_JWT_SECRET
wrangler secret put AUTH_SHARED_SECRET
# OAuth (optional)
wrangler secret put AUTH_GOOGLE_ID
wrangler secret put AUTH_GOOGLE_SECRET
wrangler secret put AUTH_GITHUB_ID
wrangler secret put AUTH_GITHUB_SECRET

Then deploy:

npm run deploy

Environment Variables

Backend

Frontend

Local Development

Backend

cd backend
uv venv --python 3.12 && source .venv/bin/activate
uv pip install -e ".[dev]"
cp .env.example .env   # fill in secrets
alembic upgrade head
uvicorn app.main:app --reload

Seed demo data (optional):

python -m scripts.seed --reset

Frontend

cd web-client
npm install
cp .env.example .env.local   # fill in secrets
npm run dev

Stack

Backend (backend/)

• Language: Python 3.12
• Framework: FastAPI (async)
• ORM: SQLModel on SQLAlchemy 2 async
• Migrations: Alembic (real migration files; no metadata auto-create)
• DB drivers: asyncpg (runtime), psycopg[binary] (migrations)
• Auth: own credentials store with bcrypt hashes, verify/reset tokens via verification_tokens table, OAuth upsert endpoint
• Bearer: HS256 JWT (PyJWT) minted by the web client and verified by every request
• Email: Resend SDK
• LLM client: pydantic-ai with OpenAIModel + OpenAIProvider pointed at OpenRouter
• Caching: SHA256 hash of (model, job_description, cv_text) against Application.analysis_hash; bypass with ?force=true
• Budget: per-user daily token cap in token_usage; over-cap returns 429 DAILY_TOKEN_LIMIT
• Tracing: Logfire instrumentation for FastAPI, SQLAlchemy, httpx, pydantic-ai
• Lint / typecheck / test: ruff, pyright, pytest
• Container: python:3.12-slim, boot.sh runs alembic upgrade head then uvicorn

Frontend (web-client/)

• Framework: Next.js 16 (App Router)
• Runtime: React 19
• Auth: Auth.js v5 (NextAuth) with Credentials + Google + GitHub providers, JWT session strategy
• Token mint: jose HS256 — every session carries a 15-minute bearer that the FastAPI backend validates
• Styling: Tailwind CSS v4
• Language: TypeScript (strict)
• Deploy adapter: @opennextjs/cloudflare → Cloudflare Workers
• Theme: High-Contrast Minimalist (Deep Black #09090B, White #FFFFFF, Slate #64748B)
• Typography: Geist Sans (UI) / Geist Mono (LLM output, metrics)

Rubrion Store

Rubrion Store is a white-label marketplace platform — multi-tenant storefronts, vendor onboarding, product catalogue, and a CMS-driven front-end with transparent infrastructure costs. Run it for your community or your customers.

Built on Medusa + Mercur for commerce and marketplace logic, with a Next.js B2C storefront and a Sanity Studio for content management.

Monorepo layout:

Live demo: rubrion.store
Repository: github.com/rubrion/store

Architecture

Rubrion Store is a Turborepo monorepo built on Medusa + Mercur.

Project structure

├── apps/
│   ├── admin/          # Admin dashboard extensions
│   ├── vendor/         # Vendor portal extensions
│   └── storefront/     # Next.js B2C storefront + Sanity Studio
├── packages/
│   └── api/            # Medusa backend
│       ├── src/
│       │   ├── api/         # Custom API routes
│       │   ├── jobs/        # Background jobs
│       │   ├── links/       # Module links
│       │   ├── modules/     # Custom modules
│       │   ├── scripts/     # CLI scripts
│       │   ├── subscribers/ # Event subscribers
│       │   └── workflows/   # Business workflows
│       └── medusa-config.ts
├── blocks.json         # Mercur blocks configuration
└── turbo.json

Services

Extending with Mercur Blocks

Add pre-built marketplace features using the Mercur CLI:

bunx @mercurjs/cli add block-name

Configure block sources in blocks.json:

{
  "aliases": {
    "workflows": "packages/api/src/workflows",
    "links": "packages/api/src/links",
    "api": "packages/api/src/api",
    "modules": "packages/api/src/modules"
  },
  "registries": {}
}

CMS Guide

Sanity Studio is hosted on Sanity (App Studio). The storefront pulls content via GROQ and revalidates via webhook on publish.

• Studio URL: https://www.sanity.io/@oJjkNT4GN/studio/blgyxui5ep83evp3wncgrwyy
• Storefront URL: https://rubrion.store
• Sanity project: rubrion-store (id l54kfv7o)
• Dataset: production

Accessing Studio

1. Open the Studio URL above.
2. Log in with your Sanity account.
3. If access is denied, ask the project owner to invite you: sanity.io/manage → rubrion-store → Members → Invite member → role Editor or Viewer.

The sidebar shows: Home pages, Navigation, Landing pages, Static pages — each per locale (BR / ES / US).

Daily editing flow

Home page

1. Sidebar → Home pages → choose locale.
2. Fill Hero (heading, paragraph, image, up to 3 buttons) + Sections (Banner / CTA / Featured products).
3. Set SEO if needed. Click Publish.

Landing page

1. Sidebar → Landing pages → + Create.
2. Fill Internal title, Locale, Slug (e.g. summer-sale).
3. Publish → live at https://rubrion.store/us/summer-sale.

Navigation

1. Sidebar → Navigation → choose locale.
2. Edit Header links, Footer groups, Legal links. Publish.

Static pages

1. Sidebar → Static pages → + Create.
2. Fill title, locale, slug (e.g. privacy), body (rich text).
3. Publish → live at https://rubrion.store/us/pages/privacy.

Cache revalidation

Two paths run in parallel:

• Webhook (instant) — on Publish, Sanity fires a signed webhook → /api/revalidate. Handler invalidates Next.js cache tags scoped to the doc (~1 second).
• ISR fallback (60 s) — if the webhook ever fails, Next.js auto-revalidates after 60 s on the next request.

Verify webhook health: sanity.io/manage → rubrion-store → API → Webhooks → Attempts tab. Look for HTTP 200 responses.

Updating Studio schema

cd apps/storefront

# 1. Edit schema files under sanity/schemas/
# 2. Edit sanity/structure.ts if sidebar layout changed
# 3. Validate locally
bunx sanity schema validate

# 4. Deploy schema + Studio
bun run studio:deploy

For iterating locally without deploying:

bun run studio:dev   # local Studio at http://localhost:3333

CMS env vars (Marketplace service on Railway)

Common operations

Deployment

Backend — Railway

The Medusa backend (packages/api) is deployed to Railway as a Node.js service.

1. Create a Railway project and point it at the repo.
2. Set Root Directory to the repo root.
3. Set the required env vars (database URL, JWT secret, cookie secret, Redis URL).
4. Railway builds and starts the service automatically on push.

Storefront — Railway

The Next.js storefront (apps/storefront) is deployed to Railway.

Required env vars:

Admin + Vendor — Railway

Deploy apps/admin and apps/vendor as separate Railway services. Both are static React + Vite builds; configure the build command to bun run build and the publish directory to dist/.

Sanity Studio

Studio is hosted on Sanity's managed infrastructure. Deploy schema changes with:

cd apps/storefront
bun run studio:deploy

Local Development

Prerequisites

• Bun
• PostgreSQL (or a Postgres connection string)
• Redis

Setup

1. Clone the repo and copy the env template:

cp packages/api/.env.template packages/api/.env

2. Update the .env file:

DATABASE_URL=postgres://user:password@localhost:5432/mercur
REDIS_URL=redis://localhost:6379
JWT_SECRET=your-super-secret-jwt-key
COOKIE_SECRET=your-super-secret-cookie-key

3. Install dependencies and start:

bun install
bun dev

4. Open the services:

Follow the on-screen instructions to log in and create your first admin user.

Build

bun run build

Rubot

Rubot is a multi-tenant, LLM-driven chat-agent scaffold built on Cloudflare Workers (gateway + middleware) and Railway (Python agents). Specialist agents are forked from a common template, registered in an orchestrator, and routed by an LLM planner — all connected through a short-lived, HMAC-signed bearer chain that never exposes long-lived credentials to the model.

Services:

Two deployment modes:

• Bearer mode (RUBOT_DATA_AUTH=bearer) — full multi-tenant identity: manager accounts, PIN-based sender→tenant binding, HMAC-signed short-lived data bearers, per-tenant agent toggles.
• Open mode (RUBOT_DATA_AUTH=open) — auth chain bypassed; all services share one implicit tenant. Suitable for single-tenant or development deployments.

Recommended reading order: Architecture → Local Development → Creating a New Agent → Deploy. Env Vars is a reference cheat-sheet.

Architecture

rubot is a five-layer stack. Each layer has a single responsibility and a clear wire contract with its neighbours.

Layers

Request lifecycle

1. user sends a message to the chat-source adapter (WhatsApp/Slack/Telegram/...)

2. adapter POSTs /v1/chat/completions to rubot-gateway with
     Authorization: Bearer GATEWAY_API_KEY
     X-Chat-Source-Session-Id: <unique-per-conversation>
     X-Chat-Source-Sender-Id:  <unique-per-end-user>

3. rubot-gateway
     - mints X-Rubot-Trace-Id (32-char hex) if absent
     - resolves session → (tenant_id, short-lived data bearer)
         lookup session_bearers → if expired/missing, self-heal via
         rubot-middleware Service Binding to refresh or bind
     - forwards request to rubot-orchestrator with
         X-Tenant-Id, X-Rubot-Data-Bearer, X-Rubot-Trace-Id

4. rubot-orchestrator
     - preflight: GET rubot-middleware /api/data/<tenant>/connections
         returns which providers the tenant has linked
     - capabilities fan-out: GET <each-agent>/v1/capabilities
         (cached 30 min)
     - planner LLM picks one or more agents to dispatch to
     - calls each agent: POST <agent>/v1/chat/completions
         with the same X-* headers
     - merges responses; returns OpenAI-shaped completion

5. specialist agent (forked from rubot-agent-template)
     - runs pydantic_ai Agent
     - tools call rubot-middleware /api/example-provider/data/<tenant>/...
         with X-Rubot-Trace-Id, X-Rubot-Data-Bearer forwarded
     - emits structured log envelope per step
     - emits agent_log_v1 payload on completion

6. rubot-middleware
     - validates incoming bearer (HMAC verify, < 900 sec TTL)
     - calls upstream data source (HTTP/OAuth/JDBC/...)
     - returns JSON to the agent

7. response flows back up: agent → orchestrator → gateway → adapter → user

Every hop carries X-Rubot-Trace-Id. Every log line has the same envelope shape. Filter by trace_id and the full lifecycle is one query.

Wire contract — specialist agent

Required inbound headers:

• Authorization: Bearer ORCHESTRATOR_API_KEY
• X-Tenant-Id
• X-Rubot-Data-Bearer
• X-Rubot-Trace-Id (set by gateway; middleware fills if absent)

Bearer format (data bearer)

mbr.v1.<tenantIdB64url>.<expSec>.<sigB64url>

HMAC-SHA256 over <tenantId>.<expSec> with the secret BEARER_SIGNING_SECRET. TTL clamp 60–900 seconds. Stateless verification (no DB lookup). Same secret shared between rubot-gateway (mint) and rubot-middleware (verify).

Structured log envelope

Every log line (Python or TS) is a JSON object:

{
  "timestamp": "2026-05-27T13:24:01.123Z",
  "log_level": "INFO",
  "service": "rubot-agent-template",
  "component": "app.agent.tools",
  "environment": "production",
  "deployment_hash": "a1b2c3d4",
  "tenant_id": "tenant-abc",
  "chat_source_session_id": "sess-...",
  "sender_id": "user-...",
  "trace_id": "f0e1d2c3b4a5...",
  "event_type": "tool.call.completed",
  "message": "Fetched 42 rows from example-provider",
  "extra": { "rows": 42, "elapsed_ms": 117 },
  "agent": null  // populated only on agent.log events
}

On agent.log, the agent field carries an agent_log_v1 payload: dimensions (provider/model), conversation (user message, assistant response, system prompt snapshot, history), execution (steps with tokens/cost/timing), problem_signals (tool errors, context overflow, …).

Security model

Each long-lived API key lives in env / secrets storage. The short-lived data bearer is minted per request, scoped to a tenant, capped at 15 minutes. Stateless verification means no extra DB round-trip in the hot path.

Open mode (RUBOT_DATA_AUTH=open)

For local development and simple deployments that don't need tenant-scoped data access, set RUBOT_DATA_AUTH=open across all services. In this mode the entire bearer chain is bypassed: no minting, no forwarding, no verification. Agents access all available data sources without authentication. BEARER_SIGNING_SECRET is not required.

X-Tenant-Id still propagates (using RUBOT_OPEN_TENANT at the gateway) for logging and routing, but is not enforced at the middleware.

Manager + provisioning subsystem (RUBOT_DATA_AUTH=bearer only)

Bearer mode ships a small three-actor identity model so a dashboard user can hand out scoped access without having to know what a minted bearer is:

• Manager — human dashboard user. One row in managers (email + PBKDF2 password hash + email-confirmed flag + reset/confirm tokens). Authenticated by the HMAC-signed rubot_session cookie, which is signed with SESSION_SIGNING_SECRET (deliberately a different key from BEARER_SIGNING_SECRET — manager-session compromise must not pivot to data-bearer forgery).
• Tenant — tenants.tenant_id row. A manager owns N tenants via manager_tenants(manager_id, tenant_id). isManagerOwnerOf gates every per-tenant admin endpoint.
• Sender — chat-source identity (Telegram chat id, WhatsApp E.164, Slack user id, etc.). identity_bindings(sender_id → tenant_id) resolves the tenant_id that /api/internal/bind-session mints a bearer for.

The PIN flow ties the three together:

1. Manager logs into the dashboard → POST /api/provision/generate {tenant_id} returns a 6-digit PIN (5-minute TTL, single-use, stored in the PROVISIONING KV with both <pin> → tenant_id and tenant:<tenant_id> → { pin, expiresAt } keys).
2. Manager hands the PIN to the end user out-of-band.
3. End user (any transport) sends the PIN to a public endpoint that forwards POST /api/provision/consume { pin, sender_id }. The route looks up the PIN, upserts identity_bindings(sender_id → tenant_id), deletes both KV keys.
4. Future chat turns flow through /api/internal/bind-session {session_id, sender_id} — the sender → tenant_id lookup uses the freshly-inserted binding, so the bearer is minted for the correct tenant transparently.

The entire subsystem is unmounted in open mode — /api/auth/*, /api/provision/*, and /api/tenant/* all return 404. Open-mode deployments don't have a notion of tenant ownership or PIN-bound sender identity to begin with, so administering those routes would be meaningless.

Tenant-admin API + per-tenant agent filter

Bearer mode also exposes /api/tenant/* (gated by the same 404 short-circuit) for the dashboard to drive integrations, agents, and sender bindings:

• GET/POST/DELETE /api/tenant/:tenantId/integrations[/...] — manage integration_tokens rows (paste-API-key in v1; OAuth start flow is a roadmap stub).
• GET /api/tenant/:tenantId/agents and POST .../:agentId/toggle — per-tenant tenant_agents.enabled. Backed by the KNOWN_AGENTS_JSON middleware env var as the source of truth for which agents are registered globally.
• GET/DELETE /api/tenant/:tenantId/senders[/...] — list / revoke identity_bindings.
• GET /api/tenant/:tenantId/usage — placeholder (see docs/observability.md).

The orchestrator picks up tenant_agents automatically: the existing preflight GET /api/data/:tenantId/connections now returns an agents array alongside connections. The orchestrator router intersects its registry × available providers × enabled_agents, so toggling an agent off in the dashboard takes effect on the next planner round-trip without redeploying anything.

The dashboard itself (workers/rubot-client/) is documented in docs/dashboard.md.

Account approval + super-admin

Manager accounts pass through four states before they can act on tenants:

register → email_confirmed=0
        ─→ confirm-email → email_confirmed=1, approved=0   (pending super-admin)
                        ─→ super-admin approves → approved=1
        (revoke at any time → approved=0)

The approved=1 gate is enforced by requireApprovedManager in src/utils/session.ts, used by every /api/tenant/* and the manager-session branches of /api/provision/*. Login itself works at approved=0 (cookie minted) so the dashboards can render a clear "Pending approval" state instead of an opaque error.

A second flag is_superadmin lives on the same row. Super-admins access /api/admin/* (always mounted in both modes) to approve / revoke / promote / demote others. The first super-admin is bootstrapped via the SUPERADMIN_EMAIL middleware env var: the first register whose email matches lands directly in email_confirmed=1, approved=1, is_superadmin=1 and no confirmation email is sent. Every state change writes an account_audit row.

The super-admin dashboard (workers/rubot-superadmin/) is documented in docs/superadmin.md. rubot-open-client (open-mode operator dashboard) is documented at the top of its src/ tree; v1 only ships the auth surface and the pending-approval screen there.

Why this shape

• multi-tenant isolation (tenant_id pinned at the edge, propagated by header)
• LLM-driven routing across many specialists (capabilities + planner)
• end-to-end traceability (one trace_id, one log envelope shape)
• short-lived data access (no long-lived bearer ever near the LLM)
• swappable providers (orchestrator never knows specific providers; it asks middleware which ones are linked, then routes by capability summary)

Creating a new specialist agent

Each specialist is a fork of agents/rubot-agent-template/, the abstract scaffold that wires BaseAgent, RubotLoggingMiddleware, OpenAI-compatible endpoints, and trace propagation.

Plan a few things before coding:

1. source_id — slug, no spaces, unique across registered agents. Used in:
   • agent_name (key in rubot_config/agents.yaml, env var prefix)
   • source_id (in /v1/capabilities)
   • service name (<source-id>-agent or whatever convention you pick)
   • key in the orchestrator's AGENT_REGISTRY_JSON
2. LLM provider + model — OpenAI, Anthropic, Groq, Mistral, or any provider via custom base_url. Default is openai:gpt-4o-mini from the defaults: block of agents.yaml.
3. Data sources — which upstream APIs/DBs the agent calls. Through rubot-middleware (preferred — gives you tenant scoping and short-lived bearers) or direct.
4. Tools — one tool per capability (get_<thing>, query_<thing>). Have the list before writing code.

Step 1 — fork the scaffold

cp -r agents/rubot-agent-template agents/<source-id>-agent
cd agents/<source-id>-agent

The structure you get:

agents/<source-id>-agent/
├── Dockerfile
├── pyproject.toml
├── start.sh
└── app/
    ├── main.py
    ├── config.py
    ├── models.py
    ├── prompt.txt
    └── agent/
        ├── deps.py
        ├── tools.py
        ├── template_agent.py    # rename to <source-id>_agent.py
        └── runner.py

Step 2 — required renames

pyproject.toml

name = "rubot-agent-template" → name = "<source-id>-agent"

app/agent/template_agent.py

• Rename file to app/agent/<source-id>_agent.py
• class TemplateAgent → class <SourceId>Agent
• agent_name = "template" → agent_name = "<source-id>"
• All template_agent variable usages → <source-id>_agent

app/agent/runner.py

Update the import to point at the renamed module.

app/main.py

• configure_logger(service="rubot-agent-template") → service="<source-id>-agent"
• _CAPABILITIES:
  • source_id="template" → "<source-id>"
  • name="Template Agent" → user-friendly name
  • summary=... → precise description of what this agent answers. The orchestrator's LLM planner reads this string verbatim to decide whether to dispatch to you. List the data types and question shapes the agent handles, explicitly.
• model="template" in ChatCompletionResponse → your source_id

Dockerfile

• COPY agents/rubot-agent-template/... → COPY agents/<source-id>-agent/...

app/config.py

• Rename TEMPLATE_API_KEY / TEMPLATE_BASE_URL etc. to provider-specific env vars (<PROVIDER>_API_KEY).
• Add any extra config the agent needs.

Step 3 — model config (optional)

If openai:gpt-4o-mini works, do nothing. The agent boots from the defaults: block of agents.yaml with a logged warning.

For a different model, edit shared-packages/packages/rubot-config/rubot_config/agents.yaml:

agents:
  <source-id>:
    provider: openai
    model: gpt-5.1
    max_tokens: 8000
    reasoning_effort: high

Per-deployment overrides via env vars:

AGENT_<SOURCE_ID>_MODEL=gpt-4o
AGENT_<SOURCE_ID>_MAX_TOKENS=4096

(Agent-name normalization: hyphens → underscores, uppercase.)

Step 4 — write the system prompt

app/prompt.txt. A solid specialist prompt has:

• Persona — what the agent is expert at.
• Hard rules — never fabricate, never quote numbers without a source, cite when relevant.
• Output shape — opening, body, closing. Keep it tight.
• Placeholders — {reference_date} and any context the system_prompt callback injects.

Step 5 — implement tools

In app/agent/tools.py, one async function per capability:

async def get_things(
    ctx: RunContext[AgentDeps],
    arg: str,
) -> dict[str, Any]:
    """One-line description; this docstring becomes the tool description."""
    url = f"{settings.MIDDLEWARE_BASE_URL}/api/example-provider/data/{ctx.deps.tenant_id}/things"
    headers = _forward_trace_headers({
        "Authorization": f"Bearer {ctx.deps.data_bearer}",
    })

    async with httpx.AsyncClient(timeout=15) as client:
        r = await client.get(url, headers=headers, params={"arg": arg})
        r.raise_for_status()
        return r.json()

Rules:

• In open mode (RUBOT_DATA_AUTH=open), ctx.deps.data_bearer is empty. Only add the Authorization header when the bearer is non-empty. The manager + PIN subsystem (/api/auth/*, /api/provision/*) is also unmounted in open mode — sender→tenant binding is bypassed entirely, and the gateway uses RUBOT_OPEN_TENANT instead. If your agent needs per-sender isolation, deploy in bearer mode and follow the manager bootstrap in docs/local-dev.md.
• Always use _forward_trace_headers() on outbound httpx calls — it reads trace_id_var and adds X-Rubot-Trace-Id.
• Log tool.call.started / .completed / .failed with get_logger().
• On error, return {"error": "..."} instead of raising. The LLM handles error dicts gracefully; raises kill the run.
• Never log the data_bearer.

Register tools in <source-id>_agent.py:

<source-id>_agent.tool(get_things)

Step 6 — smoke test

cd <repo-root>/rubot
./scripts/dev-setup.sh <source-id>-agent
source .venv/bin/activate
cd agents/<source-id>-agent

# minimal env
cat > .env <<EOF
OPENAI_API_KEY=sk-...
ORCHESTRATOR_API_KEY=
MIDDLEWARE_BASE_URL=http://localhost:8788
EOF

uvicorn app.main:app --reload --port 8000

In another terminal:

curl http://localhost:8000/
curl -H "X-Tenant-Id: dev" http://localhost:8000/v1/capabilities
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "X-Tenant-Id: dev" \
  -H "X-Rubot-Data-Bearer: fake-bearer" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"<test question>"}]}'

Check:

• /v1/capabilities returns the right source_id and summary.
• /v1/chat/completions returns a non-empty answer.
• stdout logs are JSON envelopes (not plain text).
• trace_id is the same across every event of one request.
• An agent.log event with _schema=agent_log_v1 lands after each run.

Step 7 — register with the orchestrator

Edit agents/rubot-orchestrator's env (.env locally, env vars on production):

AGENT_REGISTRY_JSON={"template":"http://localhost:8000","<source-id>":"http://localhost:8002"}

For production, point at the deployed service URL.

Step 8 — deploy

See deploy.md. Pre-deploy checklist:

• Branch merged.
• Env vars set on the deploy host (see env-vars.md).
• AGENT_REGISTRY_JSON on the orchestrator updated to include the new agent's URL.
• If you changed agents.yaml, that shared-package change is shipped (rebuilt Docker image or tagged release of an extracted shared repo).

PR checklist

• agents/<source-id>-agent/ is self-contained — no edits to other agents.
• No imports of app.agent.template_agent anywhere.
• No literal "template" or "Template Agent" in code except comments explaining the fork.
• _CAPABILITIES.summary reads like routing prose — precise, no marketing.
• Tools emit tool.call.* events and forward trace_id.
• .env not committed.
• If you touched shared-packages/, that change is its own commit/PR.

Anti-patterns

• ❌ Import rubot_config.BaseAgent but instantiate pydantic_ai.Agent directly — you lose the auto-emitted agent_log payload.
• ❌ Use print() or logging.basicConfig() instead of rubot_logger.get_logger().
• ❌ Omit RubotLoggingMiddleware — trace_id won't appear and cross-service correlation is impossible.
• ❌ Pass data_bearer as a query string. Always header.
• ❌ Hardcode tenant_id or a customer name. Tenants come from the X-Tenant-Id header.
• ❌ Multiple sequential upstream calls without timeouts. Use asyncio.gather + per-call timeout.
• ❌ Write a new rubot_logger/ module inside the agent. Always import from the shared package.

Dashboard (rubot-client)

Manager-facing admin UI for the bearer-mode rubot scaffold. Astro on Cloudflare Workers; talks to rubot-middleware over a Service Binding.

Topology

Browser ──▶ rubot-client (Astro SSR + /api/proxy/[...path].ts)
                          │
                          └── Service Binding: MIDDLEWARE
                                     │
                                     ▼
                              rubot-middleware
                                /api/auth/*
                                /api/tenant/*
                                /api/provision/*
                                /api/data/*

• All sensitive routes live on rubot-middleware. rubot-client is pure UI plus a cookie-forwarding proxy at /api/proxy/[...path].ts.
• The rubot_session HMAC cookie is signed by middleware with SESSION_SIGNING_SECRET and verified by middleware on every call — rubot-client never touches the signing key.

Routes

Auth model

layouts/Dashboard.astro gates every /dashboard/* page:

1. If RUBOT_DATA_AUTH=open, render a "bearer mode required" banner and stop.
2. Else fetch /api/proxy/auth/me. 401 → redirect to /login.
3. Else if the session resolves but approved=0, render the Pending approval card (account exists, but a super-admin hasn't activated it yet). Tenant routes return 403 not_approved.
4. Else render the page with the manager email in the topbar.

/api/tenant/* endpoints additionally check requireApprovedManager → isManagerOwnerOf(manager_id, tenantId) on the middleware side — page-level UI gating is convenience, not the security boundary.

Role model

Super-admin management lives in a separate worker (rubot-superadmin), documented in docs/superadmin.md. The bootstrap super-admin comes from the middleware env var SUPERADMIN_EMAIL.

Env / bindings

No secrets live in rubot-client itself.

Local dev

# terminal A: middleware
cd workers/rubot-middleware
npx wrangler dev --port 8788

# terminal B: rubot-client
cd workers/rubot-client
npx wrangler dev --port 8788

On Workers, Service Bindings work across local wrangler dev processes when both are started with --local and bound through the same wrangler session. For the simplest path during development, hit the middleware directly through its public URL by swapping the MIDDLEWARE binding for an MIDDLEWARE_PUBLIC_BASE_URL var + a fetch() to that URL in lib/middleware.ts.

What's not in v1

• OAuth start flow for providers (paste-API-key only for now).
• Per-tenant agent enable bulk operations (one toggle per row).
• Real usage page (see observability.md).
• rubot-open-client build-out (only stock Astro shell exists; future work).

Deploy

rubot's reference deploy targets are Railway (Python agents) + Cloudflare Workers (gateway + middleware). The stack is plain enough to run on any container host + edge platform; adapt as needed.

Topology

┌─────────────────────────────────────────────────────┐
│ Cloudflare (edge)                                   │
│   rubot-gateway          rubot-middleware           │
└─────────────────┬──────────────────┬────────────────┘
                  │                  │
                  ▼ HTTPS / mTLS     ▼ HTTPS
┌─────────────────────────────────────────────────────┐
│ Railway (or any container host) — private network   │
│   rubot-orchestrator                                │
│   rubot-agent-<one>                                 │
│   rubot-agent-<two>                                 │
│   ...                                               │
└─────────────────────────────────────────────────────┘

Workers reach the Railway services via a public tunnel (Cloudflare Tunnel with Zero Trust) or direct HTTPS with IP allowlisting.

Cloudflare Workers

Each worker has a wrangler.jsonc. From its directory:

npm install
wrangler login              # once per machine
wrangler d1 create rubot_data
# copy the database_id into wrangler.jsonc
wrangler d1 execute rubot_data --file=schema.sql      # rubot-middleware only

# secrets (one-time):
wrangler secret put GATEWAY_API_KEY        # rubot-gateway
wrangler secret put ADMIN_API_KEY          # rubot-gateway
wrangler secret put BEARER_SIGNING_SECRET  # both workers (same value)
wrangler secret put MIDDLEWARE_API_KEY     # both (same value)

wrangler deploy

Per-env (staging / production), use separate names and --env:

{
  "env": {
    "staging":    { "name": "rubot-gateway-staging" },
    "production": { "name": "rubot-gateway" }
  }
}

wrangler deploy --env staging
wrangler deploy --env production

Railway services

Each Python agent ships as a Dockerfile. Repo-root build context.

Pre-flight per service:

1. Create the Railway service, point at your repo.
2. Root Directory: /rubot
3. Dockerfile Path: agents/<name>/Dockerfile
4. Watch Paths: rubot/agents/<name>/**, rubot/shared-packages/**
5. Set env vars (see env-vars.md).
6. Generate domain (private internal — agents talk over Railway's private network).

Shared packages — single repo vs. extracted:

• Single repo (default): Dockerfile copies shared-packages/ from the build context. No build secret needed.
• Extracted (future): Dockerfile uses pip install git+https://${GITHUB_TOKEN}@github.com/<org>/rubot-shared-packages.git@${SHARED_PACKAGES_REF}#subdirectory=packages/<name>. Set GITHUB_TOKEN as a Railway build secret on every service. Token needs repo read on the shared repo.

Per-environment private domains (Railway):

• Staging: <name>-staging.railway.internal:<port>
• Production: <name>.railway.internal:<port>

The orchestrator's AGENT_REGISTRY_JSON references these internal URLs:

AGENT_REGISTRY_JSON={"template":"http://rubot-agent-template.railway.internal:8000"}

Cross-environment consistency

These must match across services in the same env:

If BEARER_SIGNING_SECRET mismatches between gateway and middleware, every agent → middleware call fails 401. Verify before promoting.

Centralized logging

Every service emits one JSON line per event with a common envelope shape. Pipe them to whichever aggregator you use:

• Cloudflare Logpush → Axiom (gateway + middleware): set up in CF dashboard, no code change. Workers' observability.enabled = true is already set in wrangler.jsonc.
• Railway log drain → Axiom HTTP ingest (Python agents): set on each service's Settings → Log Drains.

Recommended Axiom dataset fields: tenant_id, trace_id, event_type, service, environment. Filter by trace_id to follow one request across all six services.

Smoke tests after deploy

# health
curl https://<gateway-url>/                              # → ok
curl https://<middleware-url>/                           # → ok

# end-to-end (use a real bearer in production)
curl -X POST https://<gateway-url>/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" \
  -H "X-Chat-Source-Session-Id: smoke-$(uuidgen)" \
  -H "X-Chat-Source-Sender-Id: smoke-sender" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hello"}]}'

Pull the response's X-Rubot-Trace-Id header, search Axiom for it. You should see events from gateway → orchestrator → agent → middleware in one trace.

Rolling back

Workers: wrangler rollback (or redeploy the previous git ref). Railway: redeploy the previous commit on the affected service. The wire contract is stable across versions, so you can rollback one service at a time without breaking neighbours — provided you haven't bumped a shared package's version simultaneously.

Env vars — reference matrix

What each service reads. Keep secrets out of .env files that get committed.

Data auth mode

In open mode:

• Gateway skips session resolution and bearer minting.
• Orchestrator and agents accept requests without X-Rubot-Data-Bearer.
• Middleware data routes skip bearer verification.
• /connections returns all known providers as connected.
• Manager auth + PIN provisioning are unmounted entirely. /api/auth/* returns 404 auth_disabled_in_open_mode; /api/provision/* returns 404 provisioning_disabled_in_open_mode. Open mode has no tenant-ownership concept, so administering them would be meaningless.

Set across all services for consistency (same constraint as other cross-service vars).

Shared (every Python service)

Agent-level config (rubot-config)

Pattern: AGENT_<NAME>_<PARAM> (uppercase, hyphens → underscores).

Examples:

AGENT_TEMPLATE_MODEL=gpt-4o
AGENT_TEMPLATE_TEMPERATURE=0.2
AGENT_TEMPLATE_PROVIDER=anthropic
AGENT_MY_AGENT_MAX_TOKENS=8000

Standard ModelSettings fields pass through (temperature, max_tokens, top_p, timeout, seed, thinking, …). Anything else is auto-prefixed with the provider name (reasoning_effort → openai_reasoning_effort).

Provider credentials (resolved by pydantic_ai directly):

• OPENAI_API_KEY
• ANTHROPIC_API_KEY
• GROQ_API_KEY
• MISTRAL_API_KEY

rubot-gateway (Cloudflare Worker)

Bindings (in wrangler.jsonc):

• DB — D1 database rubot_data
• PROVISIONING — KV namespace
• MIDDLEWARE — Service binding to rubot-middleware

Vars (in wrangler.jsonc):

• ENVIRONMENT — dev / staging / production
• RUBOT_DEPLOYMENT_HASH — commit SHA
• ORCHESTRATOR_URL — orchestrator base URL

Secrets (wrangler secret put):

rubot-middleware (Cloudflare Worker)

Bindings:

• DB — D1 database rubot_data
• PROVISIONING — KV namespace (used by /api/provision/* PIN store in bearer mode)

Vars:

• ENVIRONMENT, RUBOT_DEPLOYMENT_HASH
• RUBOT_DATA_AUTH — bearer (default) or open. See "Data auth mode" above.
• FRONTEND_URL — base URL used to build confirmation/reset links in outbound mail. Bearer mode only.
• MAIL_FROM — From address on outbound mail. Bearer mode only.
• MAIL_BRAND_NAME — display name used inside the email body. Bearer mode only.

Secrets:

rubot-orchestrator (Python / Railway)

rubot-agent-template (and forks)

rubot-client (Cloudflare Worker — Astro)

Bindings:

• MIDDLEWARE — Service binding to rubot-middleware. Required.
• ASSETS — static asset binding (auto-generated by @astrojs/cloudflare).

Vars:

• RUBOT_DATA_AUTH — must equal bearer. When open, the dashboard renders only a "bearer mode required" banner and refuses to read data.

No secrets live on rubot-client itself — it forwards the manager's rubot_session cookie to middleware on every API call. The dashboard is documented in docs/dashboard.md.

rubot-open-client (Cloudflare Worker — Astro)

Bindings:

• MIDDLEWARE — Service binding to rubot-middleware. Required.
• ASSETS — static asset binding.

Vars:

• RUBOT_DATA_AUTH — pinned to open.

v1 surface: auth pages + pending-approval screen + a landing card. Full open-mode global admin UI is future work.

rubot-superadmin (Cloudflare Worker — Astro)

Bindings:

• MIDDLEWARE — Service binding to rubot-middleware. Required.
• ASSETS — static asset binding.

Vars: none required. Works regardless of RUBOT_DATA_AUTH on middleware — /api/admin/* is always mounted because the super-admin role lives on the managers row.

The first super-admin is bootstrapped via rubot-middleware's SUPERADMIN_EMAIL env var. See docs/superadmin.md.

Cross-service constraint summary

Mismatch → 401s (bearer mode) or mixed behaviour (auth mode). Verify after every rotation.

Local development

Run rubot on your machine. Three modes, in order of how much of the stack you spin up.

Prerequisites

• Python 3.11+ (python3 --version)
• Node 18+ + npm (for the CF Workers)
• Docker + Docker Buildx (only for the Docker-build mode)

Bootstrap (once per machine)

cd rubot
./scripts/dev-setup.sh                       # shared packages only
./scripts/dev-setup.sh rubot-agent-template  # shared packages + the template agent
./scripts/dev-setup.sh rubot-orchestrator    # or the orchestrator

What the script does:

1. Creates .venv/ with Python 3.11+.
2. pip install -e ./shared-packages/packages/rubot-logger[fastapi,dev]
3. pip install -e ./shared-packages/packages/rubot-config[dev]
4. (optional) pip install -e ./agents/<name>[dev]

After bootstrap, any edit in shared-packages/packages/* is picked up immediately — no reinstall.

source .venv/bin/activate

Mode 1 — single agent, hot-reload

For iterating on one specialist agent.

source .venv/bin/activate
cd agents/rubot-agent-template
RUBOT_DATA_AUTH=open uvicorn app.main:app --reload --port 8000

Smoke test (open mode — no bearer needed):

curl http://localhost:8000/
curl -H "X-Tenant-Id: dev" http://localhost:8000/v1/capabilities
curl -X POST http://localhost:8000/v1/chat/completions \
  -H "X-Tenant-Id: dev" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hi"}]}'

With bearer mode (RUBOT_DATA_AUTH=bearer or unset), add -H "X-Rubot-Data-Bearer: <valid-minted-bearer>".

Watch stdout for JSON envelopes. trace_id is minted by the middleware on the first request.

Mode 2 — orchestrator + one agent

For testing routing.

# terminal A: specialist
source .venv/bin/activate && cd agents/rubot-agent-template
RUBOT_DATA_AUTH=open uvicorn app.main:app --reload --port 8000

# terminal B: orchestrator
source .venv/bin/activate && cd agents/rubot-orchestrator
RUBOT_DATA_AUTH=open \
  AGENT_REGISTRY_JSON='{"template":"http://localhost:8000"}' \
  MIDDLEWARE_URL=http://localhost:8788 \
  GATEWAY_API_KEY=dev-gw-key \
  ORCHESTRATOR_API_KEY=dev-orch-key \
  uvicorn app.main:app --reload --port 8001

Hit the orchestrator (open mode — no bearer needed):

curl -X POST http://localhost:8001/v1/chat/completions \
  -H "Authorization: Bearer dev-gw-key" \
  -H "X-Tenant-Id: dev" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hello"}]}'

Inspect logs in both terminals — same trace_id appears in both.

Mode 3 — full stack with Workers

For end-to-end testing including the CF Workers.

# terminal A: middleware worker
cd workers/rubot-middleware
npm install
npx wrangler dev --port 8788

# terminal B: gateway worker
cd workers/rubot-gateway
npm install
npx wrangler dev --port 8787

# terminal C: orchestrator (port 8001)
# terminal D: specialist agent (port 8000)

Hit the gateway:

curl -X POST http://localhost:8787/v1/chat/completions \
  -H "Authorization: Bearer $GATEWAY_API_KEY" \
  -H "X-Chat-Source-Session-Id: $(uuidgen)" \
  -H "X-Chat-Source-Sender-Id: +15555550100" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"hi"}]}'

trace_id minted at the gateway propagates through middleware → orchestrator → agent → middleware again, and you should see one consistent id in all four services' logs.

Bearer-mode manager + PIN bootstrap

In bearer mode, the data-route bearer chain is driven by a sender→tenant binding written by /api/provision/consume. To set that binding up end-to-end:

# 0. one-time: schema + KV namespace
cd workers/rubot-middleware
npx wrangler d1 execute rubot_data --local --file=schema.sql

# 1. register a manager (RESEND_API_KEY empty → URL logged to stderr)
curl -X POST http://localhost:8788/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"me@example.com","password":"correcthorse"}'
# → { "success": true, "data": { "pending_confirmation": true } }
# Grab the confirmation URL from the worker log; visit it in a browser
# (or curl -L) — that flips email_confirmed=1 and sets rubot_session cookie.

# 2. log in for the session cookie
curl -c /tmp/rubot.cookies -X POST http://localhost:8788/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"me@example.com","password":"correcthorse"}'

# 3. claim a tenant (manager_id from /api/auth/me)
MANAGER_ID=$(curl -s -b /tmp/rubot.cookies http://localhost:8788/api/auth/me \
  | jq -r .data.manager_id)
TENANT_ID=demo-tenant
npx wrangler d1 execute rubot_data --local \
  --command "INSERT INTO manager_tenants (manager_id, tenant_id) VALUES ('$MANAGER_ID', '$TENANT_ID');"
npx wrangler d1 execute rubot_data --local \
  --command "INSERT INTO tenants (tenant_id, secret_hash) VALUES ('$TENANT_ID', 'unused');"

# 4. generate a PIN for that tenant
curl -b /tmp/rubot.cookies -X POST http://localhost:8788/api/provision/generate \
  -H "Content-Type: application/json" \
  -d "{\"tenant_id\":\"$TENANT_ID\"}"
# → { "success": true, "data": { "pin": "172845", "tenant_id": "demo-tenant", "expires_at": ... } }

# 5. burn the PIN as the public consume endpoint (no auth)
PIN=172845
SENDER_ID=tg:123456789
curl -X POST http://localhost:8788/api/provision/consume \
  -H "Content-Type: application/json" \
  -d "{\"pin\":\"$PIN\",\"sender_id\":\"$SENDER_ID\"}"
# → { "success": true, "data": { "linked": true, "tenant_id": "demo-tenant", "sender_id": "tg:123456789" } }

# 6. the next gateway turn that arrives with X-Chat-Source-Sender-Id: tg:123456789
#    will resolve to tenant_id=demo-tenant transparently via identity_bindings.

In open mode all of the above is skipped: /api/auth/* and /api/provision/* return 404, the gateway uses RUBOT_OPEN_TENANT as the tenant_id, and the bearer chain is bypassed end-to-end.

Mode 4 — Docker build (validate production image)

To validate exactly what the deploy host will run:

# from rubot/ root
docker build \
  -f agents/rubot-agent-template/Dockerfile \
  -t rubot-agent-template:dev \
  .

docker run --rm -p 8000:8000 \
  -e RUBOT_SERVICE_NAME=rubot-agent-template \
  -e ORCHESTRATOR_API_KEY=dev-orch-key \
  rubot-agent-template:dev

curl http://localhost:8000/

If you later extract shared-packages/ to its own private repo and switch the Dockerfile to pip install git+https, also pass a BuildKit secret:

export GITHUB_TOKEN=ghp_xxxxx
docker buildx build \
  --secret id=GITHUB_TOKEN,env=GITHUB_TOKEN \
  --build-arg SHARED_PACKAGES_REF=main \
  -f agents/rubot-agent-template/Dockerfile \
  -t rubot-agent-template:dev \
  .

Running tests

# shared packages
cd shared-packages/packages/rubot-config && pytest -v
cd shared-packages/packages/rubot-logger && pytest -v

# orchestrator
cd agents/rubot-orchestrator && pytest -v

Common problems

Observability

The scaffold ships structured logging (@rubot/logger for Python, rubotLogging() middleware for the Hono workers) but deliberately omits a log sink. Pick one of the options below per deploy.

Log envelope

Every service emits the same JSON shape per event (documented in architecture.md → Structured log envelope). The fields the dashboard needs:

Sink options

Wiring the dashboard usage page

rubot/workers/rubot-client/src/pages/dashboard/[tenantId]/usage.astro currently renders a placeholder. To back it with real data:

1. Pick a sink and stand it up.
2. Add a new middleware route, e.g. GET /api/tenant/:tenantId/usage, that queries the sink and returns { traces: [...], counts: {...} }.
3. Swap the placeholder render in usage.astro for a fetch + table.

The middleware-side ownership check (isManagerOwnerOf) stays in place — the dashboard never reads usage data directly.

What about the structured log itself in dev?

@rubot/logger and rubotLogging() both write to stdout. In npx wrangler dev, that surfaces in the worker log pane. In docker-run Python services, it surfaces on stderr. Local dev needs nothing more than a terminal.

Super-admin (rubot-superadmin)

Manages dashboard access across both rubot-client (bearer mode) and rubot-open-client (open mode). Single source of truth — the managers.is_superadmin flag, scoped to no tenant.

Account states

register
   │
   ▼
email_confirmed=0 ──confirm-email──▶ email_confirmed=1, approved=0
                                              │
                                              ▼
                                    approved=1 (super-admin click)
                                              │
                              ┌───────────────┴──────────────┐
                              ▼                              ▼
                          normal manager              is_superadmin=1

The bootstrap super-admin (env SUPERADMIN_EMAIL) skips both gates — on register the row is inserted with email_confirmed=1, approved=1, is_superadmin=1.

Bootstrap

Set in rubot-middleware's wrangler.jsonc:

"vars": {
  "SUPERADMIN_EMAIL": "you@example.com"
}

Then register on any dashboard (rubot-client, rubot-open-client, or rubot-superadmin itself) using that email. The response payload comes back as { pending_confirmation: false, bootstrapped: true } and you can sign in immediately.

If SUPERADMIN_EMAIL is unset, promote a row by hand:

wrangler d1 execute rubot_data --local --command "\
UPDATE managers SET email_confirmed=1, approved=1, is_superadmin=1 \
WHERE email='you@example.com';"

Routes (always mounted, both modes)

All gated by: requireApprovedManager → is_superadmin === 1.

Audit

Every state change writes to account_audit(manager_id, actor_id, action, reason, created_at). actor_id is NULL for the bootstrap insert (action='bootstrap').

Dashboard

Routes on rubot-superadmin:

The SuperAdmin.astro layout gates every /admin/* page on session + is_superadmin=1 && approved=1. Non-super-admins hitting the URL see a 403 banner with a link back to the operator dashboards.

Eventual log integration

The two placeholder pages (/admin/logs, /admin/agent-logs) wait on a log sink (docs/observability.md). The pattern is the same one used by tenant.usage: pick a sink, add a middleware endpoint that queries it, swap the placeholder for a fetch + table. Trace IDs are already end-to-end so the backfill is a SELECT.