Guide: Creating isolated feature branch environments (Modal + Neon) for the agents server

isolated-modal-env-guide.md

Guide: Creating isolated feature branch environments (Modal + Neon) for the agents server

Creating an Isolated Feature Branch Environment (Modal + Neon)

Complete guide for deploying an isolated copy of the agents server stack for feature testing without touching dev/prod.

What Gets Created

Component	Naming Convention	Example
Neon DB branch	<branch-name>	jacob/environment-options
Modal secret	<feat-name>-env	feat-env-options-env
Modal API app	<feat-name>-agent	feat-env-options-agent
Modal sandbox app	<feat-name>-sandbox	feat-env-options-sandbox

Prerequisites

• neonctl CLI authenticated (neonctl auth)
• modal CLI authenticated (modal token set)
• Python 3.11 available (for sandbox deploy — PYENV_VERSION=3.11.14)
• Working directory: agents/ repo root

Step-by-Step

1. Create (or reuse) a Neon DB branch

# Create a new branch from development
neonctl branches create --project-id dark-union-03995279 \
  --name "jacob/my-feature" --parent development

# Get the connection string
neonctl connection-string --project-id dark-union-03995279 \
  --branch "jacob/my-feature"

2. Run migrations on the feature DB

DATABASE_URL="<connection-string-from-step-1>" \
  python -m alembic upgrade head

Run from src/db/ or ensure alembic.ini is findable.

3. Create a Modal secret

source .env  # Load API keys from local env

FEAT=feat-my-feature  # Choose a short prefix

modal secret create ${FEAT}-env \
  DATABASE_URL="<connection-string-from-step-1>" \
  ENV="development" \
  ANTHROPIC_API_KEY="$ANTHROPIC_API_KEY" \
  OPENAI_API_KEY="$OPENAI_API_KEY" \
  BROWSERSTACK_USERNAME="$BROWSERSTACK_USERNAME" \
  BROWSERSTACK_ACCESS_KEY="$BROWSERSTACK_ACCESS_KEY" \
  BROWSERSTACK_APP_URL="$BROWSERSTACK_APP_URL" \
  BROWSERSTACK_IOS_APP_URL="$BROWSERSTACK_IOS_APP_URL" \
  AGENT_RUNNER_MODE="modal_sandbox_shared" \
  AGENT_APP_NAME="${FEAT}-sandbox"

4. Create feature deploy files

src/agency/api/modal_deploy_feature.py — API server:

"""Feature branch Modal deployment for the Agent API."""
from agency.api.modal_deploy import modal_image
import modal

FEATURE_NAME = "feat-my-feature"  # Must match secret prefix

feature_app = modal.App(f"{FEATURE_NAME}-agent")

@feature_app.function(
    image=modal_image,
    min_containers=0,
    max_containers=3,
    timeout=300,
    env={
        "ENV": "development",
        "AGENT_RUNNER_MODE": "modal_sandbox_shared",
        "AGENT_APP_NAME": f"{FEATURE_NAME}-sandbox",
    },
    secrets=[
        modal.Secret.from_name("aws-secret"),
        modal.Secret.from_name("cloudflare-bot-auth"),
        modal.Secret.from_name(f"{FEATURE_NAME}-env"),
    ],
)
@modal.asgi_app()
def api():
    from agency.api.app import app as fastapi_app
    return fastapi_app

@feature_app.function(image=modal_image)
@modal.fastapi_endpoint()
def health():
    return {"status": "healthy", "branch": "my-feature"}

src/agi_agents/framework/deploy_feature.py — Agent sandbox:

"""Feature branch agent sandbox deployment."""
import os
import modal

FEATURE_NAME = os.environ.get("FEATURE_NAME", "feat-my-feature")
FEATURE_SECRET = os.environ.get("FEATURE_SECRET", f"{FEATURE_NAME}-env")

os.environ.setdefault("AGENT_APP_NAME", f"{FEATURE_NAME}-sandbox")

import agi_agents.framework.modal as _modal_cfg

_modal_cfg.AGENT_APP_NAME = os.environ["AGENT_APP_NAME"]
_modal_cfg.AGENT_APP = modal.App(_modal_cfg.AGENT_APP_NAME)
_modal_cfg.modal_secrets = [modal.Secret.from_name(FEATURE_SECRET)]

from agi_agents.framework.deploy import *  # noqa

AGENT_APP = _modal_cfg.AGENT_APP
__all__ = ["AGENT_APP"]

5. Deploy

# Deploy API server
modal deploy src/agency/api/modal_deploy_feature.py::feature_app

# Deploy agent sandbox (requires Python 3.11 for serialized functions)
PYENV_VERSION=3.11.14 FEATURE_NAME=feat-my-feature \
  python -m modal deploy src/agi_agents/framework/deploy_feature.py::AGENT_APP

6. Prepare a test user

The branched DB inherits users from development, but they likely have limited scopes. You need to:

1. Broaden a test user's scopes to cover the environment types/agents you're testing
2. Get their user API key (not the master API key — that only works for admin endpoints like /users, not /sessions)

# Connect to the feature DB
psql "<connection-string-from-step-1>"

# Broaden scopes for a test user
UPDATE users SET scopes = '{"allowed_agents":["agi-0","agi-0-claude","agi-0-qwen","agi-1-qwen","agi-2-claude","agi-2-qwen","android-1-claude","ios-1-claude"],"allowed_environment_types":["chrome-1","ubuntu-1","android-browserstack","iphone-browserstack"]}'
WHERE id = '<user-id>';

# Get a user API key (UUID format — this is what you pass as Bearer token)
SELECT u.id, u.email, k.api_key
FROM users u JOIN user_api_keys k ON k.user_id = u.id
WHERE u.verified = true LIMIT 5;

7. Test

Your API is live at:

https://agi-inc--<feat-name>-agent-api.modal.run

# Health check
curl https://agi-inc--feat-my-feature-agent-api.modal.run/health

# List environment types
curl https://agi-inc--feat-my-feature-agent-api.modal.run/v1/environments/types \
  -H "Authorization: Bearer <user-api-key>"

# Create a Chrome session
curl -X POST https://agi-inc--feat-my-feature-agent-api.modal.run/v1/sessions \
  -H "Authorization: Bearer <user-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"goal":"Go to example.com","environment_type":"chrome-1"}'

# Create an iPhone BrowserStack session (takes ~160s to provision)
curl --max-time 300 -X POST https://agi-inc--feat-my-feature-agent-api.modal.run/v1/sessions \
  -H "Authorization: Bearer <user-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"goal":"Open Safari","environment_type":"iphone-browserstack","agent_name":"ios-1-claude"}'

# Check session events (SSE stream)
curl https://agi-inc--feat-my-feature-agent-api.modal.run/v1/sessions/<session-id>/events \
  -H "Authorization: Bearer <user-api-key>"

8. Point webapp at feature API (optional)

In webapp/ui/.env.local:

NEXT_PUBLIC_MULTION_API_URL="https://agi-inc--feat-my-feature-agent-api.modal.run"

9. Cleanup

# Delete Modal apps
modal app stop feat-my-feature-agent
modal app stop feat-my-feature-sandbox

# Delete Modal secret
modal secret delete feat-my-feature-env

# Delete Neon branch
neonctl branches delete --project-id dark-union-03995279 \
  --branch "jacob/my-feature"

Architecture

Feature API (feat-xxx-agent)
  ├── ASGI app (FastAPI)
  ├── Uses feat-xxx-env secret (feature DB, API keys)
  ├── Sets AGENT_APP_NAME=feat-xxx-sandbox
  └── Spawns agent sandboxes in feat-xxx-sandbox app

Feature Sandbox (feat-xxx-sandbox)
  ├── All agent functions (agi-0, android-1, ios-1, etc.)
  ├── Uses feat-xxx-env secret (same feature DB)
  └── Completely isolated from agency-sandbox-dev/prod

Feature DB (Neon branch)
  ├── Branched from development
  ├── Own migrations, own data
  └── No impact on dev/prod databases

Key Env Vars

Variable	Purpose	Set By
AGENT_APP_NAME	Which Modal app to spawn sandboxes in	API function env={}
AGENT_RUNNER_MODE	How agents run (modal_sandbox_shared)	Secret + env
DATABASE_URL	Feature branch Neon connection	Secret
ENV	Environment label (development)	Secret
FEATURE_NAME	Used by deploy_feature.py	Shell env var

Gotchas

1. Python 3.11 required for sandbox deploy (serialized=True functions must match image Python version)
2. BrowserStack sessions take ~160s to provision iOS devices — use --max-time 300 with curl
3. Modal mount caching: If min_containers>0, old containers may serve stale code. Use min_containers=0 for feature deploys
4. Chrome/Ubuntu environments use separate Modal apps (chrome-1-environment-dev, etc.) and are shared across all features — no need to deploy them per-feature
5. Auth: use user API keys, not master key — The master API key (150c122a-...) only works for admin endpoints (/users). Session endpoints (/v1/sessions) require a user API key from the user_api_keys table. The token format is a UUID passed as Authorization: Bearer <uuid>.
6. Feature DB user scopes are stale — Users inherited from the development DB may have limited allowed_environment_types (e.g., only chrome-1). Update scopes before testing mobile environments.
7. managed-iphone-browserstack in SERVER_DRIVEN_SESSION_TYPES — If the events/messages/screenshot endpoints return "operation_not_supported" for iPhone sessions, ensure this session type is in the SERVER_DRIVEN_SESSION_TYPES frozenset in src/agency/api/routes/sessions.py.