Format: Plan for
plan-runnerskill execution. Each## Phase <code-name>section is one plan-runner invocation. Phases with### Verifysections ending in test commands are automatically verifiable.
Supex Radar — custom Python tool for single-host log file aggregation with a TUI (over SSH), live streaming, and agent-friendly observability. Launcher script at ./radar in the supex root, project code in ./devtools/radar/.
- In: single-host, file-based ingest, live tail streaming, TUI (Textual), agent interaction via tmux
- Out: multi-host aggregation, network ingests (OTLP/syslog/kafka), enterprise SIEM integration, dedicated agent API
Log Files (rotate/truncate safe)
|
File Watch/Tail Layer
|
Parser Layer (pipe | JSONL | plain)
|
Normalizer -> LogEvent model
|
Ring Buffer (in-memory, fixed capacity)
|
+-----------+--------------+
| |
Plain mode TUI mode
(stdout stream) (Textual, interactive)
./radar watch --plain ./radar watch
| |
Agent / pipe tmux pane
/ \
Human Agent
(eyes) (capture-pane + send-keys)
Note: TUI can run outside tmux; tmux is the recommended mode for agent control.
| Decision | Choice | Rationale |
|---|---|---|
| Language | Python 3.14+ | Rapid development, rich ecosystem, user preference |
| Package manager | uv |
Workspace convention |
| TUI framework | Textual + Rich | Mature SSH-friendly TUI, async-native |
| Storage | In-memory ring buffer | No persistence needed; debugging/observing tool only |
| File watching | Custom inode-aware tailer | Handles rotate/truncate without restart |
| Two run modes | Plain (stdout stream) and TUI (interactive) | Plain for quick observing; TUI for detailed inspection |
| Agent interaction | tmux (capture-pane + send-keys) in TUI mode | Agent uses the same TUI as human; no separate API |
| Runtime | tmux optional | Plain mode works anywhere; TUI is tmux-friendly and recommended in tmux for agent operation |
| Multi-instance | Multiple radars in parallel | Each independent; read-only file tailing, no shared state |
| Parsers | Pluggable registry | Each log source gets its own parser; easy to add new ones |
| Summary renderers | Pluggable per-source | Human+agent friendly one-liners; raw detail on demand |
Three plugin points, all following the same pattern — register by source name, fall back to defaults:
- Log sources — adding a new source = one config entry (file path + optional parser/renderer name). No code changes needed for sources that work with default parser/renderer.
- Parsers — each source can have a custom parser (
BaseParsersubclass) that extracts structured data from raw lines. Registry keyed by name. Default: auto-detect (pipe/JSONL/plain). - Summary renderers — each source can have a custom summary renderer that produces a concise one-liner from
LogEvent. The summary is what both humans and agents see in the log list. Raw detail is always available via toggle. Default:timestamp level source message. Hot-reloadable: renderer files are watched for changes and re-imported on save — no restart needed. This lets both humans and agents tweak the summary view while radar is running.
source is the logical source ID from config ([[source]].name), not a path basename.
This means: when supex adds a new component with a new log format, adding support to Radar is just writing a parser + renderer and registering them.
Radar uses TOML config files. Zero-config default covers standard supex development; config files override for tests or custom setups.
Resolution rules and precedence:
Effective workspace root is $SUPEX_WORKSPACE when set, otherwise current working directory.
watch [files...]accepts zero or more positional files.- If one or more files are provided, Radar runs in ad-hoc file mode (config/default source discovery is skipped).
- Else if
--configis provided, it is the active source list (built-in defaults andradar.tomlare not loaded). - If no ad-hoc files and no
--configare provided:- load
<workspace-root>/radar.tomlwhen present, - otherwise use built-in defaults.
- load
- Workspace root for relative paths is defined by
workspacein the active config; without config, built-ins resolve from the effective workspace root.
Default config (built-in, no file needed):
Built-in defaults below describe the target post-log-unify + log-layout-unify assumptions.
# Implicit when no config provided — resolves relative to effective workspace root
workspace = "."
[[source]]
name = "mcp-protocol"
path = ".tmp/logs/mcp-protocol.jsonl"
parser = "jsonl"
[[source]]
name = "mcp-stderr"
path = ".tmp/logs/mcp-stderr.log"
parser = "pipe"
[[source]]
name = "cli-driver"
path = ".tmp/logs/cli-driver.log"
parser = "pipe"
[[source]]
name = "cli-stdout"
path = ".tmp/logs/cli-stdout.log"
parser = "plain"
[[source]]
name = "cli-stderr"
path = ".tmp/logs/cli-stderr.log"
parser = "pipe"
[[source]]
name = "runtime-console"
path = ".tmp/logs/runtime-console.log"
parser = "pipe"
[[source]]
name = "runtime-stdout"
path = ".tmp/logs/runtime-stdout.log"
parser = "plain"
[[source]]
name = "runtime-stderr"
path = ".tmp/logs/runtime-stderr.log"
parser = "plain"
[[source]]
name = "vcad-sidecar"
path = ".tmp/logs/vcad-sidecar-stderr.log"
parser = "pipe"
[[source]]
name = "vcad-events"
path = ".tmp/logs/vcad-events.jsonl"
parser = "jsonl"Note: cli-stdout, runtime-stdout, and runtime-stderr intentionally use plain because these streams can contain mixed unstructured process output.
Test config example (tests/fixtures/radar-test.toml):
# Point radar at test-isolated log directory
workspace = "/tmp/supex-test-abc123"
[[source]]
name = "test-driver"
path = "logs/driver.log"
parser = "pipe"
[[source]]
name = "test-sidecar"
path = "logs/sidecar.log"
parser = "pipe"Usage:
# Default — watches standard supex logs
./radar watch
# Explicit config file
./radar watch --config tests/fixtures/radar-test.toml
# Non-default workspace: pass a config file that sets `workspace = "/path/to/workspace"`
./radar watch --config /tmp/radar-alt-workspace.toml
# Ad-hoc files mode (skips config/default source discovery)
./radar watch .tmp/logs/*.{log,jsonl}
# Invalid: ad-hoc files mode and --config together (must fail with usage error)
./radar watch .tmp/logs/*.{log,jsonl} --config tests/fixtures/radar-test.toml
Key design points:
workspacesets the root for relative paths in[[source]]entries--configfully overrides defaults (explicit sources only)- Non-default workspace selection is done via config file (
workspace = ...), not CLI flags poll_intervalis a global runtime setting (CLI/config), not per-source- Paths support globs (
path = ".tmp/logs/*") watch [files...]accepts shell-expanded file lists; config paths support internal glob expansion- Each
[[source]]can optionally specifyparserandrendererby name - Parser selection precedence: explicit
[[source]].parser> auto-detect (detect_format) >plainfallback radar watch [files...]and--configare mutually exclusive only when one or more positional files are providedradar watch(with zero positional files) uses config/default discovery;radar watch [files...]with files is ad-hoc mode
@dataclass
class LogEvent:
eid: str # Short event ID (e.g. "a3f2b7") for cross-reference
timestamp: datetime # Parsed or inferred
level: str # DEBUG, INFO, WARN, ERROR, FATAL
source: str # Logical source ID (from config `[[source]].name`)
source_path: str # Physical origin path for diagnostics/detail view
message: str # Human-readable summary line
structured: dict | None # Parsed key-value / JSON payload
raw: str # Original untouched line(s)
multiline: bool # Whether this spans multiple raw linessupex/
├── radar # Convenience launch script (bash, executable)
├── devtools/radar/ # Python project
│ ├── pyproject.toml
│ ├── src/radar/
│ │ ├── __init__.py
│ │ ├── models.py # LogEvent dataclass + enums
│ │ ├── tailer.py # File watch/tail with rotate detection
│ │ ├── parsers/
│ │ │ ├── __init__.py # Parser registry + base class
│ │ │ ├── pipe.py # Pipe-separated format
│ │ │ ├── jsonl.py # JSON lines format
│ │ │ └── plain.py # Plain text fallback
│ │ ├── normalizer.py # Raw parsed data -> LogEvent
│ │ ├── buffer.py # In-memory ring buffer
│ │ ├── tui/
│ │ │ ├── __init__.py
│ │ │ ├── app.py # Textual App main class
│ │ │ ├── views.py # Live list, detail panel, filter bar
│ │ │ └── renderers.py # summary_renderer, raw_renderer
│ │ └── cli.py # CLI entry point (typer)
│ └── tests/
│ ├── conftest.py
│ ├── test_models.py
│ ├── test_cli_modes.py
│ ├── test_tailer.py
│ ├── test_parsers.py
│ ├── test_buffer.py
│ └── test_renderers.py
- Project lives in
devtools/radar/(part of supex repo) - Convenience launch script
./radarin supex root (same pattern as./mcpand./supex) - Python: type hints, dataclasses, async where needed
- Tests: pytest
| Risk | Mitigation |
|---|---|
| Inconsistent timestamps across files | Fallback to receive-time; warn in TUI |
| Large JSON payloads slow rendering | Truncate in summary view; full in raw view |
| Multiline exception block mis-assembly | Indentation/pattern heuristics + configurable continuation regex |
| Ring buffer overflow loses old events | Expected behavior for debugging tool; capacity configurable |
| Polling lag under bursty load | Keep --plain stream running in parallel for lossless observation; tune polling interval and batching |
Current state uses multiple locations and is a candidate for cleanup in a dedicated log-layout phase.
Current roots in practice:
$SUPEX_WORKSPACE/.tmp/logs/for wrapper/driver-side logs<repo>/.tmp/for selected runtime launcher outputs
| File | Source | Format |
|---|---|---|
supex-mcp-protocol.jsonl |
./mcp wrapper (tee) |
JSONL (MCP request/response) |
supex-mcp-stderr.log |
./mcp wrapper (tee) |
Plain text (driver stderr) |
supex-cli-stdout.log |
./supex wrapper (tee) |
Plain text |
supex-cli-stderr.log |
./supex wrapper (tee) |
Plain text |
supex-cli.log |
Python driver CLI | Plain text (asctime - name - level - message) |
sketchup_console.log |
Ruby ConsoleCapture | Pipe-separated (timestamp|level|source|message) |
sketchup_out.txt |
launch-sketchup.sh |
Plain text (SketchUp stdout) |
sketchup_err.txt |
launch-sketchup.sh |
Plain text (SketchUp stderr) |
vcad-sidecar-stderr.log |
./vcad-sidecar wrapper |
Plain text (Rust tracing) |
Typical active session: multiple files simultaneously (exact count depends on enabled actors).
Note: ./vcad-sidecar wrapper redirects stderr from scripts/launch-vcad-sidecar.sh process into vcad-sidecar-stderr.log.
Python driver also emits structured JSON events on stderr (vcad_logging.py) with fields: timestamp, level, request_id, node_id, event, error_code.
Single root: $SUPEX_WORKSPACE/.tmp/logs/
Flat per-actor filenames under the root (no subdirectories), e.g.:
$SUPEX_WORKSPACE/.tmp/logs/mcp-protocol.jsonl$SUPEX_WORKSPACE/.tmp/logs/mcp-stderr.log$SUPEX_WORKSPACE/.tmp/logs/cli-stdout.log$SUPEX_WORKSPACE/.tmp/logs/cli-stderr.log$SUPEX_WORKSPACE/.tmp/logs/cli-driver.log$SUPEX_WORKSPACE/.tmp/logs/runtime-console.log$SUPEX_WORKSPACE/.tmp/logs/runtime-stdout.log$SUPEX_WORKSPACE/.tmp/logs/runtime-stderr.log$SUPEX_WORKSPACE/.tmp/logs/vcad-sidecar-stderr.log$SUPEX_WORKSPACE/.tmp/logs/vcad-events.jsonl
Target source-to-file map:
| Source ID | Target file | Parser |
|---|---|---|
mcp-protocol |
mcp-protocol.jsonl |
jsonl |
mcp-stderr |
mcp-stderr.log |
pipe |
cli-stdout |
cli-stdout.log |
plain |
cli-stderr |
cli-stderr.log |
pipe |
cli-driver |
cli-driver.log |
pipe |
runtime-console |
runtime-console.log |
pipe |
runtime-stdout |
runtime-stdout.log |
plain |
runtime-stderr |
runtime-stderr.log |
plain |
vcad-sidecar |
vcad-sidecar-stderr.log |
pipe |
vcad-events |
vcad-events.jsonl |
jsonl |
Note: built-in default config remains the executable source of truth; this map is a quick reference.
- tmux control (
capture-pane+send-keys) is considered proven in practice; primary remaining risk is polling cadence under burst load.
Recommended execution order:
Phase scaffoldPhase log-unifyPhase log-layout-unifyPhase tailerPhase parsersPhase bufferPhase tuiPhase render-hot-reloadPhase cli-polishPhase perf-polishPhase ssh-tmux-polish
Rationale: implementing log format/layout unification early keeps downstream ingest/parser/TUI phases aligned to a single source-of-truth log structure.
Bootstrap the radar project in devtools/radar with uv, define the LogEvent data model and enums, create a minimal CLI entry point, and add the ./radar convenience launch script in the supex root. After this phase, ./radar --help works from the supex root.
Convenience bash wrapper following the same pattern as ./mcp and ./supex:
#!/usr/bin/env bash
# Supex Radar — log aggregator TUI
set -e
SCRIPT_PATH="${BASH_SOURCE[0]}"
while [ -L "$SCRIPT_PATH" ]; do
SCRIPT_DIR="$(cd "$(dirname "$SCRIPT_PATH")" && pwd)"
SCRIPT_PATH="$(readlink "$SCRIPT_PATH")"
[[ "$SCRIPT_PATH" != /* ]] && SCRIPT_PATH="$SCRIPT_DIR/$SCRIPT_PATH"
done
SUPEX_ROOT="$(cd "$(dirname "$SCRIPT_PATH")" && pwd)"
exec uv run --project "$SUPEX_ROOT/devtools/radar" radar "$@"Mark executable (chmod +x).
[project]
name = "radar"
version = "0.1.0"
requires-python = ">=3.14"
dependencies = [
"textual>=0.80",
"rich>=13",
"typer>=0.12",
]
[project.optional-dependencies]
dev = ["pytest>=8", "pytest-asyncio>=0.23"]
[project.scripts]
radar = "radar.cli:main"Define LogEvent dataclass as specified in the overview. Add Level enum (DEBUG, INFO, WARN, ERROR, FATAL) with comparison ordering. Add FilterSpec dataclass for holding active filter state (source, level, time range, regex pattern).
Typer-based CLI with subcommands:
radar watch [files...]— main watch entrypoint (minimal scaffold behavior in this phase)radar parse <file>— minimal parse/dump command for parser pipeline smoke checks--configoption for future TOML config file- early validation: reject
watch [files...](with one or more files) combined with--config
Behavior note:
radar watchwith zero positional files runs in config/default discovery mode.radar watch [files...]with one or more files runs in ad-hoc mode and does not combine with--configsource lists.- If ad-hoc files and
--configare both provided, CLI exits with a clear usage error (non-zero) explaining the conflict.
test_models.py: LogEvent creation, Level ordering, FilterSpec defaults- verifies
sourceis logical source ID andsource_pathstores physical origin path
- verifies
test_cli_modes.py: CLI mode selection and conflict handlingradar watch(zero files) succeeds in config/default moderadar watch <file>(one or more files) enters ad-hoc moderadar watch <file> --config ...fails with usage error
# Run verify commands from repository root
# Launch script works from supex root
./radar --help
# Tests pass
cd devtools/radar && uv run pytest tests/test_models.py -v
cd devtools/radar && uv run pytest tests/test_cli_modes.py -vAdjust logging across supex subsystems to make log files easier for Radar to parse. Not a forced single format — each subsystem keeps its character — but consistent enough that parsers can be simple and reliable. Changes are in the supex repo (driver, runtime, sidecar, scripts).
| Subsystem | Format | Timestamp | Problem for Radar |
|---|---|---|---|
| Python driver | asctime - name - levelname - message |
2026-02-23 14:30:45,123 |
Non-ISO timestamp and inconsistent separator for Radar parser |
| Python vcad events | JSON embedded in log line | Unix float in JSON | Double-wrapped: log format wraps JSON string |
| Ruby ConsoleCapture | HH:MM:SS.LLL|severity|SketchUp|message |
Time only, no date | Missing date; source always "SketchUp" |
| Rust sidecar | tracing_subscriber default | ISO-8601 with tz | Good as-is, but target format differs from Python |
| Launch scripts | [ERROR]/[CONSOLE] prefix via sed |
None (from source) | ANSI color codes in output; no structured fields |
Python driver (driver/src/supex_driver/):
- Unify format to:
%(asctime)s|%(levelname)s|%(name)s|%(message)s - Set
asctimedatefmt to ISO-8601:%Y-%m-%dT%H:%M:%S.%f(or uselogging.Formatterwithdefault_msec_format) - Apply to both MCP server (stderr) and CLI (file) logging
- vcad structured events: log JSON lines directly (no outer pipe wrapper) via dedicated
vcad-eventsstream (%(message)s-only logger), targetingvcad-events.jsonlin the unified layout
Ruby ConsoleCapture (runtime/src/supex_runtime/console_capture.rb):
- Add date to timestamp:
%Y-%m-%dT%H:%M:%S.%Linstead of%H:%M:%S.%L - Use actual source where available (e.g.
"Bridge","Runtime") instead of always"SketchUp" - Keep pipe-separated format — it's already good
Rust sidecar (vcad/sidecar/src/main.rs):
- Switch tracing_subscriber to pipe-separated format:
timestamp|level|target|message - Use
tracing_subscriber::fmt().with_timer(...)for ISO-8601 timestamps - Or: use
fmt::format::Formatcustomization to output pipe-separated fields - Minimal change — just format, not the content of log messages
Launch scripts (scripts/):
launch-sketchup.sh: no required changes for Radar compatibilitylaunch-vcad-sidecar.sh: no changes needed (sidecar format handled above)
After this phase, structured supex logs use either:
- Pipe-separated:
ISO-8601-timestamp|level|source|message(Python, Ruby, Rust) - JSONL: one JSON object per line (MCP protocol, vcad structured events)
runtime-stdout and runtime-stderr are treated as plain process streams in Radar defaults.
This phase aligns runtime log formats with the built-in default config parser mapping.
Radar ships with three built-in parsers (pipe/jsonl/plain) that handle all supex sources out of the box.
# Python driver format
(cd driver && uv run python -c '
import logging, os, re, tempfile
from pathlib import Path
from supex_driver.cli.main import _setup_logging
workspace = tempfile.mkdtemp(prefix="radar-log-unify-")
os.environ["SUPEX_WORKSPACE"] = workspace
_setup_logging()
logger = logging.getLogger("radar.verify")
logger.info("format-check")
line = Path(workspace, ".tmp", "logs", "supex-cli.log").read_text().strip().splitlines()[-1]
assert "|INFO|radar.verify|format-check" in line, line
assert re.match(r"^\d{4}-\d{2}-\d{2}T", line), line
')
# Ruby console format includes date
# Uses test-only introspection of private formatter helper (not a public runtime API contract)
(cd runtime && ruby -e '
require_relative "src/supex_runtime/console_capture"
capture = SupexRuntime::ConsoleCapture.allocate
line = capture.send(:idea_log_line, "I", "verify").strip
abort("bad format: #{line}") unless line.match(/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}\|I\|[^|]+\|verify$/)
')
# Rust sidecar builds
(cd vcad/sidecar && cargo build 2>&1 | tail -1)
rg -n 'tracing_subscriber::fmt|with_timer|Format' vcad/sidecar/src/main.rs
# Launch script behavior: terminal coloring is allowed, raw log files must remain parseable
test -f scripts/launch-sketchup.shMove all supex log producers to one workspace-relative root with flat actor-prefixed filenames:
$SUPEX_WORKSPACE/.tmp/logs/[actor]-[name].(log|jsonl)
Phase log-unify completed (format unification first, path unification second).
- Canonical root:
$SUPEX_WORKSPACE/.tmp/logs/ - Canonical filenames:
mcp-protocol.jsonlmcp-stderr.logcli-stdout.logcli-stderr.logcli-driver.logruntime-console.logruntime-stdout.logruntime-stderr.logvcad-sidecar-stderr.logvcad-events.jsonl
- Extension convention:
*.jsonlfor structured JSON line streams*.logfor text/pipe streams
- Hard cutover (no backward compatibility):
- no dual-write
- no legacy path fallback
- no compatibility shims
- Update wrappers/runtime/sidecar launch paths to write only under canonical root.
- Update Radar built-in default sources to canonical flat filenames.
- New log producers must follow canonical flat filename conventions.
# Run from repo root
: "${SUPEX_WORKSPACE:=$(pwd)}"
# Warning: cleanup below is destructive for listed log files.
# Use only in a disposable dev/test workspace.
# 0) Clean previous artifacts so checks validate fresh output from this run
mkdir -p "$SUPEX_WORKSPACE/.tmp/logs"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/mcp-protocol.jsonl"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/mcp-stderr.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/cli-stdout.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/cli-stderr.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/cli-driver.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/vcad-sidecar-stderr.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/vcad-events.jsonl"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/runtime-console.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/runtime-stdout.log"
rm -f "$SUPEX_WORKSPACE/.tmp/logs/runtime-stderr.log"
rm -f "$SUPEX_WORKSPACE/.tmp/sketchup_console.log"
rm -f "$SUPEX_WORKSPACE/.tmp/sketchup_out.txt"
rm -f "$SUPEX_WORKSPACE/.tmp/sketchup_err.txt"
# 1) Spawn representative producers (commands/examples may vary by environment)
# ./mcp ...
# ./supex ...
# ./vcad-sidecar ...
# Example minimal invocations (environment-dependent):
# ./mcp <<< '{"jsonrpc":"2.0","id":1,"method":"ping"}' >/dev/null 2>&1 || true
# ./supex --help >/dev/null 2>&1 || true
# ./vcad-sidecar --help >/dev/null 2>&1 || true
# These three are the mandatory subset for PASS in this verify scenario.
# Ensure commands actually emit protocol/stdout/stderr so required files are created.
# 2) Confirm only canonical tree is used
test -d "$SUPEX_WORKSPACE/.tmp/logs"
find "$SUPEX_WORKSPACE/.tmp/logs" -maxdepth 1 -type f | head
! find "$SUPEX_WORKSPACE/.tmp/logs" -mindepth 1 -type d | grep -q .
# Additional files are allowed (future producers), but layout must stay flat
# and mandatory canonical subset below must exist.
# 2b) Mandatory canonical files for the spawned producer subset (PASS criteria)
test -f "$SUPEX_WORKSPACE/.tmp/logs/mcp-protocol.jsonl"
test -f "$SUPEX_WORKSPACE/.tmp/logs/mcp-stderr.log"
test -f "$SUPEX_WORKSPACE/.tmp/logs/cli-stdout.log"
test -f "$SUPEX_WORKSPACE/.tmp/logs/cli-stderr.log"
test -f "$SUPEX_WORKSPACE/.tmp/logs/cli-driver.log"
test -f "$SUPEX_WORKSPACE/.tmp/logs/vcad-sidecar-stderr.log"
# 2c) Optional runtime/structured files (environment-dependent; informational only)
test -f "$SUPEX_WORKSPACE/.tmp/logs/runtime-console.log" || true
test -f "$SUPEX_WORKSPACE/.tmp/logs/runtime-stdout.log" || true
test -f "$SUPEX_WORKSPACE/.tmp/logs/runtime-stderr.log" || true
test -f "$SUPEX_WORKSPACE/.tmp/logs/vcad-events.jsonl" || true
# 3) Ensure no new radar-tracked logs are written outside canonical root
! test -f "$SUPEX_WORKSPACE/.tmp/sketchup_console.log"
! test -f "$SUPEX_WORKSPACE/.tmp/sketchup_out.txt"
! test -f "$SUPEX_WORKSPACE/.tmp/sketchup_err.txt"Build the file watch/tail layer that reliably streams new lines from multiple log files, handling inode changes on rotate and file truncation without data loss or crash.
class FileTailer:
"""Watches one file for new content. Handles rotate and truncate."""
def __init__(self, path: Path, poll_interval: float = 0.2):
self.path = path
self.offset: int = 0
self.inode: int | None = None
async def read_new_lines(self) -> list[str]:
"""Return new lines since last read. Detect rotate/truncate."""
def _detect_rotate(self) -> bool:
"""Inode changed or file smaller than offset -> rotated."""
def _detect_truncate(self) -> bool:
"""Same inode but size < offset -> truncated (e.g. logrotate copytruncate)."""
@dataclass
class TailSource:
source: str # logical source ID from config
source_path: Path
class MultiTailer:
"""Manages multiple FileTailers, yields (source, line, source_path) tuples."""
def __init__(self, sources: list[TailSource], poll_interval: float = 0.2): ...
async def stream(self) -> AsyncIterator[tuple[str, str, str]]:
"""Infinite async generator of (source_id, raw_line, source_path)."""Key behaviors:
- Accepts source descriptors with logical source ID + path (not raw paths only)
TailSourceentries are produced by config resolution after glob expansion, sorting, deduplication, and path validation- Track file by inode (not just path) — detect rotation when inode changes
- On rotate: re-open path, reset offset to 0, read from start of new file
- On truncate: reset offset to 0, continue reading
- Poll-based with globally configurable interval (default 200ms)
- Graceful handling of missing files (warn, retry on next poll)
- Multiline assembly is NOT done here — tailer emits raw lines only
test_tailer.py:- Basic tail: append lines, verify they appear
- Rotate: rename file + create new, verify new lines are read
- Truncate: write, truncate, write again — verify no crash, new lines appear
- Missing file: verify warning, no crash
- Multiple files: verify interleaved output
cd devtools/radar && uv run pytest tests/test_tailer.py -vImplement pluggable parsers for pipe-separated, JSONL, and plain text formats, plus a normalizer that converts parsed output into LogEvent instances. Support multiline continuation (stacktraces).
class BaseParser(ABC):
@abstractmethod
def parse_line(self, raw: str) -> ParsedRecord | None:
"""Parse a single line. Return None if not recognized."""
def is_continuation(self, line: str) -> bool:
"""Return True if line is a continuation of previous record (multiline)."""
return False
@dataclass
class ParsedRecord:
timestamp: datetime | None
level: str | None
message: str
structured: dict | None
def detect_format(sample_lines: list[str]) -> BaseParser:
"""Auto-detect log format from first N lines."""Pipe-separated format: timestamp|level|source|message|optional_json_payload
This is the canonical order for unified pipe logs; emitters should follow it consistently.
JSON lines: each line is a JSON object. message is optional.
- Preferred fields:
timestamp,level,message. - For protocol/event payloads without
message, parser derives a summary from known keys (method,event,error,result) and preserves full payload instructured. - Malformed JSON records are routed to plain-parser handling by ingest/normalizer fallback logic.
Fallback: attempt to extract timestamp and level from common patterns (syslog, ISO-8601 prefix, etc.). Rest is message. Multiline continuation: lines starting with whitespace or common stacktrace patterns ( at ..., Caused by:, Traceback).
Input normalization should tolerate ANSI escape sequences (e.g. by stripping ANSI before parsing), while preserving original content in raw when needed.
class Normalizer:
def __init__(self, parser: BaseParser, source: str): ...
def feed_line(self, raw: str) -> LogEvent | None:
"""Feed a raw line. Returns LogEvent when a complete record is ready.
Buffers continuation lines internally."""
def flush(self) -> LogEvent | None:
"""Flush any buffered multiline record."""eid generation:
- Generated by
Normalizerwhen finalizing aLogEvent. - Mandatory algorithm: first 6 lowercase hex chars of
sha256(source + timestamp_iso + first_raw_line). - No alternative algorithm is allowed in v1 (deterministic cross-reference contract).
eidis for cross-reference/search convenience (not a globally unique ID); rare collisions are acceptable.
Multiline strategy:
Normalizerholds a buffer of continuation lines- When a new non-continuation line arrives, flush the buffer as one
LogEvent(withmultiline=True), start new buffer rawfield contains all original lines joined with\nmessagefield contains the first line's parsed message
test_parsers.py:- Pipe parser: standard line, missing fields, extra payload
- JSONL parser: valid JSON, malformed JSON (fallback to plain)
- Plain parser: syslog format, ISO-8601, no timestamp
- ANSI handling: lines containing ANSI escape sequences are parsed correctly
- Auto-detection: pipe vs JSONL vs plain
- Multiline: Java stacktrace, Python traceback
- Normalizer: feed sequence of lines, verify correct LogEvent assembly
- EID: present, stable for same input, changes when source/timestamp/first_raw_line changes
cd devtools/radar && uv run pytest tests/test_parsers.py -vImplement in-memory ring buffer for live event storage and the ingest pipeline that connects tailer -> normalizer -> buffer. No persistence — radar is a debugging/observing tool.
class RingBuffer:
"""Fixed-size in-memory buffer of LogEvents."""
def __init__(self, capacity: int = 10_000): ...
def append(self, event: LogEvent) -> None: ...
def get_recent(self, n: int) -> list[LogEvent]: ...
def filter(self, spec: FilterSpec) -> list[LogEvent]: ...
def search(self, pattern: str) -> list[LogEvent]:
"""Regex/substring search over message + raw fields."""
@property
def count(self) -> int: ...
@property
def sources(self) -> set[str]: ...When buffer is full, oldest events are silently evicted (standard ring buffer behavior). Capacity is configurable via CLI flag.
Pipeline orchestrator that connects tailer -> normalizer -> buffer:
class IngestPipeline:
"""Connects MultiTailer -> Normalizer -> RingBuffer."""
def __init__(self, tailer: MultiTailer, buffer: RingBuffer): ...
async def run(self) -> None:
"""Main ingest loop. Runs until cancelled."""Auto-detects parser per source file (via detect_format from parsers module). Creates one Normalizer per source.
Parser and renderer selection is keyed by logical source (config name), not by file basename.
MultiTailer is constructed from resolved source config entries (source + source_path).
Parser selection behavior in ingest:
- If source config specifies
parser, use it (no auto-detect). - Else run
detect_formaton source samples. - If detect fails or parser errors for all samples, fallback to
plainparser and emit a warning withsourceandsource_path.
Glob handling in ingest/config loading:
- Expand glob paths before tailer construction.
- Sort expanded paths lexicographically for deterministic startup order.
- Deduplicate expanded paths.
- Keep logical
sourceID from config; each matched file gets its ownsource_path.
test_buffer.py:- RingBuffer: append, overflow/eviction, get_recent, filter, search
- IngestPipeline: end-to-end from file lines to buffered events
- source filtering uses logical source ID (
[[source]].name), independent of file basename - parser selection precedence: explicit parser override, auto-detect path, and plain fallback path
- glob expansion:
path = ".tmp/logs/*"expands deterministically and ingests all matched files
cd devtools/radar && uv run pytest tests/test_buffer.py -vBuild the Textual-based TUI application with live log streaming, filter controls (source, level, time, regex), detail panel, and summary/raw view toggle. Must work over SSH; tmux is recommended for agent operation.
class RadarApp(App):
"""Main Textual application."""
BINDINGS = [
("q", "quit", "Quit"),
("f", "toggle_filter", "Filter"),
("tab", "toggle_view", "Summary/Raw"),
("/", "search", "Search"),
("j", "scroll_down", "Down"),
("k", "scroll_up", "Up"),
("g", "scroll_top", "Top"),
("G", "scroll_bottom", "Bottom"),
]Components:
LogListView: scrollable list of log events (virtual list for performance)- Each row shows: timestamp | level (colored) | source | message (truncated)
- Selected row highlights; press Enter to open detail
DetailPanel: expanded view of single event- Toggle between summary (rendered) and raw (original text)
FilterBar: input fields for source, level dropdown, time range, regex- Live filtering (debounced, ~200ms)
StatusBar: total events, filtered count, active sources, stream rate
Pluggable summary renderers — each source can register a custom one-liner renderer. Summary is what both humans and agents see in the list view. Raw is always available via detail toggle.
class SummaryRenderer(ABC):
"""Base class for source-specific summary renderers."""
@abstractmethod
def render(self, event: LogEvent) -> RenderableType: ...
class DefaultSummaryRenderer(SummaryRenderer):
"""Default: timestamp | level (colored) | source | message (truncated)."""
class MCPProtocolRenderer(SummaryRenderer):
"""MCP JSONL: method name, tool, direction (req/res), duration."""
class ConsoleLogRenderer(SummaryRenderer):
"""Pipe-separated SketchUp console: level + source + message."""
# Registry: source name -> renderer instance
RENDERERS: dict[str, SummaryRenderer] = {}
def raw_renderer(event: LogEvent) -> RenderableType:
"""Original raw text with syntax highlighting. No truncation.
Always available via detail panel toggle — not pluggable."""
Renderer lookup uses `event.source` (logical source ID from config).Design constraints:
- Virtual scrolling for large datasets (don't load all events into DOM)
- Level color convention: DEBUG=dim, INFO=blue, WARN=yellow, ERROR=red, FATAL=red bold — applied by default, but summary renderers own their coloring (can highlight individual words, paths, durations, etc.)
- SSH-safe: no mouse-only interactions, all actions have keyboard shortcuts
- Responsive: works in 80x24 minimum terminal size
Streaming mode (default):
- New events appear at bottom, view auto-scrolls to follow
- No cursor visible in the list
- Filter bar still works — only matching events are shown
Browsing mode (activated by pressing j/k or arrow keys):
- Streaming pauses — view stays fixed so the selected row doesn't escape
- Cursor visible, moves with
j/k/arrows Enteropens detail panel for selected event (summary/raw toggle viaTab); cursor navigation (j/k) and filtering still work with panel open — detail updates to follow selection- New events still arrive into the buffer but the view doesn't scroll
Escexits browsing mode: cursor disappears, view jumps to end of list, streaming resumes
TUI receives events via async queue from IngestPipeline. RingBuffer serves both the live view and filtered/search results.
When Radar runs in tmux, both humans and agents interact with the same TUI pane:
Agent reads the screen:
tmux capture-pane -t radar -p # plain text snapshot of current viewAgent sends keystrokes:
tmux send-keys -t radar 'f' # open filter bar
tmux send-keys -t radar 'ERROR' Enter # type filter, confirm
tmux send-keys -t radar 'q' # quitAgent workflow examples:
Quick observing (plain mode):
- Agent runs
./radar watch --plain— summary lines stream to stdout - Agent pipes/reads stdout, spots interesting event
- Each line includes a short event ID (e.g.
[e:a3f2b7]) for cross-reference
Detailed inspection (TUI mode):
- Agent runs
./radar watch --pane-name radar-agentin a tmux pane - Uses
tmux capture-pane/tmux send-keysto interact - To find an event seen in plain mode: press
/and search by event ID (a3f2b7) or timestamp - Cursor jumps to matching event,
Enteropens detail
Recommended operational pattern (agent-safe):
- Keep
./radar watch --plainrunning in a background pane/window as a lossless stream. - Use
./radar watch --pane-name radar-agentin a second pane for interactive filtering and inspection. - If TUI polling falls behind during bursts, plain mode still preserves visibility of all incoming lines.
Event IDs for cross-reference:
- Every
LogEventgets a short ID (e.g. first 6 hex chars ofsha256(source+timestamp_iso+first_raw_line)) - Shown in both plain mode output and TUI list view
- Searchable in TUI via
/— enables quick jump from plain mode observation to TUI detail
TUI design for agent-friendliness:
- Status bar with consistent format: total count, filtered count, active sources, rate
- Level tags always in same column position (easy to regex from capture-pane output)
- Timestamps in ISO-8601 (machine-parseable)
- No animations or transient decorations that interfere with capture-pane reads
- Keybindings are single characters (easy to send via
send-keys)
./radar launcher:
--plainflag: stream summary lines to stdout (no TUI, no tmux needed)--pane-name NAMEflag: customize tmux pane name (default:radar)- If TUI mode inside tmux: optionally rename pane
- If TUI mode outside tmux: run directly in the current terminal (no auto-launch behavior in v1)
- Multiple instances supported: each runs independently with its own ring buffer and file tailers
Manual verification — launch TUI with sample log files and verify:
- Live streaming works
- Filter by level/source/regex works
- Summary/raw toggle works
- Keyboard navigation works
tmux capture-pane -t radar -preturns parseable texttmux send-keys -t radarcontrols the TUI correctly--plainand TUI can run in parallel over the same files without crash; both continue receiving new events
# Smoke test: app imports and basic widget construction
cd devtools/radar && uv run python -c "from radar.tui.app import RadarApp; print('OK')"Implement hot-reload for summary renderer modules so renderer output updates on file save without restarting Radar.
Phase tui completed.
- Watches renderer module files used by source renderer registry.
- On file change, reloads the module and swaps renderer instance in registry.
- Applies to summary renderers only (raw renderer remains non-pluggable).
- Successful reload updates subsequent rows/detail summaries immediately.
- Reload failures (syntax/import/runtime during init) do not crash app:
- Keep last known-good renderer for affected source.
- Emit warning in status/log stream with source + error summary.
- After fixing file and saving again, reload retries and recovers automatically.
RendererReloaderservice in TUI layer:- tracks watched files and mtime/hash
- performs debounced reload
- updates renderer registry atomically
RendererLoadResultfor structured success/error reporting
test_renderers.py:- reload success path: file update changes rendered output without restart
- syntax error path: keeps last known-good renderer and records warning
- recovery path: after fix, renderer reloads and new output is used
Manual verification:
- Run Radar TUI.
- Edit a renderer file while app is running; verify output changes without restart.
- Introduce a syntax error; verify app keeps running with warning and old renderer behavior.
- Fix syntax error; verify renderer recovers after next save.
cd devtools/radar && uv run python -c "from radar.tui.renderers import RENDERERS; print(type(RENDERERS).__name__)"
cd devtools/radar && uv run python -c "import radar.tui.renderers as r; print(bool(getattr(r, 'RENDERERS', None)))"Polish CLI ergonomics for day-to-day use: file selection, pre-set filters, and capacity tuning.
radar watch .tmp/logs/*— ad-hoc glob expansion across*.logand*.jsonl- Supports both shell-expanded CLI files and config-based glob paths (no duplicate source entries)
radar watch --level ERROR --source cli-driver— pre-set filters--sourceexpects logical source IDs ([[source]].name), not filenames/paths- Pre-set filters apply in both TUI mode and
--plainmode --capacity Nflag for ring buffer size- Improve help text/examples for common startup patterns (
--plain,--config)
cd devtools/radar && uv run radar --help
cd devtools/radar && uv run radar watch --helpTune TUI responsiveness under high event volume and large in-memory buffers.
Phase tui completed.
- Profile with large log corpus (100k+ events)
- Virtual scrolling in TUI: only render visible rows
- Debounce filter input (200ms)
- Lazy loading for detail panel
- Record measured outcomes for stream/filter responsiveness
Manual verification:
- Run with large corpus (100k+ events)
- Verify scroll responsiveness remains usable
- Verify filter interactions remain responsive under load
- Record benchmark command lines used and observed timings in phase notes/output
- Include at least one ingest/startup timing and one interactive filter timing
cd devtools/radar && uv run python -c "print('perf-polish smoke')"Finalize SSH/tmux usability and ensure robust agent interaction in terminal environments.
Phase tui completed.
- Verify all keybindings work in tmux (no conflicts with tmux prefix)
- Test with common terminal emulators (iTerm2, Terminal.app, alacritty)
- Ensure 256-color fallback (detect truecolor support, degrade gracefully)
- Mouse scroll support where available (but never required)
- Verify
tmux capture-pane -t radar -pproduces clean, parseable output - No ANSI artifacts in captured text (Textual must render cleanly for capture)
- Pane naming works correctly (
--pane-name) - No tmux auto-launch in v1; tmux usage remains explicit by operator choice
Manual verification:
- Run in tmux over SSH with real log files
- Verify agent workflow: capture-pane -> send-keys -> capture-pane
- Verify all keybindings
cd devtools/radar && uv run radar --help