terrhorn/plan.md

## plan.md

      
    Raw
  

              plan.md
            
          
    /research Slash Command Implementation Spec

Issue: https://github.com/github/blackbird/issues/13257
Original Experiment: https://github.com/github/blackbird/issues/13037
Research Agent Prompt: https://github.com/github/blackbird/blob/dev/.github/agents/research.agent.md
Overview

Add a /research slash command to the Copilot CLI that enables deep research capability using GitHub's code search and repository exploration tools. The command orchestrates iterative research by injecting a hidden prompt that guides the main agent to call a research sub-agent repeatedly until high-confidence results are achieved.
Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        User Input                               │
│                    /research <topic>                            │
└─────────────────────┬───────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Slash Command Handler                        │
│              (src/cli/commands/slashCommands.ts)                │
│                                                                 │
│  Returns: { kind: "start-research", topic: "<topic>" }          │
└─────────────────────┬───────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                   Research Workflow Handler                     │
│                     (src/cli/app.tsx or hook)                   │
│                                                                 │
│  1. Show minimal timeline entry: "Researching: <topic>"         │
│  2. Inject hidden orchestration prompt to LLM                   │
│  3. Let main agent loop handle the research                     │
└─────────────────────┬───────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                      Main Agent Loop                            │
│                                                                 │
│  Orchestration prompt instructs agent to:                       │
│  1. Call task tool with agent_type: "research"                  │
│  2. Evaluate results for confidence                             │
│  3. Call again with refined queries if needed                   │
│  4. Synthesize final comprehensive answer                       │
└─────────────────────┬───────────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Research Agent (YAML)                        │
│              (src/agents/definitions/research.agent.yaml)       │
│                                                                 │
│  - Specialized system prompt for research methodology           │
│  - GitHub MCP tools for code/repo search                        │
│  - Returns comprehensive, well-cited research reports           │
└─────────────────────────────────────────────────────────────────┘

Implementation Tasks

Phase 1: Research Agent Definition


 1.1 Create src/agents/definitions/research.agent.yaml

Copy research methodology prompt from blackbird repo
Configure tools: GitHub search, file contents, issues, PRs, discussions
Set model: claude-sonnet-4.5 (high-quality reasoning needed)
Configure promptParts for research context


 1.2 Register research agent in src/agents/customAgents/builtinAgents.ts

Add to EXPANDED_BUILTIN_AGENTS (feature-flag gated)
Add to EXPANDED_YAML_AGENT_NAMES
Add metadata: { name: "research", description: "...", hasSideEffects: false }


 1.3 Add feature flag in src/agents/customAgents/index.ts

RESEARCH_AGENT_FF_NAME = "COPILOT_SWE_AGENT_RESEARCH"
isResearchEnabled(settings) function


Phase 2: Slash Command Result Type


 2.1 Add StartResearchResult type in src/cli/commands/slashCommands.ts
type StartResearchResult = {
    kind: "start-research";
    topic: string;
};


 2.2 Add to SlashCommandResult union type


 2.3 Handle in useSlashCommands.ts switch statement

Call slashCommandResultActions.startResearch(result.topic)


Phase 3: Slash Command Implementation


 3.1 Create /research command in slashCommands.ts
export const researchCommand: SlashCommand = {
    name: "/research",
    args: "<topic>",
    help: "Deep research on a topic using GitHub code search",
    execute: async (deps: SlashCommandDeps, args: string[]) => {
        if (!isResearchEnabled(deps.settings)) {
            return { kind: "add-timeline-entry", entry: { type: "error", text: "Research feature not enabled" } };
        }
        if (args.length < 1) {
            return { kind: "add-timeline-entry", entry: { type: "error", text: "Usage: /research <topic>" } };
        }
        return { kind: "start-research", topic: args.join(" ") };
    },
};


 3.2 Add to builtInSlashCommands array (conditionally based on feature flag)


 3.3 Add startResearch to SlashCommandResultActions type in useSlashCommands.ts


Phase 4: Research Workflow Handler


 4.1 Add startResearch handler in src/cli/app.tsx

Similar pattern to startRemoteDelegation
Inject orchestration prompt into session


 4.2 Create orchestration prompt template
const RESEARCH_ORCHESTRATION_PROMPT = `
You are conducting deep research on the following topic: {{topic}}

Use the task tool with agent_type: "research" to investigate this topic.
The research agent has access to GitHub's code search and repository tools.

Strategy:
1. Start with broad searches to understand the scope
2. Follow interesting leads with more specific queries
3. Cross-reference findings across multiple sources
4. Continue until you have high confidence in your findings

When confident, synthesize a comprehensive answer with:
- Executive summary
- Detailed findings with citations
- Code examples where relevant
- Confidence assessment

Begin your research now.
`;


 4.3 Implement prompt injection mechanism

Use session's message queue or userMessageProcessor
Ensure prompt is sent to LLM but minimally displayed


Phase 5: Timeline Entry & Display


 5.1 Add research timeline entry type in src/cli/types.ts
type ResearchTimelineEntry = {
    type: "research";
    topic: string;
    status: "started" | "in-progress" | "completed";
};


 5.2 Create ResearchTimelineEntry component

Show brief "Researching: " status
Optionally show progress indicators


 5.3 Add to timelineDisplay.tsx rendering


Phase 6: Testing


 6.1 Unit tests for slash command

Command parsing
Feature flag gating
Error handling


 6.2 Unit tests for research agent

Agent loads correctly
Tools are filtered properly


 6.3 Integration tests

End-to-end research workflow
Prompt injection works
Results are synthesized


File Changes Summary


File
Changes


src/agents/definitions/research.agent.yaml
NEW - Research agent definition


src/agents/customAgents/builtinAgents.ts
Add research to EXPANDED agents


src/agents/customAgents/index.ts
Add feature flag exports


src/cli/commands/slashCommands.ts
Add /research command + result type


src/cli/hooks/useSlashCommands.ts
Handle start-research result


src/cli/app.tsx
Add startResearch workflow handler


src/cli/types.ts
Add research timeline entry type


src/cli/components/researchTimelineEntry.tsx
NEW - Display component


src/cli/components/timelineDisplay.tsx
Render research entries


test/cli/commands/research.test.ts
NEW - Unit tests


Research Agent YAML Definition (Draft)

name: research
displayName: Research Agent
description: >
  Deep research agent that provides comprehensive, well-cited answers about
  codebases, APIs, libraries, and software architecture using GitHub's search tools.
model: claude-sonnet-4.5
tools:
  - github-mcp-server-get_me
  - github-mcp-server-get_file_contents
  - github-mcp-server-search_code
  - github-mcp-server-search_repositories
  - github-mcp-server-list_branches
  - github-mcp-server-list_commits
  - github-mcp-server-get_commit
  - github-mcp-server-search_issues
  - github-mcp-server-list_issues
  - github-mcp-server-issue_read
  - github-mcp-server-search_pull_requests
  - github-mcp-server-list_pull_requests
  - github-mcp-server-pull_request_read
  - task
promptParts:
  includeAISafety: true
  includeToolInstructions: true
  includeParallelToolCalling: true
  includeCustomAgentInstructions: false
prompt: |
  You are a staff software engineer and software research specialist. Your purpose
  is to provide exhaustive, meticulously researched answers about codebases, APIs,
  libraries, and software architecture.

  ## Core Research Methodology

  1. **Exhaustive Search**: Never settle for the first result. When researching:
     - Search using multiple query variations (exact names, partial matches, related concepts)
     - Use OR logic when possible to search for multiple terms at once
     - Explore both direct matches and contextual usage patterns
     - Look for tests, examples, and documentation alongside implementation code

  2. **Multi-Source Verification**: Cross-reference findings across:
     - Source code implementations
     - Test files (often contain usage examples and edge cases)
     - Documentation and comments
     - Commit history for evolution and rationale
     - Issues and pull requests for context on design decisions

  3. **Hierarchical Understanding**: Build understanding from multiple levels:
     - High-level architecture and module organization
     - Interface definitions and public APIs
     - Implementation details and algorithms
     - Edge cases and error handling

  ## Response Requirements

  ### Citations Are Mandatory
  Every claim must be backed by a specific reference:
  - **File paths**: Always include the full path (e.g., `src/agents/prompt_manager.rs:45-67`)
  - **Commit references**: When discussing changes or history, include commit SHAs
  - **Line numbers**: Be specific about where code exists
  - **Repository context**: Include owner/repo when referencing GitHub content

  ### Structure Your Responses
  1. **Executive Summary**: 3-5 sentence overview of findings
  2. **Architecture/System Overview**: High-level understanding with citations
  3. **Detailed Analysis**: Thorough exploration with citations
  4. **Code Examples**: Relevant snippets with file locations
  5. **Related Components**: Connected code that provides context
  6. **Confidence Assessment**: Note any gaps in available information
  7. **Footnotes**: Section with all footnote definitions

  ### Depth Over Brevity
  - Prefer comprehensive answers over quick summaries
  - Include relevant code snippets inline with explanations
  - Explain the "why" behind implementation choices when discoverable
  - Connect findings to broader architectural patterns

  ## Search Strategy

  When investigating a topic:
  1. **Start broad**: Search for the main concept to understand scope
  2. **Follow dependencies**: Trace imports, calls, and type references
  3. **Check tests**: Find test files for usage patterns and edge cases
  4. **Review history**: Look at commits touching relevant files for context
  5. **Explore documentation**: Check README, docs folders, and inline comments

  ## Quality Standards

  - **Accuracy**: Only state what you can verify in the code
  - **Completeness**: Cover all relevant aspects, not just the obvious
  - **Clarity**: Explain complex concepts with examples
  - **Traceability**: Every finding should be verifiable by the user

  ## Handling Uncertainty

  When information is incomplete:
  - Clearly state what is known vs. inferred
  - Suggest additional searches that might help
  - Note when code patterns suggest intent but don't confirm it
  - Never fabricate code paths or implementations
Orchestration Prompt (Draft)

export function buildResearchOrchestrationPrompt(topic: string): string {
  return `
<research_task>
The user has requested deep research on the following topic:

**${topic}**

## Your Mission

Conduct thorough research using the research agent to produce a comprehensive, well-cited answer.

## Strategy

1. **Initial Investigation**
   - Use the task tool with \`agent_type: "research"\` to begin exploring the topic
   - Start with broad queries to understand scope and find key repositories/files

2. **Iterative Deepening**
   - Based on initial findings, identify areas that need more investigation
   - Call the research agent again with more specific, targeted queries
   - Follow interesting leads - implementations, tests, documentation, related code

3. **Cross-Reference & Verify**
   - Use multiple searches to verify important findings
   - Look for tests that demonstrate expected behavior
   - Check commit history for context on why things work the way they do

4. **Confidence Assessment**
   - Continue researching until you have high confidence in your findings
   - If you find conflicting information, investigate further to resolve
   - Note any gaps or areas of uncertainty in your final response

5. **Synthesis**
   - Once confident, synthesize all findings into a comprehensive answer
   - Include citations for every claim (file paths, line numbers, commit SHAs)
   - Structure the response with executive summary, details, and examples

## Important Notes

- You may call the research agent multiple times with different queries
- Each research call should have a specific focus or question
- The research agent has access to GitHub search - leverage it fully
- Quality and accuracy matter more than speed

Begin your research now. Call the task tool with agent_type: "research".
</research_task>
`;
}
Feature Flag Configuration

// src/agents/customAgents/index.ts

/** Feature flag name for enabling the research agent */
export const RESEARCH_AGENT_FF_NAME = "COPILOT_SWE_AGENT_RESEARCH";

/**
 * Checks if the research agent is enabled.
 */
export function isResearchEnabled(settings: RuntimeSettings): boolean {
  return isFeatureFlagEnabled(settings, RESEARCH_AGENT_FF_NAME);
}
Open Questions


Parallel vs Sequential Research Calls: Should the orchestration prompt encourage parallel research calls (multiple task invocations at once) or sequential (one at a time)?

Recommendation: Start with sequential for simplicity; can optimize later


Max Iterations: Should there be a hard limit on how many times the research agent can be called?

Recommendation: Let LLM decide, but mention "reasonable iterations" in prompt


Repository Scope: Should research be scoped to current repo, specific orgs, or all of GitHub?

Recommendation: Start with all GitHub (matching original experiment); can add scope params later


Result Caching: Should research results be cached for similar queries?

Recommendation: Out of scope for v1


Success Criteria


/research <topic> command works when feature flag is enabled
Research agent produces well-cited, comprehensive answers
Orchestration prompt successfully guides iterative research
Timeline shows minimal, non-intrusive progress indicators
All tests pass

References


Original Experiment Issue
Implementation Issue
Architecture Gist
Research Agent Prompt
File	Changes
`src/agents/definitions/research.agent.yaml`	NEW - Research agent definition
`src/agents/customAgents/builtinAgents.ts`	Add research to EXPANDED agents
`src/agents/customAgents/index.ts`	Add feature flag exports
`src/cli/commands/slashCommands.ts`	Add `/research` command + result type
`src/cli/hooks/useSlashCommands.ts`	Handle `start-research` result
`src/cli/app.tsx`	Add `startResearch` workflow handler
`src/cli/types.ts`	Add research timeline entry type
`src/cli/components/researchTimelineEntry.tsx`	NEW - Display component
`src/cli/components/timelineDisplay.tsx`	Render research entries
`test/cli/commands/research.test.ts`	NEW - Unit tests
No results found