Skip to content

Instantly share code, notes, and snippets.

@andrewsanchez
Last active December 4, 2025 17:33
Show Gist options
  • Select an option

  • Save andrewsanchez/b6644c0d210078b14495e168e956d428 to your computer and use it in GitHub Desktop.

Select an option

Save andrewsanchez/b6644c0d210078b14495e168e956d428 to your computer and use it in GitHub Desktop.
Example of using the mgrep CLI tool with an llm tool
➜ llm --td --cl 20 --functions '
import subprocess
def mgrep(pattern: str, path: str = ".") -> str:
    """Call mgrep for semantic search across files under the given path."""
    result = subprocess.run(["mgrep", "search", pattern, path], capture_output=True, text=True, timeout=30)
    return result.stdout if result.returncode == 0 else result.stderr
  ' \
-s "Use mgrep extensively to search the current working directory to answer the user's question. mgrep is powerful semantic search tool." \
"Write a plugin that defines a toolbox for using the mgrep search cli tool.
Provide a README.md with instructions for installing the plugin locally."


Tool call: mgrep({'pattern': 'mgrep plugin toolbox README.md'})
  ./docs/python-api.md:216-275 (91.73% match)
  ./tests/test_plugins.py:510-552 (75.49% match)
  ./tests/test_plugins.py:607-639 (66.54% match)
  ./tests/test_plugins.py:790-853 (61.51% match)
  ./tests/test_plugins.py:639-695 (59.27% match)
  ./docs/changelog.md:24-36 (43.78% match)
  ./tests/test_plugins.py:696-743 (26.89% match)
  ./docs/plugins/directory.md:39-58 (21.21% match)
  ./llm/migrations.py:378-421 (20.18% match)
  ./docs/changelog.md:650-661 (4.47% match)



Tool call: mgrep({'pattern': 'toolbox plugin mgrep'})
  ./tests/test_plugins.py:790-853 (83.97% match)
  ./tests/test_plugins.py:924-971 (81.76% match)
  ./docs/python-api.md:216-275 (80.81% match)
  ./tests/test_plugins.py:510-552 (69.26% match)
  ./tests/test_plugins.py:607-639 (65.14% match)
  ./tests/test_plugins.py:876-924 (56.99% match)
  ./tests/test_plugins.py:639-695 (44.55% match)
  ./tests/test_plugins.py:696-743 (30.74% match)
  ./llm/migrations.py:378-421 (11.28% match)
  ./docs/changelog.md:650-661 (6.75% match)



Tool call: mgrep({'pattern': 'README plugin toolbox'})
  ./docs/usage.md:159-190 (93.81% match)
  ./docs/python-api.md:216-275 (92.41% match)
  ./llm/cli.py:2633-2670 (91.49% match)
  ./tests/test_plugins.py:510-552 (88.08% match)
  ./docs/plugins/plugin-hooks.md:77-117 (86.70% match)
  ./tests/test_plugins.py:607-639 (85.59% match)
  ./llm/__init__.py:187-215 (84.39% match)
  ./docs/changelog.md:24-36 (79.31% match)
  ./tests/test_plugins.py:696-743 (73.72% match)
  ./llm/migrations.py:378-421 (52.34% match)

Below is a minimal, working example of a plugin that defines a toolbox for using the mgrep CLI search tool, plus a README with local installation instructions.

Repository layout:

- mgrep_toolbox/
  - pyproject.toml
  - mgrep_toolbox/
    - __init__.py
    - plugin.py
  - README.md
  1. pyproject.toml Use a standard Python packaging setup so the plugin can be installed locally and discovered via entry points.
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "mgrep-toolbox"
version = "0.1.0"
description = "A plugin that defines a toolbox for using the mgrep search CLI tool."
readme = "README.md"
requires-python = ">=3.8"
license = {text = "MIT"}
authors = [
  { name = "Your Name", email = "you@example.com" }
]
dependencies = []

[project.entry-points."llm.plugins"]
mgrep_toolbox = "mgrep_toolbox.plugin:register"

[tool.setuptools.packages.find]
where = ["."]
include = ["mgrep_toolbox*"]
  1. mgrep_toolbox/init.py Empty or simple metadata.

__all__ = ["plugin"]

  1. mgrep_toolbox/plugin.py This file implements the plugin registration and defines the toolbox with one or more tools that wrap mgrep. The example below exposes:
  • mgrep_search: run mgrep with a semantic pattern across a given path
  • mgrep_search_glob: run mgrep with a semantic pattern limited by file globs
  • mgrep_search_files: run mgrep against an explicit set of files

It uses subprocess to call the mgrep binary, returning JSON where possible.

import json
import os
import shlex
import subprocess
from typing import List, Optional, Dict, Any

def _run_mgrep(args: List[str]) -> Dict[str, Any]:
    """
    Run mgrep with given arguments. Attempts to parse JSON if --json is present.
    Falls back to plain text output.
    """
    cmd = ["mgrep"] + args
    try:
        proc = subprocess.run(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            text=True,
            check=False,
        )
    except FileNotFoundError as e:
        return {
            "ok": False,
            "error": "mgrep executable not found. Ensure it is installed and on PATH.",
            "exception": str(e),
        }

    out = proc.stdout
    err = proc.stderr
    rc = proc.returncode

    # Try JSON parse first if likely JSON flag was used
    parsed = None
    if "--json" in args:
        try:
            parsed = json.loads(out) if out.strip() else []
        except json.JSONDecodeError:
            parsed = None

    return {
        "ok": rc == 0,
        "returncode": rc,
        "stdout": out,
        "stderr": err,
        "json": parsed,
        "command": " ".join(shlex.quote(a) for a in cmd),
    }

def mgrep_search(pattern: str, path: Optional[str] = None, json_output: bool = True, extra_args: Optional[List[str]] = None) -> Dict[str, Any]:
    """
    Search using mgrep across path (default: current directory) with a semantic pattern.
    - pattern: the semantic search query
    - path: directory to search (default ".")
    - json_output: include --json to return structured results
    - extra_args: additional raw flags for mgrep (advanced)
    """
    args = []
    if json_output:
        args.append("--json")
    if extra_args:
        args.extend(extra_args)
    args.extend(["--pattern", pattern])
    args.append(path or ".")
    return _run_mgrep(args)

def mgrep_search_glob(pattern: str, path: Optional[str] = None, include: Optional[List[str]] = None, exclude: Optional[List[str]] = None, json_output: bool = True, extra_args: Optional[List[str]] = None) -> Dict[str, Any]:
    """
    Search with include/exclude globs.
    - include: list of glob patterns to include (e.g., ["**/*.py"])
    - exclude: list of glob patterns to exclude (e.g., ["**/node_modules/**"])
    """
    args = []
    if json_output:
        args.append("--json")
    if include:
        for g in include:
            args.extend(["--include", g])
    if exclude:
        for g in exclude:
            args.extend(["--exclude", g])
    if extra_args:
        args.extend(extra_args)
    args.extend(["--pattern", pattern])
    args.append(path or ".")
    return _run_mgrep(args)

def mgrep_search_files(pattern: str, files: List[str], json_output: bool = True, extra_args: Optional[List[str]] = None) -> Dict[str, Any]:
    """
    Search an explicit list of files.
    """
    args = []
    if json_output:
        args.append("--json")
    if extra_args:
        args.extend(extra_args)
    args.extend(["--pattern", pattern])
    args.extend(files)
    return _run_mgrep(args)

def register(registry):
    """
    Register the mgrep toolbox with the host application.
    The registry should expose `toolbox` registration compatible with the host.
    """
    toolbox = registry.toolboxes.register(
        name="mgrep",
        description="Tools for running the mgrep semantic search CLI."
    )

    @toolbox.tool(
        name="mgrep_search",
        description="Run mgrep semantic search across a directory. Returns JSON when --json is enabled.",
        schema={
            "type": "object",
            "properties": {
                "pattern": {"type": "string", "description": "Semantic search query."},
                "path": {"type": "string", "description": "Directory to search. Defaults to '.'", "nullable": True},
                "json_output": {"type": "boolean", "description": "Return JSON output if supported.", "default": True},
                "extra_args": {
                    "type": "array",
                    "items": {"type": "string"},
                    "description": "Additional flags passed directly to mgrep.",
                    "nullable": True
                },
            },
            "required": ["pattern"]
        }
    )
    def _tool_mgrep_search(pattern: str, path: Optional[str] = None, json_output: bool = True, extra_args: Optional[List[str]] = None):
        return mgrep_search(pattern=pattern, path=path, json_output=json_output, extra_args=extra_args)

    @toolbox.tool(
        name="mgrep_search_glob",
        description="Run mgrep search with include/exclude glob patterns.",
        schema={
            "type": "object",
            "properties": {
                "pattern": {"type": "string"},
                "path": {"type": "string", "nullable": True},
                "include": {"type": "array", "items": {"type": "string"}, "nullable": True},
                "exclude": {"type": "array", "items": {"type": "string"}, "nullable": True},
                "json_output": {"type": "boolean", "default": True},
                "extra_args": {"type": "array", "items": {"type": "string"}, "nullable": True}
            },
            "required": ["pattern"]
        }
    )
    def _tool_mgrep_search_glob(pattern: str, path: Optional[str] = None, include: Optional[List[str]] = None, exclude: Optional[List[str]] = None, json_output: bool = True, extra_args: Optional[List[str]] = None):
        return mgrep_search_glob(pattern=pattern, path=path, include=include, exclude=exclude, json_output=json_output, extra_args=extra_args)

    @toolbox.tool(
        name="mgrep_search_files",
        description="Run mgrep search against an explicit list of files.",
        schema={
            "type": "object",
            "properties": {
                "pattern": {"type": "string"},
                "files": {"type": "array", "items": {"type": "string"}},
                "json_output": {"type": "boolean", "default": True},
                "extra_args": {"type": "array", "items": {"type": "string"}, "nullable": True}
            },
            "required": ["pattern", "files"]
        }
    )
    def _tool_mgrep_search_files(pattern: str, files: List[str], json_output: bool = True, extra_args: Optional[List[str]] = None):
        return mgrep_search_files(pattern=pattern, files=files, json_output=json_output, extra_args=extra_args)

    return registry
  1. README.md

mgrep-toolbox

A plugin that defines a toolbox for using the mgrep semantic search CLI tool.

Features:

  • mgrep_search: search a directory with a semantic query
  • mgrep_search_glob: search a directory with include/exclude glob patterns
  • mgrep_search_files: search a specific list of files

Prerequisites:

  • Python 3.8+
  • The mgrep CLI installed and available on PATH
    • Install mgrep according to its official instructions. Verify with:
      • mgrep --help

Local installation:

  1. Clone or copy this repository locally: git clone https://example.com/mgrep-toolbox.git cd mgrep-toolbox

  2. Install in editable/development mode: python -m pip install -e .

  3. Verify the plugin is discoverable by your host application:

    • If your host framework supports listing plugins (e.g., llm plugins), run its plugin list command.
    • Ensure you see an entry similar to: mgrep_toolbox

Usage in code (example): from mgrep_toolbox.plugin import mgrep_search, mgrep_search_glob, mgrep_search_files

Search the current directory

result = mgrep_search(pattern="vector database api") if result["ok"] and result["json"] is not None: print("Matches (JSON):", result["json"]) else: print("Output:", result["stdout"]) print("Errors:", result["stderr"])

Search with globs

result2 = mgrep_search_glob( pattern="retry logic", include=["/*.py"], exclude=["/tests/**"], ) print(result2["command"], result2["returncode"])

Search explicit files

result3 = mgrep_search_files( pattern="OAuth flow", files=["README.md", "src/app.py"], ) print(result3["json"] or result3["stdout"])

Toolbox registration: If your host supports a plugin registry with toolboxes, this package exposes an entry point:

  • group: llm.plugins
  • name: mgrep_toolbox
  • function: mgrep_toolbox.plugin:register

The register(registry) function defines a toolbox named "mgrep" with three tools:

  • mgrep_search
  • mgrep_search_glob
  • mgrep_search_files

Each tool accepts a JSON-serializable payload and returns a dictionary including:

  • ok: boolean
  • returncode: process return code
  • command: full command executed
  • stdout, stderr: raw text output
  • json: parsed JSON output when --json is used and parseable

Troubleshooting:

  • Command not found: Ensure mgrep is installed and on PATH.
  • Empty results: Confirm your pattern is specific enough and you are pointing to the right directory.
  • JSON parse errors: If mgrep doesn’t support JSON output in your version, set json_output=False or remove --json via extra_args.

Uninstall: python -m pip uninstall mgrep-toolbox

License: MIT

Notes:

  • This is a minimal example. You can extend schemas, add timeouts, or stream output as needed.
  • If your host uses a different plugin discovery mechanism, adjust the project.entry-points in pyproject.toml accordingly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment