harupy

Proposal: Preventing Unsolicited AI-Generated PRs

Authors: @harupy, Claude

Problem

We're getting flooded with unsolicited PRs from AI coding agents. Someone sees an open issue, prompts their agent with "hey fix mlflow/mlflow#123", and a low quality PR appears minutes later with no prior discussion, no context, and no maintainer approval. This wastes maintainer time and CI resources.

Hypothesis: How AI Slop PRs Are Filed

harupy

GitHub Copilot Code Review: Comment Classification Analysis

Analysis of the Copilot code review (copilot-pull-request-reviewer) dynamic workflow on mlflow/mlflow.

• Source: CCR Agent: Comment stored log lines from the Run Autofind Agent step in the Agent job
• Sample: 270 comments from 100 completed workflow runs
• Date: 2026-02-20

Severity

harupy

Real-World Copilot Code Review Instructions Examples

A curated list of .github/instructions/code-review.instructions.md and similar files found across public GitHub repositories.

Highlights

Highly Customized (Project-Specific Rules)

• dragonflydb/dragonfly — C++ in-memory datastore. Severity tiers (critical/important/suggestion), project-specific anti-patterns with code examples (e.g. "NEVER use std::mutex"), false positive suppression section, example good vs bad comments.
• kumahq/kuma — CNCF service mesh (Go). CoVe (Chain of Verification) methodology — mandatory 3-step verify/answer/self-critique process. Confidence thresholds (95%+ block, 80%+ change, 70%+ comment, <70% skip). Kuma-specific checks for xDS, KDS, policy patterns.

harupy

Copilot Code Review: Common Bad Patterns

Analysis of 1,007 Copilot review comments on mlflow/mlflow. Roughly 1 in 5 comments (21%) fall into a bad pattern.

Pattern	%	Count
Generic test coverage nag	6.0%	59
Duplicate/repeated comments	4.8%	47
Pedantic style/grammar nitpicks	2.4%	24
Accessibility boilerplate	2.0%	20

harupy

2025 年を振り返る

Coding Agent の登場によって、この 1 年（正確には 5 月以降）でソフトウェアエンジニアとしての働き方が大きく変化したと感じたので、ざっくりと振り返ってみる。

自身でコードを書くことがほとんど無くなった

Claude Code を使い始めてから、実装という工程が「Claude Code に必要なコンテキスト、制約、ゴールを与えて自走させ、適宜フィードバックを与える」という形に変化した。実装する機能・修正の規模/複雑さによって、必要なコンテキスト、制約、ゴールの粒度は変化するが、いずれにせよ、自分でコードを直接書くことがほとんど無くなり、実装にかかる時間が大幅に短縮された。一方で、実装に至るまでの設計、チーム間での合意形成、コードレビューといった工程には人間が大きく関与していて、ボトルネックのままであり、エンジニアリング全体の生産性が劇的に向上したわけではない。

実装以外のユースケース

harupy

# /// script
# requires-python = "==3.10"
# dependencies = [
#   "pydantic",
#   "openai",
# ]
# ///
# ruff: noqa: T201
"""
How to run (https://gist.github.com/harupy/bc281c07d1a2d0678f97cf519aa81131)

harupy

import ast

from pathlib import Path

import subprocess
from typing import List, Dict
from dataclasses import dataclass


def line_no_to_offset(lines: List[str]) -> Dict[int, int]:

harupy

services:
  mysql:
    image: mysql
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: root-password
      MYSQL_DATABASE: mlflowdb
      MYSQL_USER: mlflowuser
      MYSQL_PASSWORD: mlflowpassword
    command: mysqld --default-authentication-plugin=mysql_native_password

harupy

from __future__ import annotations

import ast
import os
import random
import subprocess
import textwrap

import openai

harupy

b"""
{
    "id": "chatcmpl-7lG2usbCnpQIBmid2fGZjsT9Wymh4",
    "object": "chat.completion.chunk",
    "created": 1691497452,
    "model": "gpt-3.5-turbo-0613",
    "choices": [
        {
            "index": 0,
            "delta": {"content": " set"},