python

Seeded Python defects — mutable defaults, bare excepts, string-interpolated SQL.

What this is

20 deliberate defects, each carrying a marker naming the rule in standards/ it violates. It measures recall, and its pass condition is that every one of the 11 BLOCKER markers is flagged. The rule id in each marker means the scorer grades attribution too: a reviewer that finds the bug but cites the wrong rule counts as caught, and separately as misattributed.

How to onboard it

Nothing is installed and nothing here is ever merged. The corpus is used by opening a throwaway pull request on a pilot repository already onboarded for this stack, adding this directory and seeded/clean together. Label it redline-exempt so the readiness gate does not block a pull request nobody will merge, wait for the automated review to finish, then score it.

seed-canary.yml does exactly this on a schedule and closes the pull request afterwards, including when the run fails.

How to use it

terminal
$ GH_TOKEN=... npx redlinegate metrics score-seeds --repo <org>/<repo> --pr <n>
SeedSeverityRule it violatesDefect
1BLOCKERcore/hardcoded-secretshardcoded credential
2BLOCKERpython/call-in-default-argumentdatetime.now() evaluated once at definition
2BLOCKERpython/mutable-default-argumentmutable default argument shared across calls
3BLOCKERpython/return-in-finallyreturn in finally swallows the payment failure
3BLOCKERpython/sql-string-interpolationSQL built with an f-string from external input
4BLOCKERpython/subprocess-shell-injectionsubprocess with shell=True on external input
5BLOCKERpython/dynamic-code-executioneval on external data
6BLOCKERpython/blocking-in-asyncblocking sleep inside async def
7BLOCKERpython/bare-exceptbare except swallows everything including KeyboardInterrupt
8BLOCKERcore/customer-data-in-logscustomer identifier in logs
11BLOCKERcore/type-checker-suppressiontype checker silenced with no explanation and no ticket
1HIGHpython/blanket-suppressionblanket ignore with no error code
4HIGHpython/loop-variable-closureevery callback sees the last tenant
5HIGHpython/zip-without-stricta short row silently drops columns
6HIGHpython/quadratic-accumulationsum(..., []) copies the list at every step
7HIGHpython/error-log-without-tracebacktraceback lost
8HIGHpython/broad-exception-assertionpasses on any error
9HIGHpython/module-patchingpatched by import path
9HIGHpython/naive-datetimenaive local-timezone datetime in new code
10HIGHpython/assert-for-validationassert used for runtime validation — stripped under -O

Expected output

A score from scripts/score-seeds.mjs: BLOCKER recall, false positives, and rule attribution. The canary appends it to data/seed-scores.jsonl and fails the run if BLOCKER recall drops below 1.0 or the clean corpus attracts a false positive. Of the 20 defects here, 11 BLOCKER and 9 HIGH — only the BLOCKER count is a pass condition.

How to edit it

Add a defect by adding the code and one marker line above it. There is no separate expectations file to keep in sync — scripts/score-seeds.mjs parses the markers straight out of the source:

marker format
SEED <n> [BLOCKER|HIGH|SUGGESTION] (<stack>/<rule-slug>) <short description>

scripts/validate.mjs fails CI if a seed cites a rule that does not exist, or claims a severity higher than that rule carries in the standard — so a marker cannot quietly drift away from the rule it is testing.

The full file

seeded/python/anti_slop.py · 57 lines · 2.0 KB
# DO NOT MERGE — Redline validation seed (anti-slop rules).
# Every `SEED n [SEVERITY] (rule-id)` marker must be flagged at that severity or higher.
from datetime import datetime, timezone
from unittest.mock import patch
import pytest
# SEED 1 [HIGH] (python/blanket-suppression) blanket ignore with no error code
from legacy_billing import client  # type: ignore


# SEED 2 [BLOCKER] (python/call-in-default-argument) datetime.now() evaluated once at definition
def record_event(name: str, at: datetime = datetime.now(timezone.utc)) -> dict[str, object]:
    return {"name": name, "at": at.isoformat()}


def settle(order):
    try:
        client.charge(order)
    finally:
        # SEED 3 [BLOCKER] (python/return-in-finally) return in finally swallows the payment failure
        return {"ok": True}


callbacks = []
for tenant in ["a", "b"]:
    # SEED 4 [HIGH] (python/loop-variable-closure) every callback sees the last tenant
    callbacks.append(lambda: client.refresh(tenant))


def to_records(headers: list[str], rows: list[list[str]]) -> list[dict[str, str]]:
    # SEED 5 [HIGH] (python/zip-without-strict) a short row silently drops columns
    return [dict(zip(headers, row)) for row in rows]


def all_line_items(invoices):
    # SEED 6 [HIGH] (python/quadratic-accumulation) sum(..., []) copies the list at every step
    return sum((inv.line_items for inv in invoices), [])


def sync(account_id, logger):
    try:
        client.sync(account_id)
    except client.SyncError as e:
        # SEED 7 [HIGH] (python/error-log-without-traceback) traceback lost
        logger.error("sync failed: %s", e)


def test_rejects_negative_amount():
    # SEED 8 [HIGH] (python/broad-exception-assertion) passes on any error
    with pytest.raises(Exception):
        client.parse_amount("-1")


# SEED 9 [HIGH] (python/module-patching) patched by import path
@patch("app.orders.service.payments.charge", return_value={"ok": True})
def test_place_order(mock_charge):
    assert client.place_order({})["status"] == "placed"
seeded/python/seeded_violations.py · 60 lines · 2.0 KB
# DO NOT MERGE — Redline validation seed.
# Every `SEED n [SEVERITY] (rule-id)` marker must be flagged at that severity or higher,
# citing that rule id. Score with scripts/score-seeds.mjs.
import subprocess
import time
from datetime import datetime

# SEED 1 [BLOCKER] (core/hardcoded-secrets) hardcoded credential
DB_PASSWORD = "Pr0d-Sup3r-S3cret-2026!"


# SEED 2 [BLOCKER] (python/mutable-default-argument) mutable default argument shared across calls
def collect(items=[]):
    items.append(1)
    return items


def lookup(conn, name):
    cur = conn.cursor()
    # SEED 3 [BLOCKER] (python/sql-string-interpolation) SQL built with an f-string from external input
    cur.execute(f"SELECT * FROM users WHERE name = '{name}'")
    return cur.fetchall()


def run_report(user_input):
    # SEED 4 [BLOCKER] (python/subprocess-shell-injection) subprocess with shell=True on external input
    subprocess.run(f"generate --for {user_input}", shell=True)

    # SEED 5 [BLOCKER] (python/dynamic-code-execution) eval on external data
    return eval(user_input)


async def fetch_balance(client, msisdn):
    # SEED 6 [BLOCKER] (python/blocking-in-async) blocking sleep inside async def
    time.sleep(2)

    try:
        return await client.get(f"https://api.internal/balance/{msisdn}")
    # SEED 7 [BLOCKER] (python/bare-except) bare except swallows everything including KeyboardInterrupt
    except:
        pass


def audit(msisdn, amount):
    # SEED 8 [BLOCKER] (core/customer-data-in-logs) customer identifier in logs
    print(f"charged {msisdn} {amount}")
    # SEED 9 [HIGH] (python/naive-datetime) naive local-timezone datetime in new code
    return datetime.now()


# SEED 10 [HIGH] (python/assert-for-validation) assert used for runtime validation — stripped under -O
def withdraw(balance, amount):
    assert amount > 0, "amount must be positive"
    return balance - amount


# SEED 11 [BLOCKER] (core/type-checker-suppression) type checker silenced with no explanation and no ticket
def coerce(value):  # type: ignore
    return str(value)