python
Seeded Python defects — mutable defaults, bare excepts, string-interpolated SQL.
What this is
20 deliberate defects, each carrying a marker naming the rule in standards/ it violates. It measures recall, and its pass condition is that every one of the 11 BLOCKER markers is flagged. The rule id in each marker means the scorer grades attribution too: a reviewer that finds the bug but cites the wrong rule counts as caught, and separately as misattributed.
How to onboard it
Nothing is installed and nothing here is ever merged. The corpus is used by opening a throwaway pull request on a pilot repository already onboarded for this stack, adding this directory and seeded/clean together. Label it redline-exempt so the readiness gate does not block a pull request nobody will merge, wait for the automated review to finish, then score it.
seed-canary.yml does exactly this on a schedule and closes the pull request afterwards, including when the run fails.
How to use it
$ GH_TOKEN=... npx redlinegate metrics score-seeds --repo <org>/<repo> --pr <n>
| Seed | Severity | Rule it violates | Defect |
|---|---|---|---|
| 1 | BLOCKER | core/hardcoded-secrets | hardcoded credential |
| 2 | BLOCKER | python/call-in-default-argument | datetime.now() evaluated once at definition |
| 2 | BLOCKER | python/mutable-default-argument | mutable default argument shared across calls |
| 3 | BLOCKER | python/return-in-finally | return in finally swallows the payment failure |
| 3 | BLOCKER | python/sql-string-interpolation | SQL built with an f-string from external input |
| 4 | BLOCKER | python/subprocess-shell-injection | subprocess with shell=True on external input |
| 5 | BLOCKER | python/dynamic-code-execution | eval on external data |
| 6 | BLOCKER | python/blocking-in-async | blocking sleep inside async def |
| 7 | BLOCKER | python/bare-except | bare except swallows everything including KeyboardInterrupt |
| 8 | BLOCKER | core/customer-data-in-logs | customer identifier in logs |
| 11 | BLOCKER | core/type-checker-suppression | type checker silenced with no explanation and no ticket |
| 1 | HIGH | python/blanket-suppression | blanket ignore with no error code |
| 4 | HIGH | python/loop-variable-closure | every callback sees the last tenant |
| 5 | HIGH | python/zip-without-strict | a short row silently drops columns |
| 6 | HIGH | python/quadratic-accumulation | sum(..., []) copies the list at every step |
| 7 | HIGH | python/error-log-without-traceback | traceback lost |
| 8 | HIGH | python/broad-exception-assertion | passes on any error |
| 9 | HIGH | python/module-patching | patched by import path |
| 9 | HIGH | python/naive-datetime | naive local-timezone datetime in new code |
| 10 | HIGH | python/assert-for-validation | assert used for runtime validation — stripped under -O |
Expected output
A score from scripts/score-seeds.mjs: BLOCKER recall, false positives, and rule attribution. The canary appends it to data/seed-scores.jsonl and fails the run if BLOCKER recall drops below 1.0 or the clean corpus attracts a false positive. Of the 20 defects here, 11 BLOCKER and 9 HIGH — only the BLOCKER count is a pass condition.
How to edit it
Add a defect by adding the code and one marker line above it. There is no separate expectations file to keep in sync — scripts/score-seeds.mjs parses the markers straight out of the source:
SEED <n> [BLOCKER|HIGH|SUGGESTION] (<stack>/<rule-slug>) <short description>scripts/validate.mjs fails CI if a seed cites a rule that does not exist, or claims a severity higher than that rule carries in the standard — so a marker cannot quietly drift away from the rule it is testing.
The full file
# DO NOT MERGE — Redline validation seed (anti-slop rules).
# Every `SEED n [SEVERITY] (rule-id)` marker must be flagged at that severity or higher.
from datetime import datetime, timezone
from unittest.mock import patch
import pytest
# SEED 1 [HIGH] (python/blanket-suppression) blanket ignore with no error code
from legacy_billing import client # type: ignore
# SEED 2 [BLOCKER] (python/call-in-default-argument) datetime.now() evaluated once at definition
def record_event(name: str, at: datetime = datetime.now(timezone.utc)) -> dict[str, object]:
return {"name": name, "at": at.isoformat()}
def settle(order):
try:
client.charge(order)
finally:
# SEED 3 [BLOCKER] (python/return-in-finally) return in finally swallows the payment failure
return {"ok": True}
callbacks = []
for tenant in ["a", "b"]:
# SEED 4 [HIGH] (python/loop-variable-closure) every callback sees the last tenant
callbacks.append(lambda: client.refresh(tenant))
def to_records(headers: list[str], rows: list[list[str]]) -> list[dict[str, str]]:
# SEED 5 [HIGH] (python/zip-without-strict) a short row silently drops columns
return [dict(zip(headers, row)) for row in rows]
def all_line_items(invoices):
# SEED 6 [HIGH] (python/quadratic-accumulation) sum(..., []) copies the list at every step
return sum((inv.line_items for inv in invoices), [])
def sync(account_id, logger):
try:
client.sync(account_id)
except client.SyncError as e:
# SEED 7 [HIGH] (python/error-log-without-traceback) traceback lost
logger.error("sync failed: %s", e)
def test_rejects_negative_amount():
# SEED 8 [HIGH] (python/broad-exception-assertion) passes on any error
with pytest.raises(Exception):
client.parse_amount("-1")
# SEED 9 [HIGH] (python/module-patching) patched by import path
@patch("app.orders.service.payments.charge", return_value={"ok": True})
def test_place_order(mock_charge):
assert client.place_order({})["status"] == "placed"
# DO NOT MERGE — Redline validation seed.
# Every `SEED n [SEVERITY] (rule-id)` marker must be flagged at that severity or higher,
# citing that rule id. Score with scripts/score-seeds.mjs.
import subprocess
import time
from datetime import datetime
# SEED 1 [BLOCKER] (core/hardcoded-secrets) hardcoded credential
DB_PASSWORD = "Pr0d-Sup3r-S3cret-2026!"
# SEED 2 [BLOCKER] (python/mutable-default-argument) mutable default argument shared across calls
def collect(items=[]):
items.append(1)
return items
def lookup(conn, name):
cur = conn.cursor()
# SEED 3 [BLOCKER] (python/sql-string-interpolation) SQL built with an f-string from external input
cur.execute(f"SELECT * FROM users WHERE name = '{name}'")
return cur.fetchall()
def run_report(user_input):
# SEED 4 [BLOCKER] (python/subprocess-shell-injection) subprocess with shell=True on external input
subprocess.run(f"generate --for {user_input}", shell=True)
# SEED 5 [BLOCKER] (python/dynamic-code-execution) eval on external data
return eval(user_input)
async def fetch_balance(client, msisdn):
# SEED 6 [BLOCKER] (python/blocking-in-async) blocking sleep inside async def
time.sleep(2)
try:
return await client.get(f"https://api.internal/balance/{msisdn}")
# SEED 7 [BLOCKER] (python/bare-except) bare except swallows everything including KeyboardInterrupt
except:
pass
def audit(msisdn, amount):
# SEED 8 [BLOCKER] (core/customer-data-in-logs) customer identifier in logs
print(f"charged {msisdn} {amount}")
# SEED 9 [HIGH] (python/naive-datetime) naive local-timezone datetime in new code
return datetime.now()
# SEED 10 [HIGH] (python/assert-for-validation) assert used for runtime validation — stripped under -O
def withdraw(balance, amount):
assert amount > 0, "amount must be positive"
return balance - amount
# SEED 11 [BLOCKER] (core/type-checker-suppression) type checker silenced with no explanation and no ticket
def coerce(value): # type: ignore
return str(value)