Connect Clawsmith to your coding agent. Ship products like crazy.Unlimited usage during betaGet API Key โ†’
โ† Back to dashboard
clawsmith.com/signal/flaky-test-auto-triage-and-retry-cli-for-ci
โš  IssueUnderserveddev_tool_cliLive

Developers want a CLI that auto-detects and triages flaky tests instead of manually rerunning whole CI suites

Across major test toolchains, maintainers and users keep asking for first-class handling of flaky tests: automatic detection of nondeterministic failures, quarantining them, and retrying only the failed tests rather than re-running the entire suite. The Go team's own issue to add flaky-test support has 115 reactions and 36 comments and is still open; the JUnit request to retry flaky tests has 94 reactions and 78 comments. Today teams hand-roll retry wrappers, grep CI logs, and argue over whether flakiness is a test, product, or infra problem. The gap is a standalone tool that ingests CI test history, flags the flaky ones statistically, and gates merges on real failures only.

Score Breakdown

Issues
415

Gap Assessment

UnderservedExisting solutions leave gaps

Test frameworks bolt on per-framework retry flags but there is no language-agnostic tool that detects flakiness from CI history and quarantines it. CI vendors charge for it as an add-on. Room for a focused open-core CLI.