AI Test Triage

When an automated build fails, the test report is usually long, technical and hard to skim quickly. Someone still has to open it, find the actual error among hundreds of lines, and work out whether it is a real bug, a broken test or a flaky environment.

Drop it into any GitHub Actions workflow and it reads the test results the moment a build fails, then posts a short summary of each failure, its likely cause and a suggested next step, right on the pull request. It works with any testing tool that outputs the standard JUnit XML format, which covers most Java, Python and JavaScript test runners. It never changes the build result, and without an API key it still lists the failures.

You can try the same AI in your browser on our test triage demo: pick a sample error or paste your own. To see it on a real pipeline with nothing to install, there is an example repo with four tests that fail on purpose, each for a different reason, and an open pull request that shows the comment it posts.

We also measured it. On 40 hand-written, labelled test failures, run three times, it classified 39 correctly. On 10 tricky cases, where the error name points the wrong way, it found the real cause every time, while a simple keyword rule found none. That is an early test, not proof, and we publish where it disagreed with us.

Recent releases keep the output clean: v1.0.1 removes long dashes from the AI text and moves to a newer setup step, and v1.0.2 stops the AI from accidentally mentioning GitHub users when it writes code names such as @BeforeEach. The live demo is also checked by an automatic test every morning.