AI wrote your tests. Would they catch a bug?
Published
Mutation testing changes code on purpose and checks whether tests notice. Stryker automates this for JavaScript and TypeScript. A free-shipping rule shows how two passing tests can miss an exact boundary.
Explanation & code
Check requirements and read AI-written tests. Stryker adds evidence; it does not replace review. Some mutations behave the same, and a perfect score cannot prove every requirement is covered. The example uses whole-dollar totals.
Understand it. Then fix it.
What is mutation testing?
Temporarily change the code, then run the tests against the changed version. If a test fails, it noticed the change. Stryker is a tool that automates this for JavaScript and TypeScript. It is useful when reviewing AI-written tests, but it does not replace reading them.
Start with the requirement
In this example, shipping is free at 50 dollars or more. The >= operator includes exactly 50. The original code is correct.
function shipsFree(total: number) {
return total >= 50;
}Two tests pass, but a mutant survives
Stryker changes >= to >. This changed version is a mutant. At 49 both versions return false; at 51 both return true. These tests still pass, so this mutant survives. At exactly 50, the original returns true and the mutant returns false.
expect(shipsFree(49)).toBe(false);
expect(shipsFree(51)).toBe(true);
// Original: 50 >= 50 → true
// Mutant: 50 > 50 → falseTest exactly 50
The expectation comes from the requirement: shipping starts at 50. This test passes against the original and fails against the mutant. Stryker calls that mutant killed. Keep the original production code and add the test. You can also write boundary tests directly without Stryker.
expect(shipsFree(50)).toBe(true);Run Stryker with Vitest
Add Stryker core and its Vitest runner to an existing Vitest project. Initialize the configuration, choose Vitest, then run it. Start with a small, important module. The reproduced demo uses core and runner 10.0.0, Vitest 5.0.2 and Node 22.23.3.
npm i -D @stryker-mutator/core @stryker-mutator/vitest-runner
npx stryker init
npx stryker runScope the code to mutate
Configuration excerpt: target the shipping module and use the existing Vitest tests.
{
"testRunner": "vitest",
"mutate": ["src/shipping.ts"]
}Read the change, not just the score
In this tiny demo, four of five mutants were detected before the boundary test and all five afterward. This is evidence for the example, not a general benchmark. Some survivors are equivalent in the supported domain. No score proves all requirements are covered. AI-generated tests still need review.
Code blocks are teaching excerpts. Keep the surrounding error handling and application requirements.
Save the code excerpts ↓Read the full transcript
AI wrote my tests. They pass. I barely read them. What is mutation testing, and can it help? Mutation testing changes code to check whether tests catch it. Stryker automates this for JavaScript and TypeScript. Our rule: free shipping at fifty dollars or more. Greater than or equal includes exactly fifty. I tested forty-nine: false. Fifty-one: true. Both tests pass. So what did I miss? Stryker temporarily changes greater than or equal to just greater than. This changed version is called a mutant. It runs your tests against that version. Forty-nine is still false. Fifty-one is still true. The mutant survives: those tests cannot tell the versions apart. Exactly fifty changed, and we never checked it. Add a test for fifty, expecting true because shipping starts there. The original returns true. The mutant returns false, so this test fails against the mutant. Stryker calls that mutant killed. Your test caught the change. Keep the original code and the new test. You can also write boundary tests without Stryker. In a Vitest project, add Stryker core and its Vitest runner. Initialize the configuration, then run Stryker. Start with a small, important module. AI can miss a rule too. Check the requirements and read the tests. Stryker adds a check, not a replacement for review. Some mutants behave the same. Even a perfect score cannot prove every requirement is covered. So my tests checked forty-nine and fifty-one. Fifty was on paid leave.
