Mutation Testing for AI-Generated Code and Tests
AI coding assistants can generate a working implementation in a few seconds. They can also generate a test suite that passes in the same amount of time. That speed is useful, but a green test run does not tell us whether the tests would catch a real defect. Mutation testing gives us a practical way to check that question. A mutation tool makes small changes to the production code, runs the test suite, and reports which changes the tests detected. The changed programs are called mutants. A test suite that fails when a mutant is present has killed that mutant. A surviving mutant shows a behavior that the tests did not distinguish. ...