Tests with AI

Yes, and it does it quickly, but with a catch: if you ask it for tests for code that already exists, it tends to check what the code does, not what it should do. The result is tests that always pass, very nice coverage and no guarantee that the program is right.

Where it does help is with the heavy lifting: preparing test data, covering variations of the same case and keeping tests up to date when the code changes. The important cases, the ones that describe what the program has to do and what must never happen, are best written, or at least decided, by yourself.

What happened when I left the tests to the agents is in “Automated tests written by artificial intelligence” and in the series “A paid app built with AI”.