Basilisk's 100% Python Conformance Score Was Not Real
I recently built a Python type checker called Basilisk and submitted it to the Python typing repo. It passed on the first attempt with a perfect score, but the score was wrong. The code matched the text of the specific tests rather than implementing the specification. I used AI to build the checker and didn’t pay enough attention to how it passed the conformance tests. What’s worse is that I then went on to advertise the results on social media and the Basilisk website. This is a public apology to all the people involved and the Python community in general. On 5 August I asked the typing maintainers to remove Basilisk from the conformance results, and they did. I’m sorry. I made a claim that I didn’t verify, and somebody else had to do the work of finding out it...
Christian Findlay
Aug 07, 2026
