AI writing detectors, such as ChatGPT, are designed to identify text produced by AI models. When these detectors analyze the U.S. Constitution, they often conclude that it was likely written by an AI.

This is, of course, impossible unless James Madison, one of the key drafters of the Constitution, had access to time travel and chatGPT. So, what is the reason behind these false positives?

The use of AI writing tools, particularly generative AI like ChatGPT, has stirred up a whirlwind in the educational sector. There's a growing concern that these tools could disrupt traditional teaching methods, particularly the use of essays to assess a student's understanding of a topic. In response, educators are turning to AI writing detectors, such as GPTZero, ZeroGPT, and OpenAI's Text Classifier, in an attempt to maintain the status quo. However, these tools have proven to be unreliable due to their propensity for false positives.

For instance, when GPTZero is fed a section of the U.S. Constitution, it declares that the text is "likely to be written entirely by AI." Similar results have been observed with other AI detectors, leading to a wave of confusion and humor about the possibility of the founding fathers being robots. Even religious texts, like sections from The Bible, have been flagged as AI-generated.

To comprehend why these tools make such glaring errors, we need to delve into the underlying principles of AI detection. AI writing detectors use AI models that have been trained on a vast body of text, along with a set of rules to determine if the writing is human or AI-generated. For example, GPTZero uses a neural network trained on a diverse corpus of human-written and AI-generated text. It then uses properties like "perplexity" and "burstiness" to evaluate the text.