ChatGPT sparks surge of AI detection tools

Illustration: Aïda Amer/Axios

The rise of generative AI tools is creating parallel demand for a new class of systems that can help distinguish AI-generated text and images from those created by humans.

Why it matters: Educators, in particular, are concerned about students turning in work created by an AI system. But experts are also worried about how generative AI can create a flood of misinformation and impersonation. Detection tools, if they can be made sufficiently accurate, could help.

Driving the news: OpenAI released a free web-based tool Tuesday designed to help distinguish AI-generated text from that written by humans. But it isn't alone in trying to address this issue. A number of startups, organizations and individuals have also released or are developing AI-detection software.

  • GPTZero was created by college student Edward Tian and works similarly to the new tool from OpenAI, assessing the overall likelihood that a piece of content was machine-generated.
  • Self-funded startup is testing a product with educational institutions that analyzes content paragraph-by-paragraph rather than just providing an overall score.
  • has a tool to help writers detect if their writing will pass as human, since search engines penalize content they believe to be machine-generated.

Yes, but: While detection tools show potential, the current tools are prone to making wrong calls, particularly when used on a writing sample that's very different from the ones used to train the system.

  • The tools will no doubt improve, but so will the means for evading detection, likely creating something of a cat-and-mouse game.
  • "We believe that techniques for detecting AI content will improve, and continue to surpass human ability to discern between machine- and human-written content," a representative told Axios. "With that said, there is clearly a limit at which sufficient human editing will always obscure traces of AI-generated content."
  • OpenAI and others are also exploring other approaches, such as watermarking their systems' creations to make it easier to spot machine-generated content.

The big picture: Generative AI has hit a tipping-point of popularity, and that means all manner of computer-created content is about to explode.

  • The challenge will be developing enough systems — both human and automated — to deal with the flood of additional content at a time when misinformation is already a significant problem.

In action: Here's how several detectors performed when we asked it to analyze a portion of yesterday's Login newsletter.

  • OpenAI: "The classifier considers the text to be very unlikely AI-generated."
  • GPT Zero: "Your text is most likely human written but there are some sentences with low perplexities."
  • Fictitious.AI: This system broke the newsletter items down paragraph by paragraph, correctly discerning that all but the last part I submitted had less than a 5% chance of being AI-generated. For some reason, though, it decided the last line of a story about TikTok had an 85% chance of being AI-written. (It was us.) Line in question: "Don't be surprised to see such talks eventually resume, given how much there is at stake for the company and its investors."
  • "Fantastic!" (I.e., we passed as human.)

And here's how the services evaluated a section of a ChatGPT-generated essay on the history of baseball cards. (It was a pretty good history, speaking as someone who knows a bit about the gum-infused cardboard collectibles.)

  • OpenAI: "The classifier considers the text to be unclear if it is AI-generated."
  • GPT Zero: "Your text is likely to be written entirely by AI."
  • Fictitious.AI: The service categorized each paragraph as at least 88% likely to have been AI written with some listed as high as 98%.
  • "You should edit your text until there’s less detectable AI content."

Source: Read Full Article