We’re thrilled to introduce Ghostbuster, the state-of-the-art method for detecting AI-generated text! Ghostbuster works by finding the probability of generating each token in a document under several weaker language models, then combining functions based on these probabilities as input to a final classifier. This means Ghostbuster is particularly useful for detecting text potentially generated by an unknown model or a black-box model, such as the popular commercial models ChatGPT and Claude, for which probabilities aren’t available.
Our recent paper proves that Ghostbuster is extremely effective; when trained and tested on the same domain, it achieved an impressive 99.0 F1 across all three datasets, outperforming other models by a significant margin. Out of domain, Ghostbuster achieved 97.0 F1 averaged across all conditions, outperforming DetectGPT by 39.6 F1 and GPTZero by 7.5 F1. And when it comes to prompt variants, Ghostbuster outperformed all other tested approaches with 99.5 F1.
We also tested Ghostbuster’s robustness to edits, such as swapping sentences or paragraphs, reordering characters, or replacing words with synonyms. Most changes at the sentence or paragraph level didn’t significantly affect performance, and Ghostbuster was best on longer documents. To protect non-native English speakers, we evaluated Ghostbuster’s performance on non-native English speakers’ writing, where it achieved over 95% accuracy on two of three tested datasets.
At the end of the day, Ghostbuster is an invaluable tool for teachers, consumers, and anyone else who needs to detect AI-generated text. Whether you’re looking to protect yourself against ghostwriters or detect AI-generated text on the web, Ghostbuster is the perfect tool for the job. And best of all, you can try it out for yourself right now at ghostbuster.app!
