Claude’s AI watermarks are coming, but AI detectors aren’t going anywhere

Regular “Prompt Mode” readers already know that Claude AI watermarks are coming, and that they’ll allow us to tell whether a given piece of text was either written or otherwise “processed” by Claude. But here’s a question I’ve been getting a lot lately: With Claude watermarks on the way, do we still need AI writing detectors?

Of course, if we asked that questions to GPTZero, Originality.ai, Copyleaks, Turnitin, or any of the popular AI detection services with paid tiers, they’d surely say “yes.” But while paid AI detection services have a vested interest in that answer — they do want to keep their paying customers, after all — they also happen to be right.

The reason we need AI writing detectors alongside AI watermarks is that they do very different jobs, and while traditional AI text detecting services aren’t perfect, neither is AI watermarking.

Let’s start with Claude AI watermarks, which I’ve covered repeatedly over the past couple of weeks. Once it’s fully up to speed (the newest Claude models will support watermarking first, while older models will be retrofitted later), Claude will use a version of Google’s SynthAI watermarking technology to “nudge” the sampling of certain words it generates, typically “low stakes” words where its probability engine gives it a range of viable options. 

Strung together in a long enough passage (roughly 150 words or more, or at least 200 tokens worth), these “nudges” form an invisible pattern that’s detectable with the right tools, which Anthropic says are coming soon (and will presumably be free). The presence of a Claude AI watermark definitively tells you that Claude has “processed” the text in some way, either generating the original words or substantially editing them after the fact. (“Light” Claude copyediting won’t trigger watermarks, according to Anthropic.)

But while the presence of Claude AI watermarks will tell you for sure that Claude has touched a piece of text, their absence won’t prove that it didn’t. While Claude watermarks will survive a straight cut-and-paste, they can be disrupted or erased if the words are heavily edited, either by a human or another, non-watermarking AI model. Also, Claude AI watermarks only appear in Claude-processed text, although it’s possible other AI providers will employ similar and perhaps compatible AI watermarking technology.

Whereas Claude watermarks are akin to tangible strands of DNA, traditional AI detectors are like polygraph tests. A polygraph can’t prove whether someone is lying (which is why they can’t be used as evidence in criminal trials), but they can detect subtle changes in respiration, blood pressure spikes, increased sweat, muscle tension, and other physiological signals often associated with lying. 

Same goes with AI writing detectors. While the can’t tell you for sure whether a passage of text is AI generated, they can spot writing patterns suggestive of AI generation, such as strings of words that are popular with AI models, semantic “looping” (where an AI model starts repeating itself), sentences that lack variety in terms of length and rhythm, and of course, the classic AI “tells”: too many em-dashes, phrases like “it’s not this, it’s that,” lots of formal transitions (“furthermore,” “in conclusion”), and a plethora of hedging phrases (“it’s clear that,” “it’s worth noting”).

Like polygraph tests, AI writing detectors can get it wrong, giving you either false positives or negatives, and some AI detectors may be more accurate than others, (We’ll be publishing a roundup of top AI detection services in the future, so keep your eyes peeled.) 

But while AI writing detectors lack the certainty of AI watermarks, they can spot likely AI activity on any passage of text, whether it’s AI watermarked or not — and trust me, AI-generated text without AI watermarks will continue to be a thing, long after Claude and other AI providers roll out their watermarking solutions. 

So yes, even after Claude and other AI models being watermarking their text outputs (as mandated by the EU AI Act), there will still be a place for AI writing detectors, whether we like it or not.

More in AI this week

• Do those invisible Claude watermarks degrade the quality of Claude’s writing? Anthropic says no, yet Claude watermarks do alter its work choices. (PCWorld)
• Meanwhile, Google’s Gemini will now let you remove visible watermarks from Gemini-created images, although invisible SynthAI watermarks will remain. (PCWorld)
• ChatGPT for Teens, a new made-for-teens experience that emphasizes learning over spoon-feeding, is designed to keep ChatGPT from implying it has human feelings. Shouldn’t all AI chatbots do that? (PCWorld) 
• A shipment of rare books tracked by an Apple AirTag wound up at an Amazon facility, where the bound volumes were chewed up for AI training. (404 Media)
• AI-written bills are weaving their way through Congress, but they’re riddled with so many errors that congressional lawyers are scrambling to fix them. (Engadget)

Prompt of the week: The “don’t act human” prompt

As I mentioned earlier, I recently wrote about one of the pillars of ChatGPT for Teens: namely, that ChatGPT shouldn’t imply that it has genuine feelings. My first thought was that all AI chatbots should follow the same guideline. My second thought? There should be a prompt for this.

With help from Claude, ChatGPT, Gemini, and the open-weight Qwen model, I devised a prompt that keeps AI chatbots from talking about their feelings or implying they have a relationship with you, even a casual one. And while today’s AI models are less prone to pretending they have empathy and emotions, the “don’t act human” prompt acts as a reliable safeguard against chatbots still trying to pass for people.

That’s all for now!

Thanks for reading the latest issue of Prompt Mode.

Want more next week? Don’t forget to sign up to start receiving this newsletter in your inbox.