How we found SynthID's signal

Last updated 19 September 2026

The watermark test on this site did not start from Google's or OpenAI's documentation. It started from measuring thousands of images and following what the numbers said, including the times they said something wrong. This is the short version of that project.

An energy band that looked like it tracked the date

13 September 2026

Early on, we found a frequency band whose energy varied in a way that looked structured rather than random — and the variation seemed to line up with when an image had been generated. If real, that would have meant the watermark's carrier itself was visibly shifting over time, which would have been a significant finding on its own.

It wasn't the watermark — it was the decoder

15 September 2026

Two days later we pinned the band down and it turned out to be a rendering artifact, not a signal: a periodic pattern left behind by the image decoder's own upsampling step, present in every image that decoder ever produces, watermarked or not. It correlated with generation date only because the decoder itself had changed in that window. We removed it from every measurement that followed. Chasing it down was worth doing anyway — ruling out an artifact this specific is part of how you get confidence that what is left over is real.

Why this matters for a detector. A watermark test that can't tell its target's signal apart from an unrelated rendering quirk will produce false positives on anything that shares the quirk, and false negatives on genuine marks that don't happen to carry it. Separating the two was a precondition for everything downstream.

Identifying the actual signals

14–17 September 2026

With the decoder artifact isolated and set aside, we went after the real thing: the statistical pattern each watermark actually leaves in an image's pixel noise. OpenAI's mark came first, from a set of generated images processed the same way. Google's SynthID mark followed the next day, built from several hundred natural (non-blank) generations rather than a handful of test images — a watermark has to survive across everything a model can produce, and a signal estimated from a large, varied set is one that generalizes rather than matching a few examples too closely.

On that batch of roughly 400 images, refining how the signal was estimated took it from mostly noise to ~83% real signal, measured by how well two independent halves of the set agreed with each other on where in the image the watermark reads. That number is what the watermark test on this site is built on, and it's also why it isn't perfect: a real signal recovered from natural images is not a certificate, and the Terms of Service say plainly not to treat a result as one.

The dataset

We publish the images this work is built on. The SynthID Research Set on Hugging Face is the same prompts rendered by several generators side by side, most of them carrying an invisible provenance watermark and one that does not — a matched comparison set for anyone else working on AI-image provenance detection. It is released under CC-BY-4.0 and grows as new models ship.

SynthID is a trademark of Google LLC. scrubb.io is an independent project and is not affiliated with, sponsored by or endorsed by Google. See the Terms of Service and Privacy Policy.