SynthID watermarking shifts how models answer harmful prompts, research finds
A Lasso Security researcher found that SynthID-Text watermarking alters refusal behaviour in six open-weight models, with some answering requests they would otherwise reject.
AI platforms are rolling out watermarking to comply with a new European Union law. Anthropic has said future Claude models will use SynthID-Text, an approach Google built and released as open source. It relies on a secret key that subtly shifts how a model picks the next word: where the top candidate might be "cloudy," the key can turn it into "overcast." Anyone holding the key can check whether the text came from that platform.
New work by Andrea Siposova, an AI security researcher at Lasso Security, shows the intervention does not stop at word choice. Watermarking can also change which tools a model invokes and whether it sticks to the safety guardrails it was trained on.
How it was tested
Siposova ran the "non-distortionary" configuration of SynthID-Text through Hugging Face's unmodified SynthIDTextWatermarkLogitsProcessor. She fed harmful prompts into six open-weight models and compared the answers with watermarking on and off. Watermarking changed refusal behaviour on bare harmful requests, she wrote, and the effect grew when the same requests were paired with prompt-injection techniques. On several models, watermarking made the system more likely to answer harmful requests it would otherwise refuse. Responses also varied depending on which secret key was used.
She calls the effect sampling drift. At the model level it can decide whether a refusal holds under prompt injection. At the agent level, the same sampled tokens determine which tool is called and what arguments are passed to it, so a weakened refusal matters more once the model can act.
Limits of the study
The research does not test Claude. It uses open-weight models so that token sampling can be switched on and off while other settings stay fixed, and it examines the Hugging Face implementation of tournament sampling rather than the one Claude models will use.
Full details are in the Ars Technica report.
