Content Authenticity: Why Deepfake Detection Loses and Provenance Wins

For most of the history of photography, an image was weak evidence that something happened, and it was accepted as strong evidence anyway. That gap held because faking a photograph convincingly took skill, time, and equipment. The gap has closed, and the assumption built on top of it has not been replaced.

The instinctive response is detection: build a classifier that tells you whether an image or a video is synthetic. Having worked on content authenticity for a newsroom and now on digital forensics as a research problem, I think detection alone is a losing strategy, and it is worth being precise about why.

Why detection loses

Synthetic media detection is an adversarial classification problem, and adversarial classification problems favour the attacker in three structural ways.

The detector trains on yesterday’s generators. A model learns the artefacts of the systems it was trained against — particular frequency signatures, inconsistencies in facial geometry, unnatural blinking, mismatched lighting. Each generation of generator removes the artefacts the previous detector relied on. Detection is always describing a distribution that has already shifted.

Compression destroys the evidence. The traces detectors depend on are subtle and live in high-frequency detail. Every platform re-encodes on upload. A video that passes through a messaging app, then a social platform, then a screen recording has been compressed several times, and much of what a detector needs is gone — while the content remains perfectly persuasive to a viewer.

The error costs are asymmetric and both are bad. A false negative certifies a fake as genuine. A false positive hands anyone caught doing something real a ready-made defence. That second failure mode is already being exploited: the existence of plausible deniability about synthetic media means genuine evidence can be dismissed as fake. The mere possibility of forgery devalues authentic material, whether or not any forgery occurred.

Detection has a role. It is a filter, not a foundation.

The better question

The question “is this image real?” cannot be reliably answered from the pixels. The question “can this image demonstrate where it came from?” can be answered, because it is a cryptographic question rather than a perceptual one.

This is what provenance approaches do. Sign the content at the point of capture, with a key held in the device. Record subsequent edits as signed assertions. Anyone downstream can verify the chain: this came from this device at this time, and these specific transformations were applied by these parties.

The shift is from proving a negative to establishing a positive. You are no longer trying to show that something was not fabricated — which is unbounded. You are showing a verifiable history, which is finite.

Where provenance is genuinely hard

Three problems are real and unsolved, and anyone selling this as finished is overstating.

Platforms strip metadata. Most social networks discard embedded provenance data on upload for entirely reasonable reasons — privacy, bandwidth, format normalisation. A signature that does not survive the distribution path is worthless, which is precisely where robust watermarking becomes complementary rather than optional: a marker embedded in the pixel data survives what metadata does not.

Absence proves nothing. Most genuine images will carry no provenance data for years. If unsigned means suspicious, almost everything is suspicious and the signal is useless. This has to be handled as gradual accumulation of trust, not a binary.

The key infrastructure is the entire game. Provenance is only as good as the device keys, and device keys can be extracted. A signature proves a key signed it, not that the key was in honest hands.

What a newsroom should actually do

Most of the practical answer is unglamorous and has little to do with algorithms.

Provenance beats analysis, in that order. The first question about a submitted image is not what a detector says. It is where it came from, who sent it, what else they have sent, and whether the metadata story is consistent. Most fabrications fail at this stage, and they always did — this is old-fashioned verification practice, not a new discipline.

Preserve originals, untouched. Whatever arrives should be archived byte-for-byte before anything resizes or re-encodes it. Every processing step destroys forensic signal. This single operational habit does more for later verification than any tool.

Establish provenance on your own output. Signing what you publish is more tractable than verifying what you receive, and it is the half you control. It also lets others verify you, which becomes a competitive asset as the baseline erodes.

Treat detector output as one input. Give it to trained people alongside everything else. It should never be the deciding factor, and it should never be automated into a publication decision.

The part that is not technical

The most damaging effect of cheap synthetic media is not that people will believe fabrications. It is that people will stop believing anything, and will select what to believe on the basis of what they already prefer.

That is a problem of institutional trust, and no verification pipeline fixes it. What technology can supply is a chain that a motivated person is able to check. Whether anyone bothers to check depends on whether they trust the institution presenting it — which is earned in ways that have nothing to do with cryptography.

Worth keeping in view while building the tooling: the tooling is necessary and it is not sufficient.