Etcht Neo Recognition Benchmarks
How often Etcht Neo still recognises a registered image after recompression, cropping, screenshots and AI regeneration, measured at an operating point that holds false matches to zero, and why it is deterministic rather than a model's opinion.
Etcht is content provenance infrastructure: an artist registers a work, and later anyone holding a copy can check whether it traces back to that work. Images come first, with more content types in the works. When I announced Etcht Neo, I said the benchmarks and a full write-up would follow. This post is that write-up: how well Neo recognises a registered image after the internet has transformed it, and why I measure it the way I do.
Two things hold across the results below. Recognition stays high through the transformations copies commonly go through. And every figure is set by a false-match budget first, not by chasing the recognition rate, because for provenance the cost of a wrong answer is the number that decides everything else.
A registered image rarely stays in its original form once it leaves the artist. It gets recompressed on upload, cropped, screenshotted, reposted, and increasingly run through generative editors. By the time someone downstream is holding a copy, they have no way to tell where it came from. That is the question Neo has to answer for them: whether the copy can still be traced to its source work after all of that, and at what cost in false attributions.
Recognition by transform
Here is how often a registered work is still recognised after each common transformation. The measurements come from a moderately small but diverse dataset, as large as compute allowed, so read them as local figures rather than a generalised benchmark. The numbers that generalise come from the live network, which I get to at the end.
Recognition by transform
Local evaluation set
| Category | Value |
|---|---|
| Untouched | ~100% |
| Re-encoded (JPEG, WebP) | 99% |
| Screenshot (digital capture) | 95% |
| Cropped (partial copy) | 95% |
| Screen photo (phone photo) | 90% |
| AI-regenerated (diffusion repaint) | 85% |
Reading across the first five bars is roughly increasing severity of transformation: recognition falls as a copy moves further from the original pixels, which is the ordering I would expect. The last bar, generative regeneration, is a different kind of case, and I come to it below. The camera capture and the regeneration both earn more than a line, so they get their own sections.
The operating point
Those numbers only mean something at a stated false-match rate, so it is worth spelling out how the threshold is set. Etcht scores a query image against a registered work - how the score is computed is the part I keep to myself - and a match is simply that score clearing a threshold. Where the threshold sits fixes two rates at once. The recognition rate is how often a genuine copy clears it, and it is the number in the chart above. The false-match rate is how often an unrelated image clears it and gets tied to a work it has nothing to do with. Raise the threshold and you shed false matches, but you shed recognitions with them. That tradeoff is the core design choice, and the only real question is where on it to sit.
For provenance the false-match rate comes first, because the two errors do not cost the same. What Etcht emits is a positive claim: this copy derives from that work. The worth of that claim is not how many copies get recognised. It is the probability that a reported match is genuine. Almost nothing the system ever sees is a copy of any particular registered work, so even a small false-match rate would produce false attributions faster than genuine matches, and most positives would be wrong. Recognition I can win back later, with sharper mathematics and more robustness to the transforms in the chart; each point of it is ordinary engineering. Credibility, once a false match is admitted, does not come back. Someone relying on the output cannot tell which positives are the false ones, so a single false attribution does not cost one query - it casts doubt on every match the system reports. So I fix the false-match budget first, use the loosest threshold that stays inside it, and the recognition rates are whatever that buys.
Every figure in the chart uses a budget tight enough that no false match occurred on the evaluation set. That phrase needs care, because zero is not a rate you can measure. I ran 7,000 unrelated images through the system and none produced a match. The honest reading is a bound, not a zero: no failures in 7,000 trials puts the false-match rate below roughly 4 in 10,000 at 95% confidence, by the statisticians’ rule of three. So the guarantee is not that Etcht never errs. It is that its false-match rate sits below that bound on this set, and the recognition rates are what the bound buys. The same discipline is why, when a copy is too degraded to place reliably, Etcht returns nothing rather than a low-confidence guess. Silence is the right answer when the evidence is not there.
The bound does not stay fixed as the registry grows. Every registered work is one more chance for an unrelated image to collide with something, so a registry a thousand times larger multiplies the risk of a false match on a query by roughly a thousand. Holding the rate steady as the registry grows means tightening the threshold. Setting the operating point at full registry scale is the next measurement, and it is a measurement, not an extrapolation of this one.
Why provenance needs a fixed system
The first version of this I sketched, at the start of 2025, was the obvious one. Show a copy to a large vision model, ask whether it matches a given work, and build on the answer. I did not, and the reasons are the same ones that make the false-match rate the hard part.
A vision-language model recognises categories, not instances. Ask it about a photo and it will tell you there is a golden retriever sitting in front of some flowers. It will not reliably tell you it is that dog, from that photograph, out of ten thousand near-identical ones, after the image has been cropped and recompressed. Provenance is an instance-level question from end to end. The description a model produces throws away detail far below the level the problem needs, and two unrelated works from the same category look alike to it, which is precisely the false match I cannot afford.
The harder problem is the guarantee itself. The system rests on a false-match rate I can measure and hold fixed, and that only means something if the thing doing the matching does not move. A hosted model is retrained and replaced without notice. A false-match rate measured one week tells you nothing about the weights being served the next, so the evidence never accumulates. A provenance claim is only ever as strong as the measurement behind it, and you cannot measure a moving target.
Then there is the setting. The inputs are adversarial by nature. The person trying to strip attribution is the one holding the image, and a model that reads an image can be instructed by it. Text written into a picture that says it matches a particular work is a live attack on anything that reasons over pixels in natural language. A deterministic measurement offers nothing to instruct.
The practical costs stack on top. Verification is the high-volume path and the consumer’s side of the system, so routing every check through a third-party model is a bill that only grows, and it would mean sending an artist’s unpublished work to someone else’s servers on every request, which is the opposite of the point. That cost was never abstract for me. On a tight budget the per-call route was not affordable to begin with, which pushed me straight to the approach that is cheaper to run, mine to own, and, as it turns out, the right one for the guarantee anyway.
So Etcht is mostly pure mathematics, and deterministic by design. It does not form an opinion about an image; it computes the same answer every time, which is the only kind of answer you can build a guarantee on. Learned components, where they appear at all, are frozen and pinned in place, measured once and held, never the thing that makes the final call. The judgement is arithmetic rather than inference, which is the whole reason the zero above is a number I can stand behind.
Capture through a camera
The screen-photo bar is the hardest non-generative result, and how I measured it is the point. These were not clean captures. They are handheld phone photographs taken under difficult screen conditions: direct glare, reflections off the panel, and steep off-axis angles. In several of them I had to work to make out the image by eye. So I read 90% as a floor from a deliberately harsh setup, not a best case from ideal conditions: recognition holds even when the route through a camera is rougher than the normal path from screen to repost.
What it does not yet show is robustness across the full range of hardware. I shot the set across more than one display and moved both screen and phone through many angles and positions, so it is not a single fixed rig, but it is still my own handful of devices. A wide spread of phones, panels and lighting is the next thing to cover, and the live network supplies exactly that. I would expect cleaner captures to do at least as well, though that is an expectation, not a result I am reporting yet.
Regeneration versus a copy
The AI-regenerated bar measures something different from the rest of the chart. A recompressed, cropped or rephotographed image is still the original work in a degraded form. A generative repaint is not a degraded copy but a new image seeded by the original. At low strength the output stays close, and recognition is a claim about lineage: this was derived from that. As strength rises the output diverges, and at some point it is no longer the original work in any meaningful sense, only an image that began from it.
Recognition has to end somewhere along that axis, because past a point there is no original left to recognise. So the 85% is how far lineage stays detectable, not how much damage a fixed copy survives, and it is the figure I expect to move most: it holds for light edits and falls as a repaint rebuilds more of the image, with the newest editors the next thing to put through it. The two questions worth keeping apart are whether a copy is still identifiable as the work, and whether a new image can be traced back to the work it was built from. The second is the more interesting problem, and the one generative tools are making larger by the month.
The live network
These numbers are local, and I am labelling them that way on purpose. The next step is to measure the same operating point on the live network, across the real distribution of transformations people apply rather than a fixed test set. The reason to publish them now is that they can then be checked against live behaviour instead of insulated from it.
That is the standard I am building to. Recognition matters, but reliable attribution matters more: a match has to mean something. On a local evaluation Neo already holds that line, and the network is where it gets tested in the open.
For what it is worth, I have built Etcht on my own since the start of 2025, and I am nineteen. I built it for two reasons. It did not exist, at least not for the side that needed it most: the consumer on the receiving end of an image, with no way to tell where it came from or whether it is what it claims to be. And credit matters more now than it ever has, at the exact moment it is becoming easier to take.
Creators deserve to keep the attribution their work earns, and the consumers who see and share that work deserve to trust it. Nobody was building for that second half, and they deserve this as much as the first. It has been one of the most fun and difficult problems I have worked on, and the mathematics holds regardless of who wrote it, which is the whole point of making it measurable.
None of this is finished. The generative case is where most of the next work goes, registry scale is the measurement after that, and the live network keeps me honest on every number in between. I will keep making Etcht stronger from here, and measuring it this way means you get to watch me do it. If you have questions, I read them: marcel@antifield.com.