Every serious ML team trains on data that comes from somewhere. A radiology model learns from X-rays contributed by dozens of hospital sites, each with its own scanners, its own radiologists, and its own labeling habits. A factory vision model retrains on camera feeds pulled from customer production lines. A drone navigation model consumes reference imagery licensed from a satellite provider and re-rendered into training tiles. In every one of these cases the training set is not a single thing. It is a merge of streams from many independent sources, and any one of those sources can fail you. Usually by accident. Occasionally, and increasingly, on purpose.
The industry has built good tools for checking datasets and good tools for checking models. Almost nobody continuously checks the sources. This post explains why that gap is costing teams real time and model quality today, and how we score source trust at Agentiks.
The problem you already have
Forget attackers for a moment. Sources go bad on their own, constantly, for boring reasons. A hospital site swaps a scanner and every image after the swap has subtly different characteristics. A labeling vendor loses its best reviewers to turnover and quality sags over a quarter. An upstream API changes a default and a field quietly shifts meaning. None of these produce data that looks wrong sample by sample. All of them degrade the model that trains on it.
The uncomfortable part is where that failure is visible. It is invisible at the sample level, because each individual sample still looks plausible. And it is usually invisible at the dataset level too, because one drifting source among thirty barely moves the aggregate statistics that dataset-quality tools monitor. The place the signal actually lives is the source: this particular stream, compared against its own history and against its peers, started behaving differently on a particular day. If nothing in your stack works at that granularity, the failure surfaces weeks later as a model regression, and someone spends days of archaeology tracing it back to the batch, the vendor, or the site that caused it.
There is also an accountability version of the same gap. When an auditor, a customer, or your own postmortem asks why you trusted a data source and what you knew about it at the time, most teams have no evidence-backed answer. That question is starting to arrive with legal weight behind it, and it is not answerable retroactively: the evidence either was being collected at ingestion time or it does not exist.
Trust is a property of sources, not of snapshots. So that is where we put it.
Why isn’t this already being done?
Not for lack of tooling. The data stack is crowded with products that are each excellent at their own layer. The clearest way to see the gap is to ask which layer of the training pipeline each of them actually examines.
The dataset layer
Cleanlab scores individual samples and annotators for label quality, and observability products like Anomalo and Monte Carlo alert when a table’s distribution shifts or a pipeline breaks. All of them examine the merged pool of data after it has been collected: a point-in-time check of the aggregate, and a single misbehaving source among thirty is exactly what aggregate statistics are worst at showing.
The model layer
Security vendors such as Protect AI, HiddenLayer, and Cisco AI Defense examine the trained artifact and its runtime traffic: tampered model files, adversarial inputs, unsafe outputs. Valuable work, but it sits downstream of training. By the time a data problem is visible at this layer, the model has already learned it.
The metadata layer
Provenance standards like C2PA and the Data and Trust Alliance’s data provenance standards examine the paperwork that travels with data: signed attribution recording where it came from and how it was produced. Genuinely useful, and explicitly not a trust judgment. C2PA’s own documentation is careful to say it makes no claim about whether the attributed source should be believed.
The source layer
The entities actually producing the data, the hospital site, the camera line, the vendor feed, the labeling team, are the one layer none of these tools continuously examine. Everyone inspects what the sources produced, or records who they were. Nobody watches how they behave.
To be fair to the research world: the idea of scoring contributors by their behavior is not new. Federated learning (a training setup where many clients train a shared model without pooling their raw data) has a decade of academic work on client reputation systems, and crowdsourcing platforms have estimated per-worker label reliability since the 1970s. The concept exists. What does not exist is a product that applies it where most enterprise training actually happens: a centralized ingestion pipeline pulling from many external sources. The dataset, model, and metadata layers are all covered. The source layer, a continuously evolving trust score for each origin that decides whether its data gets into training at all, is the layer we build.
How Agentiks scores a source
Every source in an Agentiks-instrumented pipeline carries a trust score between 0 and 1, recomputed continuously as it submits data and as detectors observe it. The score is a weighted blend of behavioral signals:
- Consistency.Statistical drift detectors watch each source’s behavior over time. A source whose data suddenly shifts character loses trust.
- Flag history. A record of incidents and quarantines, weighted by severity and recency. When deeper detection layers downstream (for example, our embedding-space detectors that spot coordinated content across sources) flag a source, the penalty lands here.
- Track record depth. Age and submission volume. A source cannot become highly trusted off ten submissions, no matter how clean they look.
- Cross-source agreement.Whether this source’s labels agree with independent sources labeling comparable content.
- Historical accuracy.How often the source’s data proves correct when checked against ground truth. This dimension is built and tested, and it starts carrying weight the moment a deployment wires in a ground-truth feed to verify against. Until then it runs dormant, because we would rather run it at zero weight than pretend to measure accuracy nobody is verifying.
The blend then decays. Recent behavior is weighted exponentially more than old behavior, with a half-life of about three weeks. This is a security decision, not a smoothing detail: it is the structural answer to trust grooming, the attack where someone submits clean data for months to build a high score and then flips to poison. Under decay, groomed trust evaporates on roughly the same timescale it was earned, and the moment behavior turns, the drift and agreement signals pull the score down fast.
Scores map to tiers, and tiers have teeth. New sources start in probation with a prior based on how strongly they authenticated, and nothing with fewer than 50 submissions leaves probation regardless of its score. An established, highly trusted source can vouch for a newcomer, but vouches are capped, budgeted per voucher, and cannot be chained, so a ring of fake identities cannot bootstrap itself. At the other end, the pipeline’s gate reads the tier on every decision: low-trust sources get heavier scrutiny and quarantine, and a source with no score at all is treated as probation rather than given the benefit of the doubt. The system fails closed.
The hard part: not punishing the innocent
A scoring system that can keep data out of training is itself a target, and it is also a false-positive hazard. Most of our engineering effort went into guardrails that keep the score honest in both directions.
- Hysteresis. The threshold to regain a tier sits above the threshold to lose it, so a source hovering at a boundary cannot flap between tiers on noise.
- No single-witness convictions.One detector family alone cannot destroy a source. Until a second, independent signal corroborates, a lone detector’s penalty is capped and the composite score is floored. Two independent witnesses lift the cap entirely.
- Penalties that heal when they were wrong. A drift alarm opens a case rather than passing final sentence. If the source then sails through a window of escalated inspection with nothing corroborating, half the penalty is refunded. If content-level detection confirms the alarm, the penalty becomes permanent. The refund is never full, on purpose: an adversary probing the boundary should pay for every probe.
- Victim discrimination.The subtlest failure we designed for: when a coordinated ring of fake sources mass-duplicates an honest source’s content, that honest source lands inside the ring’s detected cluster and would naturally be punished with it. We compute a per-member victim score from signals the ring cannot easily fake, such as what fraction of the source’s lifetime output is implicated and who published first, and withhold the coordination penalty from sources that look like victims.
In a controlled test on our staging cluster, we simulated exactly this attack: a synthetic ring of fake sources flooding duplicated content around one honest victim. Every ring member’s trust collapsed, the coordination detector caught 96 percent of ring samples, and the honest victim’s score never fell below the guardrail floor. That is one controlled run against one synthetic attack pattern, and we present it as exactly that, not as a fleet statistic.
The problem after this one
Everything above earns its keep against ordinary failure, the kind every multi-source pipeline already has. But there is a second reason to put trust at the source layer, and it is the problem we think teams will be forced to care about next: data poisoning, deliberately planting corrupted or booby-trapped samples in training data so that the resulting model misbehaves. It is already a named category in the OWASP Top 10 for LLM applications, and three research results say the obvious defenses will not hold.
It is cheap: Carlini and colleagues showed in Poisoning Web-Scale Training Datasets Is Practical that controlling 0.01 percent of a web-scale dataset costs about 60 dollars, because the URLs those datasets point to can simply be bought when their domains expire. It does not dilute: a joint study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found that roughly 250 malicious documents were enough to backdoor language models from 600 million to 13 billion parameters, so the amount of poison needed is close to constant rather than a percentage of your data. And it is invisible to inspection: Nightshade corrupts a concept inside an image generation model with fewer than 100 poisoned images that look completely normal to a human reviewer.
In other words, a viable poisoning campaign will not be a noticeable fraction of your data and will not look wrong. The one thing an attacker cannot easily fake is a source’s behavior over time, which is exactly what this layer watches. That is the quiet payoff of putting trust at the source: the machinery you install for mundane integrity today is the same machinery that gives you a real defense on the day someone makes your data fail on purpose.