Aller au contenu

Actualités

The Certainty of Uncertainty: When AI Models Are Confidently Wrong

Why AI models can be confidently wrong when sorting waste, and what that means for reliable recycling.

By now, most of us have learned to treat the answers of generative AI with a healthy dose of skepticism. We have all seen ChatGPT invent a reference, a fact, or a place with complete assurance. In July 2026, two hikers in the Tatra Mountains had to be rescued by helicopter after following a “shortcut” suggested by ChatGPT: the route confidently led them onto climbing terrain they could not cross. The model did not say “I am not sure.” It simply answered, convincingly, and wrongly.

For the general public, this skepticism towards AI predictions is relatively recent. At Matvision, however, it has always been the central engineering reality. Working with artificial intelligence on a high-speed sorting line requires a permanent, critical assessment of what the model predicts. More importantly, it requires engineering a system that remains reliable precisely where a standalone classifier would otherwise be confidently wrong.

The premise is straightforward: a confidence score is not a probability. That gap is measurable, it is correctable, and a system that knows when to say nothing is worth far more than one that always answers.

1. The confidence score is not what it looks like

Measurement and material-characterization systems are standard across modern recycling. Among the most widely used are handheld XRF and LIBS analyzers, which give an immediate indication of a material’s composition and suggest a corresponding grade directly in the field.

But these devices are calibrated for specific families of materials. Their algorithms compare a measurement against a predefined database of reference grades, and in most cases the instrument is programmed to always return an answer, even when the piece falls outside the scope of its calibration.

A sorting system’s neural network behaves the same way, but at a larger scale. It may report a confidence score of 0.95 for stainless steel grade 304, which an operator intuitively reads as: “this piece has a 95% probability of being 304.”

The mathematical reality is different. Modern classifiers use a softmax function, which converts the network’s raw outputs into a set of values that sum to 1.0:

P(classi)=exp(zi)∑j=1Kexp(zj)

The reported confidence score is simply the maximum value within that normalized distribution. Because the values are constrained between 0 and 1, they look like probabilities. In reality, they express nothing more than a relative ranking among the specific classes the model knows. The softmax operation forces this ranking whether or not the true material is one of them.

Consider a simple analogy: take a model trained only to differentiate cats from dogs, and show it an image of a giraffe. The network will not hesitate. It may answer “dog” at 99% confidence. What we would expect from a sensible model is something near a 50/50 split meaning “I have never seen this”; that is exactly what a standard softmax cannot produce. It must distribute all its probability mass over the classes it knows, and it does so with manufactured certainty.

On the line, the same thing happens. When a fragment belongs to an alloy grade absent from the training set, the model reports a high value for the closest known class, and the operator reads it as a probability of being that grade. This illusion of certainty is dangerous. Once an operator discovers it, trust in the system collapses. Not because the model is inaccurate, but because its confidence was never trustworthy to begin with.

2. Miscalibration is measurable

Stating that confidence does not equal probability is straightforward. The practical question is whether that gap can be measured systematically. It can.

A model is considered well-calibrated when its stated confidence matches its observed accuracy. Over a large sample set, among all predictions made at 70% confidence, roughly 70% should be correct. Weather forecasting provides a clear operational parallel: if a meteorologist forecasts a 10% chance of rain across 100 days, and it rains on 50 of them, the forecast is not merely inaccurate; it is miscalibrated. The system systematically misrepresents its own uncertainty.

That example is worth keeping in mind, because miscalibration runs in two directions:

Real models rarely sit cleanly in one camp. A trained detector is often underconfident on its low-confidence predictions and overconfident on its high-confidence ones at the same time, which is exactly why calibration must be measured rather than assumed.

To evaluate this behaviour, the standard tool is the reliability diagram. Predictions are grouped into confidence bins. For each bin, the average predicted confidence is plotted against the observed accuracy. In a perfectly calibrated model, the curve follows the diagonal axis. Points above it indicate underconfidence; points below indicate overconfidence. The shape of the curve tells you which problem you have.

The summary metric for this discrepancy is the Expected Calibration Error (ECE), which calculates the weighted average gap between confidence and accuracy across all bins:

ECE=∑m=1M|Bm|n|acc(Bm)−conf(Bm)|

with:

acc(Bm)=1|Bm|∑i∈Bm1(y^i=yi)
conf(Bm)=1|Bm|∑i∈Bmp^i

A perfectly calibrated model has an ECE near zero. An off-the-shelf detector trained on real waste is typically far from it, and that gap is far from harmless when those numbers drive an ejection decision.

The root cause is structural. Standard training objectives penalize wrong predictions; they do not penalize miscalibrated ones. Two models with identical accuracy can have completely different reliability curves (Guo et al., 2017).

Reliability diagram comparing confidence and accuracy
Source: adapted from Figure 5 in Hu et al. (2023), licensed under CC BY 4.0.

3. Why early results are misleading

There is a moment in every AI sorting project where everything seems to work. You collect a few thousand pieces, train on a part of them, and test on the rest. Accuracy is high, confidence scores look reasonable, and the natural conclusion is: “It works!”

This deserves caution. Waste streams have enormous intrinsic variability, and an initial dataset rarely covers it. The pieces collected in one week, from one supplier, under one set of surface conditions, are a narrow slice of what the line will see over months. Class distributions in that slice rarely match the bulk stream either: rare but valuable materials are underrepresented, and the model drifts toward the majority class while remaining confident about it.

The problem amplifies with time. Calibration learned during training assumes the data distribution stays stable, and in production it never does: suppliers change, surface conditions change with the season, and sensors age. A model well-calibrated in April can be poorly calibrated by July. By tracking calibration through continuous ECE monitoring, this invisible drift becomes a quantitative metric that can be acted on.

4. Correcting calibration after training

The good news is that once miscalibration is measured, it can be corrected, and the most effective method is also the simplest.

Temperature scaling (Guo et al., 2017) divides the network’s raw outputs by a single scalar T before the softmax normalization:

p^i=exp(zi/T)∑j=1Kexp(zj/T)

That symmetry is what makes it the most adaptable correction available: it handles miscalibration in either direction, requires no retraining, adds a single parameter, and costs essentially nothing at inference. In the comparative studies that established it, it consistently removes a large share of the calibration gap while leaving accuracy untouched.

Finding an optimal T is less trivial than it looks. The obvious approach is to choose the T that minimizes ECE on a held-out set, but ECE relies on hard binning and an absolute value, which makes it non-differentiable and impossible to optimize by gradient descent. The workaround is implementing a smooth, differentiable surrogate in which each prediction contributes to neighbouring bins through a soft kernel rather than landing in exactly one. T can then be optimized directly.

A single global T value may prove insufficient in scrap streams that feature extreme class imbalances. Implementing class-specific temperature parameters yields significantly better calibration across both majority and minority fractions (Liu et al., 2022).

Effect of the temperature parameter on a softmax distribution
Source: adapted from Figure 2 in Maracani et al. (2024), licensed under CC BY 4.0.

5. Correcting it at the source, and combining both

Post-hoc correction fixes the symptom. Calibration can also be embedded directly into the training process itself.

One common intervention is label smoothing (Müller et al., 2019), which replaces rigid zero-and-one training targets with softened ones. The model is no longer rewarded for pushing its outputs to absolute certainty. While effective for suppressing overconfidence, applying label smoothing to a model that naturally leans underconfident can worsen calibration.

A more direct approach incorporates calibration into the objective function. By segmenting model outputs into a four-quadrant matrix (accurate vs. inaccurate against certain vs. uncertain), the loss function specifically penalizes the high-confidence, incorrect predictions that lead directly to contaminated sorting fractions:

CertainUncertain
AccurateDesirableMiscalibrated
InaccurateMiscalibrated (costly error)Desirable

In practice, these methodologies are complementary. Training the model to be honest about its own uncertainty, then correcting whatever systematic bias remains at inference, gives a better-calibrated system than either approach alone.

One point deserves emphasis for an industrial reader: calibration is not a trade-off against accuracy. The model detects the same objects. It simply stops overstating how sure it is.

6. Knowing what you don’t know

A well-calibrated confidence score is a much better metric, but it is still just one number, and it still cannot say “this piece is unlike anything I was trained on.” Resolving this requires explicit Out-of-Distribution (OOD) detection: evaluating how far a new input lies from the learned data manifold, rather than asking the classifier how sure it is.

Reconstruction metrics via Variational Autoencoders (VAEs). An autoencoder is trained solely on representative in-distribution material. It compresses each input into a compact latent representation and reconstructs it, and in doing so it learns what “normal” looks like for that stream. At inference, a piece that resembles the training data is reconstructed faithfully; an unfamiliar one is not. The deviation, measured through reconstruction error, through reconstruction probability (An & Cho, 2015), or through a distance in latent space, becomes an outlier score that is entirely independent of the classifier’s opinion. A new alloy, an unexpected contaminant, or a material the line has never processed can be flagged as outside the known distribution rather than forced into the nearest known class. Importantly, because raw likelihoods from deep generative models can occasionally assign high values to unfamiliar data (Nalisnick et al., 2019), robust implementations rely on reconstruction probability and latent space distance metrics rather than raw likelihood alone.

Architecture of a variational autoencoder: encoder, latent space, decoder
Source: architecture of a Variational Autoencoder (VAE) showing the encoder, latent bottleneck parameters, and decoder reconstruction. Van Otten (2023), Spot Intelligence.

Model uncertainty. Running stochastic forward passes via Monte Carlo (MC) Dropout (Gal & Ghahramani, 2016) assesses internal model stability. If repeated passes over the same input generate divergent class predictions, epistemic uncertainty is high, signaling that the network lacks the structural knowledge to classify the piece reliably. Ensembles behave similarly: when several independently trained models vote and only some agree, that disagreement is valuable information.

Conformal prediction sets. Rather than a point estimate with a confidence attached, conformal methods output a set of candidate classes carrying a statistical coverage guarantee, calibrated on a held-out set and adaptive to how difficult each input is (Bethell et al., 2024). Easy pieces get a single answer; ambiguous ones get a set. This provides the exact operational trigger needed to route an object to a secondary analytical step rather than guessing.

In practice, an outlier flagged as an outlier costs a single pick cycle. An outlier classified with misplaced confidence costs a contaminated batch.

7. The Matvision approach

This calibration discipline defines both our engineering philosophy and the physical architecture of how we work with recyclers and build sorting machines.

When a recycler comes to us with a new material stream, our role is not to promise the moon. It is to be objective about what field reality allows. We run structured test campaigns on representative material and report the performance the system will actually deliver on that stream, not the performance a small, flattering dataset would suggest. Announcing a realistic recovery rate and purity, then achieving it in production, is worth more than an impressive number that collapses three months in.

This framework is built into the core software pipeline of our sorting machine: resolving the gap between confidence and correctness, tracking the need to measure calibration rather than assume it, accounting for the drift that comes with real waste streams, and deploying the capability to flag what the system does not recognise. Not as a feature added at the end, but as part of what makes a sorting decision trustworthy in the first place.

References

First published on 12 October 2026 in Sorting Tomorrow, the Matvision newsletter on LinkedIn.