Every model rated itself higher than the humans did

For my bachelor thesis I built a Figma plugin that reads interface frames and names the deceptive patterns in them. Behind it sits a study: over 12 leading language models against rule-based systems. The language models won.

Then I read how sure each model was of its own answers. Every one of them rated itself higher than the human evaluators did.

The gap, model by model

Average rating from 0 to 1, by humans and by each model itself
Model	Human rating and the model's own rating	Gap
GPT-4.1	Human 0.783, model 0.923	+0.140
GPT-4o	Human 0.741, model 0.893	+0.152
o1	Human 0.708, model 0.875	+0.167
Claude 3.7 Sonnet	Human 0.658, model 0.860	+0.202

Fig. 1 · Average rating on a scale of 0 to 1 · Thesis fig. 8, p. 24

Fig. 1 · Average rating, 0 to 1 · Source: thesis fig. 8, p. 24

Average rating from 0 to 1, by humans and by each model itself
Model	Human rating	Model's own rating	Gap
GPT-4.1	0.783	0.923	+0.140
GPT-4o	0.741	0.893	+0.152
o1	0.708	0.875	+0.167
Claude 3.7 Sonnet	0.658	0.860	+0.202

The gap widens as the quality of the answer drops.

Overconfidence persisted regardless of output quality.

Across 147 scored responses the average human score was 5.9 out of 10. 39 of them reached 8 or more, 28 scored 4 or less.

That is why the plugin never gives a bare verdict. It shows a score from 0 to 10, the evidence behind it, and a plain note that the AI can be wrong.