When Jev is 0.99 sure it is right; below 0.8 it is a coin flip
Jev 1.13 on Banking77, 308 messages, grouped by the probability of its top choice. Bar: share of answers that were correct.
top probability 0.99 to 1.00
98% correct| 149 messages (48%)
top probability 0.95 to 0.99
81% correct| 37 messages (12%)
top probability 0.90 to 0.95
77% correct| 30 messages (10%)
top probability 0.80 to 0.90
75% correct| 24 messages (8%)
top probability 0.70 to 0.80
52% correct| 21 messages (7%)
top probability 0.50 to 0.70
40% correct| 35 messages (11%)
top probability below 0.50
42% correct| 12 messages (4%)
Black tick: the mean probability Jev gave in that band. A calibrated model puts the tick at the end of the bar.