Calibration of probability predictions from machine‐learning and statistical models

摘要: AIM: Predictions from statistical models may be uncalibrated, meaning that the predicted values do not have nominal coverage probability. This is easiest seen with probability predictions in machine‐learning classification, including common species occurrence probabilities. Here, a of, say, .7 should indicate out of 100 cases these environmental conditions, and hence same probability, present 70 absent 30. INNOVATION: A simple calibration plot shows this necessarily case, particularly for overfitted or algorithms use non‐likelihood target functions. As consequence, ‘raw’ such model could easily off by .2, are unsuitable averaging across types, resulting maps substantially distorted. The solution, flexible regression, can applied whenever deviations observed. MAIN CONCLUSIONS: ‘Raw’, uncalibrated calibrated before interpreting them probabilistic way.