The Comparison and Evaluation of Forecasters.

摘要: In this paper we present methods for comparing and evaluating forecasters whose predictions are presented as their subjective probability distributions of various random variables that will be observed in the future, e.g. weather who each day must specify own probabilities it rain a particular location. We begin by reviewing concepts calibration refinement, describiing relationship between notion refinement sufficiency comparison statistical experiments. also consider question interrelationships among discuss which an observer should combine from two or more different forecasters. Then turn our attention to concept proper scoring rule forecasters, relating refinement. Finally, conditions under one fore- caster can exploit another forecaster obtain better score. describe some appropriate com- paring repeatedly whether not events occur terms those events. The ideas here relevant almost any situation make such probabilistic predictions, regardless subject matter substantive area being forecast. might economist at beginning quarterly period makes about unemployment, rate inflation, Gross National Product quarter based on values economic indicators; even using large-scale econometric model United States economy hundreds relations. field, weatherman television station announce his during day. For ease exposition, discussions context after x there least certain amount given location specified time interval refer occurrence well-specified event simply "rain". Thus, begin- ning end he observes actually occurred. is called prediction shall realistic, simultaneously simplifying, assumption