Asymmetry and Interdependence when Evaluating U.S. Energy Information Administration Forecasts

Main points

• When evaluating US Energy Information Agencies (EIA) forecasts of the world petroleum market, it is important to take a multivariate perspective, considering asymmetric loss and allowing for time-variation.
• Forecasts for total demand, total supply, total stock withdrawals and the oil prices are biased, with biases that change over time and differ across variables.
• A loss function that takes into account asymmetry and interdependence can rationalise these biases.
• The implied asymmetric loss gives less weight to under-prediction of both demand and supply, while for oil prices, we document significant regime changes in the implied loss due to asymmetry.
• The EIA forecasts dominate a simple random walk benchmark when evaluated using symmetric and independent loss in the form of MSE statistical criteria.
• Yet, when allowing for asymmetry and interdependence that rationalize the EIA forecasts, the performance of the EIA forecasts worsens and is comparable to the random walk benchmark.

We evaluate US Energy Information Agencies (EIA) forecasts of the world petroleum market, emphasising the importance of joint evaluation with different costs of under and over predicting and also how these change over time. Forecasts for total demand, total supply, total stock withdrawals and the oil prices are biased, with biases that change over time and differ across variables. A loss function that takes into account over and under predicting and the interdependence of the forecasts can rationalise these biases.