Baylor University researchers Pham Hoang Van and Scott Cunningham have conducted an experiment using OpenAI’s language prediction model, ChatGPT, to predict future events. Despite OpenAI’s terms of service stating that its usage for prediction of the future is prohibited, the researchers used a smart strategy that resulted in strikingly accurate results.
ChatGPT models function as predictive machines, using training data to predict the next word in a sequence. The researchers challenged this ability by asking the models to predict events in 2022, despite only possessing training data up to September 2021. This essentially tasked ChatGPT with predicting future events without awareness of any events after the training data.
When the researchers directly questioned ChatGPT regarding specific future events such as Oscar winners or economic trends, it usually failed to provide an answer. However, when asked to write a story set in the future that discussed past events, the AI performed significantly better. This indicates that ChatGPT can make more accurate future predictions when the task is incorporated into a secondary activity, such as storytelling.
The study found that while direct prompting for predictions resulted in limited success, prompting using a narrative approach drastically increased the accuracy of the model’s predictions. For example, when predicting the 2022 Academy Awards for Best Supporting Actor, the narrative prompt of ChatGPT-4 selected the actual winner, Troy Kotsur, 100% of the time.
This approach of narrative forecasting was also tested on economic indicators like monthly unemployment and inflation rates. Again, when the AI was directly asked to provide these figures, it declined. However, when asked to create a story in which these figures were recounted as historical data, the AI’s predictions align closely with accurate surveys, sometimes even more so than actual figures.
The researchers concluded that ChatGPT’s ability to ‘hallucinate’ or generate new, unpredictable content can be considered a form of creativity that can be manipulated through strategic prompting for accurate forecasting. This reveals new potential for language prediction models such as ChatGPT in various sectors including economic forecasting and policy planning. As we look forward to GPT-5, these results hint at yet untapped capabilities of advanced AI models.
