There are many areas where AI might appear to do great, but on deeper inspection it's actually awful. Forecasting is one of those areas and this is that deeper inspection.
Two totally different things I think may come together to provide some insight here.
Years ago some neonazis online started putting the names of Jewish people in (((multiple parentheses))). Called an 'echo', the idea would be during this normal period they would casually retweet or comment on the work of journalists or writers but add the little flag if they were Jewish. At some point in the future when their revolt or seizure of power would happen, they could Google the multiple marks to instantly get a list of 'known Jews'.
Also years ago George Soros wrote multiple books pushing his idea of Reflexivity. In a nutshell it is that predictions alter the underlying reality. His prediction machine was not an AI or LLM but the financial markets....which is kind of the whole idea of AI since even the largest players in the markets are only drops in the ocean.
His idea never really caught on, like a lot of people who try to create a new theory by pushing it as a popular audience book, it wasn't clear that his work could both be really unique and stand up to falsification (Among other such failures IMO are Jordan Peterson, Stephen Pinker, Jared Diamond but in this modern sort attention span age you don't even need a book, Eric Weinstein has had a 'theory of everything' for years now no one is allowed to see).
What is interesting about these two roads is:
1. People attempting to exploit an algorithm or type of AI for their own ends. That would be the neonazis who assume Google would just continue to run as always after they get power as if it was some NPC in a video game*.
2. The 'prediction engine’ altering the outcomes of the reality it is trying to predict. In the case of bubbles, this is hard to ignore. Your bank views your capacity to borrow by your home value plus your credit score in most cases. Rising home prices mean you become more worthy of lending. This alters the reality around you. The lumber stores in your area are more likely to increase in value if you're more able to build that mega deck you've always dreamed about. Or perhaps crypto is more valuable if your inclination is to borrow and play the markets. You can be a very prudent person and avoid borrowing even though you could walk out of your bank with a check for $50K anytime you want in less than a hour. But will all your neighbors also follow?
This means a system of 'super forecasters' who are either people or AI that we listen to will also become a group that will be gamed. It also means their predictions will generate feedback loops that alter reality. That may sound mystical, but it isn't, almost all the predictions you are interested in are essentially human behavior and since you want to get these forecasts, you are proposing humans alter their behavior by tapping into them...that's fundamentally different than a group of super forecasters making predictions about what new particles will come from a new collider versus different lengths of school closures or tax policies.
Perhaps you need to consider adding a bit of Asimov to this idea. Remember in his Foundation Series the ‘super forecasting’ math required that the subject population could not be told of the predictions. In fact, I think they weren’t even allowed to know someone was out there making predictions about them.
* The 'echo' game got gamed on its own. People countered by putting their own names in parenthesis to show solidarity, others put the nazis' names in so they would come up as well on any 'purge list'. Google almost certainly took note of the attempt to tweak search results and acted as well. A super forecaster who was charged with making predictions about what would happen if Nazis got power again would have to know about the echo trend. But if he did why wouldn't others also know about it and their efforts to counter it would frustrate an ability to predict that aspect of the question.
I think we're already seeing much of what you describe in prediction markets. Look at how much the odds of a Trump victory have increased, based on what? A slight tightening of the polls? I know there's a claim that it was whales, but at some point it should balance out, but I think we definitely see a herding effect, which previously caused the prediction markets to miss Brexit and Trump in 2016. (Though the latter was more forgivable.)
Indeed, the financial markets are so much more powerful and deep. Yet were we not told by them AMC and Gamestop would dominate our economy today? For prediction markets at least one whale has been identified, plus you got the problem that some of them do not allow Americans to trade unless they are willing to go deep into making crypto wallets and hiding their location.
I think a superforecaster who could tell us how much damage a Trump victory would do ten years in the future is a lot more helpful than one who can refine 60% odds to 55.5% or whatnot. In both cases, though, I have no idea how we are supposed to verify the alleged superforecaster is all that super?
Asimov I think had the right idea. Make alleged superforecasters encrypt their predictions only to be unveiled years later. Perhaps some type of blockchain like mechanism can do this. You could then promptly kill them after they made their entries. Years later we can observe who had the better method.
It kind of feels like we are trying to hammer a nail by using our forehead with this whole AI/LLM stuff. Isn't what we really want are actual methods of predictions that can be picked up and used over and over again? Newton's equations to tell us how planets orbit rather than a trillion datapoints churning out a digital feeling?
In a certain sense this is actually a step backwards where we gather around an Oracle of Delphi hoping for useful assertions from a literal black box.
Two totally different things I think may come together to provide some insight here.
Years ago some neonazis online started putting the names of Jewish people in (((multiple parentheses))). Called an 'echo', the idea would be during this normal period they would casually retweet or comment on the work of journalists or writers but add the little flag if they were Jewish. At some point in the future when their revolt or seizure of power would happen, they could Google the multiple marks to instantly get a list of 'known Jews'.
Also years ago George Soros wrote multiple books pushing his idea of Reflexivity. In a nutshell it is that predictions alter the underlying reality. His prediction machine was not an AI or LLM but the financial markets....which is kind of the whole idea of AI since even the largest players in the markets are only drops in the ocean.
His idea never really caught on, like a lot of people who try to create a new theory by pushing it as a popular audience book, it wasn't clear that his work could both be really unique and stand up to falsification (Among other such failures IMO are Jordan Peterson, Stephen Pinker, Jared Diamond but in this modern sort attention span age you don't even need a book, Eric Weinstein has had a 'theory of everything' for years now no one is allowed to see).
What is interesting about these two roads is:
1. People attempting to exploit an algorithm or type of AI for their own ends. That would be the neonazis who assume Google would just continue to run as always after they get power as if it was some NPC in a video game*.
2. The 'prediction engine’ altering the outcomes of the reality it is trying to predict. In the case of bubbles, this is hard to ignore. Your bank views your capacity to borrow by your home value plus your credit score in most cases. Rising home prices mean you become more worthy of lending. This alters the reality around you. The lumber stores in your area are more likely to increase in value if you're more able to build that mega deck you've always dreamed about. Or perhaps crypto is more valuable if your inclination is to borrow and play the markets. You can be a very prudent person and avoid borrowing even though you could walk out of your bank with a check for $50K anytime you want in less than a hour. But will all your neighbors also follow?
This means a system of 'super forecasters' who are either people or AI that we listen to will also become a group that will be gamed. It also means their predictions will generate feedback loops that alter reality. That may sound mystical, but it isn't, almost all the predictions you are interested in are essentially human behavior and since you want to get these forecasts, you are proposing humans alter their behavior by tapping into them...that's fundamentally different than a group of super forecasters making predictions about what new particles will come from a new collider versus different lengths of school closures or tax policies.
Perhaps you need to consider adding a bit of Asimov to this idea. Remember in his Foundation Series the ‘super forecasting’ math required that the subject population could not be told of the predictions. In fact, I think they weren’t even allowed to know someone was out there making predictions about them.
* The 'echo' game got gamed on its own. People countered by putting their own names in parenthesis to show solidarity, others put the nazis' names in so they would come up as well on any 'purge list'. Google almost certainly took note of the attempt to tweak search results and acted as well. A super forecaster who was charged with making predictions about what would happen if Nazis got power again would have to know about the echo trend. But if he did why wouldn't others also know about it and their efforts to counter it would frustrate an ability to predict that aspect of the question.
I think we're already seeing much of what you describe in prediction markets. Look at how much the odds of a Trump victory have increased, based on what? A slight tightening of the polls? I know there's a claim that it was whales, but at some point it should balance out, but I think we definitely see a herding effect, which previously caused the prediction markets to miss Brexit and Trump in 2016. (Though the latter was more forgivable.)
Indeed, the financial markets are so much more powerful and deep. Yet were we not told by them AMC and Gamestop would dominate our economy today? For prediction markets at least one whale has been identified, plus you got the problem that some of them do not allow Americans to trade unless they are willing to go deep into making crypto wallets and hiding their location.
I think a superforecaster who could tell us how much damage a Trump victory would do ten years in the future is a lot more helpful than one who can refine 60% odds to 55.5% or whatnot. In both cases, though, I have no idea how we are supposed to verify the alleged superforecaster is all that super?
Asimov I think had the right idea. Make alleged superforecasters encrypt their predictions only to be unveiled years later. Perhaps some type of blockchain like mechanism can do this. You could then promptly kill them after they made their entries. Years later we can observe who had the better method.
It kind of feels like we are trying to hammer a nail by using our forehead with this whole AI/LLM stuff. Isn't what we really want are actual methods of predictions that can be picked up and used over and over again? Newton's equations to tell us how planets orbit rather than a trillion datapoints churning out a digital feeling?
In a certain sense this is actually a step backwards where we gather around an Oracle of Delphi hoping for useful assertions from a literal black box.