The recent claim that top AI systems like GPT and Claude failed Bridgewater’s finance tests is less about model capability and more about the nature of the data these tests require.
Bridgewater and Thinking Machines Lab highlight a finely tuned open-weight model outperforming the leading AI models when evaluating financial documents. The catch? The correct answers for these tests aren’t public, meaning that the systems GPT and Claude rely on simply cannot be trained or fine-tuned with that specific target data.
This is a critical nuance often lost in the enthusiasm for off-the-shelf solutions. Most large AI models are trained broadly on public data but struggle in domains where the ground truth is proprietary or closely guarded, like high-stakes finance. Bridgewater’s approach underscores that domain expertise combined with targeted, transparent data dramatically changes outcomes—not just raw compute or scale of models.
So, the headline that these models “failed” is misleading. The right framing is that without access to the actual answers, no model—no matter how powerful—is going to get financial document comprehension right. This signals a push back towards open-weight models and custom tuning as more effective routes for specialised industries, rather than relying on generalist AI behemoths trained on public data alone.
In other words, unlocking real AI value in finance isn’t about model size. It’s about relevance of data and openness of training sets.
Leave a Reply