Large language models, like those powering ChatGPT, are known for their unpredictable outputs. Ask the same question multiple times, and you might get a range of answers, from slightly rephrased to entirely different. This variability, or nondeterminism, stems from the complex interplay of GPU kernels—small programs running on Nvidia’s chips during inference, the process that generates AI responses. According to Horace He, a researcher at Thinking Machines Lab, these kernels introduce randomness when stitched together, leading to inconsistent results.
This unpredictability poses challenges for industries relying on AI, such as finance, healthcare, and scientific research, where consistent outputs are critical. For example, a medical diagnostic tool that provides varying interpretations of the same data could undermine trust and reliability. Thinking Machines Lab’s research aims to tackle this issue at its root, focusing on the orchestration layer of GPU processing.
A Path to Deterministic AI
The lab’s approach centers on controlling the orchestration of GPU kernels to minimize randomness. By fine-tuning how these kernels interact, Thinking Machines Lab believes it can make AI models more deterministic, ensuring that identical inputs yield identical outputs. This could lead to more reliable responses for enterprise applications, such as customer service chatbots or automated data analysis, where consistency is paramount.
Beyond enterprise use, deterministic AI could enhance reinforcement learning (RL), a method that trains models by rewarding correct answers. Inconsistent outputs create “noisy” data, complicating the RL process. By smoothing out these variations, Thinking Machines Lab’s work could make training more efficient, potentially accelerating advancements in AI capabilities.
Thinking Machines Lab’s Broader Mission
Founded in February 2025, Thinking Machines Lab has quickly gained attention, raising $2 billion in a seed round led by Andreessen Horowitz at a $12 billion valuation. The startup, led by Murati as CEO, with OpenAI co-founder John Schulman as chief scientist and Barret Zoph as CTO, aims to make AI systems more customizable and widely understood. Their mission emphasizes building multimodal AI that collaborates with users across domains like science and programming, addressing gaps in current AI systems that are often opaque and difficult to tailor.
The lab’s research blog, Connectionism, launched alongside the nondeterminism study, signals a commitment to transparency. By sharing insights into frontier AI systems, Thinking Machines Lab hopes to foster broader discourse and empower users to leverage AI effectively. Their open-source ethos, hinted at in earlier announcements, suggests future releases could benefit the wider research community.
Industry Implications and Challenges
The pursuit of consistent AI aligns with industry trends, as companies like Anthropic and Google DeepMind also explore ways to make models more predictable. However, Thinking Machines Lab’s focus on GPU kernel orchestration is a novel angle, potentially setting it apart from competitors. If successful, their approach could reduce the computational overhead of running LLMs, making AI more accessible to smaller organizations without massive data centers.
Yet, challenges remain. The lab’s $12 billion valuation places high expectations on its ability to deliver tangible products. Developing deterministic AI requires not only technical breakthroughs but also practical applications that justify the investment. Additionally, competing with established players like OpenAI and Google DeepMind, which are pouring billions into their own research, demands rapid innovation.
The Road Ahead
Thinking Machines Lab’s work on nondeterminism is a promising step toward more reliable AI. By addressing the root causes of randomness, the startup could unlock new possibilities for industries and researchers alike. Their focus on transparency and collaboration, combined with a stellar team of former OpenAI researchers, positions them as a formidable player in the AI landscape. As they prepare to unveil their first product in the coming months, the industry watches closely to see if Thinking Machines Lab can live up to its ambitious vision.
