Going Down the Stack: AI Inference, Kernel Fusion, and Model Tuning with Paul Brookes
In this MonkCast conversation, RedMonk's James Governor talks with Paul Brookes, a senior AI engineer at TurinTech, about making AI inference faster and cheaper. TurinTech predates the generative AI boom, having spent years optimizing complex code with genetic algorithms, and now points those tools at the models themselves. Brookes walks through techniques like kernel fusion and model compilation that squeeze more tokens per second out of specific hardware, drawing on the company's work with Intel on OpenVINO and vLLM. The two get into running capable open models such as Qwen on local machines, the spiraling cost of AI , and why judging engineers by tokens burned misses the point. Brookes also describes Artemis and its discovery harness, which lets agents learn from past results, and traces his own route from quantum physics into low-level performance engineering.This RedMonk conversation is sponsored by TurinTech.Show notes: https://redmonk.com/videos/paul-brookes/Chapters00:00 Introduction to AI and TurinTech01:42 Optimization Challenges in AI04:30 Working with Semiconductor Companies08:46 The Cost of AI and Local Model Deployment11:29 Local Model Performance and Infrastructure14:44 Partnerships and Breakthroughs with Intel16:54 Key Techniques for Optimizing Inference19:39 The Role of AI Engineers21:50 Bridging the Gap with Customers23:21 Harnessing AI for Continuous Improvement
