English-language podcasts

Linear Digressions

GPU Architecture for LLM Inference: Memory, Bandwidth and Hardware Explained

20 min

Listen through the publisher, then discuss the episode here or arrange an in-person Roundtable. Your city determines where to meet.

Roundtables are designed for groups of four, with a shared subject and room for everyone to speak. Come for discussion, with no dating expectation.

GPUs are the engines powering every AI conversation you've ever had — but what's actually happening inside those chips when an LLM generates a response? Understanding GPU fundamentals turns out to be surprisingly useful for making sense of why inference works the way it does, from the prefill stage that processes your entire prompt at once to the serial decode step that produces tokens one by one. If you want to understand the real constraints shaping modern AI systems, this is where to start.

Episode information is provided by the publisher. Listening takes place on the publisher’s service.

Listen at the source ↗

Featured places to meet in Palo Alto, CA

Explore these places and arrange a conversation through ThinkMatch.

City guide for Palo Alto, CA

More from Linear Digressions

Linear Digressions

How to Stay Relevant as a Data Scientist in the Age of AI | Interview with Dr. Jon Krohn

44 min

Staying relevant as a data scientist when AI is reshaping the field isn't just a career question — it's an existential one. Dr. Jon Krohn, neuroscientist-turned-data-science-educator, podcaster, and consultant, sits down to share how he thinks about navigating that challenge. From his unconventional path out of academia to the advice he gives practitioners trying to future-proof their careers, this conversation is packed with hard-won perspective. If you've ever wondered what it takes to keep growing in a field that won't stop moving, this one's for you. Links: Jon Krohn's website Y Carrot (Jon's AI consultancy) Katie on the Super Data Science Podcast, episode 1029 METR (source of the "task length doubles every seven months" charts Jon cited) Artificial Analysis (AI benchmarking site Jon cited) Serg Masís, "The Podcast Pipeline Nobody Sees: Behind the Scenes with AI Agents and AI gen" (Build Stuff) Search Engine podcast, "One man could slow down AI. He won't. Why?" --- Website: https:/

Discuss this episode

Linear Digressions

KV Cache Fundamentals

25 min

The unsung hero of every LLM conversation you've ever had? A clever optimization called the KV cache. Self-attention is computationally brutal — every token attending to every other token means costs that scale as N², and that blowup gets worse with every turn of a conversation. The KV cache is what keeps that from becoming completely intractable at scale. Today's episode goes deep on how it works, why inference is a harder and more interesting problem than it gets credit for, and what's really happening under the hood when you chat with an LLM.

Discuss this episode

Community Poll

Classroom technology should prove educational value before becoming standard.

Share your view, then see how readers voted. No sign-in required. This poll is shared across stories.

Do you agree or disagree?

Choose one answer

Common Quandary

A Common Quandary to discuss

Another conversation to explore, separate from this news story.

Your coworker bends a dumb rule to help people. Do you report it?

An underpaid coworker quietly bends a company rule to help colleagues receive benefits they were technically denied. No customer is harmed, but the records are inaccurate and you are asked to certify them.

Where should loyalty end when a rule seems unjust?

Explore this quandary