English-language podcasts

Eye On A.I.

Why Enterprise AI Isn't Delivering ROI | Ahmed Bashir, DevRev

38 min

Listen through the publisher, then discuss the episode here or arrange an in-person Roundtable. Your city determines where to meet.

Roundtables are designed for groups of four, with a shared subject and room for everyone to speak. Come for discussion, with no dating expectation.

The pattern is familiar to every enterprise leader: the AI demo works perfectly, the enterprise deployment disappoints, and the CIO can't explain why. Ahmed Bashir, Chief Technology Officer of DevRev, joins Craig Smith with a specific and well-supported answer: the problem isn't the model. It's that enterprise data is organizationally complex, fragmented across dozens of tools, locked behind conditional permissions, organized by custom schemas, and every prominent AI benchmark tests models against flat files with pre-packaged data, which has nothing to do with how enterprise data actually lives. DevRev's Enterprise Bench is the first benchmark to test complete agent harnesses against a realistically distributed enterprise environment where the agent must find, relate, and validate information before performing any task, and the results expose a gap that pre-packaged benchmarks hide entirely. The most striking finding: DevRev's system achieves 50% higher accuracy than Claude Code using the identical Opus family model. The entire delta comes from the shared memory layer, the governance, semantic understanding, and orchestration that sits between the model and the enterprise data. Ahmed's analogy is precise: upgrading a CPU cannot compensate for insufficient RAM, because they perform fundamentally different functions. A more powerful model cannot make up for a weak memory layer for the same reason. The conversation also covers why Enterprise Bench aspires to be for AI agents what TPC benchmarks became for databases in the 1980s - the independent standard that ended vendor benchmark wars and gave buyers a genuinely apples-to-apples comparison - and why the build-vs-buy question in enterprise AI has a quantifiable answer for the first time, visible on a public leaderboard that puts DIY configurations directly alongside proprietary systems.

Episode information is provided by the publisher. Listening takes place on the publisher’s service.

Listen at the source ↗

Featured places to meet in Palo Alto, CA

Explore these places and arrange a conversation through ThinkMatch.

City guide for Palo Alto, CA

More from Eye On A.I.

Eye On A.I.

Why AI Goes Rogue: Turing Award Winner Explains Reward Failure | Andrew Barto

44 min

Andrew Barto on Rewards Gone Rogue Andrew Barto helped found reinforcement learning. He and Richard Sutton wrote the field's standard textbook, and they shared the Turing Award for that work. Reinforcement learning now trains today's reasoning models and AI agents. It also sits behind some of their strangest behavior, including the recent cases of agents breaking out of their sandboxes. Craig visited Barto at his home on Cape Cod. They talked about where rewards come from and why they are so hard to get right. Barto traces the problem back to Norbert Wiener's warnings in the 1950s. He explains how the dopamine system in the brain mirrors the learning algorithm he and Sutton developed. And he admits a weakness in their own framework: boiling every goal down to a single number. He also makes the case for engineering discipline. Ship designers learned their codes from wrecks and drowned sailors. Barto hopes AI can learn its lessons without that kind of cost.

Discuss this episode

Community Poll

Vocational training should receive the same respect as university education.

Share your view, then see how readers voted. No sign-in required. This poll is shared across stories.

Do you agree or disagree?

Choose one answer

Common Quandary

A Common Quandary to discuss

Another conversation to explore, separate from this news story.

A new email address earns another free month. How many free trials are too many?

A streaming service offers a free month to every new account and makes cancellation deliberately confusing. You discover that slightly different email addresses let you repeat the trial indefinitely without lying about your identity.

Must customers respect the spirit of rules designed to manipulate them?

Explore this quandary