May 17, 2025
Summary
In this episode, Andy talks with Dr. Joe Sutherland, co-author of the new book Analytics the Right Way: A Business Leader's Guide to Putting Data to Productive Use. Joe is a leader in AI policy and practice, serving as the founding director of the Emory Center for AI Learning and lead principal investigator for the U.S. AI Safety Institute Consortium.
Andy and Joe explore what it really takes to make better decisions in a world drowning in data and exploding with AI hype. They discuss the myths of data collection, how randomized controlled trials and causal inference impact decision quality, and Joe’s “two magic questions” that help project managers stay focused on outcomes. They also dive into recent AI breakthroughs like DeepSeek, and why executives may be paralyzed when it comes to implementing AI strategy.
If you're looking for insights on how to use data and AI more effectively to support leadership and project decision-making, this episode is for you!
Sound Bites
- “What are we trying to achieve? And how would we know if we achieved it?”
- “Sometimes we’re measuring success by handing out coupons to people who already had the product in their cart.”
- “AI doesn’t replace decision-making—it demands better decisions from us.”
- “Causality is important for really big decisions because you want to know with a level of certainty that if I make this choice, this outcome is going to happen.”
- “Too often, we make decisions based on bad causal inference and wonder why the outcomes don’t match our expectations.”
- “The ladder of evidence helps you decide how much certainty you need before making a decision—and how much it’ll cost to climb higher.”
- “The truth is, we’re not ready for human-out-of-the-loop AI—we’re barely asking the right questions yet.”
- “Leadership isn’t about replacing people with AI. It’s about using AI to make your people more productive and happier.”
- “We're starting to see some evidence that when you use large language models in education, test scores go up in excess of 60%.”
- “This may be the first time the kids feel more behind than the parents when it comes to a new technology.”
Chapters
- 00:00 Introduction
- 02:00 Start of Interview
- 02:09 What Are Some Myths About Data?
- 03:49 What Is the Potential Outcomes Framework?
- 08:50 What Are Counterfactuals?
- 13:00 How Do You Personally Evaluate Causality?
- 18:22 What Are the Two Magic Questions for Projects?
- 20:45 What's Getting Traction From the Book?
- 24:26 What Can We Learn From DeepSeek's Disruption?
- 27:30 Human In or Out of the AI Loop?
- 30:41 How Joe Uses AI Personally and Professionally
- 33:33 What Is the Future of Agentic AI?
- 35:37 Will AI Replace Jobs?
- 37:18 How Can Parents Prepare Kids for the AI Future?
- 41:19 End of Interview
- 41:46 Andy Comments After the Interview
- 45:07 Outtakes
Learn More
You can learn more about Joe and his book at AnalyticsTRW.com.
For more learning on this topic, check out:
- Episode 381 with Jim Loehr about how to make wiser decisions.
- Episode 372 with Annie Duke on knowing when to quit.
- Episode 437 with Nada Sanders about future-prepping your career in the age of AI.
Thank you for joining me for this episode of The People and Projects Podcast!
Talent Triangle: Power Skills
Topics: Leadership, Decision Making, Data Analytics, Artificial Intelligence, Project Management, Strategic Thinking, Causal Inference, Agile, AI Ethics, AI in Education, Machine Learning, Career Development, Future of Work
The following music was used for this episode:
Music: Ignotus by
Agnese Valmaggia
License (CC BY 4.0): https://filmmusic.io/standard-license
Music: Synthiemania
by Frank Schroeter
License (CC BY 4.0): https://filmmusic.io/standard-license