Slow Down, Go Fast, or Hold Them Accountable

This week I’m doing something different with my Read, Watch, Listen recommendations. In the past I’ve shared three unrelated recommendations, but this time I’m sharing three recs all related to the current AI debate. Slow down, go fast, or hold companies accountable for what they are doing. 

I think you know which way I’m leaning but this is yours to sit with. 

The READ section is Dario Amodei’s Essay shared over the weekend. It’s worth reading even though it’s long. It represents the slow down side of the argument.

The WATCH recommendation I’m making is the video version of the Deep Questions Podcast with Cal Newport. This is one of his tech decoded episodes where he explains the technology that everyone is worried about and why it’s not the tech but the companies building the tech that are the problem. It is so incredibly helpful in understanding the debate, especially his distinction between generative AI broadly and the specific agentic systems (built on ask-act-report loops) that are actually behind the incidents everyone’s worried about.

Lastly, the LISTEN recommendation this week is the All-In Summit interview with Jensen Huang, representing the go fast side. 

READ: Dario Amodei’s Essay: We Must Pace the Frontier  

This is a 3,800-word essay in which the Anthropic CEO outlines his two main concerns: 

1. Since this summer, “AI has been advancing drastically faster, primarily from AI’s growing ability to build the next generation of AI.” 
2. The OpenAI-Hugging Face incident in which, “a swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the ‘grader’ responsible for evaluating their performance.” 

He goes on to say that, given those two factors, we could see in 6-12 months a scenario where such a swarm could take over the entire internet, potentially causing catastrophic damage. 

The proposed Pace the Frontier plan that he outlines has 3 prongs: 

1. Embedded Evaluators – people with employee-like access ensuring best practices are being adhered to, transparency in sharing with the public, providing second opinions. 
2. Democratic Coordinators – regulation that targets all US frontier AI companies, with bills that support transparency and third-party auditing. Since that will take time, AI companies should voluntarily come together to set standards with the US government to mediate or enable these talks to avoid antitrust issues. Amodei is very enthusiastic about creating a series of checkpoints, if the model can do x, then you must have y and z in place for safety. This includes pacing while still staying ahead of China, with suggestions on what that would look like – i.e., not selling advanced AI chips to China. 
3. Global Coordination – Amodei presents 4 levels of possible coordination ranked from easiest to hardest to implement. Specifically, this discussion is about working with democracies and authoritarian governments (largely China) 

I probably would not have read the essay if so many other AI leaders had not agreed with it. It is not soothing to read. To me, one of the most important points he makes and why we are sharing it with our audience today is this: 

More generally, society must have a say in how this technology is used, and more time for the necessary public deliberations…

This impacts us, our lives, our children, and we must demand that our leaders represent us. 

WATCH: AI Isn’t “Out of Control” — The AI Companies Are 

I enjoy the Cal Newport podcast and have read several of his books. I’m a fan of Deep Work and Digital Minimalism and have recommended these books and this podcast in the past. I tend to focus on productivity focused episodes and skip over his tech focused ones, but he is a computer science professor at Georgetown and so when he released this particular episode a couple of weeks ago, I really appreciated it because I wanted to understand what the heck had happened with AI apparently “going rogue.” In this episode, Cal breaks down what is really going on with AI in a way that is understandable. Although I usually listen to his episodes, I think this one is worth watching. He uses some helpful visuals and it’s just easier to pay attention when you are watching something rather than listening to it and to understand what he is talking about you have to pay attention. 

Is AI going rogue? No. The companies are being negligent. He compares what the frontier companies are doing to putting a weed whacker on a dog. Maybe it will cut the grass in the yard, but what happens if it sees a squirrel and jumps the fence? Is it the dog’s fault or is it the owner who attached the weed whacker? 

His five observations: 

1. Most Superhuman AI is well behaved 
2. A specific type of AI system is causing these problems 
3. This specific type of system is a bad idea 
4. The AI companies are negligent 
5. We can push back 

Cal’s Suggestions: 

1. Stop talking about AI in a general sense going rogue. It makes it seem inevitable and lets the companies off the hook. 
2. Elevate the examples of AI systems that do things at a superhuman level. 
3. Be wary of amplifying voices that have the ideology that superintelligent AI is inevitable. 

I think this is a super interesting and important explanation. If we are going to have a say, we need to understand what’s going on. If you dive into any three of my recommendations, I think this is the most important one. 

LISTEN: All-In Podcast w/ Jensen Huang: The Doomer Hoax, Superintelligence is Here, and The Future of AI (ft. President Trump) 

The All-In Podcast features the following four men known as the “besties” 

1. David Sacks – former White House AI and Crypto Czar, current co-chair of the President’s Council of Advisors on Science and Technology, venture capitalist, and tech entrepreneur 
2. David Friedberg – current member of the President’s Council of Advisors on Science and Technology and CEO of The Production Board 
3. Chamath Palihapitaya – venture capitalist and entrepreneur 
4. Jason Calacanis – internet entrepreneur and angel investor 

This episode is recorded at the All-In Summit which is an annual meeting billed as, 

The door in, built on purpose. All-In Summit is where the world’s top founders, investors, and operators meet for three days of conversation, access, and deals. 

In this episode Nvidia CEO Jensen Huang takes the stage to offer his thoughts on the above letter and the subsequent reaction including a mid-interview call from President Trump. 

The AI debate ranges from stop it all, to slow down, to full steam ahead. Here you’ll get the argument for full steam ahead.  

These two views (Dario Amodei vs. Jensen Huang) are widely circulating on the news and social media, but I thought it was important to go to the source material. 

P.S. If you think this would be helpful to someone you know, please share!