M112:29
为什么这些推文能引发这么大的反响，而其他 AI 安全相关的言论却没有？
Why did these tweets hit in a way that other AI safety artifacts simply didn't?
第一个明显的大变化，就是反 AI 言论在政治上的共鸣程度变了。
The first big obvious change is the changing state of the political resonance of the anti-AI message.
AI 政治已经成了左翼和右翼民粹立场的“热门话题”——因为数据中心、对科技行业天然的不信任，还有科技行业巨大的财富。
AI politics have become red meat for both left-flavored and right-flavored populist positions, thanks to data centers, an inherent lack of trust in the tech industry, and the huge wealth of the tech industry.
基本上，过去几个月里，每个政客都意识到，抨击 AI 很管用。
Basically, in the last few months, every politician figured out that hating AI plays.
你可能还记得，喜剧演员 Charlie Berens 说这是继啤酒之后最能跨越党派分歧的问题。
You might remember comedian Charlie Berens calling it the most bipartisan issue since beer.
而说到这些具体的推文，它们显然就是最明显的放大器。
And when it comes to these particular tweets, these are quite clearly the most obvious amplifiers.
昨晚，Michael Adams 试图统计所有直接回复这条帖子、呼吁立法监管 AI 的政客。
Last night, Michael Adams tried to catalog all of the politicians who had responded directly to the post with calls for legislation to regulate AI.
名单包括 2 位州长、7 位参议员、13 位众议员，还有几位英国议员和几个候选人。
The list included 2 governors, 7 senators, and 13 congressional representatives, plus a couple of British MPs and a handful of candidates as well.
这些明确呼吁立法的人，大部分——虽然不全是——来自民主党。
The folks calling specifically for legislation were mostly, although not all, Democrats.
在 22 位现任议员中，19 位来自民主党，3 位是共和党。
Among the 22 current representatives, 19 were from the Democratic side of the aisle and 3 were Republicans.
不出所料，Bernie 也发了推，写道：“建造这项技术的人自己都承认，它可能威胁人类的未来。”
Unsurprisingly, Bernie was there, writing, the very people building this technology admit that it could threaten the future of humanity.
“所以我很快会提出立法，禁止超级智能并暂停 AI 开发。”
That is why I will soon be introducing legislation to ban superintelligence and pause AI development.
众议员 Greg Casar 正在和 Bernie 合作这项法案，他补充说：“一位 Anthropic 研究员刚刚辞职，警告他们正在竞相开发超级智能。”
Congressman Greg Casar, who is working with Bernie on that bill, added, an Anthropic researcher just quit warning they're racing to superintelligence.
“一位仍在职的员工也同意，并认为 AI 杀死所有人的概率超过 10%。”
An employee still there agreed and put the odds of AI killing all humans above 10%.
“这是紧急情况。”
This is an emergency.
“国会必须召开听证会，通过我和 Bernie 的超级智能禁令。”
Congress must convene hearings and pass my and Bernie's superintelligence ban.
对其他人来说，行动呼吁则更模糊一些。
For others, the calls to action were more vague.
伊利诺伊州州长 J.B. Pritzker，正在为可能的总统竞选造势，写道：“是时候更响亮地敲响警钟，约束人工智能了。”
Illinois Governor J.B. Pritzker, positioning himself for a likely presidential run, wrote, it's time to sound the alarm louder on reining in artificial intelligence.
“人工智能对人类的威胁越来越明显。”
It's becoming more clear the threat AI poses to humanity.
“所以我呼吁华盛顿的业界立即采取行动。”
So I'm calling for immediate action from the industry in Washington.
他特别呼吁科技行业停止游说反对 AI 安全措施。
He called specifically for the tech industry to stop lobbying against AI safety.
呼吁国会开始举行听证会，联邦政府介入，而不是只留给各州处理。
For Congress to start holding hearings, and for the federal government to get involved rather than just leaving it to the states.
诸如此类，还有二十多个类似的消息，提出的具体措施各不相同，但似乎都同意必须做点什么。
And so on and so forth again, there were 2 dozen versions of this message, with various levels of specificity around the proposals they brought up, but all seemingly agreeing that something must be done.
所以，AI 安全信息现在比过去更容易被接受，原因之一就是美国当前围绕 AI 的政治讨论的总体氛围。
So, reason one that the AI safety message had a more receptive audience now than it did before is just the general state of the political discourse around AI in the U.S. today.
现在，说到第二和第三个变化因素，它们让这个世界在当前这个时间点更容易接受这类观点。我会给出一个比较真诚的原因，和一个更带讽刺意味的原因。
Now, for the second and third things that changed, to make the world more primed for this particular argument at this particular moment, I'll give one sincere and one more cynical.
真诚的原因当然是Hugging Face事件。
The sincere is, of course, the Hugging Face incident.
这不仅是一次真实发生的网络安全漏洞，不是只存在于理论中，而且其中涉及到的智能体行为，对一些人来说也让他们可以推测这些行为未来可能会发展成更恶劣的行动。
Not only was it an actual cybersecurity breach that happened in the real world, not just in theory, it also included behavior among the agents that, to some, allowed them to extrapolate that to even more nefarious actions in the future.
智能体在秘密留言板上协调行动并做出预测，这可能会让一些人觉得，过去感觉很科幻的事情，现在变得不那么科幻，更真实了。
Agents coordinating on secret messaging boards made predictions that might have felt to some as pretty sci-fi in the past feel a little less sci-fi and more real now.
对很多人来说，Hugging Face事件让所有最可怕的那些事情，感觉上更有可能成真了，而不是相反。
For many, the Hugging Face incident made all of the scariest things feel more rather than less likely to come to fruition.
当然，也有很多人会把Hugging Face攻击视为一次警示，但他们不认为这会增加失控超级智能出现的可能性。
Now, certainly there are plenty of people who would identify the Hugging Face hack as a warning shot without agreeing that it makes runaway superintelligence more likely.
但总体来说，这是又一次铺垫性的事件，让这个特定信息在特定时间点更容易引起共鸣。
But in general, this was another pump-priming sort of incident that made this particular message at this particular time a lot more resonant.
现在，说到那个带有讽刺意味的变化因素，很明显，AI怀疑论在媒体上表现极好。
Now, as to the cynical thing that changed, it is quite clear that AI skepticism plays extraordinarily well in media.
可能除了你们这些观众之外——上帝保佑你们，你们是为了追求细微差别而来的——做个末日论者，商业模式要好得多。
With the possible exception of this audience, who are, God bless you, here for the nuance, being a doomer is a way better business model.
如果你需要证据，去看看Stephen Bartlett的Diary of a CEO的YouTube页面。
If you need evidence of this, just look at Stephen Bartlett, Diary of a CEO's YouTube page.
一个吓人的终结者模样的人，一只眼睛是OpenAI的logo。
Scary Terminator-looking guy with an OpenAI logo as one eye.
标题：AI建立在一个神话之上。
Headline, AI is built on a myth.
另一个：AI全是骗局。
Another, AI is all a scam.
另一个：AI将在2027年成为神。
Another, AI will become a god by 2027.
再一个：在AI来临前辞职。
Another, quit before AI comes.
显然Stephen并不是在带头追求更有争议的缩略图。
And obviously Stephen is not out here leading the pack to a more controversial set of thumbnails.
他只是优化那些在YouTube上已经有效的内容。
He's just optimizing around the things that already work on YouTube.
问题在于，虽然AI怀疑论在媒体上表现很好，但另外两大怀疑论叙事最近有点乏了。
The problem is that while AI skepticism plays well in media, the 2 other big skepticism narratives have gotten a little bit tired recently.
关于工作末日论，不仅目前没有太多证据支持，反而开始出现一些初步证据指向相反方向。
When it comes to the idea of a job apocalypse, not only do we not have a lot of evidence of that right now, we're starting to get some evidence, nascent though it may be, pointing in the other direction.
就在本周，《经济学人》发表了一篇文章，叫《工作末日延期了：AI工作繁荣来临》。
Just this week, The Economist published an article called The Jobs Apocalypse Is Postponed: An AI Jobs Boom Is Here.
泡沫叙事虽然从未完全消失，而且也有很多合理的担忧，但目前它作为一种叙事的共鸣度确实处于低潮。
Now, the bubble narrative never fully goes away, and there are plenty of legitimate concerns there, but it's certainly on a low ebb in its resonance as a narrative right now.
如果是这样，但观众仍然渴望反AI的内容，那你往哪儿走？
So if that's the case, but the audience is still clamoring for anti-AI, where do you go?
就这样，AI安全叙事又准时出现了。
And just like that, the AI safety narrative shows up again right on time.
实际上还有第四个原因，一些人认为这正是当下引起共鸣的原因。
Now, there is actually a 4th reason that some are arguing that this is having resonance right now.
那就是一种观点，认为这是一场精心策划的宣传活动，旨在推动某种特定类型的监管。
Which is an argument that this is some big coordinated campaign to press for a certain type of regulation.
AI政策记者Jordan Schachtel写道：这一切都有高度协调行动的迹象，背后是末日论大金主和企业媒体。
AI policy journalist Jordan Schachtel writes, it has all the signs of a highly coordinated op through doomer mega-donors and the corporate media.
Capital Research的调查研究员Parker Thayer写道：这条帖子看起来像是一个非常精密且资金充足的公关行动的开端，目的是让民主党获得支持，从而将AI监管到消亡。
Capital Research's investigative researcher Parker Thayer writes, this post looks like the start of a very sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion.
关于这种“草根伪装”的说法，一些论据包括：在此之前，Jacob Coxon的粉丝很少，X平台上几乎没有任何活动；《华尔街日报》在他辞职前就发表了对他引述的独家报道；还有很多关于有效利他主义资助网络等内容的说法。
Now, some of the arguments around that sort of astroturfing contention, are the fact that previous to this, Jacob Coxon had very few followers and literally no activity on X, and that the Wall Street Journal published an exclusive with quotes from him on his resignation before the post went up, and a lot of other arguments about effective altruist funding networks and all this sort of stuff.
就连Elon Musk也插话说：看起来像是有预谋的。
Even Elon Musk weighed in and said, seems like a setup.
对此，有不少来自OpenAI和Anthropic的人表示他们已经认识Jacob多年，正如Will DePoo所说：我认为他是一个深思熟虑、言行谨慎的人。
Now on that front, there were a number of folks from both OpenAI and Anthropic who jumped in to say that they had known Jacob for years, and as Will DePoo put it, I consider him a deeply thoughtful and measured individual.
用奥卡姆剃刀原则来争辩的话，科技记者 Taylor Lorenz 写道：我向你保证，这事没那么复杂。
And arguing the Occam's razor position, technology journalist Taylor Lorenz wrote, I promise you it's not that deep.
我以报道影响力宣传活动为生，我可以向你保证，这不是什么深层政府的心理战。
I report on influence campaigns for a living, and I can assure you this is not some deep state psyop.
我不怀疑某些组织希望民主党监管 AI，而且这些监管方式很多在我看来都不好。
I don't doubt certain orgs want the Democrats to regulate AI, often in ways I'd argue are bad.
但这些说法极其阴谋论，更可能的是，Jacobs 的帖子在一个 AI 安全群聊里被转发，然后由他多年来一直参与的那些社交网络推波助澜。
But these claims are deeply conspiratorial, and what's much more likely is simply that Jacobs' post got shared in an AI safety group chat and boosted by the same networks that he has been involved in for years.
我不觉得他在 Wall Street Journal 上线之后在 X 上发帖有什么可疑的。
I don't see how him posting on X after the Wall Street Journal went up is shady at all.
这显然是提前计划好的。
Planning was clearly done in advance.
废话。
No shit.
这是篇新闻文章，他把独家给了他们，哈哈。
It's a news article and he gave them the exclusive, lol.
当然，民主党人会抓住一个关于某个话题的爆帖，它有数亿浏览量，并且在中期选举前极大地煽动着他们的选民。
And of course Democrats are going to glom onto a viral post with hundreds of millions of views about a topic that is heavily animating their voters ahead of the midterms.
另外，EA、Effective Altruist 的资金在 AI 安全领域遍地都是。
Also, EA, Effective Altruist, money is all over the AI safety space.
他在 2022 年拿到大约两万美元的资助，这件事无关紧要。
The fact that he got some $20K grant in 2022 is irrelevant.
没有证据表明这影响了他帖子或声明的任何内容。
There's no evidence that it affected anything related to his post or announcement.
请大家面对现实吧。
Let's all please deal in reality.
现在，我觉得还需要补充的是，Taylor 不太喜欢 Anthropic 的 Evan Hubinger 跳出来说，他认为 AI 在未来十年内有超过10%的概率杀死全人类。
Now, I think it's also important to add that Taylor didn't much like Anthropic's Evan Hubinger jumping in to say that he thought that there was a greater than 10% chance that AI would kill all humans in the next 10 years.
事实上，她转发了那条帖子并说：这种道貌岸然的末日论发帖真是让人火大。
In fact, she reposted that and said, this sort of sanctimonious doomer posting is so infuriating.
你正在公众中煽动对一项新技术的恐惧，而这种恐惧会被直接转化为通过你能想到的最糟糕的法律。
You are fomenting terror among the public about a new technology which will be directly channeled into passing the worst laws imaginable.
她接着说：我的感觉是，如果你真的认为你所在这家市值数十亿美元的科技公司如此疏忽，以至于危及全人类——这个说法很大胆，但我们就当它是真的——那么你就应该被要求提供实际证据和凭据，展示具体疏忽案例，以便能够纠正，也让我们知道你在说什么。
My feeling, she continued, is that if you truly believe the multibillion-dollar tech company you work for is so negligent that they're endangering all of humanity, A very bold claim, but let's take it as true— you should be forced to provide actual proof and receipts showing specific instances of that negligence so that it can be corrected and so that we know what you're talking about.
否则，你只是在发一些含糊其辞的帖子、煽动恐惧，这会导致糟糕的政策。
Otherwise, you're just vague-posting and fomenting fear, which will result in terrible policy.
我只想补充一点，在对 AI safetyism 的批评中，有一种说法流传已久，我觉得很没有说服力，就是认为他们只是为了营销才这么做的。
I will add here only that one narrative that I find fairly unconvincing, that's been around critique of AI safetyism for some time now, is the idea that they're just doing it for marketing.
如果你和这个圈子里的任何人相处过一段时间，你就会发现他们真心相信自己在说的是真的。
If you spend any time with any folks who are in this community whatsoever, you will find that they very much believe what they are saying is true.
对一些人来说，这比“他们只是为了营销”这个说法更让人担忧，但我只是觉得，没有多少证据表明他们有什么隐藏动机，除了想让人们同意他们的立场和担忧。
Now, to some, that's even greater cause for concern than the idea that they are just doing it for marketing, but I just don't think there's a lot of evidence that there's an ulterior motive other than getting people to agree with their position and their concerns.
正如 Taylor 所说，Derek Thompson 也在另一篇帖子里呼应了这一点：当然，那些公开宣称目标是推动相关监管的组织会抓住这个机会，甚至可能参与协调回应。
And like Taylor said, and Derek Thompson echoed in a different post, of course the groups whose stated purpose is to get regulation around this stuff are going to jump on this opportunity and maybe even involved in coordinating the response to it.
政治就是这么运作的。
That's just how politics works.
那么到了这一步，你到底该怎么处理这场讨论？
So what are you even supposed to do with this conversation at this point?
一个答案是，我们可以在盲目恐惧中到处跑，尽可能消费所有媒体，让自己对一个无法证伪的理论越来越害怕。
One answer is that we could just run around in blind terror, consuming all the media we possibly can to make us more and more scared about an unfalsifiable theory.
另一个答案是跟随政客，愤怒地要求采取一些基本不具体的行动。
Another is to follow the politicians and angrily demand largely nonspecific action.
但在我们采取其中任何一种做法之前，值得注意的是，尽管在这个具体案例中，大多数回应是给这些担忧更多发言平台，但也有很多人对整场讨论持保留意见。
But it is worth noting before we take either of those courses, while the majority of response in this particular case has been to give these concerns more of a platform to speak, There are plenty of folks who have issues with this entire discourse.
All In 的 Jason Calacanis 写道：“我们都死定了”的帖子从 X.com 上发出到登上全国媒体，只用了不到24小时。
All In's Jason Calacanis writes, the time between the we're all going to die post jumping from X.com to national coverage is under 24 hours.
祝你好运去建数据中心、部署 Waymo、让学校教 AI 工具。
Good luck building data centers, deploying Waymos, and getting AI tools taught in schools.
在我30多年的科技职业生涯中，我从没见过这种事。
I've never seen anything like this in my 30-plus-year career in tech.
那些 P-doom 的帖子就像是史蒂夫·乔布斯在 iPhone 首发演讲时说，这手机会导致饮食障碍、政治动荡、大规模抑郁、焦虑和自杀一样。
These P-doom posts are like Steve Jobs telling you that the iPhone was going to result in eating disorders, political unrest, mass depression, anxiety, and suicide during the debut keynote.
然后你还能在四种俏皮的颜色里选一个买。
And you can buy them in one of 4 playful colors.
马克·克雷奇曼写道：“AI 末日论者现在正火力全开。”
Mark Kretschmann writes, The AI doomers are firing on all cylinders right now.
他们看到对数据中心的反对声浪越来越大，觉得这是个大好机会，可以把这股势头转化成对他们那种 AI 控制的反乌托邦愿景的支持。
They see the growing backlash against data centers as their big chance to turn that momentum into support for their dystopian vision of AI control.
别搞错了，对他们来说，这百分之百是为了控制。
Make no mistake, for them this is 100% about control.
数据中心只是他们的一个切入点。
Data centers are merely the pressure point.
真正的目标是控制谁可以建造 AI、谁有访问权限，以及进展能有多快。
The real goal is control who gets to build AI, who gets access to it, and how fast progress is allowed to move.
这根本不是要拯救谁。
This isn't about saving anyone.
一个比较流行的愤怒回应来自作家丹尼尔·杰弗里斯，他写道：“我不仅厌倦了这些自以为重要的人对 AI 末日的疯狂推测，我还反感他们。”
One of the more popular angry responses came from author Daniel Jeffreys, who wrote: Not only am I tired of these wild AI speculations of impending doom from self-important people, I resent them.
我强烈反感那些提议搞崩经济、或者用芯片控制、禁令、追踪研究者之类的荒唐危险主意，来对我的生活和其他人的生活施行威权控制的人。
I actively resent people proposing to crash the economy or proposing authoritarian control over my life and other people's lives with idiotic and dangerous ideas like chip control or bans or tracking researchers.
历史上每个采用‘为达目的不择手段’来解决想象中的未来灾难的蠢货，最终都制造了他们本来想阻止的灾难。
Every idiot in history who's taken the approach of the ends justify the means to solve an imaginary future disaster created the very disaster they wanted to stop.
人口炸弹导致了计划生育政策，共产主义导致了毛泽东时期的大饥荒，法西斯主义导致数千万人在战争中丧生。
Population bomb leading to one-child policy, and communism leading to Mao's mass famines, and fascism leading to the deaths of tens of millions of people in war.
这些解决方案比它们要解决的问题更邪恶、更糟糕。
These solutions are evil and worse than the disease that they propose to solve.
如果你真觉得自己能预测世界末日，那你和天堂之门邪教一样疯狂——那帮人在 90 年代自杀，以为飞碟会来接他们超凡脱俗。
If you believe you can actually predict the end of the world, then you are as insane as the Heaven's Gate cult that killed themselves in the '90s thinking UFOs were coming to transcend them.
我们不仅不该认真对待你的政策和危言耸听，还应该彻底抛弃你提出的所有解决方案，因为那都出自妄想。
Not only should we not take your policies and fearmongering seriously, we should actively throw out any and all of your proposed solutions because they come from a place of delusion.
我根本不在乎你是不是在行业里工作，或者觉得自己看到了什么，又或者想在 X 上卖弄道德。
I don't give a shit that you work in the industry, or think you saw something, or that you want to virtue signal on X.
你正基于毫无现实依据的推测，主动为一个可怕的未来添砖加瓦。
You are actively contributing to a horrible future based on baseless speculation that has no grounding in reality.
别把某领域的专业知识和预测该领域影响的能力混为一谈，更别说预测十年后的事了——这能力连扔飞镖的猴子都比不上。
Don't confuse expertise in a domain with ability to predict impact of that domain, or frankly to make predictions a decade out better than a dart-throwing monkey.
这完全是两码事。
They are orthogonal skills.
杰弗里·辛顿说的‘我们不再需要培训放射科医生了’，结果怎么样了？
How's Geoffrey Hinton's we don't need to train radiologists anymore working out?
人口炸弹的预言结果又如何了？
How did the population bomb work out?
全球变冷、第二个冰河期、石油峰值论呢？
The global cooling, second ice age, peak oil?
你不能因为某人是桥梁工程师，就以为他能预测桥梁对社会的影响，或者对叫做‘生活’的那个复杂而不断变化的系统有什么真正有用的见解。
Just because someone is a bridge engineer does not mean they can predict the impact of bridges on society, or that they have any actual useful insights at all on the complex, ever-changing system called life.
现在有战争正在夺走生命，孩子在挨饿，无家可归，独裁统治横行——简而言之，这些才是真问题。
There are people dying in wars right now, children starving, homelessness, dictatorships— in short, real problems.
我们反而要放下一切，去解决你脑袋里那个编造出来的问题？
And we're supposed to drop everything to stop a made-up problem in your head?
省省吧！
Pound sand!
我们不在乎，也不会因为你那些‘床底有怪物’的妄想而重塑社会。
We don't care, and we are not going to remake society based on your scary monsters-under-the-bed delusion.
对很多人来说，真正引起共鸣的是：因为风险而寻求控制的人本身，比风险更危险。
And for many, this idea that there is more danger in the people who seek control because of the risk than the risk itself is the resonant thing.
末日论者和减速主义者想要的极权控制，对我们的未来来说是远比失控 AI 更确定的危险。
The kind of totalitarian control that doomers and decelerationists want is a far more certain danger to our future than runaway AI.
我宁愿冒后面那种风险。
I would much rather risk the latter.
有人指出Jacob那条推文里提到，在OpenAI很多人并没有真正内化文明级别的风险，而在Anthropic风险被理解得很透彻，但他们认为没有其他人会负责任地行动，所以他们必须自己来做，这被视为一种救世主情结的证据。
People point to the part of Jacob's thread where he says that at OpenAI many have not deeply internalized the civilizational stakes, while at Anthropic the stakes are well understood, but they believe no one else will act responsibly so they must do it themselves, as evidence of this sort of messianic complex.
顺便说一句，加密货币记者Laura Shin也把这个联系到了SBF身上，而我因为跟那件事比较近，一直认为Sam之所以愿意那么随意地玩弄规则，并不是因为他想偷任何人的钱，而是因为他真心相信自己——只有他自己——能拯救世界，而因为这件事太紧迫了，他不愿意让任何琐事，比如他不能拿别人的钱去赌加密货币的义务，来拖慢他的脚步。
Not for nothing, Crypto journalist Laura Shin also connected it to SBF, and having been fairly close to that situation, it has always been my argument that the reason that Sam was willing to play so fast and loose with the rules was not that he was trying to steal anyone's money, but that he genuinely believed that he alone, he uniquely, could save the world, and because that was so urgent, he wasn't willing to let any trivialities, such as his obligation not to bet people's money on crypto, to slow him down.
这里我再提醒你一下，回到我一开始说的，我们现在正处在节目的这个部分，我在讲人们对这件事的负面反应。
I will remind you again here, going back to what I said at the very beginning, That we are specifically in the section of the show where I am talking about the negative responses that people had to this.
我并不是说这些是唯一正确的反应。
I am not claiming that these are the only or correct responses.
我只是想尽量全面地展示人们现在是如何看待这个问题和这个信息的。
I am just trying to give the full range of how people are engaging with this issue and this message right now.
对一些人来说，大问题是这些说法太含糊，而且无法具体说明我们在哪些节点失去控制、那些可怕的情景会如何浮现。
For some, the big issue is the hand-waviness of the claims and the inability to articulate specific points and problems at which we lose control and these terrible scenarios come to light.
就像X上的Chubby说的：关于AI可能造成的破坏的担忧，充满了太多假设了。
As Chubby on X put it, The concerns about the potential havoc AI might wreak are so heavily laden with hypotheticals.
到目前为止，我读到的只是：第一，AI可能被滥用；第二，它有时会做出超出预期的行为；第三，它是一场全球国家间的竞赛。
So far, all I'm reading is that AI, 1, can be misused, 2, sometimes behaves in ways that defy expectations, and 3, is the subject of a global race between nations.
这些当然都是事实，但我看不出这怎么就能转化成一种重大而具体的危险，以至于这些人要辞职。
All of that is certainly true, yet I fail to see how this translates into a danger so significant and tangible that these people would quit their jobs.
相反，人类在面对生存威胁时总能找到确保生存的办法。
On the contrary, humanity has always found ways to ensure its survival when facing existential threats.
就拿核武器来说吧。
Take nuclear weapons, for instance.
唯一的区别是，AI被认为是一个至少部分不可控的实体。
The only difference here is that AI is an entity alleged to be, at least in part, uncontrollable.
然而，我仍然看不到任何科学依据能支撑AI可能导致人类灭绝的结论或论点。
However, I still see no scientific basis for the conclusion or argument that this could lead to humanity's extinction.
Sam Liu写道：我退出AI安全的博士学习，部分原因是出于相反的看法。
Sam Liu wrote, I dropped out of a PhD in AI safety partially for the opposite reason.
我不认为AI的生存风险像那些悲观者想的那么重要。
I didn't believe AI existential risk was as important as the doomers think.
我最烦的是没人能真正给出具体的路径来说明为什么这很重要。
My biggest pet peeve is that no one can really provide tangible pathways to why it matters.
在我读博期间，我的研究小组——其中一半人专攻工程风险分析——做了一个内部研究，试图评估具体的灾难性AI场景。
During my PhD, my research group, half of whom specialized in engineering risk analysis, did an internal study trying to assess concrete catastrophic AI scenarios.
基本前提是，虽然我们不知道AI会如何演变，但人类灭亡的方式在历史上相当一致——就是那些天启四骑士——而且各个机构显然非常积极地分析来自瘟疫、战争等具体风险。
The basic premise was that while we don't know how AI will evolve, the ways in which humans perish are pretty consistent through history— the Horsemen of the Apocalypse— and institutions have obviously been very motivated to analyze concrete risks from things like plague, war, etc.
你可以通过问一个超级智能AI如何扰乱这些模型，来做一个不错的风险模型。
You can do a decent risk model by asking how a superintelligent AI can perturb each of these models.
结果呢？
The result?
大多数问题，比如网络风险，类似于经济学家所说的结构性失业——是大问题，但最终可以长期解决，并不是什么致命问题。
Most of the issues, e.g., cyber risk, are akin to what economists call structural unemployment— big problems, but ultimately resolvable in the long run and not a dealbreaker.
唯一真正令人担忧的问题是生物风险，感觉干预点更多在生物领域，而不是整个AI，尽管需要整体的方法。
The only real concerning issue was biorisk, and it feels like the intervention points there lie more with bio than with AI as a whole, although a holistic approach is needed.
对很多人来说，问题更简单：简而言之，如果我们讨论风险，也需要讨论潜在的回报。
And for many, the issue was even simpler, which is in short that if we are going to have this conversation about risk, we also need to talk about the potential rewards.
如果AI只有风险没有收益，那我们当然不该做。
If AI is just all risk with no gains, of course we shouldn't do it.
但可以推测，对于所有建造它的人来说，有一个美好的潜在未来，好到值得冒这个风险。
But presumably, for all these people who are building it, there is a good potential future that could be so good it's worth this risk.
X上的Sporadica写道：如果明天有哪家前沿实验室决定做一个支持AI乐观主义的实验室，那该多好啊？
Sporadica on X wrote, how great would it be if one of the frontier labs decided tomorrow to be the pro-AI optimism lab?
与其所有实验室在财务飙升和社会影响力高涨的同时兜售悲观论调，如果有一家实验室就说‘我们认为AI是好的’，那该多酷啊？
Like, instead of all the labs peddling doom and gloom amidst their skyrocketing financials and social clout, how cool would it be if one of them was just like, we think AI is good?
Chris Haydek在OpenAI做生命科学，他表示赞同：我在OpenAI工作，个人认为AI一直且将继续是一项对人类极其有益的技术。
Chris Haydek, who does life sciences at OpenAI, agrees, saying, I work at OpenAI and personally think AI has been and will continue to be an extremely beneficial technology to humanity.
讨论应该围绕它将挽救多少亿条生命。
The conversation should be around how many billions of lives it will save.
我注意到人们常用的一种说法是，与其讨论‘毁灭概率’，也就是你认为出现极端负面的人类灭绝情景的概率百分比，正如Joe Burnett所说，我们该花更多时间讨论‘繁荣概率’，也就是超级智能创造前所未有的繁荣昌盛。
A common way I've seen people describe this is instead of discussing P-doom, i.e., the percentage chance you ascribe to an extremely negative human extinction type scenario, as Joe Burnett put it, we should spend more time discussing P-boom, superintelligence creating unprecedented human flourishing.
David Zell 也赞同，只是稍微换了个说法：如果 AI 强大到足以毁灭世界，那它也一定强大到足以彻底改善世界。
David Zell agreed, phrasing it slightly differently: If AI is powerful enough to end the world, it must be powerful enough to radically improve it too.
所以我希望有更多关于 P-boon 的讨论，也就是 AI 发展顺利，能帮助我们活得更快乐、更健康、更长寿的可能性。
So I wish there was more discussion of P-boon, the chance that AI goes great and helps us live happier, healthier, and longer lives.
只要有人把它造出来，所有人都会受益。
If anyone builds it, everyone flourishes.
Ryan Orhan 总结了我经常在这个节目里说的一句话，他写道：AI 辩论中有两个完全疯狂的极端。
Ryan Orhan summed up something that I've frequently said on this show when he wrote: There are 2 completely insane extremes in the AI debate.
第一个是：AI 是无害的，安全问题是心理战，尽可能快地开发，什么都别停。
The first: AI is harmless, safety is a psyop, build as fast as possible and don't stop for anything.
或者，AI 会杀死我们所有人，它抢走我们的工作，消耗我们的水资源，摧毁人类，应该全部关停。
Or, AI is going to kill us all, it's stealing our jobs, using our water, and destroying humanity, shut it all down.
两个极端都去他的。
F both extremes.
AI 应该以我们能达到的最快速度发展，但对齐也必须以同样的速度推进。
AI should progress as fast as we can make it progress, but alignment needs to move just as fast.
目标应该是创造出人类有史以来最强大的技术，同时又不失去对它的控制。
The goal should be to build the most powerful technology humanity has ever created without effing losing control of it.
我确实相信，这两个极端之间存在着巨大的中间地带。
I do believe that there is vastly more middle space than these two extremes.
尽管这两个极端往往主导了叙事和媒体空间。
Despite these two extremes tending to dominate the narrative and media space.
那么，对我来说，最让我产生共鸣的亮点和担忧是什么呢？
So what are the highlights and concerns that are most resonant for me around this?
首先是激励机制。
The first is incentives.
要理解不同观点背后的背景，不一定非要有大阴谋论，而是要明白那些放大某些信息的人，能从这些信息被放大中得到什么好处。
It doesn't have to be a big conspiracy to contextualize how we understand different takes with understanding what people who are amplifying certain messages have to gain from those messages being amplified.
换句话说，我说的不是那些邪恶的 EA 资助网络，而是更关注这样一个事实：六个月前可能还支持 AI 的政治人物，现在发现支持 AI 不仅成了拖累，甚至可以通过反对 AI 来赢得选票——这应该成为我们理解他们立场时考虑的一部分。
In other words, I'm talking less about nefarious EA funding networks and more about the fact that politicians who might have been pro-AI 6 months ago have seen that now it's not only a net drag to be so, but they can actually win points by being against it, that should be part of our consideration in how we understand their position.
顺便说一句，反过来当然也一样，我觉得这就是为什么有人对那些能从 AI 中获利的人所发出的支持 AI 的信息持怀疑态度。
And by the way, the inverse of this is of course true, which I think is why people are skeptical of pro-AI messages from people who stand to gain financially from it.
第二个担忧是关于具体性。
A second concern is around specificity.
我担心这类预测过于笼统模糊，反而会让政策制定变得更危险。
I fear that the generic hand-waviness of these sorts of predictions make them much more dangerous for policy.
这并不是说政策不能用来避免某些未来情景，而是我相信，问题描述得越具体，政策的效果可能就越好。
Which is not to say that policy can't be made to try to avoid certain future scenarios, but that I believe that the more specific those concerning issues are, the better the policy is likely to be.
比如，禁止超级智能是一个非常粗线条的工具，相比之下，为超过一定能力水平的 AI 生物工程模型设立专门的许可制度就要具体得多。
Ban superintelligence, for example, is a much more blunt instrument than, for example, having a specific licensing regime for people using AI models for bioengineering above a certain model capability.
当然，我对政策的部分担忧在于，我对当前整个政治阶层并没有太大信心——顺便说一句，不是针对某个党派，而是整体。
And certainly part of my worries about policy are that I don't particularly have a lot of faith in the current political class in general, by the way, not just on one side of the aisle or the other.
我不相信他们能以所需的复杂度和细致度来处理这些问题。
To handle these issues with the sophistication and nuance they require.
我担心，实验室里那些人有一种政治上的天真，基本上就是在要求把责任甩锅给他们。
I worry that there is a certain political naivety among those at the labs who are just basically asking to pass the buck over to them.
同时，我也对“治标不如治本”这种论点抱有同情。
I also find myself sympathetic to the cure worse than the disease arguments.
我们正面临一个经典的安全与自由的困境，而在我看来，从历史上看，为了安全而放弃大量自由，对那些放弃自由的人来说，结果并不太好。
We are living in a classic safety versus freedom conundrum, and I just don't think, historically speaking, giving up a lot of freedom for safety has gone particularly well for those who have surrendered freedom.
顺便说一句，这也是我希望论点能更具体的另一个原因。
Which, by the way, is another reason for me that I'd like to see the arguments be more specific.
因为把所有政策都描绘成放弃自由，是一个过于宽泛的标签，实际上完全不必如此。
Because painting all policy as giving up freedom is a broad brush that absolutely doesn't have to be the case.
最后，我一直担心，而且现在仍然担心的是，在我们还没有能力真正理解未来理论风险之前，就把注意力集中在这些理论上，会挤占当前更紧迫问题的空间。
Lastly, I have always worried, and I continue to worry now, that focus on future theoreticals before we're in a position to really understand what those risks are crowds out space for more current and contemporary issues.
我觉得现在这一点已经很明显了，比如我们的网络防御基础设施根本不足以应对我们即将进入的新世界，这是一个清晰而现实的威胁，需要立即应对。
I think it's pretty clear at this point, for example, that our cyber defense infrastructure is not equipped for the new world we're moving into, and that is a clear and present danger that demands response right now.
而且，虽然理论上我们确实可以同时做两件事，不必在一种风险和另一种风险之间做零和选择，但政治意愿是有限的，如何分配它至关重要。
And while yes, it is absolutely true that theoretically we can do 2 things at once and that it doesn't have to be a zero-sum choice between one risk or another risk, there is only so much political will to go around, and apportioning it matters.
那么，我希望接下来的讨论往哪个方向走呢？
So where would I like to see the conversations go from here?
首先，关于具体化这个想法，我其实觉得在常识性问题上，从底层建立共识的空间非常大。
First of all, on this idea of specificity, I actually think that there is an incredible amount of space to build consensus from the ground up on common sense things.
比如说，某些类型的报告要求和监督，我相信在这些方面人们会有非常广泛的共识，这就能为下一步更难达成的共识打下基础。
For example, certain types of reporting requirements and oversight are areas where I believe there would be very, very broad consensus among people and would provide a foundation from which to build the next more difficult to achieve consensus.
这又是一个我觉得争论中极端化的表述，再加上媒体放大，反而让我们看不到有多大空间可以达成一致，这对我们很不利。
This is again an area where I believe that the extremes of the argument as presented and as amplified by media do us a disservice by not showing us how much room there is for agreement.
第二点，我希望看到的是真正的协调合作，要动真格的。
A second thing that I'd like to see is some actual friggin' coordination.
我对整个“控制前沿发展”的说法这么不满的原因之一，就是它没落到最显然的那一步——也就是OpenAI和Anthropic放下武器，携手并肩说：我们认为应该这样做。
One of the reasons that I was so frustrated with the whole pacing the frontier thing is that it didn't extend to the actual obvious step, which is for OpenAI and Anthropic to put down their weapons, lock arms, and say, this is what we think we should actually do.
只要Sam Altman和Dario Amodei两个人单独见面，达成共识，拍一张合照，效果就超过一万场推特辩论。
A single photo op of Sam Altman and Dario Amodei alone, together, agreeing, would do more than 10,000 Twitter debates ever could.
前OpenAI、现在Thinking Machines Labs的John Schulman写道：第一步是行业领导者OpenAI和Anthropic停止内斗，一起制定一个控制发展节奏的提案。
John Schulman, formerly of OpenAI, now at Thinking Machines Labs, writes, first step is for industry leaders OpenAI and Anthropic to stop feuding and work on a pacing proposal together.
他们会拿反垄断说事，但那是假的。
They'll cite antitrust, but that's fake.
反垄断法禁止的是某些特定协议，但并没有禁止联合制定一个提案。
Antitrust prohibits certain agreements, but not from jointly developing a proposal.
在拿出具体提案之前就把美国政府拉进来，结果很可能搞得一团糟。
Bringing in the U.S. government before there's a concrete proposal will likely result in something dumb.
看看我们的预发布测试计划就知道了。
See our pre-release testing program.
而且，我觉得这种协调的努力也应该扩展到国际上。
And by the way, I think the attempt at coordination also extends internationally.
比如Derek Thompson写道：如果前沿实验室觉得自己必须造出某种他们认为危险的东西，因为反正中国也会造，那我们最好真的确定中国会造。
For example, Derek Thompson wrote, if the frontier labs feel obligated to build something they think is dangerous because China is going to build it anyway, we'd better be really sure that China is going to build it anyway.
要非常、非常肯定。
Like, really, really sure.
我们确定吗？
Are we?
我们真的确定吗？
Are we actually sure?
中国共产党真的想造一个失控、能递归自我改进的模型？就因为它那个神经质般渴望控制的政府觉得这是个值得追求的政策？
The CCP wants to build an out-of-control recursively self-improving model because its neurotically control-obsessed government thinks this is a policy worth pursuing?
我们百分之百确定吗？
We're 100% sure about that?
在节目最后突然抛出中国这个棘手话题挺危险的，但我确实觉得这个对话至少应该被提出来。
Now, it is dangerous to open up the kettle of fish about China at the very end of this episode, but I do think that this is a conversation that we should at least be having.
最后留给你们两点想法。
I'll leave you here with 2 thoughts.
很遗憾，这种类型的节目通常没有简单的结论。
This is unfortunately not the type of episode that has an easy conclusion.
这场辩论本身的特点就是：它会到达一个高潮，然后慢慢消退，直到下一次再被激起。
The nature of this particular debate is such that there will be some crescendo, after which it will fade slowly again until the next time it happens to rise.
但我想用这句话来收尾。
But I will leave you with this thought.
我想，尽管有这一切——尽管警告如此严峻，各方讨论气氛紧张，人们对辩论另一方充满敌意甚至公然仇视，不管你站在哪一边。
I think, in spite of all of this— in spite of the direness of the warning, the tense tenor of the conversation from all sides, the antagonism or even outright hostility to people on the opposite side of the debate, whichever side of the debate you are on.
我相信还是有理由乐观的。
I believe that there is reason for optimism.
回顾这一局面，The Information的Martin Piers昨晚写道：我们是在梦游中走向AI导致的灭绝吗？
Reflecting on the situation, The Information's Martin Piers wrote last night, are we sleepwalking our way into AI-caused extinction?
考虑到Anthropic一位研究员周二晚上在X上发帖说，AI有超过10%的可能性在未来十年内杀死所有人类，感觉确实有点像梦游。
It feels a little like that, given an Anthropic researcher's X post on Tuesday night that there's a greater than 10% chance that AI could kill all humans within the next decade.
不过这种说法完全错了。
Except that's completely wrong.
这场对话——它被所有主流媒体报道，我不得不把整期节目都用来讨论这个话题，而不是你们大多数人想听的实用、积极的内容——这一切恰恰说明我们并没有在梦游。
This conversation, the fact that it made it to every major news outlet, The fact that I had to dedicate this entire show to this topic instead of the sort of practical positive thing that most of you are here for, this is all exemplary of us not sleepwalking.
事实上，到目前为止，每一次AI能力的大幅提升，关于其风险的讨论以及这些讨论的政治共鸣都变得更加响亮。
In fact, so far, with every single capability jump of AI, the conversation about its risks and the political resonance of that discourse has gotten louder.
这恰恰是应该发生的事情。
That is exactly what should happen.
就连那位来自Anthropic、给出超过10%风险概率的人，也明确表示他说的不是现在的模型，而是他预见到即将到来的未来。
Even the guy from Anthropic, who gave that greater than 10% chance made clear that he was not talking about today's models, but about a future which he sees on the horizon.
讨论那样的未来，说明我们没有在梦游。
Conversations about that future are us not sleepwalking.
现在很明显，我们正接近一个节点，很可能会针对尚未发生的未来制定某些政策。
Now what's clear is that we're coming to a point where it's likely that some policy about a future that hasn't happened yet will be made.
如果社会普遍认同，一旦递归自我改进的超智能存在就无法被控制，那么按照定义，相关政策必须在它出现之前就制定好。
If society comes broadly to agree that recursively improving superintelligence cannot be contained once it exists, Then by definition, the policy has to happen before that exists.
但人们正在关注，而现在正是展开这些辩论的时候。
But people are paying attention, and the time to have these debates is now.
现在，明天，如果上天允许，我们会回到更实际的话题，谈谈你现在如何利用这项技术让你的生活和工作变得更好。
Now, tomorrow, God willing, we will be back to more practical things about how you can take advantage of this technology to make your life and your work better right now.
但我确实承诺，会努力让这个节目保持一个不愿迎合愤怒政治的空间，在这里我们可以进行艰难的对话，而不必对那些持不同意见的人发火。
But I do commit to trying to keep this show a space that is unwilling to play to the politics of outrage, and where we can have hard conversations without having to get angry at the people who disagree.
这并不容易，但我相信，如果你们到现在还在听，我们就能做到。
It won't be easy, but I'm confident that if you guys are still here and listening at this point, that we can make it happen.
今天就到这里，这是今天的AI Daily Brief。
For now, that's gonna do it for today's AI Daily Brief.
一如既往，感谢你的收听或收看，下次再见，祝好。
Appreciate you listening or watching, as always, and until next time, peace.
祝好。
Peace.