政策制定者如何处理AGI？(initial findings)

||yabo app

米里使命是“确保创造聪明的智力具有积极的影响。”

一个政策相关问题是：我们应该如何预期政策制定者处理AGI的发明，以及如何暗示投入到AGI风险缓解与其他问题的努力？

要调查这些问题，我们问道Jonah Sinick检查政策制定者如何以某种方式处理过去的事件，以某种方式对AGI的未来发明，并总结了他的研究结果。我们预先致力于在主题（具有次要编辑）上发布我们的整个电子邮件交换，就像我们的项目一样how well we can plan for future decades。下面的职位是研究结果的摘要our full email exchange (.pdf)so far.

As with our investigation of how well we can plan for future decades,我们决定在调查几个历史案件后发布我们的初始调查结果。这使我们能够在继续之前获得对项目的价值的反馈，以及改进的建议。这也意味着我们尚未能够为我们的核心问题提供任何自信的结论。

到目前为止，该项目的最重要结果是：

  1. We came up with a preliminary list of 6 seemingly-important ways in which a historical case could be analogous to the future invention of AGI, and evaluated several historical cases on these criteria.
  2. 气候变化风险似乎足够disanalogous to AI risk that studying climate change mitigation efforts probably gives limited insight into how well policy-makers will deal with AGI risk: the expected damage of climate change appears to be very small relative to the the expected damage due to AI risk, especially when one looks at expected damage to policy makers.
  3. 在调查浅薄调查之后，2008年的金融危机将足够比AGI风险更多地表达它应该给我们一些小理由担心政策制定者明智地管理AGI的发明。
  4. The risks to critical infrastructure from geomagnetic storms are far too small to be in the same reference class with risks from AGI.
  5. 消除了小型斑点仅适用于AGI的发明。
  6. Jonah performed very shallow investigations of how policy-makers have handled risks from cyberwarfare, chlorofluorocarbons, and the Cuban missile crisis, but these cases need more study before even “initial thoughts” can be given.
  7. We identified additional historical cases that could be investigated in the future.

下面给出了进一步的细节。对于来源等，请参阅our full email exchange (.docx)。

6种方式可以类似于AGI的发明

In conversation, Jonah and I identified six features of the future invention of AGI that, if largely shared by a historical case, seem likely to allow the historical case to shed light on how well policy-makers will deal with the invention of AGI:

  1. AGI可能会在有些不可预测的时间内成为一个主要威胁。
  2. 当世界与其经验非常有限时，AGI可能会成为一种威胁。
  3. A good outcome with AGI may require solving a difficult global coordination problem.
  4. 充分为AGI威胁准备可能需要提前仔细努力。
  5. 政策制定者具有强大的个人激励措施来解决AGI问题。
  6. A bad outcome with AGI would be a global disaster, and a good outcome with AGI would have global humanitarian benefit.

More details on these criteria and their use are given in the second email of our full email exchange.

气候变化的风险

People began to see climate change as a potential problem in the early 1970s, but there was some ambiguity as to whether human activity was causing warming (because of carbon emissions) or cooling (because of smog particles). The firstIPCC报告于1990年发布，规定，由于温室气体，这是大量人为的全球变暖。到2001年，这一索赔背后有一个强大的科学共识。

While policy-makers’ response to risks from climate change might seem likely to shed light on whether policy-makers will deal wisely with AGI, there are some important disanalogies:

  • 预计全球变暖的危害将在贫穷国家的弱势群体方面不成比例地下降，而不是政策制定者。因此，政策制定者的个人激励远远不那么个人动力，而不是AGI的情况。
  • 在中位数，全球变暖的人道主义损失似乎是about 20% of GDP per year for the poorest people. In light of anticipated economic development and marginal diminishing utility, this is a很多比AGI风险更小的负面的人道主义影响（甚至忽略后代）。例如，经济学家indur goklanyestimatedthat “through 2085, only 13% of [deaths] from hunger, malaria, and extreme weather events (including coastal flooding from sea level rise) should be from [global] warming.”
  • Thus, potential analogies to AGI risk come from climate change’stail risk。但是，似乎有很少的资本科学家，他们的观点与预测兼容的观点，即使概率分布的第95百分位数（达到2100）的温度增加，也会比刚开始才能使地球的某些区域无法居住地。
  • According to the第五IPCC.那the risk of human extinction from climate change seems very low: “Some thresholds that all would consider dangerous have no support in the literature as having a non-negligible chance of occurring. For instance, a ‘runaway greenhouse effect’—analogous to Venus—appears to have virtually no chance of being induced by anthropogenic activities.”

The 2008 financial crisis

乔纳对2008年金融危机进行了浅浅调查，但初步调查结果足以让我们在一些细节中描述它们。乔纳对2008年金融危机与AGI情况相关的印象是基于阅读的音乐停止后由Alan Blinder，谁是联邦储备金副主席，克林顿政府期间1.5年。当然，应该在借鉴政策制定者对金融危机的意义上明智地处理AGI的可能性的结论之前征求许多额外来源。

Blinder’s seven main factors leading to the recession are (p. 27):

  1. 夸大资产价格，特别是房屋（房屋泡沫），也具有某些证券（债券泡沫）;
  2. 整个金融体系和经济中的过度杠杆（重借款）;亚博体育苹果app官方下载
  3. LAX金融监管，无论是法律留下的不受管制，各种监管机构都履行了职责;
  4. 次次抵押和其他抵押贷款中的可耻银行惯例;
  5. 建立在这些糟糕的抵押贷款的未经管制的证券和衍生品的疯狂被子;
  6. 统计评级机构的深渊表现，帮助疯狂被子缝合在一起;和
  7. The perverse compensation systems in many financial institutions that created powerful incentives to go for broke.

通过这些因素，让我们来看看2008年金融危机与AGI的未来发明之间类比的实力：

  1. Almost tautologically, a financial crisis is unexpected, though we do know that financial crises happen with some regularity.
  2. The 2008 financial crisis was not unprecedented in kind, only in degree (in some ways).
  3. 避免2008年金融危机需要解决一个艰难的国家协调问题，而不是全球协调问题。仍然，这个类比似乎相当强烈。正如乔纳写的那样，“虽然2008年的金融危机似乎主要是我们特定的（在拥有更广泛的后果时），但有一种意义在于阻止它需要解决一个困难的协调问题。危机的原因是弥漫性，责任落在许多不同的行动者上。“
  4. 乔纳的分析不足以辨别2008年金融危机是否类似于AGI的未来发明，以便提前谨慎地致力于避免风险。
  5. 与AI风险相比，金融危机不是几乎任何参与者的生命或死亡问题。金融的许多人没有激励避免金融危机：实际上，涉及的一些关键数字被奖励奖励。但是，政府决策者对声誉造成的原因促进金融危机，许多利益群体受到金融危机的不利影响，这是合理的。
  6. Once again, the scale of the financial crisis wasn’t on a par with AI risk, but it was closer to that scale than the other risks Jonah looked at in this initial investigation.

Jonah concluded that “the conglomerate of poor decisions [leading up to] the 2008 financial crisis constitute a small but significant challenge to the view that [policy-makers] will successfully address AI risk.” His reasons were:

  1. The magnitude of the financial crisis is nontrivial (even if small) compared with the magnitude of the AI risk problem (not counting future generations).
  2. 金融危机产生了非常广泛的人，显然包括那些权力职位的大部分人（这似乎是在气候变化的情况下的那种人）。经济衰退对大多数企业和大多数工人来说都不糟糕。然而，这些演员无法识别问题，坐标和防止它。
  3. 政策制定者无法识别问题，协调和阻止的原因，它看起来与人们可能无法识别AI风险作为问题，协调和防止它的原因相关。首先，涉及的几个关键演员似乎表现出显着的过度控制和忽视尾部风险（例如夏天等忽略Brooksley出生的关于过度杠杆的警告）。如果是真，这表明权力位置的人们易受过度信养和忽视尾部风险的影响。避免过度交付并给予足够的重量尾部风险可能对缓解AI风险至关重要。其次，一个人有一种感觉，即在金融危机的情况下，公共场合的旁观者的效果和悲剧发挥了重要作用。有风险没有充分解决，因为这样做并没有属于任何现有政府机构的职权范围。这可能对应于“不是我的工作 - 其他人可以照顾它”的心态。“如果人们认为ai风险很大，那么他们可能会想到“如果没有人会照顾它，那么我会的，因为否则我会死。”但如果人们认为AI风险很小，他们可能会认为“这可能对我来说可能对我来说非常糟糕，甚至虽然有人应该照顾它，但它不会成为我。”

从地磁风暴的风险

像巨大的地磁风暴1859 Carrington Eventare infrequent, but could cause serious damage to satellites and critical infrastructure. See这个经合组织的报告概述。

乔纳调查显示出从地磁风暴的预期损失范围广，每年3000万美元至每年30亿美元。但在未来50年的预期损失中，即使这个较大的数量也达到1.5万亿美元。将此与2008年金融危机的损失进行比较（大约为50年的活动），这是estimatedto be about $13 trillion for Americans alone.

Though serious, the risks from geomagnetic storms appear to be small enough to be disanalogous to the future invention of AGI.

The eradication of smallpox

Smallpox那after killing more than 500 million people over the past several millennia, was eradicated in 1979 after a decades-long global eradication effort. Though a hallmark of successful global coordination, it doesn’t seem especially relevant to whether policy-makers will handle the invention of AGI wisely.

Here’s how the eradication of smallpox does our doesn’t fit our criteria for being analogous to the future invention of AGI:

  1. 天花没有达到一个不可预测的时间;它在根除竞选前到了千年。
  2. 在消除Smallpox之前，世界没有经验消除疾病，但许多国家已经消除了天花。
  3. Smallpox根除需要解决困难的全球协调问题，但在某种程度上是对AGI安全发明的侵害（参见本名单上的其他要点）。
  4. 在某种意义上提前准备小型露天努力，但在宣布竞选活动之前，大多数努力主要已经施加了努力。
  5. 没有Smallpox的国家有激励消除SmallPox，以便他们没有必要花钱来免疫公民，以便病毒不会（重新） - 他们的国家。例如，1968年，美国在常规的Smallpox疫苗中花了大约1亿美元。
  6. Smallpox can be thought of as a global disaster: by 1966, about 2 million people died of smallpox each year.

Shallow investigations of risks from cyberwarfare, chlorofluorocarbons, and the Cuban missile crisis

约拿的浅从cyberw调查的风险arfare revealed that experts disagree significantly about the nature and scope of these risks. It’s likely that dozens of hours of research would be required to develop a well-informed model of these risks.

To investigate how policy-makers handled the discovery that chlorofluorocarbons (CFCs) depleted the ozone layer, Jonah summarized the first 100 pages of臭氧危机：突然全球紧急情况下的15年演变（有关摘要，请参阅完整的电子邮件交换机）。这种历史案件似乎值得进一步调查，可能是政策制定者解决全球风险的案例，令人惊讶的迅速，尽管是否争议响应是争议的。

Jonah also did a shallow investigation of the古巴导弹危机。难以评估危机升级到全球核战争的可能性有何可能，但似乎似乎政策制定者在古巴导弹危机中导致了许多导致和期间的决定（见我们的全额电子邮件兑换）。乔纳结束了：

even if the probability of the Cuban missile crisis leading to an all out nuclear war was only 1% or so, the risk was still sufficiently great so that the way in which the actors handled the situation is evidence against elites handling the creation of AI well. (This contrasts with the situation with climate change, in that elites had strong personal incentives to avert an all-out nuclear war.)

However, this is only a guess based on a shallow investigation, and should not be taken too seriously before a more thorough investigation of the historical facts can be made.

可以调查的其他历史案例

We also identified additional historical cases that could be investigated for potentially informative analogies to the future invention of AGI:

  1. The 2003Iraq War
  2. The frequency with which dictators are deposed or assassinated due to “unforced errors” they made
  3. 核扩散
  4. 重组DNA
  5. 分子纳米技术
  6. Near Earth objects
  7. Pandemics and potential pandemics (e.g.艾滋病病毒那SARS.的）