AlphaGo Zerouses 4 TPUs, is built entirely out of neural nets with no handcrafted features, doesn’t pretrain against expert games or anything else human, reaches a superhuman level after 3 days of self-play, and is the strongest version of AlphaGo yet.

该体系结构已简化。以前的Alphago有一个预测出色比赛的政策网，并且一个评估位置的价值网，两者都使用MCT馈入LookAhead（随机概率加权将在游戏结束时发挥）。Alphago Zero具有一个选择动作的神经网，该网络由Paul-Christiano风格训练能力扩增，与自己一起玩游戏，以学习获胜的新概率。

As others have also remarked, this seems to me to be an element of evidence that favors the Yudkowskian position over the Hansonian position in my and Robin Hanson’sAI-Foom辩论。

正如我所记得的那样，正如我所理解的那样：

• 汉森怀疑他所说的“建筑”很重要，与（汉森说的）元素相比，诸如累积领域知识或专业公司建造的特殊用途组件（他期望是为AI服务的公司的生态学）而建立的特殊用途组件经济。
• 当我评论一下，当人类对黑猩猩的建筑改进，这对我来说很重要时，汉森回答说，这在他看来，这似乎是一次性的收益，从允许文化的知识积累。

I emphasize how all the mighty human edifice of Go knowledge, the joseki and tactics developed over centuries of play, the experts teaching children from an early age, was entirely discarded by AlphaGo Zero with a subsequent performance improvement. These mighty edifices of human knowledge, as I understand the Hansonian thesis, are supposed to bethebulwark against rapid gains in AI capability across multiple domains at once. I said, “Human intelligence is crap and our accumulated skills are crap,” and this appears to have been borne out.

Similarly, single research labs like DeepMind are not supposed to pull far ahead of the general ecology, because adapting AI to any particular domain is supposed to require lots of components developed all over the place by a market ecology that makes those components available to other companies. AlphaGo Zero is much simpler than that. To the extent that nobody else can run out and build AlphaGo Zero, it’s either because Google has Tensor Processing Units that aren’t generally available, or because DeepMind has a silo of expertise for being able to actually make use of existing ideas like ResNets, or both.

Sheer speed of capability gain should also be highlighted here. Most of my argument for FOOM in the Yudkowsky-Hanson debate was about self-improvement and what happens when an optimization loop is folded in on itself. Though it wasn’t necessary to my argument, the fact that Go play went from “nobody has come close to winning against a professional” to “so strongly superhuman they’re not really bothering any more” over two years just because that’s what happens when you improve and simplify the architecture, says您甚至不需要自我完善要获得看起来像福姆的东西。

是的，GO是一个允许自我播放的封闭系统。亚博体育苹果app官方下载人类仍然花了几个世纪的时间才能学习如何玩游戏。也许新的汉森尼亚堡垒反对快速能力增长可能是，环境有很多经验的地方，这些幅度应该很难学习，即使在AI的限制中，也足够快地吹嘘过去几个世纪以来的人类式学习;而且，人类已经在数百年的文化积累中学到了这些重要的知识，even though we know that humans take centuries to do 3 days of AI learning when humans have all the empirical bits they need;and that AIs cannot absorb this knowledge very quickly using “architecture”,even though humans learn it from each other using architecture。If so, then let’s write down this new world-wrecking assumption (that is, the world ends if the assumption is false) and be on the lookout for further evidence that this assumption might perhaps be wrong.

AlphaGo clearly isn’t a general AI. There’s obviously stuff humans do that make us much more general than AlphaGo, and AlphaGo obviously doesn’t do that. However, if even with the human special sauce we’re to expect AGI capabilities to be slow, domain-specific, and requiring feed-in from a big market ecology, then the situation we see without human-equivalent generality special sauce should not look like this.

换句话说，我在递归自我完善和从灵长类情报到人类智能的变化方面的辩论中非常重视。这并不意味着我们无法获得有关能力速度提高的信息withoutself-improvement. It doesn’t mean we can’t get info about the importance and generality of algorithmswithoutthe general intelligence trick. The debate can start to settle for fast capability gains before we even get to what I saw as the good parts; I wouldn’t have predicted AlphaGo and lost money betting against the speed of its capability gains, because reality held a more extreme position than I did on the Yudkowsky-Hanson spectrum.

(Reply from Robin Hanson.)

What is the function of a fire alarm?

One might think that the function of a fire alarm is to provide you with important evidence about a fire existing, allowing you to change your policy accordingly and exit the building.

在1968年拉坦和达利（Latane and Darley）的经典实验中，要求八组三个学生在一个房间里填写一份问卷，不久之后不久就开始充满烟雾。八个小组中有五个没有反应或报告烟雾，即使它变得足够稠密以使它们开始咳嗽。随后的操作表明，一个孤独的学生将有75％的时间做出反应。虽然一名学生陪同两个演员被告知假装冷漠的时间只有10％的时间。这项和其他实验似乎确定正在发生的事情是多元化的无知。We don’t want to look panicky by being afraid of what isn’t an emergency, so we try to look calm while glancing out of the corners of our eyes to see how others are reacting, but of course they are also trying to look calm.

(I’ve read a number of replications and variations on this research, and the effect size is blatant. I would not expect this to be one of the results that dies to the replication crisis, and I haven’t yet heard about the replication crisis touching it. But we have to put a maybe-not marker on everything now.)

A fire alarm creates common knowledge, in the you-know-I-know sense, that there is a fire; after which it is socially safe to react. When the fire alarm goes off, you know that everyone else knows there is a fire, you know you won’t lose face if you proceed to exit the building.

这fire alarm doesn’t tell us with certainty that a fire is there. In fact, I can’t recall one time in my life when, exiting a building on a fire alarm, there was an actual fire. Really, a fire alarm isweaker火灾的证据比从门下传来的烟雾的证据。

But the fire alarm tells us that it’s socially okay to react to the fire. It promises us with certainty that we won’t be embarrassed if we now proceed to exit in an orderly fashion.

It seems to me that this is one of the cases where people have mistaken beliefs about what they believe, like when somebody loudly endorsing their city’s team to win the big game will back down as soon as asked to bet. They haven’t consciously distinguished the rewarding exhilaration of shouting that the team will win, from the feeling of anticipating the team will win.

When people look at the smoke coming from under the door, I think they think their uncertain wobbling feeling comes from not assigning the fire a high-enough probability of really being there, and that they’re reluctant to act for fear of wasting effort and time. If so, I think they’re interpreting their own feelings mistakenly. If that was so, they’d get the same wobbly feeling on hearing the fire alarm, or even more so, because fire alarms correlate to fire less than does smoke coming from under a door. The uncertain wobbling feeling comes from the worry that others believe differently, not the worry that the fire isn’t there. The reluctance to act is the reluctance to be seen looking foolish, not the reluctance to waste effort. That’s why the student alone in the room does something about the fire 75% of the time, and why people have no trouble reacting to the much weaker evidence presented by fire alarms.

时不时地提议我们应该以后对人工通用情报的问题作出反应（背景在这里), because, it is said, we are so far away from it that it just isn’t possible to do productive work on it today.

(For direct argument about there being things doable today, see: Soares and Fallenstein (2014/2017); Amodei, Olah, Steinhardt, Christiano, Schulman, and Mané (2016); or Taylor, Yudkowsky, LaVictoire, and Critch (2016）。

(如果这些论文存在或者如果你是一个AI researcher who’d read them but thought they were all garbage, and you wished you could work on alignment but knew of nothing you could do, the wise next step would be to sit down and spend two hours by the clock sincerely trying to think of possible approaches. Preferably without self-sabotage that makes sure you don’t come up with anything plausible; as might happen if, hypothetically speaking, you would actually find it much more comfortable to believe there was nothing you ought to be working on today, because e.g. then you could work on other things that interested you more.)

（但是没关系。）

So if AGI seems far-ish away, and you think the conclusion licensed by this is that you can’t do any productive work on AGI alignment yet, then the implicit alternative strategy on offer is: Wait for some unspecified future event that tells us AGI is coming near; andthenwe’ll all know that it’s okay to start working on AGI alignment.

在我看来，这是错误的。这里是其中的一些。

阅读更多 ”