The Machine Intelligence Research Institute’s2015 winter fundraising drive开始于12月1日！我们目前的进步：
该驱动器将于12月份的月份运行，并有助于支持Miri的研究工作，旨在确保更智能的AI系统具有积极影响。亚博体育官网亚博体育苹果app官方下载
MIRI’s Research Focus
The field of AI has a goal of automating perception, reasoning, and decision-making — the many abilities we group under the label “intelligence。“Most leading researchers in AI预计our best AI algorithms to begin strongly outperforming humans this century in most cognitive tasks. In spite of this, relatively little time and effort has gone into trying to identify the technical prerequisites for making smarter-than-human AI systems safe and useful.
We believe that several basic theoretical questions will need to be answered in order to make advanced AI systems stable, transparent, and error-tolerant, and in order to specify correct goals for such systems. Our技术议程describes what we think are the most important and tractable of these questions.
- 高能力天花板- 人类似乎无处可行的认知能力靠近物理限制，智力中的适度优势也可能会产生对AI系统的决定性战略优势。亚博体育苹果app官方下载
- “Sorcerer’s Apprentice” scenarios- 更智能的AI系统可以提亚博体育苹果app官方下载出越来越广泛的方式来满足程序的目标。预期如何实现目标是更难的，指定正确的目标是越难的。
- 收敛的工具目标— By default, highly capable decision-makers are likely to have incentives to treat human operators adversarially.
- AI speedup effects— Progress in AI is likely to accelerate as AI systems approach human-level proficiency in skills like software engineering.
We think MIRI is well-positioned to make progress on these problems for four reasons: our initial technical results have been promising (see ourpublications), our methodology has a good track record of working in the past (seeMiri的方法), we have already had a significant influence on the debate about long-run AI outcomes (seeAssessing Our Past and Potential Impact), and we have an exclusive focus on these issues (see什么让miri分开？). MIRI is currently the only organization specializing in long-term technical AI safety research, and our independence from industry and academia allows us to effectively address gaps in other institutions’ research efforts.
General Progress This Year
6月，卢克·穆罕沃斯left MIRI在公开慈善项亚博体育官网目的研究职位。我取代了卢克作为Miri的执行董事，我很高兴地说过渡已经顺利。我们在技术研究和学术外展之间分开了我们的时间，跑了一个亚博体育官网workshop series旨在向我们的工作引入更广泛的科学观众，并赞助三周summer fellows programaimed at training skills required to do groundbreaking theoretical research.
Our fundraiser this summer was our biggest to date. We raised a total of $631,957 from 263 distinct donors, smashing our previous funding drive record by over $200,000. Medium-sized donors stepped up their game to help us hitour first two funding targets：更多的捐助者比过去的筹款人在5,000美元到50,000美元之间。我们的成功筹款人员，讲习班和研究员计划使我们能够大幅增加我们的增长，并且已经直接导致了几个新的研究员雇用。亚博体育官网
三个新的全日制研究员都加入了我们的亚博体育官网teamthis year: Patrick LaVictoire in March, Jessica Taylor in August, and Andrew Critch in September. Scott Garrabrant will become our newest research fellow this month, after having made major contributions as a workshop attendee and research associate.
Meanwhile, our two new research interns, Kaya Stechly and Rafael Cosman, have been going through old results and consolidating and polishing material into new papers; and three of our new research associates, Vanessa Kosoy, Abram Demski, and Tsvi Benson-Tilsen, have been producing a string of promising results on our research forum. Another intern, Jack Gallagher, contributed to our类型理论项目over the summer.
以适应我们的团队成长,we’ve recently hired a new office manager, Andrew Lapinski-Barker, and will be moving into a larger office space this month. On the whole, I’m very pleased with our new academic collaborations, outreach efforts, and growth.
Research Progress This Year
随着我们的研究亚博体育官网项目和合作乘以，我们更多地利用在线机制，以便在研究人员之间快速沟通和反馈。3月，我们推出了智能代理基础论坛, a discussion forum for AI alignment research. Many of our subsequent publications have been developed from material on the forum, beginning with Patrick LaVictoire’s “Löb在Miri研究中的定理介绍亚博体育官网。“
We have also produced a number of new papers in 2015 and, most importantly, arrived at new research insights.
在decision theory, Patrick LaVictoire and others have developed new results pertaining to bargaining and division of trade gains, using the proof-based decision theory framework (example). Meanwhile, the team has been developing a better understanding of the strengths and limitations of different approaches to decision theory, an effort spearheaded by Eliezer Yudkowsky, Benya Fallenstein, and me, culminating in some insights that will appear in a paper next year. Andrew Critch has proved some promising results about bounded versions of proof-based decision-makers, which will also appear in an upcoming paper. Additionally, we presented ashortened versionof our overview paper at AGI-15.
在Vingean reflection，Benya Fallenstein和研究助理R亚博体育官网amana Kumar合作“Proof-producing reflection for HOL“ （提出了at ITP 2015) and have been working on anFLI-funded霍尔定理箴言中的反思推理的实施。另外，反思性的甲骨文框架已经帮助我们更好地了解了什么样的反射，也是不可能的，产生一些nice technical results和a few insights that seem promising. We also published the overview paper “Vingean reflection。“
Jessica Taylor，Benya Fallenstein和Eliezer Yudkowsky专注于错误宽容on and off throughout the year. We released Taylor’s “普通ilizers.” (accepted to a workshop at AAAI-16) and presented the paper “Corrigibility” at a AAAI-15 workshop.
在value specification, we published the AAAI-15 workshop paper “安全自主ai的概念学习“和概述文件”价值学习问题。“With support from anFLI补助金，杰西卡泰勒正在在这一领域的更好地正式形成亚各布，最近开始向这一主题编写她的想法论研究论坛亚博体育官网。
最后，在forecasting和strategy，我们发表了“正式化收敛工具目标” (accepted to a AAAI-16 workshop) and two historical case studies: “The Asilomar Conference” and “Leó Szilárd and the danger of nuclear weapons。“Many other strategic analyses have been posted to the recently revampedai影响现场，凯特贾格雷德一直在发布关于技术发展模式的研究。亚博体育官网
筹款人的目标和未来计划
Like our last fundraiser, this will be a non-matching fundraiser with multiple funding targets our donors can choose between to help shape MIRI’s trajectory. Our successful summer fundraiser has helped determine how ambitious we’re making our plans; although we may still slow down or accelerate our growth based on our fundraising performance, our current plans assume a budget of roughly每年1,825,000美元。
Of this, about $100,000 is being paid for in 2016 through FLI grants, funded by Elon Musk and the Open Philanthropy Project. The rest depends on our fundraising and grant-writing success. We have a twelve-month runway as of January 1, which we would ideally like to extend.
Taking all of this into account, our winter funding targets are:
目标1 -$ 150k：持久稳定。在这个级别，我们将有足够的资金在2016年初维持我们的跑道，同时继续所有目前的行动，包括运行研讨会，写作论文和参加会议。
Target 2 —$450k: Maintaining MIRI’s growth rate.At this funding level, we would be much more confident that our new growth plans are sustainable, and we would be able to devote more attention to academic outreach. We would be able to spend less staff time on fundraising in the coming year, and might skip our summer fundraiser.
目标3 -$1M: Bigger plans, faster growth.在这个级别，我们将能够大大提高招聘努力并接受新的研究项目。亚博体育官网很明显，我们的捐助者的支持比我们想象的更强大，我们将相应地扩大计划和增长率。
目标4 -$6M: A new MIRI.此时，Miri将成为一个定性不同的组织。凭借这种筹资级别，我们将能够使我们的研究举措多样化，并从我们目前的议程开始分支为AI对准问题的替代攻击角度。亚博体育官网
Our largest cost ($700,000) is in wages and benefits for existing research staff and contracted researchers, including research associates. Our current priority is to further expand the team. We expect to spend an additional $150,000 on salaries and benefits for new research staff in 2016, but that number could go up or down significantly depending on when new research fellows begin work:
- Mihály Bárász, who was originally开始开始in November 2015, has delayed his start date due to unexpected personal circumstances. He plans to join the team in 2016.
- We are recruiting a specialist for our型理论型理论项目，旨在制定简单的反思推理模型的程序模式。对这个主题的兴趣最近一直在增加, which is exciting; but the basic tools needed for our work are still missing. If you have programmer or mathematician friends who are interested in dependently typed programming languages and MIRI’s work, you can send them ourapplication form。
- We are considering several other possible additions to the research team.
Much of the rest of our budget goes into fixed costs that will not need to grow much as we expand the research team. This includes $475,000 for administrator wages and benefits and $250,000 for costs of doing business. Our main cost of doing business is renting office space (slightly over $100,000).
Note that the boundaries between these categories are sometimes fuzzy. For example, my salary is included in the admin staff category, despite the fact that I spend some of my time on technical research (and hope to increase that amount in 2016).
Our remaining budget goes into organizing or sponsoring research events, such as fellows programs,Mirix活动, or workshops ($250,000). Some activities (e.g., traveling to conferences) are aimed at sharing our work with the larger academic community. Others, such as researcher retreats, are focused on solving open problems in our research agenda. After experimenting with different types of research staff retreat in 2015, we’re beginning to settle on a model that works well, and we’ll be running a number of retreats throughout 2016.
在past years, we’ve generally raised $1M per year, and spent a similar amount. Thanks to substantial recent increases in donor support, however, we’re in a position to scale up significantly.
我们的捐赠者在我们最后的筹款制定者中吹了我们的支持。如果我们能够继续我们的筹款和补助成功，我们将能够维持我们的新预算，并采取符合概述的独特机会为什么现在很重要,帮助制定议程,建立正式的工具s for the young field of AI safety engineering. And if our donors keep stepping up their game, we believe we have the capacity to scale up our program even faster. We’re thrilled at this prospect, and we’re enormously grateful for your support.
