(Two days later, Amber returns with another question.)

AMBER:嗯，说，珊瑚。当您构建一种全新的系统时，安全心态有多重要 - 例如，您希望它具有某种稳健性的属性，其中一个受到潜在不利优化压力的影响？亚博体育苹果app官方下载

CORAL:How novel is the system?

AMBER:非常新颖。

CORAL:Novel enough that you’d have to invent your own new best practices instead of looking them up?

AMBER:Right.

CORAL:That’s serious business. If you’re building a very simple Internet-connected system, maybe a smart ordinary paranoid could look up how we usually guard against adversaries, use as much off-the-shelf software as possible that was checked over by real security professionals, and not do too horribly. But if you’re doing something qualitatively new and complicated that has to be robust against adverse optimization, well… mostly I’d think you were operating in almost impossibly dangerous territory, and I’d advise you to figure out what to do after your first try failed. But if you wanted to actually succeed, ordinary paranoia absolutely would not do it.

AMBER:在other words, projects to build novel mission-critical systems ought to have advisors with the full security mindset, so that the advisor can say what the system builders really need to do to ensure security.

CORAL:(笑可悲) No.

AMBER:No?

Read more »

(AMBER, a philanthropist interested in a more reliable Internet, andCORAL, a computer security professional, are at a conference hotel together discussing what Coral insists is a difficult and important issue: the difficulty of building “secure” software.)

AMBER:因此，珊瑚，我了解您认为，在创建软件时，将该软件成为您所谓的“安全”非常重要。

CORAL:Especially if it’s connected to the Internet, or if it controls money or other valuables. But yes, that’s right.

AMBER:I find it hard to believe that this needs to be a separate topic in computer science. In general, programmers need to figure out how to make computers do what they want. The people building operating systems surely won’t want them to give access to unauthorized users, just like they won’t want those computers to crash. Why is one problem so much more difficult than the other?

CORAL:That’s a deep question, but to give a partial deep answer: When you expose a device to the Internet, you’re potentially exposing it to intelligent adversaries who can find special, weird interactions with the system that make the pieces behave in weird ways that the programmers did not think of. When you’re dealing with that kind of problem, you’ll use a different set of methods and tools.

AMBER:Any system that crashes is behaving in a way the programmer didn’t expect, and programmers already need to stop that from happening. How is this case different?

CORAL:好的，所以……想象您的系统将每次会议进行一千千字节的输入亚博体育苹果app官方下载。（尽管这本身就是我们要提出的一种假设，并问如果它得到了投入的兆字节，但不介意。^8,000可能的输入，或大约10^2,400或者。同样，为了扩展简单的可视化，请想象计算机每秒获得十亿个输入。假设只有一个Googol，10¹⁰⁰, out of the 10^2,400possible inputs, cause the system to behave a certain way the original designer didn’t intend.

如果该系统以与亚博体育苹果app官方下载输入是否不当的方式获得输入，那么它不会在宇宙结束前处于不当行为状态。另一方面，如果有一个聪明的对手了解系统，他们也许可以找到使系统不良的非常罕见的投入之一。亚博体育苹果app官方下载因此，当一个智能对手故意故意破坏它时亚博体育苹果app官方下载，一百万年中永远不会在一百万年内行为不良的系统中永远不会破裂。

AMBER:So you’re saying that it’s more difficult because the programmer is pitting their wits against an adversary who may be more intelligent than themselves.

CORAL:That’s an almost-right way of putting it. What matters isn’t so much the “adversary” part as the optimization part. There are systematic, nonrandom forces strongly selecting for particular outcomes, causing pieces of the system to go down weird execution paths and occupy unexpected states. If your system literally has no misbehavior modes at all, it doesn’t matter if you have IQ 140 and the enemy has IQ 160—it’s not an arm-wrestling contest. It’s just very much harder to build a system that doesn’t enter weird states when the weird states are being selected-for in a correlated way, rather than happening only by accident. The weirdness-selecting forces can search through parts of the larger state space that you yourself failed to imagine. Beating that does indeed require new skills and a different mode of thinking, what Bruce Schneier called “security mindset”.

AMBER:Ah, and what is this security mindset?

CORAL:我可以说一两件事，但是请记住，我们正在处理一种不完全有效的思维质量。如果我能给您一些有关安全心态的陈词滥调，这实际上会使您能够设计安全的软件，那么Internet看起来与目前的表现会大不相同。就是说，在我看来，所谓的“安全思维方式”可以分为两个组成部分，其中一个组成部分比另一个组成部分要困难得多。而且这可能会欺骗人们高估自己的安全性，因为他们可以使安全心态更容易并忽略另一半。不太困难的组成部分，我将以“普通偏执狂”一词称呼。

AMBER:Ordinaryparanoia?

CORAL:很多程序员有能力想象广告versaries trying to threaten them. They imagine how likely it is that the adversaries are able to attack them a particular way, and then they try to block off the adversaries from threatening that way. Imagining attacks, including weird or clever attacks, and parrying them with measures you imagine will stop the attack; that is ordinary paranoia.

AMBER:这不是什么安全性吗？您声称另一半是什么？

CORAL:To put it as a platitude, I might say… defending against mistakes in your own assumptions rather than against external adversaries.
Read more »