译/杜焱

我们都知道如何处理可疑包裹——那就是要多小心就有多小心！现在我们让机器人替我们去冒险。但是，如果机器人本身就是危险呢？一些评论员指出：我们应该像对待可疑包裹一样去看待人工智能，因为可能有一天它会给我们造成威胁。我们真该为此担忧吗？

爆发的智能？

[2]当被问及未来是否会和人一样聪明时，美国数学家、科幻小说作家瓦尔·温格回答道：“会的，可是这样的计算机只会短暂地出现一段时间。”

[3]他的意思是：一旦计算机发展到这个水平，势将在短时间内取得飞速发展。温格将智能这一井喷式的发展势头命名为“技术奇点”，同时他认为，从人类的角度来看，这可能不是一个好消息。

We know how to deal with suspicious packages—as carefully as possible!These days, we let robots take the risk.But what if the robots are the risk? Some commentators argue we should be treating AI (artificial intelligence) as a suspicious package, because it might eventually blow up in our faces. Should we be worried?

Exploding intelligence?

[2] Asked whether there will ever be computers as smart as people, the US mathematician and sci-f i author Vernor Vinge replied: “Yes, but only brief l y.”

[3] He meant that once computers get to this level, there’s nothing to prevent them getting a lot further very rapidly.Vinge christened this sudden explosion of intelligence the “technological singularity”, and thought that it was unlikely to be good news, from a human point of view.

[4] Was Vinge right, and if so what should we do about it? Unlike typical suspicious parcels, after all, what the future of AI holds is up to us, at least to some extent. Are there things we can do now to make sure it’s not a bomb (or a good bomb rather than a bad bomb,perhaps)?

AI as a low achiever

[5] Optimists sometimes take comfort from the fact the field of AI has very chequered past. Periods of exuberance and hype have been mixed with socalled “AI winters”—times of reduced funding and interest, after promised capabilities fail to materialise.

[6] Some people point to this as evidence machines are never likely to reach human levels of intelligence,let alone to exceed them. Others point out that the same could have been said about heavier-than-air flight.

[7] The history of that technology,too, is littered with naysayers (some of whom refused to believe reports of the Wright brothers’ success, apparently).

[4]温格的观点是否正确？如果正确，我们该如何应对？毕竟，人工智能产品不同于一般的可疑包裹，至少从某种程度上来说，人工智能的未来是由我们决定的。现在，我们能做点什么来确保人工智能产品不是一个“炸弹”（或者，如果可能的话，是一个杀伤力较小的炸弹，而不是一个危害性极强的炸弹。）？

人工智能成就平平

[5]人工智能的发展历经曲折，乐观主义者有时可从中得到安慰。人工智能的发展有过繁荣期和炒作大热期，但在这过程中，也曾多次遭遇所谓的“人工智能的冬天”，即由于预先承诺实现的功能无法兑现，导致投入资金削减，研究热度降低的阶段。

[6]有人以此为证据，指出机器永远不可能达到人类同等的智力水平，更不用说超越。也有人指出，曾有人说过飞机重于空气，不可能起飞的话。

[7]飞行技术发展的过程中，也充斥着否定者的声音（显然，其中有些人不愿相信莱特兄弟已经获得成功的新闻报道。）和飞机重于空气一样，对人工智能持否定观点的人必须正视一个事实，即：自然已经设定好规则：不要将人类的脑力与鸟类的飞行力混为一谈。

[8]想要令人信服地否定人工智能，需要一个理由证明人类技术永远达不到人工智能的水平。

[9]秉持悲观论调更容易。但需要提醒的是，所有人都知道人类的大脑存储智慧，而一部分人会致力于了解更多关于人类智慧的原理。这很难支撑人类技术永远达不到人工智能水平的说法，恰恰相反，人们正在不断增进对其的了解。

摩尔定律和弱人工智能

[10]不管是在硬件还是软件方面，我们在技术层面上似乎越来越接近人工智能的核心原理。在硬件领域，摩尔定律预测的一块芯片所负载的计算能力每两年翻一番没有丝毫减缓的迹象。

[11]在软件领域，“强人工智能”（即与人类智力相当甚至超越人类智力的人工智能）是否有可能出现，人们对此争论不休。但是，“弱人工智能”（即仅限于完成某些特定任务的人工智能）正稳步向前发展。计算机占领了一个又一个过去人们认为只有人的智力和直觉力才能够胜任的工作领域。

For human-level intelligence, as for heavier-than-air flight, naysayers need to confront the fact nature has managed the trick: think brains and birds,respectively.

[8] A good naysaying argument needs a reason for thinking that human technology can never reach the bar in terms of AI.

[9] Pessimism is much easier. For one thing, we know nature managed to put human-level intelligence in skullsized boxes, and that some of those skull-sized boxes are making progress in figuring out how nature does it. This makes it hard to maintain that the bar is permanently out of reach of artificial intelligence—on the contrary, we seem to be improving our understanding of what it would take to get there.

Moore’s law and narrow AI

[10] On the technological side of the fence, we seem to be making progress towards the bar, both in hardware and in software terms. In the hardware arena, Moore’s law, which predicts that the amount of computing power we can fit on a chip doubles every two years,shows little sign of slowing down.

[11] In the software arena, people debate the possibility of “strong AI”(artificial intelligence that matches or exceeds human intelligence) but the caravan of “narrow AI” (AI that’s limited to particular tasks) moves steadily forward. One by one, computers take over domains that were previously considered off-limits to anything but human intellect and intuition.

[12] We now have machines that have trumped human performance in such domains as chess, trivia games, flying,driving, financial trading, face, speech and handwriting recognition—the list goes on.

[13] Along with the continuing progress in hardware, these developments in narrow AI make it harder to defend the view that computers will never reach the level of the human brain. A steeply rising curve and a horizontal line seem destined to intersect!

[12]现在，机器的性能在很多领域都远远超过了人类的表现，比如国际象棋、益智游戏、飞行、汽车驾驶、金融贸易、人脸识别、语音识别、字迹识别等，这样的例子不胜枚举。

[13]在硬件不断发展的同时，弱人工智能的这些发展让人们更难相信“计算机永远不会达到人脑的智力水平”这一说法。毕竟，人工智能一直在快速向前发展，如同一条急剧上升的曲线；而人类智能处于稳定状态，是一条水平线，这两条线似乎注定要相交。

智能帮手，有什么不好？

[14]如果计算机变得跟人一样聪明，这难道不是一件好事吗？看看弱人工智能目前取得的一系列成功，这也许正好说明有些人态度消极是毫无根据的。归根到底，这些应用程序不是非常实用吗？也许国际象棋大师的自尊心会遭到一点打击，金融市场会出现些许动荡，但在上述领域中，我们并没有看到任何灾难性事件即将发生的迹象。

[15]事实确实如此，悲观主义者如是说。可是就人类未来发展而言，弱人工智能所渗透的各个领域对人类生活所造成的影响可大可小。有些领域人工智能的影响大于其他领域。（比如，在未来十年左右，如果机器人代替人类驾驶汽车，那我们的经济将会发生翻天覆地的变化。）

What’s so bad about intelligent helpers?

[14] Would it be a bad thing if computers were as smart as humans?The list of current successes in narrow AI might suggest pessimism is unwarranted. Aren’t these applications mostly useful, after all? A little damage to Grandmasters’ egos, perhaps, and a few glitches on financial markets, but it’s hard to see any sign of impending catastrophe on the list above.

[15] That’s true, say the pessimists,but as far as our future is concerned, the narrow domains we yield to computers are not all created equal. Some areas are likely to have a much bigger impact than others. (Having robots drive our cars may completely rewire our economies in the next decade or so, for example.)

[16] The greatest concerns stem from the possibility that computers might take over domains that are critical to controlling the speed and direction of technological progress itself.

[16]一些至关重要的领域，其本身就掌控着技术进步的速度和方向，它们也有可能被计算机接管，这才是人们最担心的问题。

不仅像人，而且比人聪明！

[17]超过人类自身能力的任何智能都可以像人类一样，在一些重要领域，甚至比人脑聪明得多，这让人深感欣慰。但是，再一次，那些悲观的人看到了一些坏消息。他们说，几乎人类所珍视的一切（爱、幸福，甚至生存）对我们而言都是无比重要的，因为我们人类拥有自己专属的进化史——这是我们与高等动物共同享有的历史，与人工智能这样的计算机程序并无关系。

[18]于是，似乎没有什么理由可以让我们默认智能机器可以和我们人类拥有同样的价值观。好消息是，这样可能我们也找不到任何理由认为智能机器会充满敌意，毕竟敌意也是动物才会有的情感。

Not just like us, but smarter!

[17] It would be comforting to think that any intelligence that surpassed our own capabilities would be like us, in important respects—just a lot cleverer.But here, too, the pessimists see bad news: they point out that almost all the things we humans value (love,happiness, even survival) are important to us because we have particular evolutionary history—a history we share with higher animals, but not with computer programs, such as artificial intelligences.

[18] By default, then, we seem to have no reason to think that intelligent machines would share our values. The good news is that we probably have no reason to think they would be hostile,as such: hostility, too, is an animal emotion.

[19] The bad news is that they might simply be indifferent to us—they might care about us as much as we care about the bugs on the windscreen.

[20] People sometimes complain that corporations are psychopaths, if they are not sufficiently reined in by human control. The pessimistic prospect here is that artificial intelligence might be similar, except much much cleverer and much much faster.

[19]坏消息是：它们可能也只会用冷漠的态度对待我们——我们有多不在意爬在挡风玻璃上的小虫子，它们就会有多不在意我们。

[20]人们有时会抱怨，如果管理者不能有效管理，企业会陷入混乱。悲观者认为，人工智能和企业一样，只不过人工智能智商更高，反应也更迅速。

发展阻碍

[21]现在，透过这些悲观的想法，我们可以看到人工智能的未来发展方向。问题是：设计出与人类智能相当的计算机（至少是在影响技术进步的重要领域），就意味着我们冒着将地球交到智能机器手里的风险，而这些智能机器对我们漠不关心，对生命、可持续发展的环境等我们所珍视的一切漠不关心。

[22]悲观者如是说，如果这听上去有些牵强附会，就去问问大猩猩的感受吧，问问它们与最聪明的物种——人类——争夺资源时感受如何。基本上可以这样认为，大猩猩会灭绝不是因为人类对它们恶意满满，而是因为受人类控制的自然环境已经不适合它们继续存活下去。

Getting in the way

[21] By now you see where this is going, according to this pessimistic view. The concern is that by creating computers that are as intelligent as humans (at least domains that matter to technological progress), we risk yielding control over the planet to intelligences that are simply indifferent to us, and to things that we consider valuable—things such as life and a sustainable environment.

[22] If that sounds far-fetched, the pessimists say, just ask gorillas how it feels to compete for resources with the most intelligent species—the reason they are going extinct is not (on the whole) because humans are actively hostile towards them, but because we control the environment in ways that are detrimental to their continuing survival.

“也许，这些消极的观点都是不对的！”

[23]正如悲观主义者所说，不可否认，人们很难做出预测，尤其是预测未来！但是我们在日常生活中也非常认真地对待各种不确定性，尤其是在危如累卵的时刻。

[24]归根到底，这也是为什么我们会使用价格昂贵的机器人去检查那些可疑包裹。（尽管我们明白只有极少数的包裹中有炸弹。）

[25]如本文所述，人工智能未来也会“爆炸”，那将会是人类碰到的最后一颗炸弹，因为自此人类将不复存在。这样看来，即使我们有充分的理由相信人工智能爆炸的风险很小，人们心怀疑虑的态度也很合理。

[26]就目前而言，即使是这种程度的宽慰似乎也无法实现，因为我们对人工智能不够了解，因此无法信心满满地进行风险评估。（毕竟，自我感觉乐观和有充足的理由保持乐观不是一回事。）

“The pessimists might be wrong!”

[23] Of course—making predictions is difficult, as they say, especially about the future! But in ordinary life we take uncertainties very seriously, when a lot is at stake.

[24] That’s why we use expensive robots to investigate suspicious packages, after all (even when we know that only a very tiny proportion of them will turn out to be bombs).

[25] If the future of AI is “explosive”in the way described here, it could be the last bomb the human species ever encounters. A suspicious attitude would seem more than sensible, then, even if we had good reason to think the risks are very small.

[26] At the moment, even that degree of reassurance seems out of our reach—we don’t know enough about the issues to estimate the risks with any high degree of confidence. (Feeling optimistic is not the same as having good reason to be optimistic, after all.)

What to do?

[27] A good first step, we think, would be to stop treating intelligent machines as the stuff of science fiction, and start thinking of them as a part of the reality that we or our descendants may actually confront, sooner or later.

[28] Once we put such a future on the agenda we can begin some serious research about ways to ensure outsourcing intelligence to machines would be safe and beneficial, from our point of view.

[29] Perhaps the best cause for optimism is that, unlike ordinary ticking parcels, the future of AI is still being assembled, piece by piece, by hundreds of developers and scientists throughout the world.

[30] The future isn’t yet fixed, and there may well be things we can do now to make it safer. But this is only a reason for optimism if we take the trouble to make it one, by investigating the issues and thinking hard about the safest strategies.

[31] We owe it to our grandchildren—not to mention our ancestors, who worked so hard for so long to get us this far!—to make that effort. ■

接下来怎么办？

[27]我们认为，不能再把智能机器当作科幻小说的素材，要开始把它们看作是我们这一代或我们的后代可能迟早要面对的现实中的一部分，这是我们首先要做的事。

[28]一旦把这样的未来放在议事日程上，我们就可以开始认真地研究一些方法，确保将智能赋予机器安全且有益。

[29]也许，可以让我们保持乐观的最好理由是，与普通的包裹不同，人工智能的未来仍需要世界各地成百上千的开发者和科学家一块一块去组装。

[30]未来变数满满，而现在我们能做的就是让未来更安全。但是，如果我们认真对待问题，深入了解并认真思考最安全的策略，上面所述也只是我们乐观的理由。

[31]我们做这一切都是为了我们的子孙后代，更不用说我们的祖先为我们现有的成就而做出的努力。 □