This is a dream reinvented with every breaking wave of technology, and with AI, the wave is a big one.

这个梦想随着每一次技术浪潮的涌现而被重新构想，而人工智能的浪潮尤为巨大。

The AI we have is already smart, at least in some ways, but could it ever be conscious?

我们现有的人工智能在某些方面已经相当智能，但它是否可能拥有意识呢？

Will a robot ever gaze at a sunset and experience the beautiful colors, the reds and the oranges?

机器人是否会凝视日落，感受到那美丽的色彩，那些红色和橙色？

Will it feel a sense of beauty or a rush of joy?

它会感受到美，或是体验到一阵喜悦吗？

Or will computers, however smart they get, always remain dark on the inside, always an object and never a subject?

还是说，无论计算机变得多么智能，它们内部始终是一片黑暗，永远只是客体而非主体？

Whether AI can be conscious is one of the most consequential questions we face in our time.

人工智能是否能有意识，是我们这个时代面临的最具深远影响的问题之一。

If computers can be conscious or sentient or aware, we'd be entering a new era in human history.

如果计算机能够拥有意识、感知或觉知，我们将进入人类历史的新纪元。

We'd have new entities that have their own inner lives, new inventions that matter for their own sakes and not only for their effects on us.

我们将拥有拥有自己内心生活的新实体，这些新发明不仅因其对我们产生影响而重要，更因其自身的存在而具有意义。

Conscious AI might suffer the click of a mouse, perhaps in ways we wouldn't even recognize.

有意识的人工智能可能会因鼠标的点击而受苦，其方式或许是我们甚至无法识别的。

And if silicon can be sentient, then maybe our messy flesh and blood bodies will soon be superseded by machines that never age and cannot die.

如果硅基物质也能拥有意识，那么我们这些血肉之躯或许很快就会被永不衰老、不会死亡的机器所取代。

Now over the last few years, progress in AI has been simply astonishing, and who knows what's around the corner.

过去几年里，人工智能的进展简直令人震惊，谁知道接下来还会出现什么突破。

Many experts think that conscious AI is possible, quite a few think it's inevitable, and some, some think it's here already.

许多专家认为有意识的人工智能是可能实现的，相当一部分人认为这不可避免，还有一些人甚至觉得它已经存在了。

I think they're wrong.

我认为他们错了。

I want to tell you why and why this matters so much.

我想告诉你们为什么，以及为什么这个问题如此重要。

So I've been studying brains, minds and consciousness for nearly thirty years, and one thing I've learned is that to answer the question can AI be conscious, we need to start by looking within ourselves at the makeup of our own human minds.

我研究大脑、思维和意识已近三十年，我学到的一点是：要回答人工智能能否拥有意识这个问题，我们需要先从自身入手，审视人类心智的构成。

Now we humans, we tend to see the world in our own terms.

我们人类倾向于以自己的方式看待世界。

We know we're conscious and we like to think that we're intelligent.

我们知道我们有意识，也乐于认为自己很聪明。

So we think the two go together, and this is why some people think that consciousness might just glimmer into existence as AI gets smarter and smarter.

因此我们认为这两者是相辅相成的，这也是一些人认为随着人工智能越来越聪明，意识可能会悄然出现的原因。

But consciousness and intelligence are different things.

但意识和智能是两回事。

Intelligence is all about doing it's solving a crossword puzzle, assembling some furniture, navigating a tricky family situation.

智能关乎做事——比如解填字游戏、组装家具、处理棘手的家庭状况。

Consciousness on the other hand, it's all about feeling and being.

而意识则关乎感受和存在。

It's the difference between normal wakeful awareness and the oblivion of general anesthesia.

这就像清醒的觉知与全身麻醉下的无意识之间的区别。

It's the bitter tang of coffee.

它是咖啡的苦涩。

It's the warmth of a log fire.

是篝火的温暖。

The joy of seeing a loved one.

是见到所爱之人的喜悦。

Just because consciousness and intelligence go together in us does not mean that they go together in general.

仅仅因为意识和智能在我们身上共存，并不意味着它们普遍相伴。

The assumption that they do well, that's a reflection of our own psychology, not an insight into the nature of reality.

认为它们密不可分的假设，其实反映了我们自身的心理，而非对现实本质的洞察。

Take language models like Claude or GPT trained on vast quantities of written texts.

以Claude或GPT等语言模型为例，它们基于海量文本训练而成。

They reflect back to us an image of ourselves, of our collective digitized past.

它们向我们反射出我们自身的影像，以及我们集体数字化的过去。

We talk about ourselves endlessly, and so do they.

我们无休止地谈论自己，他们也是如此。

We wonder about consciousness and the meaning of it all, and so it seems do they.

我们思考意识及其意义，他们似乎也在这样做。

But language models are not conscious.

但语言模型并没有意识。

they simulate consciousness, we project consciousness into them in the same way we might project faces into clouds or even the image of mother teresa in a cinnamon bun.

它们模拟意识，而我们则将意识投射到它们身上，就像我们可能在云中看到人脸，甚至在一个肉桂卷上看到特蕾莎修女的形象一样。

i know you can see that there she is.

我知道你能看到她在那里。

can you mother teresa in a cinnamon bun?

你能在一个肉桂卷上看到特蕾莎修女吗？

we are built to be seduced like narcissus by our own reflections, and so we see ourselves in our algorithms.

我们天生就像纳西索斯一样，容易被自己的倒影所诱惑，因此我们在算法中看到了自己。

I'm always struck that nobody really worries whether DeepMind's AlphaFold is conscious.

我一直感到惊讶的是，没有人真正担心DeepMind的AlphaFold是否有意识。

AlphaFold predicts the structure of proteins rather than words and sentences, but under the hood, it's not much different from Claude or GPT algorithms running on silicon and trained on vast reservoirs of data.

AlphaFold预测的是蛋白质的结构，而非文字和句子，但在本质上，它与运行在硅芯片上、通过海量数据训练的Claude或GPT算法并无太大不同。

AlphaFold just doesn't pull our psychological strings in the same way.

AlphaFold只是没有以同样的方式触动我们的心理弦。

So if we think that Claude is conscious, but AlphaFold is not, that says more about us than it says about AI.

因此，如果我们认为Claude有意识，而AlphaFold没有，这更多地反映了我们自身的看法，而非AI的本质。

But how can I be so sure those systems like Claude or GPT are not conscious?

但我怎么能如此确定像Claude或GPT这样的系统没有意识呢？

Well, nothing is for certain, sure, when it comes to consciousness.

当然，在意识这个问题上，没有什么是一定的。

But the very idea of conscious AI depends on a deeper assumption, a kind of myth, really.

但AI有意识这一想法本身，依赖于一个更深层的假设，实际上是一种迷思。

and this is the myth that the brain is a computer that just happens to be made of meat rather than metal.

这个迷思就是：大脑是一台计算机，只不过碰巧由血肉而非金属构成。

Now consciousness in this story is a special algorithm, a collection of computations that just happens to be carried out in the wetware of the brain in ours, but which could equally be carried out in silicon in ai.

在这个故事中，意识是一种特殊的算法，是一系列计算，这些计算碰巧在我们大脑的湿件中运行，但同样可以在AI的硅基硬件中执行。

But the computer is just one in a long line of technological metaphors that we've reached for when trying to understand the deep complexity of the brain.

但计算机只是我们在试图理解大脑深层复杂性时，所采用的一系列技术隐喻中的最新一个。

One time, the brain was a system of plumbing; later it was a telephone exchange, and for the last few decades, it's been a computer.

曾几何时，大脑被比作管道系统；后来又被比作电话交换机；而过去几十年里，它被比作计算机。

And this most recent metaphor has been extremely powerful.

这个最新的隐喻极具影响力。

But it is still a metaphor.

但它终究只是一个隐喻。

And we will always get into trouble when we confuse a metaphor with the thing itself, the map with the territory.

当我们把隐喻和事物本身、地图和实际地形混为一谈时，总会陷入困境。

for one thing in a real brain there's no sharp separation between the mindware and the wetware, unlike the separation that you get between software and hardware in a computer.

首先，在真实的大脑中，心智软件和生物硬件之间没有明显的分离，不像计算机中软件和硬件那样界限分明。

And this really matters because in a computer you can describe and understand everything about an algorithm whether it's a language model or a word processor without worrying about all the silicon shenanigans going on underneath.

这一点至关重要，因为在计算机中，你可以描述和理解一个算法的方方面面——无论是语言模型还是文字处理软件——而无需担心底层硅芯片的各种复杂运作。

The computation, the algorithm is all that matters.

计算、算法才是关键所在。

But for brains, you just cannot separate what they do from what they are.

但对于大脑来说，你无法将它们的功能与它们的本质分离开来。

And this means that what they do is unlikely to be a matter of computation of algorithm alone.

这意味着，大脑的功能不太可能仅仅依赖于计算或算法。

Look closely at a brain, at any brain, and it becomes less and less plausible that all that's going on is just turning some numbers into other numbers in this endless dance of zeros and ones.

仔细观察大脑，任何大脑，都会越来越难以相信其中发生的一切仅仅是在零和一的无限舞蹈中将一些数字转换成另一些数字。

Yes, there are neural circuits which exchange signals and may do computation or at least something like it.

是的，大脑中有神经回路交换信号，可能进行着计算或至少类似计算的活动。

But there's so much else that escapes the confines of the digital neurotransmitter chemicals course through the brain circuitry, electromagnetic fields sweep through the cortex like weather systems.

但还有太多其他因素超越了数字的范畴：神经递质化学物质在大脑回路中流动，电磁场像天气系统一样扫过大脑皮层。

Even a single neuron is such a beautiful biological machine, a far cry from the simplified cartoon-like neurons that power today's AI.

即使是一个单一的神经元，也是一个如此美妙的生物机器，远非当今人工智能所依赖的那种简化的卡通式神经元可比。

the brain is not, or at least not just a computer made of meat, and so consciousness is very unlikely to be a matter of computation alone.

大脑并非，或者至少不只是一台由血肉构成的计算机，因此意识极不可能仅仅是计算的问题。

and if this is true, then conscious ai is off the table, at least for ai as we know it today.

如果这是真的，那么有意识的AI就无从谈起，至少就我们今天所知的AI而言。

Let me put it another way.

让我换个方式说。

What if we simulated every last detail about the brain in some massive supercomputer?

如果我们在一台巨大的超级计算机中模拟大脑的每一个细节呢？

Now, if the fine details of the brain do matter for consciousness, well, wouldn't this be enough for consciousness to happen inside a machine?

现在，如果大脑的精细细节确实对意识至关重要，那么这难道不足以让意识在机器中产生吗？

Well, a computer simulation of a hurricane does not create real wind.

然而，计算机模拟的飓风并不会产生真正的风。

Computer simulation of a black hole doesn't suck the earth into its algorithmic singularity.

计算机模拟的黑洞也不会将地球吸入其算法奇点。

Making these simulations more detailed can make them more useful, but it does not make them any more real.

让这些模拟更加详细可以使它们更有用，但并不会让它们变得更真实。

we could have a simulation of the brain and you could make it as detailed as you want.

我们可以模拟大脑，并且可以尽可能详细地模拟它。

this might make it more useful, but it's not going to make it any more conscious.

这可能会让它更有用，但并不会让它变得更有意识。

Now consciousness has to be said does remain a bit mysterious, but perhaps one reason for this is that we've been so constrained by our metaphors, by the idea that it just has to be some kind of information processing.

现在必须说，意识确实仍有些神秘，但或许原因之一是我们过于受限于我们的比喻，受限于它必须是某种信息处理的想法。

After all, if you think the brain literally is a computer, then what else could it really be?

毕竟，如果你认为大脑本质上就是一台计算机，那它还能是什么呢？

But once we see the brain more clearly for what it really is, many new possibilities arise.

但一旦我们更清楚地看到大脑的真实本质，许多新的可能性就会出现。

And my own view developed over many years is that consciousness is intimately connected to our nature as living creatures.

我多年来形成的观点是，意识与我们作为生物的本质紧密相连。

Unlike the abstract universe of computation, life is all about materiality.

与抽象的计算宇宙不同，生命完全关乎物质性。

Unlike algorithms, living systems are deeply embedded in flows of energy and matter, and they continually regenerate their own conditions for existence and for persistence over time.

与算法不同，生命系统深深嵌入能量和物质的流动中，并不断再生其存在和持续的条件。

I think we can draw a direct line from the molecular furnaces of metabolism, one billion biochemical reactions in every cell in every second, all the way to the neural circuits that underlie each and every experience that we have, whether it's the sight of a blue sky or a pang of envy.

我认为我们可以从新陈代谢的分子熔炉——每个细胞每秒发生十亿次生化反应——一直追溯到支撑我们每一次体验的神经回路，无论是看到蓝天还是感到一丝嫉妒。

Every conscious experience is imbued, however, subtly with a tinge of aliveness with some core relevance for our future survival prospects.

然而，每一次意识体验都微妙地浸透着生命的色彩，与我们的未来生存前景有着核心关联。

And at the heart of every experience beneath even emotion is this simple shapeless and formless but fundamental feeling of being alive.

在每一次体验的核心，甚至在情感之下，是这种简单、无形但根本的活着的感觉。

And in this story, It's life, not computation, that breathes the fire into the equations of experience.

在这个故事中，是生命，而非计算，为体验的方程式注入了活力。

And if this is right, then conscious AI will need to be living AI.

如果这是正确的，那么有意识的人工智能必须是活的人工智能。

Let me bring things together.

让我来总结一下。

First we're built to see consciousness where it isn't, thanks to deep-seated psychological biases that bundle language, intelligence, and consciousness together.

首先，我们天生倾向于在不存在意识的地方看到意识，这源于根深蒂固的心理偏见，将语言、智能和意识捆绑在一起。

Second, the brain is not or not just a computer, so consciousness is unlikely to be a matter of computation of algorithm alone.

其次，大脑不是或不仅仅是计算机，因此意识不太可能仅仅是算法计算的问题。

Brains are the kinds of things which you can't separate what they do from what they are, and silicon is not up to the job.

大脑这类事物无法将其功能与本质分离，而硅材料无法胜任这一任务。

And third, many other things about our biological brains and bodies might matter for consciousness, including a deep connection between consciousness and life.

第三，我们生物大脑和身体的许多其他方面可能对意识至关重要，包括意识与生命之间的深刻联系。

Artificial intelligence is computer software.

人工智能是计算机软件。

It is not a living mind.

它不是一个活生生的心智。

It might give the impression of being conscious, but it is vanishingly unlikely that it actually is.

它可能给人以有意识的印象，但实际上它几乎不可能真正拥有意识。

I want to close by returning to why this matters so much.

最后，我想回到为什么这个问题如此重要来结束讨论。

Take the idea of AI welfare.

以人工智能福利这个概念为例。

Now there are already influential groups advocating that AI systems should have their own rights, based mainly on the idea that they might be or become conscious.

现在已经有颇具影响力的团体主张，人工智能系统应该拥有自己的权利，这主要是基于它们可能具有或即将获得意识的观点。

Now, if real artificial consciousness is on the way, maybe through some other technology or other pathway, then this is entirely justified.

如果真正的人工意识即将到来，或许是通过其他技术或途径，那么这种主张是完全合理的。

We humans have a terrible track record in our ethical treatments of nonhuman animals and of other humans.

我们人类在伦理上对待非人类动物和其他人类方面的记录非常糟糕。

And we don't want to make the same mistake again.

我们不想再犯同样的错误。

And this is one reason why even trying to build conscious AI is a very bad idea.

这也是为什么尝试构建有意识的人工智能是一个非常糟糕的想法的原因之一。

But if conscious AI is just an illusion created by design, as I think it is, then by extending rights to these systems, we'd be sacrificing our ability to control, to regulate them, and perhaps even to turn them off, and for no good reason at all.

但如果人工智能的意识只是设计创造出的幻觉，正如我所认为的那样，那么通过赋予这些系统权利，我们将牺牲控制、监管它们的能力，甚至可能无法将其关闭，而且毫无正当理由。

And this is one reason why even AI that merely seems to be conscious is very dangerous for our society too, and unlike real artificial consciousness.

这也是为什么即使是仅仅看起来有意识的人工智能，对我们的社会来说也非常危险的原因之一，这与真正的人工意识不同。

Conscious seeming AI is either already here or coming very, very soon.

看起来有意识的人工智能要么已经存在，要么即将到来。

there are other reasons we should avoid creating ai that seems to be conscious.

我们还有其他理由应该避免创造看起来有意识的人工智能。

it makes us more psychologically vulnerable.

它使我们在心理上更加脆弱。

we might be more likely to do what an ai says if we believe that it really feels for us, that it really understands us, even if what it's telling us to do is very bad for us.

如果我们相信人工智能真的能感同身受、真正理解我们，即使它告诉我们要做的事情对我们非常不利，我们也更可能听从它的指示。

And finally, the very idea of conscious AI undermines our human nature.

最后，有意识的人工智能这一概念本身就在削弱我们的人性。

The mirror of AI goes both ways.

人工智能的镜子是双向的。

We see ourselves in our algorithms.

我们在算法中看到了自己。

but we also see our algorithms in ourselves, and when we do, when we think of the mind as a collection of computations floating free from their basis in biology, we reduce and we diminish what it is to be a living breathing human being in a real world.

但我们也从自身中看到了算法的影子。当我们这样做时，当我们把心灵视为脱离生物学基础的自由计算集合时，我们就在贬低和削弱作为一个在真实世界中呼吸、生活的活生生人类的意义。

Frankenstein，which Mary Shelley wrote when she was just nineteen，it's often taken as a cautionary tale，a warning against the hubris of bringing something to life，a Promethean sin like stealing fire from the gods.

玛丽·雪莱在年仅十九岁时创作的《弗兰肯斯坦》，常被视为一则警示故事，警告人们不要狂妄自大地赋予生命，这是一种像从众神那里偷火一样的普罗米修斯式罪恶。

The idea of conscious AI is a new Promethean dream wrapped up in a silicon rapture.

有意识的人工智能这一概念，是一个包裹在硅基狂喜中的新普罗米修斯之梦。

And if conscious ai is possible, then so is the prospect of uploading our own conscious minds and floating off to eternity in a silicon cloud of living or at least existing forever in the pristine circuits of some future supercomputer.

如果有意识的人工智能是可能的，那么上传我们自己的意识、在硅基云中永恒漂浮，或者至少在未来的超级计算机的原始电路中永远存在的前景也就成为可能。

Now, the seductive power of this vision of being at such a pivotal point in the history of life on earth, i suppose it's understandable.

现在，这种处于地球生命历史如此关键节点的愿景具有诱人的力量，我想这是可以理解的。

And back in the real world, talk of conscious AI does other things too.

回到现实世界，关于有意识人工智能的讨论还会带来其他影响。

Conjures this sense of technological wonder and magic, which might keep share prices aloft and regulators at bay.

它唤起了人们对科技奇迹与魔法的遐想，这或许能维持股价高企，并让监管机构望而却步。

But we should resist.

但我们应该抵制这种想法。

The sacrament of the algorithm is most likely an empty dream, delivering not posthuman paradise but silicon oblivion.

算法的神圣仪式很可能是一场空梦，带来的并非后人类天堂，而是硅基的遗忘。

We need a different story.

我们需要一个不同的叙事。

one in which we're more part of nature, not apart from it, with consciousness more closely tied to living flesh and blood, not to the dead sand of silicon.

在这个叙事中，我们更贴近自然而非与之分离，意识更紧密地与鲜活的血肉相连，而非与硅的死寂沙粒为伍。

A I might claim the prize of intelligence at least in some ways, but consciousness, consciousness remains ours to celebrate and to share with other living creatures.

人工智能或许能在某些方面摘得智能的桂冠，但意识——意识仍属于我们，值得我们去颂扬，并与其他生命共享。

so let's not sell our minds so easily to our machine creations.

所以，不要轻易将我们的心智出卖给机器造物。

if we do, we not only overestimate them, we underestimate ourselves.

如果我们这样做，不仅高估了它们，也低估了我们自己。

thank you, thank you, thank you very much.

谢谢，谢谢，非常感谢。