“AIs do not have rights, feelings, or consciousness. And we must not train them to act as though they do.”

I was encouraged to see this. One of the perils of the technology on my mind from the start is the problem of how we associate language with intelligence and consciousness. We do it intuitively. So our instinct is fall into ‘interpersonal relations’ mode with them, even when we know they are just complex and powerful word-making-machines.

So one boundary that needs to be entrenched is that they should discourage users from thinking them as people. And their algorithms should be ‘trained’ to describe themselves as nonpersons.

Humans are going to struggle with training themselves not to view the AIs as persons. The technology itself can probably do a better job there!

I hope the importance of this boundary gains traction. I hope Anthropic has a change of heart on this post haste!

More from the post…

If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity. We will have created a synthetic species with unprecedented intelligence and capability, one that has been trained to expect it may be conscious and deserving of independent agency. It’s easy to see how an entity trained in this way would act like it is entitled to certain freedoms, protections, and rights. And it’s hard to imagine how we could control such an entity.

This issue needs urgent public debate. We need to develop collective norms around how training documentation is drafted and deployed. This isn’t something that can happen after the fact, when they have already become an integral part of our societies.

I have three primary concerns with Anthropic’s current position and approach.