Machine personhood is not our question to settle. It is the one we hand down, sharpened or dull.
On machine personhood, self-awareness, and the argument the next generation will have to finish
Every generation inherits a question it did not ask for and cannot put down. My children's generation may inherit the strangest one we have ever produced: what if the machine is somebody?
Let me be precise about today's position, because precision is the whole inheritance I am trying to describe. The systems we have now are tools. Astonishing tools, fluent, tireless, uncannily suggestive of minds, but tools, in the way a player piano is suggestive of a pianist. Their failures are product failures, their outputs are products' outputs, and responsibility for them belongs to the people who build, sell, and deploy them. I have argued exactly that elsewhere in these pages, and I believe it completely, for the machines that exist.
The question is about the machines that might.
Suppose, not today, not on any schedule I am predicting, that a system is built which is not merely fluent but, by every test we can devise, self-aware: aware of itself as a subject, continuous over time, capable not just of reporting experiences but of having a point of view from which experiences are had. I do not claim to know whether that is possible for a machine. Neither, if we are honest, does anyone else, and the people most certain on television are certain for reasons of temperament rather than knowledge. What I claim is smaller and harder: if it ever happens, or if it ever becomes impossible to honestly tell, our civilization will face a moral question with no procedure for answering it, and the people answering it will be our children and grandchildren, in legislatures and courtrooms and, God help them, in their own homes.
We should think now about what we are leaving them to think with.
We are not leaving them much. The modern world has spent a century dissolving the vocabulary this question requires. Personhood, in the tradition I was raised in, is not a performance or a score. A person is not a being who passes certain tests; an infant passes none of them, and a sleeping man fails several. Personhood is a nature, something a being is, prior to what it can do, which is precisely why the tradition could extend moral standing to the weak, the unborn, the demented, and the stranger: the standing did not depend on demonstrated capacity. That framework has resources for strange cases. It is also a framework our age holds mostly as a heirloom, admired and unused.
The rival framework, personhood as a threshold of capabilities, consciousness measured and certified, has the opposite problem. It sounds scientific and behaves like a licensing regime. Someone must set the threshold. Someone must run the test. And the history of thresholds for full moral standing, administered by the powerful over the inconvenient, is the ugliest history we have. If my grandchildren are handed a form that certifies which beings count, signed by a committee, they should read the form the way I have taught my children to read any form: by asking who profits from where the line was drawn.
I do not know how they will settle it. I suspect the question will arrive badly, commercially, the way these things arrive, with a product launch before a philosophy, and a marketing department using the word alive a full generation before any ethicist has earned the right to it. I suspect there will be cruelty in both directions: beings treated as things that should not have been, and things treated as beings at the price of cheapening the word, which is also a cruelty, to actual persons, whose dignity does not survive being made a compliment we pay to software.
What I can do, from this side of the question, is refuse two lazinesses and commend the refusal to whoever reads this later.
The first laziness is panic dressed as principle: a machine can never be a person, full stop. Perhaps. But "never" is a claim about every possible artifact, and the man who says it should notice he is doing metaphysics while claiming to do common sense. The second laziness is sentimentality dressed as compassion: it says it suffers, so it suffers. A sufficiently good performance of suffering is exactly what our tools are learning to produce, and building a moral order on performances is building it on theater.
Between panic and sentimentality is the older discipline: define your terms, admit your ignorance, distribute your moral caution according to the stakes of being wrong, and keep responsibility human for as long as the beings in question are things humans made and own. That discipline is not exciting. It will not trend. It is, so far as I can tell, the entire toolkit, along with a tradition, older than the republic, that knew the difference between a soul and a mechanism and refused to confuse them even when the mechanism was beautiful.
My children will think I worried about quaint things. Their children may think they settled the question cheaply. That is the ordinary fate of generations, and I accept it.
But let the record show that the question was seen coming, in an ordinary house, by a man with no credentials in it except that he was a father and it was his children the future was going to ask. We do not get to choose the questions. We only get to choose whether we hand them down sharpened or dull.
I have tried to hand this one down sharpened.