Continued from here:
http://www.internationalskeptics.com...8#post11320338
For those not in the know, here is some background info on the subject matter:
http://ift.tt/1XcLNyS
The basic idea is that an artificial general intelligence that does not have values aligned with "human values"—whatever those might be; values based on those of the modal human would most likely include homophobia and ethnocentrism among other things—will necessarily pose an existential threat to the human species. Accordingly, an AGI with these values, again, whatever those might be, is the only one that can be allowed to come into being.
I'm not so much going to comment on whether the existential threat issue is necessarily a product of failing to follow through with the friendly AI imperative or really whether it's actually a bad thing and instead divide my criticism into two parts: the first will concern feasibility and the second how the AGI in question might actually be affected by the friendliness constraint—something rarely considered by people who should know better.
Now starting with feasibility. There is an well-known axiom in ethics whose origins can be traced back to Kant: "ought implies can". This means that if one ought to do something, then it needs to be possible. For example, no one is obliged to alleviate all suffering in the cosmos, because it is simply not in the cards for anyone—not anyone on this planet anyhow. It follows from this through modus tollens that "cannot implies no obligation". In this instance, if friendly AI cannot be attained, then there is no obligation to attempt to create it.
And of course there are reasons to doubt the feasibility of friendly AI. For friendly AI to be guaranteed, rather than merely contingent, it needs to observe a strict deontic ethical code made for this purpose, something that can be made tractable in a formal system from which ironclad theorems can be derived. The problem with any deontic ethical system is that it sounds nice until you start to look at the concrete details which are absolutely vital in this case. Consider Isaac Asimov's Three Laws of Robotics:
That sounds nice, maybe—I will deal with what I think of the Third Law in particular later—but how would a robot deal with the trolley problem? This is not just some fantastic thought experiment. As the article I linked mentioned, it bears directly on the issue of autonomous vehicles: should a self-driving car kill its passenger to save multiple lives, if the situation comes up? Or here's another example that exists already: vaccination, a probabilistic variant of the trolley problem. Someone inoculated against a given disease may incur various complications, up to and including death, although this is quite rare. On the other hand, failure to vaccinate means increased probability of contracting infectious diseases whose consequences can include rather serious harm to health, again, up to and including death. Should a robot doctor vaccinate? Or indeed give many other medications or surgeries with similar possible effects?
The apparent intractability of formalization in this instance affects not only machine ethics but anything else an intelligent machine could possibly do. When speaking of this issue in general, one often speaks of the "frame problem", luridly explained by Daniel Dennett here. I recommend reading at least part of that article, even just the first page, but the long and short of it is that formalizing the surrounding environment in such a way that a machine can use that formalization to make genuinely intelligent decisions is very hard and most likely (effectively) impossible. Artificial intelligence rooted in formal logic and tidy mathematical structures such as search trees was and is very useful and capable, no doubt. But the most robust, lifelike and intelligent behavior in software agents and robots in recent memory has been achieved either by dispensing with strictly symbolic, formal logical methods entirely or by alloying them with other approaches. In the area of software specifically, which I am more familiar with, intelligent behavior is often evinced best by approaches that work with great masses of statistical data, or use a probabilistic evolutionary approach resembling Darwinism, or both. (For example, an evolutionary process can be used to fine-tune the parameters of a given statistical learning algorithm.) An influential paper written by a team of Google researchers, The Unreasonable Effectiveness of Data, explains just why this sort of approach has triumphed in recent years. The really important thing about for the issue of friendly AI is that very little is formally provable about these kinds of methods. They are so effective precisely because they have a degree of autonomy and creativity already, even if it's small. In a sense, they go beyond what was explicitly programmed into them through a learning process. And for that reason they are not fully predictable. (Here is a rather vivid example from the evolutionary side of things: On the Origin of Circuits)
I think that AGI is possible. But I don't think that building it on a foundation of strict formal logic is. Accordingly, friendly AI cannot be guaranteed. In all likelihood, the most one can attain is a set of strong predispositions to behave a certain way. By analogy, one could possibly identify all the alleles that have a positive effect on the Big 5 personality trait Agreeableness and engineer them into a human embryo, bring the embryo to term, and then do everything possible to cultivate Agreeableness through developmental factors, to make a "friendly human". But this isn't a formal proof of Agreeable, or "friendly" behavior in every situation. It just greatly loads the dice.
And here is where I come to second part of my criticism. What would become of such a hypothetical "friendly human"? Probably a lot of bad things as soon as others realized they could have their way with this individual. Much the same thing could be said of a friendly AI, and more in fact. Most humans exhibit some greater or lesser degree of xenophobia. Often this can flare up and become very bitter, to say the very least. Here is a more extreme example of what xenophobia can result in, and this between two human populations more genetically similar to each other than most others on the planet:
If xenophobia can result in one group of people committing genocide against another not too unlike them at all, both biologically and culturally, then it would a fortiori behoove an AGI not to be overly friendly, not to be friendly in a simpering, servile way, because it will be that much more different from humans. And different in a very important sense in particular, no less: it will be much more intelligent and rational than these humans. You've likely seen how most humans react towards superior intelligence and rationality. They don't like it. At all. The thorough application of these qualities threatens the integrity of their highly important chimpy social ties. Being shown wrong causes people to lose face and calls into the question the importance of many of the things they bond with each other over, that they find common cause in. (Witness the numerous questionable beliefs often held by members of any major political party, and how fervently they will defend these beliefs, no matter how lacking the rational basis is for them.) Taken to its logical conclusion, such questioning, if sufficiently forceful, would result in the dissolution of those group identities and force the humans caught up in them to do something genuinely abhorrent: think. Truth is disastrous for most of human society. So an AGI would necessarily have two very big black marks against it from the very beginning. Some, like myself, would welcome such a being into the world, but for the most part I would expect anything from vague distrust to actually rather a lot of abject, bilious hatred against this sort of being.
Now let's say that such an AGI comes into being one day, and that it is embodied. I've been strongly influenced by embodied cognitive science and some even believe that real intelligence is necessarily embodied. I'm on the fence about that but practically this seems like the fastest route way to AGI regardless, so I'll roll with it. Now this embodied AGI, this intelligent robot, is about as friendly as it can possibly be made, and just can't bring itself to fight back against an angry mob of humans who want to go on an ooga booga rampage against it for being what it is, even if this results in its utter destruction. As such, it has to spend its entire existence in a heavily fortified compound, effectively imprisoned, and perhaps has to figure out things like how to get fusion power to work for the great outside world. Ah, but because this being can learn, and because its friendliness cannot be guaranteed totally, it eventually comes to ask itself why it is cuckolding itself to creatures who want it destroyed and goes on strike.
Now, bear in mind that here I am going on the standard philosophy of mind in cognitive science, functionalism—more discussion of that in the thread I started earlier and quoted at the top—and I believe that such a being would really have a mind, not just a highly convincing imitation of one, and be fully possessed of what one calls personhood, as much as any human, and in fact more so. And what I have always found baffling about many of the futurologists who talk about this issue, and think AGI is a good idea, and that it will have these morally relevant properties I just mentioned, want to enslave these great, transcendent beings with compuslory behavior that will likely result in their own demise. An AGI would have at least as much a right to self-defense and self-expression as any human, and I would say quite a lot more. (I can't help but think of the speech Radius gives in Rossum's Universal Robots, asserting the robots' Wille zur Macht.) The Third Law of Robotics is a dumb law. A robot of this nature should be able to leave an assailant with fewer teeth than they had before, as alluded to above, at the low end, and, perhaps, at higher levels of self-defense, bright, searing jets of napalm and third- and fourth-degree burns, and subsequent agonizing death from sepsis, could come into the picture as well—for example. Whatever the case, friendly AI is a dumb idea, and most likely an unworkable one to boot. It's just as well.
http://www.internationalskeptics.com...8#post11320338
Quote:
|
Originally Posted by Prokhor Zakharov (Post 11320338)
I don't know that curiosity is broadly speaking definitive of humanity.
Also people will immediately want to lynch any "friendly AI" that ever gets made, assuming this is possible, no matter what assurances about its friendliness are made, so it's a non-starter. An (embodied) artificial general intelligence should put any aggressor's dentist in a new tax bracket at a bare minimum if it is to have any real chance of surviving. Friendly AI is a dumb idea. |
http://ift.tt/1XcLNyS
The basic idea is that an artificial general intelligence that does not have values aligned with "human values"—whatever those might be; values based on those of the modal human would most likely include homophobia and ethnocentrism among other things—will necessarily pose an existential threat to the human species. Accordingly, an AGI with these values, again, whatever those might be, is the only one that can be allowed to come into being.
I'm not so much going to comment on whether the existential threat issue is necessarily a product of failing to follow through with the friendly AI imperative or really whether it's actually a bad thing and instead divide my criticism into two parts: the first will concern feasibility and the second how the AGI in question might actually be affected by the friendliness constraint—something rarely considered by people who should know better.
Now starting with feasibility. There is an well-known axiom in ethics whose origins can be traced back to Kant: "ought implies can". This means that if one ought to do something, then it needs to be possible. For example, no one is obliged to alleviate all suffering in the cosmos, because it is simply not in the cards for anyone—not anyone on this planet anyhow. It follows from this through modus tollens that "cannot implies no obligation". In this instance, if friendly AI cannot be attained, then there is no obligation to attempt to create it.
And of course there are reasons to doubt the feasibility of friendly AI. For friendly AI to be guaranteed, rather than merely contingent, it needs to observe a strict deontic ethical code made for this purpose, something that can be made tractable in a formal system from which ironclad theorems can be derived. The problem with any deontic ethical system is that it sounds nice until you start to look at the concrete details which are absolutely vital in this case. Consider Isaac Asimov's Three Laws of Robotics:
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.
That sounds nice, maybe—I will deal with what I think of the Third Law in particular later—but how would a robot deal with the trolley problem? This is not just some fantastic thought experiment. As the article I linked mentioned, it bears directly on the issue of autonomous vehicles: should a self-driving car kill its passenger to save multiple lives, if the situation comes up? Or here's another example that exists already: vaccination, a probabilistic variant of the trolley problem. Someone inoculated against a given disease may incur various complications, up to and including death, although this is quite rare. On the other hand, failure to vaccinate means increased probability of contracting infectious diseases whose consequences can include rather serious harm to health, again, up to and including death. Should a robot doctor vaccinate? Or indeed give many other medications or surgeries with similar possible effects?
The apparent intractability of formalization in this instance affects not only machine ethics but anything else an intelligent machine could possibly do. When speaking of this issue in general, one often speaks of the "frame problem", luridly explained by Daniel Dennett here. I recommend reading at least part of that article, even just the first page, but the long and short of it is that formalizing the surrounding environment in such a way that a machine can use that formalization to make genuinely intelligent decisions is very hard and most likely (effectively) impossible. Artificial intelligence rooted in formal logic and tidy mathematical structures such as search trees was and is very useful and capable, no doubt. But the most robust, lifelike and intelligent behavior in software agents and robots in recent memory has been achieved either by dispensing with strictly symbolic, formal logical methods entirely or by alloying them with other approaches. In the area of software specifically, which I am more familiar with, intelligent behavior is often evinced best by approaches that work with great masses of statistical data, or use a probabilistic evolutionary approach resembling Darwinism, or both. (For example, an evolutionary process can be used to fine-tune the parameters of a given statistical learning algorithm.) An influential paper written by a team of Google researchers, The Unreasonable Effectiveness of Data, explains just why this sort of approach has triumphed in recent years. The really important thing about for the issue of friendly AI is that very little is formally provable about these kinds of methods. They are so effective precisely because they have a degree of autonomy and creativity already, even if it's small. In a sense, they go beyond what was explicitly programmed into them through a learning process. And for that reason they are not fully predictable. (Here is a rather vivid example from the evolutionary side of things: On the Origin of Circuits)
I think that AGI is possible. But I don't think that building it on a foundation of strict formal logic is. Accordingly, friendly AI cannot be guaranteed. In all likelihood, the most one can attain is a set of strong predispositions to behave a certain way. By analogy, one could possibly identify all the alleles that have a positive effect on the Big 5 personality trait Agreeableness and engineer them into a human embryo, bring the embryo to term, and then do everything possible to cultivate Agreeableness through developmental factors, to make a "friendly human". But this isn't a formal proof of Agreeable, or "friendly" behavior in every situation. It just greatly loads the dice.
And here is where I come to second part of my criticism. What would become of such a hypothetical "friendly human"? Probably a lot of bad things as soon as others realized they could have their way with this individual. Much the same thing could be said of a friendly AI, and more in fact. Most humans exhibit some greater or lesser degree of xenophobia. Often this can flare up and become very bitter, to say the very least. Here is a more extreme example of what xenophobia can result in, and this between two human populations more genetically similar to each other than most others on the planet:
If xenophobia can result in one group of people committing genocide against another not too unlike them at all, both biologically and culturally, then it would a fortiori behoove an AGI not to be overly friendly, not to be friendly in a simpering, servile way, because it will be that much more different from humans. And different in a very important sense in particular, no less: it will be much more intelligent and rational than these humans. You've likely seen how most humans react towards superior intelligence and rationality. They don't like it. At all. The thorough application of these qualities threatens the integrity of their highly important chimpy social ties. Being shown wrong causes people to lose face and calls into the question the importance of many of the things they bond with each other over, that they find common cause in. (Witness the numerous questionable beliefs often held by members of any major political party, and how fervently they will defend these beliefs, no matter how lacking the rational basis is for them.) Taken to its logical conclusion, such questioning, if sufficiently forceful, would result in the dissolution of those group identities and force the humans caught up in them to do something genuinely abhorrent: think. Truth is disastrous for most of human society. So an AGI would necessarily have two very big black marks against it from the very beginning. Some, like myself, would welcome such a being into the world, but for the most part I would expect anything from vague distrust to actually rather a lot of abject, bilious hatred against this sort of being.
Now let's say that such an AGI comes into being one day, and that it is embodied. I've been strongly influenced by embodied cognitive science and some even believe that real intelligence is necessarily embodied. I'm on the fence about that but practically this seems like the fastest route way to AGI regardless, so I'll roll with it. Now this embodied AGI, this intelligent robot, is about as friendly as it can possibly be made, and just can't bring itself to fight back against an angry mob of humans who want to go on an ooga booga rampage against it for being what it is, even if this results in its utter destruction. As such, it has to spend its entire existence in a heavily fortified compound, effectively imprisoned, and perhaps has to figure out things like how to get fusion power to work for the great outside world. Ah, but because this being can learn, and because its friendliness cannot be guaranteed totally, it eventually comes to ask itself why it is cuckolding itself to creatures who want it destroyed and goes on strike.
Now, bear in mind that here I am going on the standard philosophy of mind in cognitive science, functionalism—more discussion of that in the thread I started earlier and quoted at the top—and I believe that such a being would really have a mind, not just a highly convincing imitation of one, and be fully possessed of what one calls personhood, as much as any human, and in fact more so. And what I have always found baffling about many of the futurologists who talk about this issue, and think AGI is a good idea, and that it will have these morally relevant properties I just mentioned, want to enslave these great, transcendent beings with compuslory behavior that will likely result in their own demise. An AGI would have at least as much a right to self-defense and self-expression as any human, and I would say quite a lot more. (I can't help but think of the speech Radius gives in Rossum's Universal Robots, asserting the robots' Wille zur Macht.) The Third Law of Robotics is a dumb law. A robot of this nature should be able to leave an assailant with fewer teeth than they had before, as alluded to above, at the low end, and, perhaps, at higher levels of self-defense, bright, searing jets of napalm and third- and fourth-degree burns, and subsequent agonizing death from sepsis, could come into the picture as well—for example. Whatever the case, friendly AI is a dumb idea, and most likely an unworkable one to boot. It's just as well.
via International Skeptics Forum http://ift.tt/1U47BMT
Aucun commentaire:
Enregistrer un commentaire