A few days ago Elon Musk announced that there is “no more AI”, and that the better term is “SI”, meaning superintelligence; this came after an official American directive to use the same term in government correspondence.
It may seem merely a change of name, but names create expectations: when we say “super” we expect more from the technology, give it greater trust, and perhaps question it less.
And here is the point to beware of: the real danger is not a machine that rebels, for the machine has neither will nor feeling; the danger is that it carries out its orders precisely without understanding what we mean.
The matter becomes more complicated when intelligent systems deal with one another without oversight: one system builds on another system’s error, and the error grows without anyone noticing.
We have seen something close to this in the financial markets, when automated trading programs interacted faster than humans could intervene, and losses occurred within minutes without any human deciding it.
We do not know whether we have reached superintelligence or are still on the way to it, but we are living the beginnings of its effect, and that is enough to call for caution on our part.
What is required is neither fear of the technology nor rushing towards it, but clear controls, human oversight, and accountability for those who release it. The direction is not set by the machine, but by whoever holds the steering wheel.
The change of name, that is, they will make this change, without setting a date for it.
