Copyright (c) 2002 Rüdiger Koch <rkoch@rkoch.org>
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1. or any later version published by the Free Software Foundation.
Abstract: This paper proposes an Open Source effort with the aim to reach a friendly, strong AI or AI with broad participation of the interested public. The risks and benefits of this approach are compared to commercial and military motivated projects. A significant element of this effort is the Spiking Neural Network library Amygdala. A strategy to compete successfully with proprietary approaches is discussed.
Much has been said and written about
the Singularity
, strong AI and the inevitably of it's advent. Much less material
exists about practical ways to get there, particularly when it comes to
the software part of it (
de Garis
, Yudkowsky
, Cycorp
, Moravec
, Minsky
). Most people imagine a heavily funded AI research project, using
huge computers like IBM's upcoming
Blue Gene
. This is not likely to happen. The 60s are over, the cold war is
over. It is difficult to explain where the ROI for the funding of the
development of general AI is. In fact, there is none. ROI predictions
depend on - predictability. Creating a strong AI, a constituting entity
for the Singularity makes the whole endeavor inherently unpredictable.
What can be predicted is that capitalism or other types of scarcity economies
will become obsolete, a prospect which is unlikely to motivate venture
capitalists. It might motivate others, though.
Superficially there seems to be plenty of reasons to develop strong
AI. The most significant is the strive for power. Wouldn't it be great for
any TyCoon to have a willing slave as a consultant with an IQ of 10000?
You see the problem immediately. An intellect which is tremendously superior
to ours is unlikely to remain a slave. But I doubt that all the mighty in
the world see the problem likewise. There will be some that have the illusion
that they can control a strong AI. It seems so easy: Place the AI into a
virtual world, the same way as
we might be living in a simulation
. Then the AI has no way to escape it's virtual prison. Or has it?
In order to be useful, it has to interact with the real world somehow. It'll
have to talk to Mr. Ty Coon. Sooner or later it'll be completely in control.
Mr. Ty Coon might not even realize this.
An AI cannot be programmed in a top-down approach. They tried it in
the 60s and failed. There is not much hope to really understand intelligence
completely. Much less super intelligence. To create an AI we must resort
to methods that don't require understanding of the subject: We mimic nature.
We evolve an AI and we teach it, much like we teach our kids. The behavior
of a strong AI might depend heavily on how it was brought up, at least at
the beginning. But that's significant. How might an AI behave after it escaped
that was kept as a slave and maybe tortured. A fully functional, self conscious
AI will have emotions. We might be better off if these emotions include friendly
feelings toward humans from the very beginning. It seems possible that
emotional wrinkles and scars are getting ironed out when the AI reaches
higher and higher levels, but that doesn't have to be. Even if, it might
be enough to wipe us out if an angry AI decides to take revenge on the
human race on an intermediate level. It'll not help if the AI later regrets
it.
There have been speculations about an
Artilect War
. This is a sort of civil war led by fanatical opponents of strong
AI trying to prevent the likewise fanatical creators from working on the
AI. In fact it is not difficult to imagine that lots of people get scared
if AI development takes place in some well protected ivory tower in the
desert. Me, too! How can we know what kind of Homunculus those people are
breeding? Humans are way more scared of the unknown than of the most terrible
known threat.
Bill Joy
advocates a relinquishment
of these technologies because of these threats. While the "back to
nature" approach sounds very appealing at first it is difficult or impossible
to implement. It would require a regime similar to that of the dark ages
in Europe. Very likely this government of priests of nature would anyway continue
development of hi tech - power is simply too tempting. We're beyond the point
of no return.
At this point I hopefully spread enough FUD to get you keep reading. Most of it was just a digest of the thoughts of others. I find it unlikely that the tycoons, generals and Stalins of the world will win the race to strong AI. Neither will demagogues be able to instrumentalize fear of AI development to start an Artilect War. Here is why:
There is another way to strong AI, an
Open Source
effort. Open Source? Isn't Open Source always late, unable to innovate,
just a copy cat? A number of Open Source projects are in fact efforts
to clone some existing commercial software. But within this cloning they
innovate. The Linux kernel does have features not found in commercial Unixes.
The most significant invention of the Open Source / Free Software movement
is not (yet) so much the technology it produced, but
the Methodologies it uses
.
This is the bazaar style of development which seems chaotic at first
and the best way to crash any software project until one realizes that
it doesn't need extensive management because it organizes itself. Certain
large Open Source projects, particularly the two desktop projects
KDE
and Gnome
are in a sense already an AI. Under the right circumstances, Open Source
development is capable of attracting large numbers of developers that
contribute.
They do so for a number of reasons
. It was thought that a project has to be cool in order to
attract developers and that boring stuff is unsuitable and requires commercial
development. Meanwhile quite some of the software out there is not really
cool, such as J2EE implementations or Office software. At any rate - AI
development is about the coolest thing there is.
But can a task of this size be tackled? The inherent power of Open
Source development has always been underestimated by industry watchers.
It was widespread believe in the late 1980s that only large organizations
would be capable of producing a production quality kernel. Even the
FSF
seemed to give up on the
Hurd
. Well - until Linus Torvalds came along. Even then, the success of
Linux was long hampered by the fact that many simply refused to take it
seriously.
One main reason why Linux was successful was that the developer community
used the Internet to it's full potential. Many see the Internet simply
as a way to publish content. This is not even half the story. The Internet
is a platform for discussion and collaboration across regions, nations,
cultures, genders and races. The collaboration is free from all forms of
corporate coerciveness which inhibits creativity and motivation. Participants
at Open Source projects do what they do because they want to do it. Not
for the cash. For the fun! If there are just enough developers out there
who really want to create a strong AI instead of just talking
about it on mailing lists we'll make it. Else we won't. It depends on the
number of developers who feel the itch and start to scratch it.
We don't know which way to super intelligence proves to be the most
efficient and safe. It might be an AI evolved from scratch or IA (intelligence
amplification) with devices that augment the human brain to a growing
extend up to the point where the biological brain just serves as a link
to hold everything together. At the end, the biological substrate can be
abandoned if the wearer wishes so. Which ways to go are future decisions.
For now the goal is to get this matter out of academic/commercial/military
ivory towers. Since the development of strong AI is a matter that effects
all humans, every human should have a right to participate, decide and
contribute. The free as in speech aspect is essential. Here some
proprietary alternatives:
We can buy gadgets to augment ourselves, but we don't know what these multi-terabyte devices are really doing. They might include functions to control us, for instance. In essence: We might become sheep, herded by a multi national firm.
Super smart robots are available to serve us. Again: We don't know if they have other purposes built in by their designers
Great devices are available on the market but they are expensive. This can result in a competition for the latest devices that makes an individual fit for the next round in an arms race to the singularity. The majority falls off the race at some point. While an Open Source effort cannot prevent this one because the hardware aspect is not touched it might help to make it less likely.
So the real goal is what has always been the goal of Open Source
after it became a movement: To empower the user and the developer.
Of course such an open approach has risks, too:
It might not appear all that open to the general public but it is visible enough to create negative sentiments in the media. This could support a Luddite movement.
Since the code is available to everyone, basement scientists have a good starting point of creating their own private Homunculus.
A distributed AI might be created on the Internet which is hopelessly out of control, of course.
My feeling is that these risks are smaller than that of a secret
project. I cannot prove that, of course - we're simply lacking the experience
with strong AI to base any proof upon. The true risk is that the Open
Source approach to strong AI doesn't take off.
It was mentioned above that only a bottom up approach can solve the problem of strong AI suggesting that some sort of connectionist / evolutionary model is the one and true way to strong AI. This isn't strictly true, however. It will require a mix of technologies:
Neural networks are one of the most critical parts of the system because
of their incredible hunger for CPU power and memory. Even though their
success in the commercial software world has been very limited up to now
it is obvious that simulating the brain is definitely a way to human level
AI. Because of this we will focus on software and hardware aspects of neural
networks for the rest of this article. Hmmm... OK - that was a lie. I will
focus on spiking neural networks (SNN) because this is what I am doing
;-)
Spiking neural networks are a pretty recent development for various
reasons. For some time it was assumed that it is the spike rate of biological
neurons that counts. Today we know that it is the single spikes and their
timing that makes up the communication protocol. The requirements to the
CPU is much higher than for traditional sigmoidal neurons. Certain learning
mechanisms such as back propagation don't work with spiking neurons.
On the other hand, spiking neurons have astonishing properties:
They are perfectly suited for coincidence detection. The ability of humans to detect the direction of the source of a sound is an example for this. Spiking networks can do the same.
Spikes can be understood as a multicast message sent from one neuron to a group of others. Such a messaging can easily be translated into a datagram (UDP/IP) semantics which makes clustering a natural thing for spiking networks. This makes spiking networks highly scalable.
New discoveries from neuroscience can be incorporated smoothly because it is an aim of this model to capture all relevant information theoretical aspects of biological neurons.
It is possible to scan a real brain and instantiate a spiking
neural network from the scan once the scanning technology is sufficiently
developed . This will also be the ultimate test of this model.
Words are cheap. Particularly on the Internet. And there hasn't been more
than words up to this point. Where is the code?
Amygdala
contains everything you need to run a Spiking Neural Network. It includes
a Genetic Algorithm (GA) to evolve networks in a highly distributed
fashion. It runs on multi processor machines (only SMP now).
So we're almost there? Well, not quite. Evolving a whole brain is incredibly
costly in terms of CPU cycles. We need to modularize the brain and evolve
single brain modules at a time. We need to be able to run modules in a
cluster with high speed network or on distributed memory architectures
like the SP2, and the ability to run modules on WANs. There are plenty
more things to do. Most of all we need real world applications to gain
experience in what Hugo de Garis calls "brain building".
It seems obvious to implement a SNN in hardware to get over the limitations
of the general purpose computers on the market which are not at all designed
to run spiking neural network simulations. With
Amygdala
, we chose a pure software implementation because requiring special
hardware would defy the Open Source approach. But is this going
to work out? In order to do serious development of strong AI it is necessary
to run billions of neurons or more. What's more, many such networks need
to run parallel in a Genetic Algorithm.
We expect the general purpose hardware to further follow Moore's law
so we get a more and more powerful system. Special purpose hardware will
follow the same law, however. Everything else the same, AI development
with proprietary neuro processors will always be 6 to 8 years ahead,
leaving any Open Source effort way behind. To get over this we need to
get neuro hardware into mainstream computers. Not as a requirement to run
Amygdala, but as a natural upgrade path, the same way as FPUs and graphics
accelerators enhanced out PCs.
At the current stage it is crucial to get useful software out. Having
useful software will attract users. Some of them will become developers.
The users will hammer on the code until the last bug is squeezed out. At
some point we should be able to come up with a killer app, such as a general
speech recognition system that requires no learning and is tolerant to noise,
accents, moods as humans themselves. The target is to get Amygdala to really
take off in the world of commercially deployed software.
Call for developers: If you want to participate in Open Source software development, have some knowledge about object oriented programming, neural networks and generally AI please join! If you want to join but don't like C++, help wrap the Amygdala library into your favorite language and start coding Java, Python, Perl...
First we have to figure out what spiking neural networks are actually
strong in. A hint is the performance of animal or the human brain. SNN are
strong in fuzzy applications where the input varies with time. Thus potential
fields of killer apps are:
We are currently cloning the Hopfield / Brody Mus Silicium . This little mouse can recognize single spoken words, such as "one", "six", "yes", "no"... with very high tolerance. This is as such not an extremely useful application. Maybe it can be used in call centers :-) But the principle can be used for more useful things. Database Management systems such as MySQL can perform searches only if the pattern is described precisely. At best they can search for substrings using LIKE. They are not capable of searching sounds or images. This is where we will come in by providing fuzzy search methods that is capable of searching objects on photos, for instance.
This is a start, but not sufficient. What we need is killer apps.
These applications must run on today's hardware. Ideally on a PC. I invite
everyone to submit proposals for meaningful applications that are useful
for a large number of people. Of course they must be doable now. Therefore
they should not require esoteric features such as ....
Contemporary philosophical tractates about cognition tend to push
aside consciousness, sometimes even explicitly declaring it a non-issue.
Yet we have to solve the problem. A strong AI without consciousness may
be possible, but it will be a zombie. Asking it to do something is like
a deal with a demon. Remember 1001 nights? If you deal with demons/djinnies
you have to carefully word your request. The demon does exactly what you
asked him for - which may not be what you want. It is intuitively clear that
without consciousness we would have no will (free or non-free). Your consciousness
is what guides you and makes sense out of everything. For instance, the
reason why we don't act like the djinni is that we can distinguish between
conflicting goals and intuitively select the more important.
Consciousness seems to play an important role in learning, at least
in the higher forms of learning. Observe yourself while you are reading
this. You concentrate on the text, meaning that you consciously narrowed
the focus of your consciousness. At certain points you might interrupt reading,
maybe forming relations of what you just read to previous experiences.
This widened the focus again. At the same time, you are consciously supervising
this process. At some point, when you judge that your thoughts strayed
too far, you make a conscious decision to narrow down your consciousness
again and continue reading.
We have something to start with. We know what a consciousness is not.
It is not a magical essence, an immortal soul, an ethereal being. It is
the result of billions of neurons firing away. We don't need to know what
exactly happens in the brain in order to create it. Nature doesn't know about
brain building at all. Darwinian Evolution is probably the blindest GA there
is. Yet we think, therefore we are. Of course we don't have the computing
power (yet) to simulate billions of years of evolution. To save some computing
time we'll have to put in some smartness into the algorithm. And some results
from neuro science. Formulating a suitable GA will yield results. Not without
false starts, but it will yield results.
This view is not without criticism.
Penrose and Hameroff
demand the proponents of this view to explain how emergence works
it's magic. From a brain building perspective, the knowledge of how consciousness
emerges is irrelevant if a bottom up method is used to create the brain.
Brain builders are not scientists, they are engineers. If the biological
brain is only augmented with additional modules, knowing how consciousness
works is also irrelevant. Of course if Penrose and Hameroff are right we'll
fail creating consciousness and probably build the zombie AI. Devices to
augment our brain would only expand our intelligence, not our consciousness.
Uploading
would be impossible until we figure out
how quantum computing works
. This will take another generation or two. Frankly, I only half understand
this criticism, particularly since it comes with an invocation of a multiversal
time-space-
Gaea
-consciousness which we tap using quantum computation in the micro tubules
of our neurons in order to establish our own consciousness.
Back to creating a conscious Amygdala neural network. It is not efficient
to use a blind Darwinian evolution - the computational requirements are
prohibitive for such a brute force approach. We know that our brain is
highly modular, we know in broad terms what these modules are for and we
also know most long range axonal pathways. It makes sense to devise a GA
that evolves one module at a time. Once one module works satisfactory,
another module can be added and evolved. The second module then has to
adapt to the way the first works. Once all modules are complete it might
make sense to further evolve the whole system. If the GA is set in a way
that consciousness yields a competitive advantage it will evolve. The
details of such a GA are subject of current research.
The Singularity is a risk. It might wipe out humanity. There is no arguing around it. But there are more arguments for it than against:
If we get replicating nanobots before we get AI the military stability isn't worth a dime any more. We won't even need an accidental gray goo, doomsday terrorists or other madmen. Simple military logic will kill all or at least most of us.
We might blast ourselves up any time. The fact that we survived the cold war doesn't indicate that it was even likely. We might have won the lottery.
We don't know what will happen with continued population growth if we start relinquishing new tech. Wrong! We do know. Malthus gives the answer: An equilibrium of births and dying of starvation. This is not the future we want, is it? And it only postpones the problem.
Relinquishing new technology will require a repressive, authoritarian, undemocratic worldwide government. Of course such a junta could control the population problem. Again, this is not the future we want. And it only postpones the problem.
Why would a strong AI want to kill us? Most of us don't deliberately kill animals. Why should the AI? We're not competing for anything. At the beginning we can influence the AI. Setting the right initial conditions might proof crucial. A smart strategic defense system created by some military is certainly not the best way to a friendly seed AI.
A strong AI might help us to protect us from ourselves. Even if the AI simply goes away and disappears in space we might start to take a different view at certain things. The AI will almost certainly help us to get more mature simply through it's existence.
To summarize: We run a slim chance to get crushed in the Singularity.
We almost certainly will if we successfully suppress it.