Forfeiting
The Future
Powerful
New Technologies Threaten Life on Earth and Raise Moral Issues
ACCUSTOMED
TO living
with routine scientific break-throughs, we have yet to come to terms
with the fact that the most compelling new technologies - robotics,
genetic engineering and nanotechnology - pose a different kind of
threat than the technologies that came before. Specifically, robots,
engineered organisms and nanobots share a dangerous amplifying factor:
they can self-replicate. A bomb is blown up only once, but one altered
gene can become many, and quickly get out of control.
While
replication in a computer or a computer network can be a nuisance,
at worst it disables a machine or takes down a network or network
service. But self-replication in the new technologies runs a much
greater risk: a risk of substantial damage in the physical world.
Each
of these new technologies also offers untold promise: the vision
of near immortality; genetic engineering that may soon provide treatments,
if not outright cures, for most diseases; and nanotechnology and
nanomedicine which can apparently address yet more ills. Together
these technologies could significantly extend our average life-span
and improve the quality of our lives. Yet, with each of these technologies,
a sequence of small, individually sensible advances leads to an
accumulation of great power, and, concomitantly, great danger.
What
was different in the twentieth century? Certainly, the technologies
underlying the weapons of mass destruction - nuclear, biological
and chemical - were and are a powerful and enormous threat. But
building nuclear weapons required, at least for a time, access to
both rare raw materials and highly protected information.
The
twenty-first-century technologies, however, are within the grasp
of individuals. They do not require large facilities and rare materials.
Knowledge alone will enable the use of them. Thus, we have the possibility
of not just weapons of mass destruction, but of knowledge-enabled
mass destruction, hugely amplified by the power of self-replication.
It
is always hard to see the bigger impact of technology whilst in
the vortex of change, but failing to understand the consequences
of our inventions while we are in the rapture of discovery and innovation
seems to be a common fault of scientists and technologists. We have
long been driven by an overarching desire to know: that is the nature
of science's quest, not stopping to notice that the progress to
newer and more powerful technologies can take on a life of its own.
Because
of the recent rapid and radical progress in molecular electronics
and related nanoscale technologies, by 2030 we are likely to be
able to build machines a million times as powerful as the personal
computers of today. As this enormous computing power is combined
with the manipulative advances of the physical sciences and the
new, deep understandings in genetics, enormous transformative power
is being unleashed. These combinations open up the opportunity to
redesign the world completely, for better or worse. The replicating
and evolving processes that have been confined to the natural world
are about to become realms of human endeavour.
Given
the incredible power of these new technologies, shouldn't we proceed
with great caution?
THE
DREAM OF ROBOTICS is that intelligent machines can do our work
for us, allowing us lives of leisure, restoring us to Eden. How
soon could such an intelligent robot be built? The coming advances
in computing power seem to make it possible by 2030, and once an
intelligent robot exists, it is only a small step to a robot species
- to an intelligent robot that can make evolved copies of itself.
A
second dream of robotics is that we will gradually replace ourselves
with our robotic technology, achieving near immortality by downloading
our consciousnesses. We are beginning to see intimations of this
in the implantation of computer devices into the human body. But
if we are downloaded into our technologies, what are the chances
that we will thereafter be ourselves or even human?
Genetic
engineering promises to revolutionize agriculture by increasing
crop yields while reducing the use of pesticides; to create tens
of thousands of novel species of bacteria, plants, viruses and animals;
to replace reproduction, or supplement it, with cloning; to create
cures for many diseases, increasing our life-span. We know with
certainty that the profound changes in the biological sciences are
imminent and will challenge all our notions of what life is.
Technologies
such as human cloning have, in particular, raised our awareness
of the profound ethical and moral issues we face. If, for example,
we were to re-engineer ourselves into several separate and unequal
species using the power of genetic engineering, then we would threaten
the notion of equality that is the very cornerstone of our democracy.
THE
MANY WONDERS of nanotechnology were first imagined by the Nobel-laureate
physicist Richard Feynman in a speech in 1959, in which he described
how manipulation of matter at the atomic level could create a utopian
future of abundance, where just about everything could be made cheaply,
and almost any disease or physical problem could be solved using
nanotechnology and artificial intelligences.
Imagine
some of the changes that might take place in a world where we had
molecular-level 'assemblers'. Assemblers could make possible incredibly
low-cost solar power, cure cancer and the common cold by augmentation
of the human immune system, could clean up the environment, create
inexpensive pocket supercomputers, and restore extinct species.
The
enabling breakthrough to assemblers seems quite likely within the
next twenty years. Molecular electronics should mature quickly and
become enormously lucrative within this decade, causing a large
incremental investment in all nanotechnologies.
But
we can't simply do our science and not worry about the ethical issues.
Unfortunately, as with nuclear technology, it is far easier to create
destructive uses of nanotechnology than constructive ones. Nanotechnology
has clear military and terrorist uses, and you need not be suicidal
to release a massively destructive nanotechnological device: such
devices could be built to be selectively destructive, affecting
for example, only a certain geographical area or a group of people
who are genetically distinct.
An
immediate consequence of the Faustian bargain in obtaining the great
power of nanotechnology is that we run a grave risk - the risk that
we might destroy the biosphere on which all life depends. For example,
as Eric Drexler explained in his book, Engines of Creation: "'Plants'
with 'leaves' no more efficient than today's solar cells could out-compete
real plants, crowding the biosphere with an inedible foliage. Tough
omnivorous 'bacteria' could out-compete real bacteria; they could
spread like pollen in the wind, replicating swiftly and reducing
the biosphere to dust in a matter of days.
Dangerous
replicators could easily be too rapidly-spreading, tough and small
to stop. We have trouble enough controlling viruses and fruit flies.
We cannot afford these kinds of accident with self-replicating assemblers."
THESE
POSSIBILITIES ARE all undesirable. The only realistic alternative
is relinquishment: to limit development of the technologies that
are too dangerous, by limiting our pursuit of certain kinds of knowledge.
Although humankind inherently 'desires to know', if open access
to, and unlimited development of, knowledge henceforth puts us all
in clear danger of extinction, then common sense demands that we
re-examine our reverence for knowledge.
If
we could agree, as a species, what we wanted, where we were headed
and why, then we could make our future much less dangerous - then
we might understand what we could and should relinquish. If the
course of humanity could be determined by our collective values,
ethics and morals, and if we had gained more collective wisdom over
the past few thousand years, then a dialogue to this end would be
practical, and the incredible powers that we are about to unleash
would not be nearly so troubling.
One
would think that we might be driven to such a dialogue by our instinct
for self-preservation. Individuals clearly have this desire, yet
as a species our behaviour seems not to be in our favour. The new
Pandora's boxes of genetics, nanotechnology and robotics are almost
open, yet we seem hardly to have noticed. Ideas can't be put back
in a box: unlike uranium or plutonium, they don't need to be mined
and refined; they can be freely copied. Once they are out, they
are out.
Verifying
relinquishment will be a difficult problem, but not an unsolvable
one. We are fortunate to have already done a lot of relevant work
in the context of the Biological Weapons Convention and other treaties.
Verifying compliance will also require that scientists, technologists
and engineers adopt a strong code of ethical conduct, resembling
the Hippocratic oath, that they cease and desist from work creating,
developing and manufacturing knowledge-enabled technologies of mass
destruction.
Where
can we look for a new ethical basis to set our course? We would
do well to consider a new book by His Holiness the Dalai Lama called
Ethics for the New Millennium. As is perhaps well-known but little-heeded,
the Dalai Lama argues that the most important thing is for us to
conduct our lives with love and compassion for others, and that
our societies need to develop a stronger notion of universal responsibility
and of our interdependency. He proposes a standard of positive ethical
conduct for individuals and societies and further argues that we
must understand what it is that makes people happy, and acknowledge
the strong evidence that neither material progress nor the pursuit
of the power of knowledge is the key - that there are limits to
what science and the scientific pursuit alone can do.
Bill
Joy
Bill
Joy's article is reprinted from Resurgence Magazine. For
a free sample copy, please contact:
Lynn Batten, Resurgence, Ford House, Hartland, Devon, EX39 6EE,
UK
Tel:
+ 44 (0) 1237 441293
A
full-length version of this article first appeared in the April
2000 edition of Wired magazine.
Click
the link below to support the work of our volunteer Internet publication:
|