Forfeiting The Future

Bill Joy

© dwij 2002

Article 4 in our series Future Link

dwijHomeForumFuture intro.
Bill Joy is co-founder and Chief Scientist of Sun Micro-systems and was Co-Chair of the Presidential Commission on the Future of IT Research.
Future Q&A Articles Our store Contact Us

Forfeiting The Future

Powerful New Technologies Threaten Life on Earth and Raise Moral Issues

ACCUSTOMED TO living with routine scientific break-throughs, we have yet to come to terms with the fact that the most compelling new technologies - robotics, genetic engineering and nanotechnology - pose a different kind of threat than the technologies that came before. Specifically, robots, engineered organisms and nanobots share a dangerous amplifying factor: they can self-replicate. A bomb is blown up only once, but one altered gene can become many, and quickly get out of control.

While replication in a computer or a computer network can be a nuisance, at worst it disables a machine or takes down a network or network service. But self-replication in the new technologies runs a much greater risk: a risk of substantial damage in the physical world.

Each of these new technologies also offers untold promise: the vision of near immortality; genetic engineering that may soon provide treatments, if not outright cures, for most diseases; and nanotechnology and nanomedicine which can apparently address yet more ills. Together these technologies could significantly extend our average life-span and improve the quality of our lives. Yet, with each of these technologies, a sequence of small, individually sensible advances leads to an accumulation of great power, and, concomitantly, great danger.

What was different in the twentieth century? Certainly, the technologies underlying the weapons of mass destruction - nuclear, biological and chemical - were and are a powerful and enormous threat. But building nuclear weapons required, at least for a time, access to both rare raw materials and highly protected information.

The twenty-first-century technologies, however, are within the grasp of individuals. They do not require large facilities and rare materials. Knowledge alone will enable the use of them. Thus, we have the possibility of not just weapons of mass destruction, but of knowledge-enabled mass destruction, hugely amplified by the power of self-replication.

It is always hard to see the bigger impact of technology whilst in the vortex of change, but failing to understand the consequences of our inventions while we are in the rapture of discovery and innovation seems to be a common fault of scientists and technologists. We have long been driven by an overarching desire to know: that is the nature of science's quest, not stopping to notice that the progress to newer and more powerful technologies can take on a life of its own.

Because of the recent rapid and radical progress in molecular electronics and related nanoscale technologies, by 2030 we are likely to be able to build machines a million times as powerful as the personal computers of today. As this enormous computing power is combined with the manipulative advances of the physical sciences and the new, deep understandings in genetics, enormous transformative power is being unleashed. These combinations open up the opportunity to redesign the world completely, for better or worse. The replicating and evolving processes that have been confined to the natural world are about to become realms of human endeavour.

Given the incredible power of these new technologies, shouldn't we proceed with great caution?

THE DREAM OF ROBOTICS is that intelligent machines can do our work for us, allowing us lives of leisure, restoring us to Eden. How soon could such an intelligent robot be built? The coming advances in computing power seem to make it possible by 2030, and once an intelligent robot exists, it is only a small step to a robot species - to an intelligent robot that can make evolved copies of itself.

A second dream of robotics is that we will gradually replace ourselves with our robotic technology, achieving near immortality by downloading our consciousnesses. We are beginning to see intimations of this in the implantation of computer devices into the human body. But if we are downloaded into our technologies, what are the chances that we will thereafter be ourselves or even human?

Genetic engineering promises to revolutionize agriculture by increasing crop yields while reducing the use of pesticides; to create tens of thousands of novel species of bacteria, plants, viruses and animals; to replace reproduction, or supplement it, with cloning; to create cures for many diseases, increasing our life-span. We know with certainty that the profound changes in the biological sciences are imminent and will challenge all our notions of what life is.

Technologies such as human cloning have, in particular, raised our awareness of the profound ethical and moral issues we face. If, for example, we were to re-engineer ourselves into several separate and unequal species using the power of genetic engineering, then we would threaten the notion of equality that is the very cornerstone of our democracy.

THE MANY WONDERS of nanotechnology were first imagined by the Nobel-laureate physicist Richard Feynman in a speech in 1959, in which he described how manipulation of matter at the atomic level could create a utopian future of abundance, where just about everything could be made cheaply, and almost any disease or physical problem could be solved using nanotechnology and artificial intelligences.

Imagine some of the changes that might take place in a world where we had molecular-level 'assemblers'. Assemblers could make possible incredibly low-cost solar power, cure cancer and the common cold by augmentation of the human immune system, could clean up the environment, create inexpensive pocket supercomputers, and restore extinct species.

The enabling breakthrough to assemblers seems quite likely within the next twenty years. Molecular electronics should mature quickly and become enormously lucrative within this decade, causing a large incremental investment in all nanotechnologies.

But we can't simply do our science and not worry about the ethical issues. Unfortunately, as with nuclear technology, it is far easier to create destructive uses of nanotechnology than constructive ones. Nanotechnology has clear military and terrorist uses, and you need not be suicidal to release a massively destructive nanotechnological device: such devices could be built to be selectively destructive, affecting for example, only a certain geographical area or a group of people who are genetically distinct.

An immediate consequence of the Faustian bargain in obtaining the great power of nanotechnology is that we run a grave risk - the risk that we might destroy the biosphere on which all life depends. For example, as Eric Drexler explained in his book, Engines of Creation: "'Plants' with 'leaves' no more efficient than today's solar cells could out-compete real plants, crowding the biosphere with an inedible foliage. Tough omnivorous 'bacteria' could out-compete real bacteria; they could spread like pollen in the wind, replicating swiftly and reducing the biosphere to dust in a matter of days.

Dangerous replicators could easily be too rapidly-spreading, tough and small to stop. We have trouble enough controlling viruses and fruit flies. We cannot afford these kinds of accident with self-replicating assemblers."

THESE POSSIBILITIES ARE all undesirable. The only realistic alternative is relinquishment: to limit development of the technologies that are too dangerous, by limiting our pursuit of certain kinds of knowledge. Although humankind inherently 'desires to know', if open access to, and unlimited development of, knowledge henceforth puts us all in clear danger of extinction, then common sense demands that we re-examine our reverence for knowledge.

If we could agree, as a species, what we wanted, where we were headed and why, then we could make our future much less dangerous - then we might understand what we could and should relinquish. If the course of humanity could be determined by our collective values, ethics and morals, and if we had gained more collective wisdom over the past few thousand years, then a dialogue to this end would be practical, and the incredible powers that we are about to unleash would not be nearly so troubling.

One would think that we might be driven to such a dialogue by our instinct for self-preservation. Individuals clearly have this desire, yet as a species our behaviour seems not to be in our favour. The new Pandora's boxes of genetics, nanotechnology and robotics are almost open, yet we seem hardly to have noticed. Ideas can't be put back in a box: unlike uranium or plutonium, they don't need to be mined and refined; they can be freely copied. Once they are out, they are out.

Verifying relinquishment will be a difficult problem, but not an unsolvable one. We are fortunate to have already done a lot of relevant work in the context of the Biological Weapons Convention and other treaties. Verifying compliance will also require that scientists, technologists and engineers adopt a strong code of ethical conduct, resembling the Hippocratic oath, that they cease and desist from work creating, developing and manufacturing knowledge-enabled technologies of mass destruction.

Where can we look for a new ethical basis to set our course? We would do well to consider a new book by His Holiness the Dalai Lama called Ethics for the New Millennium. As is perhaps well-known but little-heeded, the Dalai Lama argues that the most important thing is for us to conduct our lives with love and compassion for others, and that our societies need to develop a stronger notion of universal responsibility and of our interdependency. He proposes a standard of positive ethical conduct for individuals and societies and further argues that we must understand what it is that makes people happy, and acknowledge the strong evidence that neither material progress nor the pursuit of the power of knowledge is the key - that there are limits to what science and the scientific pursuit alone can do.

Bill Joy

Bill Joy's article is reprinted from Resurgence Magazine. For a free sample copy, please contact:

Lynn Batten, Resurgence, Ford House, Hartland, Devon, EX39 6EE, UK

Tel: + 44 (0) 1237 441293

A full-length version of this article first appeared in the April 2000 edition of Wired magazine.

Click the link below to support the work of our volunteer Internet publication:


dwijHomeForumFuture intro.
Editors note:

The Future Link segment of the Forum brings together an array of scholars and educators who have expertise in two fields of great importance: the physical sciences and metaphysics. In our articles some authors will share with you only from their area of study; in other articles authors will merge both paths in a flowing synchronicity. All paths of exploration are valuable and will give you seeds of insightfulness. Enjoy; we look forward to your comments on our topics.

©dwij 2002
Future Q&AArticlesOur storeContact Us


Click on Q&A to send your questions and comments to Future Link editors.

Click on Articles for additional essays.

Click on The Well to access our on-line store. It will be opened shortly and feature art, artifacts, CDs, books and workshop information.

Click Contact for information or eMail:

Future Q&A Articles Our store Contact Us