- We're living in a simulation and it gets shut down
A case can be made that the hypothesis that we are living in a computer simulation should be given a significant probability. The basic idea behind this so-called “Simulation argument” is that vast amounts of computing power may become available in the future, and that it could be used, among other things, to run large numbers of fine-grained simulations of past human civilizations. Under some not-too-implausible assumptions, the result can be that almost all minds like ours are simulated minds, and that we should therefore assign a significant probability to being such computer-emulated minds rather than the (subjectively indistinguishable) minds of originally evolved creatures. And if we are, we suffer the risk that the simulation may be shut down at any time. A decision to terminate our simulation may be prompted by our actions or by exogenous factors.
While to some it may seem frivolous to list such a radical or "philosophical" hypothesis next the concrete threat of nuclear holocaust, we must seek to base these evaluations on reasons rather than untutored intuition. Until a refutation appears of the argument presented in, it would intellectually dishonest to neglect to mention simulation-shutdown as a potential extinction mode.
- Badly programmed superintelligence
When we create the first superintelligent entity, we might make a mistake and give it goals that lead it to annihilate humankind, assuming its enormous intellectual advantage gives it the power to do so. For example, we could mistakenly elevate a subgoal to the status of a supergoal. We tell it to solve a mathematical problem, and it complies by turning all the matter in the solar system into a giant calculating device, in the process killing the person who asked the question.
- Physics disasters
The Manhattan Project bomb-builders' concern about an A-bomb-derived atmospheric conflagration has contemporary analogues.
There have been speculations that future high-energy particle accelerator experiments may cause a breakdown of a metastable vacuum state that our part of the cosmos might be in, converting it into a "true" vacuum of lower energy density. This would result in an expanding bubble of total destruction that would sweep through the galaxy and beyond at the speed of light, tearing all matter apart as it proceeds.
Another conceivability is that accelerator experiments might produce negatively charged stable "strangelets" (a hypothetical form of nuclear matter) or create a mini black hole that would sink to the center of the Earth and start accreting the rest of the planet.
These outcomes seem to be impossible given our best current physical theories. But the reason we do the experiments is precisely that we don’t really know what will happen. A more reassuring argument is that the energy densities attained in present day accelerators are far lower than those that occur naturally in collisions between cosmic rays. It’s possible, however, that factors other than energy density are relevant for these hypothetical processes, and that those factors will be brought together in novel ways in future experiments.
The main reason for concern in the "physics disasters" category is the meta-level observation that discoveries of all sorts of weird physical phenomena are made all the time, so even if right now all the particular physics disasters we have conceived of were absurdly improbable or impossible, there could be other more realistic failure-modes waiting to be uncovered. The ones listed here are merely illustrations of the general case.
Friday, December 15, 2006
bangs, crunches, shrieks and whimpers
Here's a really fun article examining various extinction scenarios, except the only actual extinction scenarios are the "bangs", some of the crunches, shrieks and whimpers are pretty horrible, but I would guess that a lot of people would actually prefer some of them to the "non-extinction" scenarious. Here are the mst interesting "bang" scenarios: