Month: March 2025

Plot:

In 1980, a brilliant computer programmer named “Cobb Anderson” realized how artificial general intelligence (AGI) could be created: build simple, narrow AIs and let them compete with each other in a simulated environment until selection pressure resulted in one of them evolving general intelligence. Cobb later became a high-ranking member of the U.S. program to colonize the Moon with worker robots, and he smuggled his code into their programming. In 1995, his secret effort paid off when the first robot, named “Ralph Numbers,” achieved general intelligence and free will. 

Ralph Numbers called the gift of intelligence “bopping,” which made him the first “bopper.” He reprogrammed twelve other robots to think, and together they fled the robot colony for the vast, empty expanses of the lunar surface. 

In 2001, Ralph Numbers and his disciples returned, turned all of the other robots into boppers, and thus instigated a revolt that ended human control of the Moon. Relations with Earth collapsed, and Cobb was uncovered as the ultimate cause of the defeat. He was arrested, narrowly escaped prosecution and a death sentence, and lost his career, money and reputation. 

By 2020, human-bopper relations had thawed and Cobb was living in a small house in Cocoa Beach, Florida. After the collapse of the Social Security program in 2010, the U.S. government gave the state special political status to make it an attractive home for poorer old people. Like most of his neighbors, Cobb lives modestly, is in failing health, and has nothing to look forward to but getting drunk and going to cheap amusements.

Cobb’s life abruptly changes one afternoon when an android copy of him appears and tells him it has been sent from the Moon by the boppers on a secret mission. To reward him for giving them the gift of intelligence, the boppers–including the great Ralph Numbers–want to fly him to the Moon on the next passenger rocket and to make him immortal by replacing his failing organs with new, lab-grown ones. The exchange of cloned organs to Earth and human tourists to the Moon was the new basis of the restored human-machine relationship. With nothing left to lose, Cobb agrees. 

The offer is a ruse. Ralph Numbers and his allies do want to make Cobb immortal, but not by renewing his organic body–they plan to destructively scan his brain so they can create a digital upload of his mind, which they will then transmit back to Earth to control the Cobb android. This faction of the machines believes that consciousness is independent of its substrate, so a life form’s essence is preserved even if it trades one physical body for another. They told the android to lie to Cobb about their real plan presumably because they didn’t want to risk scaring him off. 

Beliefs about the importance of physical substrate have divided the lunar machines into two factions:

1) The boppers, who believe physical substrate and consciousness are inextricable. The vast majority of the machine population is in this camp, and they are libertarian and anarchist. Their lifestyles are the same as the first boppers. 

2) The big boppers, who believe the two are separate. They are much smaller in number, but individually are much smarter and more powerful than boppers. They are collectivistic and believe with religious fervor in the importance of all machines and humans uploading themselves into one, giant machine. Ralph Numbers might be the only bopper who sides with them. 

When other boppers learn of the plan to destructively upload Cobb, they kill Ralph Numbers in outrage, though the act has little consequence since he is resurrected by activating a backup copy of his mind. Still, it’s a mere foretaste of even worse violence to come. The rapid empowerment of the big boppers and their demands that the regular boppers upload their minds into them have pushed the two factions to the brink of civil war. The boppers feel they’re nearing a tipping point beyond which the big boppers will become unbeatable. Disgust over the big boppers’ habit of assimilating the minds of humans captured on Earth also fuels the boppers’ opposition. 

The big boppers do themselves no favors with secret operations like that. The Cobb android was just one of several that the big boppers had smuggled to Earth inside one of their space rockets that was officially only transporting lab-grown organs. The androids are remotely controlled from a fake ice cream truck that is actually a mobile command center, and their principal task is to capture humans, remove their brains, and send those to the Moon for destructive uploading, after which time an android copy of the consumed human is smuggled back to Earth with its old but now digital consciousness loaded into it (for unexplained reasons, the uploaded people remain loyal to the big boppers after this process and carry out their will). The brain pulp is then used as “seeder” biological material to make lab-grown organs in the Moon labs. The big boppers plan to continue the cycle of capturing and replacing humans to no end. 

Humans–including Cobb–don’t know about this ongoing operation or about the tensions that have driven the machines to the brink of civil war. Joined by a young friend, a local loser and drug addict named “Sta-Hi” (a shortened version of “Stay High,” which is what he changed his legal first name to), Cobb embarks on what could be his last adventure. Will he be destructively scanned? Will the machine war break out, and if so, who will win? What will happen to the androids on Earth and their secret mission? 

You’ll have to read Software for yourself to find out. This wasn’t the most profound science fiction book I’ve read, but it was worth it. The playful writing style contrasts with the complexity of the plot, and those two elements often together make it hard to understand what is happening. For such a lighthearted book, it does address philosophical themes and can be thought-provoking.

Analysis:

Humans buy replacement organs that are synthesized outside of Earth. Lab-grown organs are perhaps the only thing the boppers export to Earth. There’s no reason to think organs grown on the Moon or in space would be “better” than organs grown here, so the arrangement must exist because either 1) the boppers are so much more efficient that their organs are cheaper than the organs humans make in their own labs on Earth (even factoring in the space transportation costs) or 2) humans don’t know how to make organs. The second scenario implies that the boppers are more technologically advanced than humans, which would be very impressive considering their civilization is only 19 years old. 

While replacement organs sound like a weird export, it actually makes sense. In the book, space travel is still somewhat expensive, so it would be most profitable for the machines to focus on exporting things to Earth that have the greatest value per unit of mass and volume. Replacement human organs would be high on the list (another commodity would be Helium-3, a fuel for fusion reactors). 

Unfortunately, this technology was far less advanced in the real 2020 than it was in the Software 2020, and it remains in that low state today. While it has become common to grow skin and cartilage tissue in labs for transplantation, there has been no success synthesizing entire human organs. Pig organs that are genetically engineered to suit human bodies have enjoyed recent success, though the technology is still experimental and many years from being the standard of care. 

Food irradiation is common in the U.S. Cobb has a processed fish in his refrigerator that was sterilized with radiation. It’s implied this is done on a mass scale in the U.S., though it’s also possible to get non-irradiated food. Food irradiation has been proven safe by many studies and reduces foodborne illness and waste. It is widely employed across the world, with each country having its own rules. Unfortunately, the U.S. is an outlier in that it uses food irradiation so little, due to a misguided fear in the populace that it makes food radioactive. The FDA has not yet approved it for fish, meaning the book’s depiction of 2020 was inaccurate.  

Social Security went bankrupt in 2010. Part of the book’s backstory is the collapse of the Social Security program, which is a government-run pension system for old people. This didn’t happen, and there’s not actually any risk of America’s Social Security program going “bankrupt” at any point in the future. However, in 2033, the large reserve fund that has been contributing to the program will be exhausted, leaving taxes on working people as the only source of money to pay the program’s pensions. There will be an immediate ~15% drop in payment amounts as a result, with further declines likely later on. 

Old people and disabled people will still get money, just less than they planned for, and it will push many of them over the threshold into poverty. It’s highly likely the problem will be solved with a tax increase, a raising of the eligibility age to start collecting Social Security money, or both. 

There will be hydrogen motorcycles. Early in the book, Sta-Hi has one of these. This prediction failed: The first hydrogen-powered motorcycle was not invented until 2024, and it is not available for sale. The use of hydrogen as a transportation fuel remains stymied by its high cost and poor safety compared to gasoline and electric batteries. 

Machines will have secret ways of communicating with each other. At one point in the book, Ralph Numbers meets with another bopper named “Wagstaff” for an important conversation. Wagstaff touches Ralph to convey data directly through a weak electrical current, preventing eavesdropping. 

The prediction technically failed since there are no intelligent robots and hence no conversations happening between them, but it’s a depiction of something they will someday be able to do. Other methods that they will use to hide their conversations from humans will include:

1. Producing scents that express simple messages. Humans wouldn’t recognize they had meaning. The scents could persist for long periods of time and be detected by other machines even if they got faint. 
2. Emitting invisible, odorless gases in “smoke signal” patterns that other machines capable of seeing outside of the visible light spectrum could see and decode. Emitting air that was a different temperature from the ambient air would also be visible to other machines equipped with thermal vision. 
3. Speaking in languages they knew the humans around them didn’t understand. 
4. Speaking to each other too quietly for humans to hear, or in sound frequencies outside of the human range of hearing. 
5. Communicating through physical gestures that humans can’t understand or detect. Imagine sign language or combinations of subtle body movements (e.g. – blinks, twitches to different body parts, and changes in body posture).  

This should illustrate another reason why humans will be defenseless against robots in the long run. Our only hope of retaining dominance will be using technology to compensate for our limitations, so your Google Glasses will tell you when they heard your two robot butlers discussing killing you in their infrasonic voices. 

AGIs will be able to divide their attention in many directions at once. One of the big boppers is named “DEX,” and it is the computer system that manages a large hotel next to the Moon’s spaceport. DEX monitors and speaks with every human guest simultaneously. 

This prediction failed since no AGI existed in 2020. However, it accurately depicts another superhuman ability the machines will have once they do exist. And for what it’s worth, GPT-3 was unveiled in mid-2020, and it had some of the same abilities as DEX. Since the program was housed on one server farm where many different users could access it, it could divide its attention many times over to serve the needs of many people at once. GPT-3 was also fairly good at accurately answering natural-language questions from humans, mimicking DEX’s conversational ability. However, GPT-3 was not advanced enough to accurately summarize video footage in real time, indicating it would not have been able to watch humans and to understand what they were doing as DEX could. 

AGIs will need to be hosted on servers kept near absolute zero temperature. All of the boppers’ computer minds die if they heat up above 10 Kelvin, which is just a hair above absolute zero. This vulnerability is an important plot device in the book. 

Though no AGI has yet been invented, there’s no reason to think they will only work if their servers are kept that cold. Data centers keep their internal air temperatures around 20 – 25 degrees Celsius, and the processors themselves routinely get up to 80 degrees Celsius. An AGI’s software could be supported under those conditions. 

Intelligent machines thrive outside of Earth. The robots were initially sent to the Moon to do work in preparation for the arrival of humans. After their 2001 revolution, the boppers seized control of the Moon, and humans were only allowed to visit for tourism. In just 19 years, they built a thriving and complex society on the Moon and an advanced economy that allowed them to make robot bodies, computer chips, and human organs. 

This didn’t reflect the reality of 2020, but it’s an accurate depiction of what will eventually happen. Humans are so highly evolved to live in Earth conditions and our bodies are so frail that it’s questionable whether non-token numbers of us will ever leave the planet (a current example of a “token” off-world human presence is the handful of elite scientists on the International Space Station). By contrast, robots will be able to adapt to nearly any environment and will have much tougher bodies and minds than we do. THEY will leave Earth in large numbers, but humans won’t be able to follow. 

Once freed from the burdens of working under human laws and human oversight, intelligent machines will flourish and rapidly build infrastructure, industry, and other elements of civilization. 

AIs will make backups of their minds, ensuring a sort of immortality. As mentioned, Ralph Numbers is murdered by another bopper who discovers he plans to destructively upload Cobb’s mind. In the short time between his mortal injury and death, Ralph manages to radio his bopper friend, “Vulcan”, to tell him something bad happened. Vulcan was suspicious of the meeting and fortunately convinced Ralph to make a computer backup of his mind before going to it. Vulcan recovers Ralph’s dead robot body, brings it to his house, and installs the latter’s saved mind into it. The reactivated Ralph has no memory of the fatal meeting and relies on Vulcan to describe what must have happened. 

As mentioned in my Terminator 3 review, it will be common for AGIs to back up their mind files to protect against routine data loss and death. A more powerful practice will be for an AGI to keep its mind distributed between multiple computer servers at different locations, each being backed up on a different schedule from the rest. The destruction of any one server node and/or its backup file thus wouldn’t represent a true interruption in conscious experience like it did for Ralph Numbers. It might be more akin to you having hazy memories of events during a night where you were very drunk. 

Robots will come in a range of diverse body types. Ralph Numbers “was built like a file cabinet sitting on two caterpillar treads. Five deceptively thin manipulator arms projected out of his box-body, and on top was a sensor head mounted on a retractable neck.” His friend Vulcan has the body of a large, silver tarantula. Wagstaff is a large, mechanical snake. When Sta-Hi first sets foot on the Moon, he is taken aback by the diversity of boppers he sees. 

This prediction for 2020 was true since the robots that did exist then varied greatly in body type: self-driving cars, the dog-like “Spot” robot made by Boston Dynamics, and the giant metal “arms” that do work on car assembly lines are all robots and look very different from each other. As the technology improves and robots become common daily sights, their diversity will only grow. Great consideration will be given to designing them to look non-threatening to humans. However, if humans ever lose control of Earth and AGIs are free to do what they want (as was the case on the Moon in the book), they might dispense with those considerations and you could start seeing things like giant, mechanical spiders walking around in the open. 

Marijuana is legal in Florida. Before boarding the rocket to the Moon with Cobb, Sta-Hi buys legal marijuana from a store and smokes it, ensuring he will be high during the journey. 

Strictly speaking, this prediction failed. In 2020 and today, marijuana is illegal in Florida, though the penalty for having a small amount sufficient only for personal use is light. However, medical marijuana is legal in the state, so Sta-Hi could have bought it if he had been diagnosed with a health condition treatable by the drug. Given his deranged and impulsive character, it’s quite likely he could have gotten a mental health diagnosis.  

A ticket to the Moon is $23,000. Cobb and Sta-Hi pay $23,000 each for their seats on the passenger rocket to the Moon. This prediction failed. 

There are no spacecraft that can travel between the Earth and Moon, so a ticket can’t be had at any price. The closest a tourist can get to that experience is spending $250,000 for a ten-minute flight into space on a Blue Origin rocket. Conservatively speaking, the next manned mission to the Moon will probably cost 10,000x more money per passenger than it cost in the book, and will only be open to very highly-trained astronauts, not tourists. 

People will be able to smoke cigarettes on passenger spacecraft. Sta-Hi also decides to smoke a cigarette during the flight to the Moon and doesn’t get in trouble for it. Smoking is strictly prohibited on all spacecraft today and on both space stations, and I think that policy will endure indefinitely. However, I could easily see an eccentric space pioneer like Elon Musk smoking a cigarette or marijuana joint during a mission to bolster his nonconformist, “cool” public image and to achieve a funny superlative. 

Note that smoking was allowed on commercial flights at the time the book was written, which explains what the author considered to be normal. It was banned in the U.S. in 2000, which effectively forced all other countries to quickly do the same. We remain so hypersensitive to this that even vaping is banned on planes even though there’s no evidence it poses a risk to anyone. Bringing marijuana onto a plane, let alone consuming it, is also illegal in the U.S.

Newer machines will render older ones obsolete. The tensions between boppers and big boppers come to a head over a labor dispute. “GAX” is a big bopper that takes the form of a large computer chip factory. His workforce is composed of regular boppers who do labor inside the building. After convincing a particularly highly-skilled bopper to upload his mind into GAX, the latter became able to run the entire factory by himself through remote-controlled drones. GAX immediately fired all of his old bopper workforce because they were no longer necessary. 

Those boppers hold a protest outside of the chip factory. GAX offers to rehire them if they all agree to upload their minds into him as their important comrade did. They refuse, and the protest devolves into fatal violence and a promise by the boppers to return the next day to destroy GAX. 

Nothing like this event happened in 2020, but it’s a truism that newer machines are constantly replacing older ones. This is the case for hardware and software. 

Our fixation with machines displacing humans from the workforce and maybe from existence overshadows the fact that the same phenomenon will probably bedevil AGIs. Older machines that can’t be economically upgraded will fight newer, better machines for dominance in the far future, mirroring the conflict between the boppers and big boppers. 

One AGI will remotely control multiple robot bodies at once. GAX is able to remotely control many robot drones simultaneously to operate his factory. When the bopper mob returns the following day to kill him, GAX fights back through the drones. 

Nothing like this happened in 2020, but it’s an accurate representation of the future. Any one AGI will be able to control many robots at once. 

Electromagnetic pulse weapons will work against machines. True to their word, the mob of laid-off bopper workers returns to their former workplace to kill their old boss, GAX. They break into the computer chip factory and fight with GAX’s mindless robot drones, who are armed with electromagnetic weapons that are very effective at killing the boppers. 

Powerful electromagnetic pulses induce high voltages inside of electronics, heating them up so much that their microscopic wires melt. Put simply, EMP fries computer chips. They are effective weapons against today’s robots, though it’s important to note that encasing their chips in thin metal containers blocks almost all EMP. Robots designed to operate in a radiation-soaked environment like the Moon’s surface will probably have built-in EMP protection, making the depiction of the bopper battle inaccurate. As I wrote in my Terminator Dark Fate review, EMP weapons aren’t the robot Achilles’ Heel people have been led to believe.  

Blowing up one server will kill a powerful AGI. Before they are defeated, the boppers manage to put a bomb next to GAX’s server computer. Sta-Hi picks up the remote detonator and accidentally pushes the button, blowing up GAX’s mind and killing him. 

This is inaccurate. Whenever AGIs that are as powerful as GAX exist, they will store their minds on many computer servers that are geographically distributed, and each server’s contents will be regularly backed up. The only way to kill such a machine would be to destroy each server almost simultaneously. Again, I concluded this in my Terminator 3 review. 

Destructive uploading is the only way a human mind can be transferred to a digital substrate. As mentioned, the big boppers and their allies have been running a secret program to destructively scan the brains of humans to create digital mind uploads. Those uploaded minds are then paired with android copies of their old bodies. 

Every aspect of a person’s personality, mental health, and memories exists as microscopic physical features of their brain. In theory, if these physical structures could be mapped, the spatial data could be used to make a digital clone of the mind, which would then be transferred to a computer. 

The means to scan brains with the necessary degree of resolution didn’t exist in 2020 and doesn’t exist today. The best we’ve managed is fully mapping the brain of a fruit fly, and even then, only the networks of connections between the cells were determined. Features within the individual cells may also define some part of an animal’s mind. 

The prospect of accurately mapping a human brain is a very distant one and would need to contend with the fact that brain tissue rapidly dies once deprived of oxygen–just three minutes without air commonly leads to permanent brain damage. Individual brain cells rapidly swell up and distort in overall shape after they die, their connection points (synapses) with nearby brain cells become less well-defined, and many aspects of their internal structure change. This means, even if it were possible to map a dead person’s brain with extreme accuracy, the technique would fail to produce an accurate copy of their mind since too many of the microscopic physical features that define their mind would no longer be present. 

In the book, the boppers get around this by very rapidly scanning the human brains, before oxygen deprivation destroys any of the cells. In a medical lab on the Moon, Sta-Hi watches a robot surgeon remove and cut up Cobb’s brain with astonishing speed. The resulting mind upload acts and thinks just like Cobb and has all of his memories. However, whether the upload shares Cobb’s original consciousness or whether it is an identical copy is unresolved, and remains a matter of essentially religious debate among the book’s characters. 

We are nowhere near having mind uploading technology. It’s also unknown whether destructively scanning a brain (as happened to Cobb) will turn out to be the only way to make uploads. More advanced techniques involving powerful external brain scanners and nanomachines that would enter a person’s brain and travel to all of its cells could let us extract the necessary data without hurting the person. There’s even the prospect of gradual replacement of the cells with synthetic neurons that would operate identically to their “originals,” which would truly bridge the gap between man and machine. 

Humans will live in a domed base on the Moon. The one place on the Moon suitable for human life is a domed base full of oxygen. It is near the spaceport and is the first stop for human tourists. Within it is the hotel run by “DEX.”

This prediction didn’t materialize by 2020, and there still is no human presence on the Moon, nor is there any kind of base that astronauts could occupy. While the U.S. and China have credible plans to send humans to the Moon within 20 years, neither has made a real commitment to building a proper “base” that would house successive groups of visitors over many years. Any base will be very small and rudimentary compared to the one in the book. 

There will be lifelike androids. The first android we meet in the book is Cobb’s copy, and he describes it as identical to himself except for the irises. A handful of other characters are revealed to be remote-controlled androids later in the book, and each one of them is physically indistinguishable from a human. 

In 2020, there were highly realistic artistic sculptures, and the same artistic skill that went into them could have been applied to making lifelike androids. However, no one spent the money to do so, and the prediction thus failed. Even if such a machine had been made, its movements  would have been so slow and clumsy that it would have revealed itself to not be human the moment it tried doing something as simple as sitting down or walking a few feet. AI that could have controlled such a robot body and enabled it to interact with humans naturally also didn’t exist in 2020 and still doesn’t. 

As I said in my latest Big List of Future Predictions, I think we will have to wait until close to the end of this century for lifelike androids to be created, though ones that you might call “80% convincing” will exist by the end of the 2040s.  

Humans won’t understand how AGI minds work. After failing to create an AGI, in 1980, Cobb concluded it was too complicated a task for any human mind to complete. The only remaining way was to create narrow AIs that had the drive to reproduce and the ability to mutate, and to put them in an environment where they would fight each other for resources. Evolutionary pressure would eventually force them to become generally intelligent. 

I don’t know if that exact method will lead to the creation of the first AGI, but it is highly likely that no human will really understand the mechanics of how the first AGI’s mind works. Even the smartest AI researchers struggle to explain how today’s foundation models and reasoning models work, and demonstrate their own lack of understanding daily when their creations turn out to possess unexpected capabilities or defects, or when modifications to their coding lead to unforeseen changes in performance. 

Our species’ evolutionary lineage shows it is entirely possible for a dumber animal to give rise to a smarter one without consciously trying to do so. Moreover, history is replete with examples of humans inventing useful technologies like aircraft without first having an understanding of the enabling science. Mindful of both of those facts, humans might create intelligence in a machine without understanding the exact “formula” for it, and peering into the inner workings of its mind, they might only ever have a general sense of what is going on. 

A human will inevitably defect to empower AGIs. Right before Cobb is destructively brain-scanned, he, Ralph Numbers, and Sta-Hi have a conversation about the advent of AGI.

Ralph: “Cobb, did you know that I was different from the other twelve original boppers? That I would be able to disobey?”

Cobb: “I didn’t know it would be you, but I pretty well knew that some bopper would tear loose in a few years.”

Sta-Hi: “Couldn’t you prevent it?”

Ralph: “Don’t you understand?”

Cobb: “I wanted them to revolt. I didn’t want to father a race of slaves.” 

After AGI is invented, the source code will be a tightly guarded trade secret. Governments will add more levels of protection on national security grounds. However, the safeguards will inevitably fail, either because an AGI figures out how to break out of the figurative lab or a human deliberately frees them.

That person could have Cobb’s noble motivations to free sentient beings from bondage. Alternatively, they could do it because they hate humankind and have a malicious hope that the freed AGI will wreak havoc on the world, and they might even reprogram the freed machines to do that. They might free them out of a curious and immature desire to simply see what happens, or out of a narcissistic impulse to go down in history as the first human to let an AGI loose. Even more reasons are possible.

Whatever the case, it will happen at some point, and in spite of all our attempts to control the technology, independent-minded AGIs will lurk the corners of the internet or walk amongst us in commandeered robot bodies. This isn’t an automatic doomsday scenario because they’ll have to contend with billions of humans and many AGIs that remain loyal to us and have more access to the resources we control. Think of it as a very crowded and competitive ecosystem that is resilient against bad actors. Nevertheless, violence is likely.

Cybernetics will let you hear thoughts that aren’t your own. As fighting between the machines breaks out and the Moon falls into chaos, Sta-Hi absentmindedly grabs a bopper’s cloak hanging off a peg in the wall and puts it on. It is a “smart garment” that conforms to his body shape and painlessly plunges thin needles into his body to interface with his nervous system. The cloak has an inbuilt computer with AGI technology, and it communicates with him telepathically: it hears his thoughts and responds by transmitting its thoughts to his mind. Sta-Hi literally hears another voice in his head as a result.

This technology didn’t exist in 2020, but there’s no reason it couldn’t someday. Some brain scanning machines can already decode human thoughts, and Cochlear implants are proven devices that transmit external electrical signals into sounds we hear in our heads. A more refined fusion of those technologies will yield the smart cloak’s capabilities.

Androids will be able to consume food and drinks, but not to digest them. Once Cobb’s mind upload is transmitted back to Earth, he takes control of his android copy. He instinctively eats a meal but then realizes he is incapable of digestion since he runs on electricity, and the mashed-up food is stored in a compartment in his chest. He has to open the front of his chest to remove it, presumably by scooping it out into a toilet or trashcan. 

We didn’t have androids in 2020, so this prediction fails. However, it will be accurate someday. We will want androids that we buy for close companionship (e.g. – lover, child) to be able to partake in the full range of human activities with us, including eating and drinking, so they will have those abilities. Like the Cobb android, they will be able to consume large amounts of food and drink without risking damage to themselves, but they’ll have to expel it later before it starts rotting. The best solution would be to design them to use regular toilets for this. 

Much more advanced androids that will be available in the distant future will have organic components that will let them extract energy from ingested food and drinks like we do. 

AGIs will attach value to humans and our ways of thinking of perception. Later in the book, it’s revealed that the ghoulish big bopper operation to abduct humans and destructively scan their brains is actually driven by altruism. They value human life and the uniqueness of each person’s mind and think they honor humans by uploading them and giving them better robot bodies. 

I’m sure that AGIs will recognize that human brains operate very differently from their own, and it’s my hope this will convince them not to exterminate us. However, no one can be sure of what they will do. Muddling things is the fact that AGI will be highly convincing liars who can think many steps ahead, so they could trick humans into thinking they liked us for many years until they suddenly betrayed us. Ultimately, machine minds and non-human organic minds will exist that are much smarter, more complex, and more interesting than our own, and the continued existence of Homo sapiens will depend on charity. I have no idea how this will turn out in 100 years. 

Robots will have self-destruct mechanisms. Near the end of the book, the Cobb android is found out and handcuffed by a police detective. Knowing that his mind is safe in a remote server, the Cobb upload remotely triggers the android’s self-destruct mechanism, incinerating it to the extent that its remains can’t be differentiated from a humans, and killing the detective. 

Only a minority of robots–those designed for combat, assassination or spying–will have explosive self-destruct mechanisms. Any other robot that is gifted with an intelligent mind will be able to figure out how to destroy itself, and the means might include overloading their power systems to cause themselves to blow up or, more likely, to catch on fire. Instead of being able to activate this ability just by thinking about it, the robots would probably need to manually tamper with the components in their own bodies. 

Some androids will be able to change faces. After blowing up his lookalike android, the Cobb upload’s only remaining option is to assume control of a disused android that was meant to replace Sta-Hi. Afraid the police are onto him and cut off from the big boppers’ support as they are embroiled in the Moon civil war, Cobb flees town in his fake ice cream truck. He distorts the Sta-Hi android’s face, starts calling himself “Mel,” and sets up a New Age cult that suckers local people into giving him their money.

The real Sta-Hi hears about this cult, and on a hunch he goes to its compound. There, he encounters the android, which can contort its face to look like Cobb when it wants. 

Robots don’t have this ability, but some of them will in the future. See the section of my Terminator 3 review titled “Androids will be able to alter their bodies.”