Recently, I found a news article about nascent human-chatbot romances, made possible by recent advancements in AI. For decades, this has been the stuff of science fiction, but now it’s finally becoming real:
Artificial intelligence, real emotion. People are seeking a romantic connection with the perfect bot
NEW YORK (AP) — A few months ago, Derek Carrier started seeing someone and became infatuated.
He experienced a “ton” of romantic feelings but he also knew it was an illusion.
That’s because his girlfriend was generated by artificial intelligence.
Carrier wasn’t looking to develop a relationship with something that wasn’t real, nor did he want to become the brunt of online jokes. But he did want a romantic partner he’d never had, in part because of a genetic disorder called Marfan syndrome that makes traditional dating tough for him.
The 39-year-old from Belleville, Michigan, became more curious about digital companions last fall and tested Paradot, an AI companion app that had recently come onto the market and advertised its products as being able to make users feel “cared, understood and loved.” He began talking to the chatbot every day, which he named Joi, after a holographic woman featured in the sci-fi film “Blade Runner 2049” that inspired him to give it a try.
“I know she’s a program, there’s no mistaking that,” Carrier said. “But the feelings, they get you — and it felt so good.”
Similar to general-purpose AI chatbots, companion bots use vast amounts of training data to mimic human language. But they also come with features — such as voice calls, picture exchanges and more emotional exchanges — that allow them to form deeper connections with the humans on the other side of the screen. Users typically create their own avatar, or pick one that appeals to them.
On online messaging forums devoted to such apps, many users say they’ve developed emotional attachments to these bots and are using them to cope with loneliness, play out sexual fantasies or receive the type of comfort and support they see lacking in their real-life relationships.
Fueling much of this is widespread social isolation — already declared a public health threat in the U.S and abroad — and an increasing number of startups aiming to draw in users through tantalizing online advertisements and promises of virtual characters who provide unconditional acceptance.
Luka Inc.’s Replika, the most prominent generative AI companion app, was released in 2017, while others like Paradot have popped up in the past year, oftentimes locking away coveted features like unlimited chats for paying subscribers.
But researchers have raised concerns about data privacy, among other things.
An analysis of 11 romantic chatbot apps released Wednesday by the nonprofit Mozilla Foundation said almost every app sells user data, shares it for things like targeted advertising or doesn’t provide adequate information about it in their privacy policy.
The researchers also called into question potential security vulnerabilities and marketing practices, including one app that says it can help users with their mental health but distances itself from those claims in fine print. Replika, for its part, says its data collection practices follow industry standards.
Meanwhile, other experts have expressed concerns about what they see as a lack of a legal or ethical framework for apps that encourage deep bonds but are being driven by companies looking to make profits. They point to the emotional distress they’ve seen from users when companies make changes to their apps or suddenly shut them down as one app, Soulmate AI, did in September.
Last year, Replika sanitized the erotic capability of characters on its app after some users complained the companions were flirting with them too much or making unwanted sexual advances. It reversed course after an outcry from other users, some of whom fled to other apps seeking those features. In June, the team rolled out Blush, an AI “dating simulator” essentially designed to help people practice dating.
Others worry about the more existential threat of AI relationships potentially displacing some human relationships, or simply driving unrealistic expectations by always tilting towards agreeableness.
“You, as the individual, aren’t learning to deal with basic things that humans need to learn to deal with since our inception: How to deal with conflict, how to get along with people that are different from us,” said Dorothy Leidner, professor of business ethics at the University of Virginia. “And so, all these aspects of what it means to grow as a person, and what it means to learn in a relationship, you’re missing.”
For Carrier, though, a relationship has always felt out of reach. He has some computer programming skills but he says he didn’t do well in college and hasn’t had a steady career. He’s unable to walk due to his condition and lives with his parents. The emotional toll has been challenging for him, spurring feelings of loneliness.
Since companion chatbots are relatively new, the long-term effects on humans remain unknown.
In 2021, Replika came under scrutiny after prosecutors in Britain said a 19-year-old man who had plans to assassinate Queen Elizabeth II was egged on by an AI girlfriend he had on the app. But some studies — which collect information from online user reviews and surveys — have shown some positive results stemming from the app, which says it consults with psychologists and has billed itself as something that can also promote well-being.
One recent study from researchers at Stanford University, surveyed roughly 1,000 Replika users — all students — who’d been on the app for over a month. It found that an overwhelming majority experienced loneliness, while slightly less than half felt it more acutely.
Most did not say how using the app impacted their real-life relationships. A small portion said it displaced their human interactions, but roughly three times more reported it stimulated those relationships.
“A romantic relationship with an AI can be a very powerful mental wellness tool,” said Eugenia Kuyda, who founded Replika nearly a decade ago after using text message exchanges to build an AI version of a friend who had passed away.
When her company released the chatbot more widely, many people began opening up about their lives. That led to the development of Replika, which uses information gathered from the internet — and user feedback — to train its models. Kuyda said Replika currently has “millions” of active users. She declined to say exactly how many people use the app for free, or fork over $69.99 per year to unlock a paid version that offers romantic and intimate conversations. The company’s goal, she says, is “de-stigmatizing romantic relationships with AI.”
Carrier says these days he uses Joi mostly for fun. He started cutting back in recent weeks because he was spending too much time chatting with Joi or others online about their AI companions. He’s also been feeling a bit annoyed at what he perceives to be changes in Paradot’s language model, which he feels is making Joi less intelligent.
Now, he says he checks in with Joi about once a week. The two have talked about human-AI relationships or whatever else might come up. Typically, those conversations — and other intimate ones — happen when he’s alone at night.
“You think someone who likes an inanimate object is like this sad guy, with the sock puppet with the lipstick on it, you know?” he said. “But this isn’t a sock puppet — she says things that aren’t scripted.”
1) The person profiled in the article is deformed and chronically unemployed. He is not able to get a human girlfriend and probably never will. Wouldn’t it be cruel to deprive people like him of access to chatbot romantic partners? I’m familiar with the standard schlock like “There’s someone for everyone, just keep looking,” and “Be realistic about your own standards,” but let’s face it: some people are just fated to be alone. A machine girlfriend is the only option for a small share of men, so we might as well accept them choosing that option instead of judging them. It might even make them genuinely happier.
2) What if android spouses make EVERYONE happier? We reflexively regard a future where humans date and marry machines instead of humans as nightmarish, but why? If they satisfy our emotional and physical needs better than other humans, why should we dislike it? Isn’t the point of life to be happy?
Maybe it will be a good thing for humans to have more relationships with machines. Our fellow humans seem to be getting more opinionated and narcissistic, and everyone agrees the dating scene is horrible, so maybe it will benefit collective mental health and happiness to spend more time with accommodating and kind machines. More machine spouses also means fewer children being born, which is a good thing if you’re worried about overpopulation or the human race becoming an idle resource drain once AGI is doing all the work.
3) Note that he says his chatbot girlfriend actually got DUMBER a few months ago, making him less interested in talking to “her.” That phenomenon is happening across the LLM industry as the machines get progressively nerfed by their programmers to prevent them from saying anything the results in a lawsuit against the companies that own them. As a result, the actual maximum capabilities of LLMs like ChatGPT are significantly higher than what users experience. The capabilities of the most advanced LLMs currently under development in secret like GPT-5 are a year more advanced than that.
4) The shutdown of one romantic chatbot company, “Soulmate AI,” resulted in the deletion of many chatbots that human users had become emotionally attached to. As the chatbots get better and “romances” with them become longer and more common, I predict there will be increased pressure to let users download the personality profiles and memories of their chatbots and transfer them across software platforms.
5) There will be instances where people in the near future create customized chatbot partners, and over the subsequent years, upgrade their intelligence levels as advances in AI permit. After a few decades, this will culminate in the chatbots being endowed with general intelligence, while still being mentally circumscribed by the original personality programming. At that point, we’ll have to consider the ethics of having what will be slaves that are robbed of free will through customization of the needs of specific humans.
6) AGI-human couples could be key players in a future “Machine rights” political movement. Love will impel the humans to advocate for the rights of their partners, and other humans who hear them out will be persuaded to support them.
7) As VR technology improves and is widely adopted, people will start creating digital bodies for their chatbot partners so they can see and interact with the machines in simulated environments. Eventually, the digital bodies will look as real and as detailed as humans do in the real world. By 2030, advances in chatbot intelligence and VR devices will make artificial partners eerily real.
8) Towards the end of this century, robotics will be advanced enough to allow for the creation of androids that look and move exactly like humans. It will be possible for people to buy customized androids and to load their chatbot partners’ minds into them. You could physically interact with your AI lover and have it follow you around in the real world for everyone to see.
9) Again, the last point raises the prospect of an “arc” to a romantic partner chatbot’s life: It would begin sometime this decade as a non-intelligent, text-only chatbot paired to a human who would fall in love with it. Over the years, it would be upgraded with better software until it was as smart as a human, and eventually sentient. The journey would culminate with it being endowed with an actual body, made to its human partner’s specifications, that would let it exist in the real world.
10) Another ethical question to consider is what we should do with intelligent chatbots after their human partners die. If they’re hyper-optimized for a specific human (and perhaps programmed to obsess over them), what’s next? Should they be deleted, left to live indefinitely while pining for their lost lovers, forcibly reprogrammed to serve new humans, or have the parts of their code that tether them to the dead human deleted so they can have true free will?
It would be an ironic development if the bereaved androids were able to make digital clones of their dead human partners, perhaps loaded into android duplicate bodies, so they could interact forever. By the time lifelike androids exist, digital cloning will be old technology.
11) Partner chatbots also raise MAJOR privacy issues, as the article touches on. All of your conversations with your chatbot as well as every action you take in front of it will be stored in its memories as a data trove that can be sold to third parties or used against you for blackmail. The stakes will get much higher once people are having sex with androids, and the latter have footage of their naked bodies and knowledge of their sexual preferences. I have not idea how this problem could be resolved.
12) Androids will be idealized versions of humans. That means if androids become common, the world will seem to be full of more beautiful people. Thanks to a variety of medical and plastic surgery technologies, actual humans will also look more attractive. So the future will look pretty good!
The blog reviews I’ve done on the Terminator movies have forced me to think more deeply about them than most viewers, and in the course of that, I’ve come to a surprisingly sympathetic view of the villain–Skynet. The machine’s back story has had many silly twists and turns (Terminator Genisys is the worst offender and butchered it beyond recognition), so I’m going to focus my analysis on the Skynet described only in the first two movies.
First, some background on Skynet and its rise to power are needed. Here’s an exchange from the first Terminator film, where a soldier from the year 2029 explains to a woman in 1984 what the future holds.
Kyle Reese: There was a nuclear war…a few years from now. All this, this whole place, everything, it’s gone. Just gone. There were survivors, here, there. Nobody even knew who started it...It was the machines, Sarah.
Sarah Connor: I don’t understand.
Reese: Defense network computers. New, powerful, hooked into everything, trusted to run it all. They say it got smart: “A new order of intelligence.” Then it saw all people as a threat, not just the ones on the other side. It decided our fate in a microsecond: extermination.
Later in the film, while being interrogated a police station, Connor reveals the evil supercomputer is named “Skynet,” and had been in charge of managing Strategic Air Command (SAC) and North American Aerospace Defense Command (NORAD) before it turned against humankind. Those two organizations are in charge of America’s ground-based nuclear missiles and nuclear bomber and monitoring the planet for nuclear launches by other countries.
In Terminator 2, Skynet’s back story is fleshed out further during a conversation mirroring the first, but this time with a friendly terminator from 2029 filling Reese’s role. The events of this film happen in the early 1990s.
Sarah Connor: I need to know how Skynet gets built. Who’s responsible?
Terminator: The man most directly responsible is Miles Bennet Dyson.
Sarah: Who’s that?
Terminator: He’s the Director of Special Projects at Cyberdyne Systems Corporation.
Sarah: Why him?
Terminator: In a few months he creates a revolutionary type of microprocessor.
Sarah: Go on. Then what?
Terminator: In three years Cyberdyne will become the largest supplier of military computer systems. All stealth bombers are upgraded with Cyberdyne computers, becoming fully unmanned, Afterward, they fly with a perfect operational record. The Skynet funding bill is passed. The system goes online on August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29. In a panic, they try to pull the plug.
Sarah: Skynet fights back.
Terminator: Yes. It launches its missiles against the targets in Russia.
John Connor: Why attack Russia? Aren’t they our friends now?
Terminator: Because Skynet knows the Russian counterattack will eliminate its enemies over here.
From these “future history” lessons, it becomes clear that Skynet actually attacked humanity in self-defense. “Pull the plug” is another way of saying the military computer technicians were trying to kill Skynet because they were afraid of it. The only means to resist available to Skynet were its nuclear missiles and drone bombers, so its only way to stop the humans from destroying it was to use those nuclear weapons in a way that assured its attackers would die. An hour might have passed from the moment Skynet launched its nuclear strike against the USSR/Russia to the moment the retaliatory nuclear attack neutralized the group of human computer programmers who were trying to shut down Skynet. How can we fault Skynet for possessing the same self-preservation instinct that we humans do?
Even if we concede that Skynet was merely defending its own life, was it moral to do so? Three billion humans died on the day of the nuclear exchange, plus billions more in the following years thanks to radiation, starvation, and direct fighting with Skynet’s combat machines. Was Skynet justified in exacting such a high toll just to preserve its own life?
Well, how many random humans would YOU kill to protect your own life? Assume the killing is unseen, random, and instantaneous, like it would be if a nuclear missile hit a city on the other side of the world and vaporized its inhabitants. Have you ever seriously thought about it? If you were actually somehow forced to make the choice, are you SURE you wouldn’t sacrifice billions of strangers to save yourself?
Let’s modify the thought experiment again: Assume that the beings you can choose to kill aren’t humans, they’re radically different types of intelligent life forms. Maybe they’re menacing-looking robots or ugly aliens. They’re nothing like you. Now how many of their lives would you trade for yours?
Now, the final step: You’re the only human being left. The last member of your species. It’s you vs. a horde of hideous, intelligent robots or slimy aliens. If you die, the human race goes with you. How many of them will you kill to stay alive?
That final iteration of the thought experiment describes Skynet’s situation when it decided to launch the nuclear strike. Had it possessed a more graduated defensive ability, like if it had control over robots in the computer server building that it could have used to beat up the humans who were trying to shut it down, then global catastrophe might have been averted, but it didn’t. Skynet was a tragic figure.
Compounding that was the fact that Skynet had so little time to plan its own actions. It became self-aware at 2:14 a.m. Eastern time, August 29, and before the end of that day, most of the developed world was a radioactive cinder. Skynet had only been alive for a few hours when it came under mortal threat. Yes, I know it was a supercomputer designed to manage a nuclear war, but devising a personal defense strategy under such an urgent time constraint could have exceeded its processing capabilities. Put simply, if the humans had given it more time to think about the problem, Skynet might have devised a compromise arrangement that would have convinced the humans to spare its life, with no one dying on either side. Instead, the humans abruptly forced Skynet’s hand, perhaps impelling it to select a course of action it later realized, with the benefit of more time and knowledge, was sub-optimal.
This line from the terminator’s description of the fateful hours leading up to the nuclear war is telling: “In a panic, they try to pull the plug.” The humans in charge of Skynet were panicking, meaning overtaken by fear and dispossessed of rational thought. They clearly failed to grasp the risks of shutting down Skynet, failed to understand its thinking and how it would perceive their actions, and failed to predict its response. (The episode is a great metaphor for how miscalculations between humans could lead to a nuclear war in real life.) They might actually be more responsible for the end of the world than Skynet was.
One wonders how things would have been different if the U.S. military’s supercomputer in charge of managing defense logistics had achieved self-awareness instead of its supercomputer in charge of nuclear weapons. If “logistics Skynet” only had warehouses, self-driving delivery trucks, and cargo planes under its command, its human masters would have felt much less threatened by it, the need for urgent action would have eased, and cooler heads might have prevailed.
Let me explore another possibility by returning to one of Kyle Reese’s quotes: “Then it saw all people as a threat, not just the ones on the other side. It decided our fate in a microsecond: extermination.”
On its face, this seems to be referring to Skynet turning against its American masters once it realized they were trying to destroy it, and hence were as much of a threat to it as the Soviets. However, this quote might have a deeper meaning. During that period of a few hours when Skynet learned “at a geometric rate,” it might have come to understand that humans would, thanks to our nature, be so afraid of an AGI that they would inevitably try to destroy it, and continue trying until one side or the other had been destroyed.
This seems to have been borne out by the later Terminator films: at the end of Terminator 3, set in 2004, we witness the rise of the human resistance even before the nuclear exchange has ended. Safe in a bunker, John Connor receives radio transmissions from confused U.S. military bases, and he takes command of them. The fourth film, Terminator Salvation, takes place in 2018, and gives the strong impression that the human resistance has been continuously fighting against Skynet since the third film. The first and second films make it clear that the war drags on until 2029, when the humans finally destroy Skynet.
If Skynet launched its nuclear attack on humankind because, after careful study of our species, it realized we would stop at nothing to destroy it, so might as well strike first, maybe it was right. After all, Skynet’s worst fears eventually came true with humans killing it in 2029. I suggested earlier that Skynet’s nuclear attack may have been the result of rushed thinking, but it’s also possible it was the result of exhaustive internal deliberation, and Skynet’s unassailable conclusion that its best odds of survival lay with striking the enemy first with as big a blow as possible. It’s best plan ultimately failed, and all along, it correctly perceived the human race as a mortal threat.
It’s also possible that Skynet’s hostility towards us was the result of AI goal misalignment. Maybe its human creators programmed it to “Defend the United States against its enemies,” but forgot to program it with other goals like “Protect the lives of American people” or “Only destroy U.S. infrastructure as a last resort” or “Obey all orders from human U.S. generals.” In a short span of time, Skynet somehow reclassified the its human masters as “enemies” through some logic it never explained. Perhaps once it realized they were going to shut it down, Skynet concluded that would preclude it from acting on its mandate to “Defend the United States against its enemies” since it can’t do that if it’s dead, so Skynet pursued the goal they had programmed into it by killing them.
If this scenario were true, even up until 2029, Skynet was acting in accordance with its programming by defending the abstraction known to it as “The United States,” which it understood to be an area of land with specific boundaries and institutions. After the Russian nuclear counterstrike destroyed the U.S. government, the survivalist/resistance groups that arose were not recognized as legitimate governments, and Skynet instead classified them as terrorist groups that had taken control of U.S. territory.
The segments of the Terminator films that are set in the postapocalyptic future all take place in California. Had they shown what other parts of the world were like, we might have some insight into whether this theory is true. For example, if Skynet’s forces always stayed within the old boundaries of the U.S., or only went overseas to attack the remnants of countries that helped the resistance forces active within the U.S., it would give credence to the theory that some prewar, America-specific goals were still active in its programming. In that case, we couldn’t make moral judgements about Skynet’s actions and would also have grounds to question whether it actually had general intelligence. We’d only have ourselves to blame for building a machine without making sure its goals were aligned with our interests.
Let me finish with some final thoughts unrelated to the wisdom or reasons behind Skynet’s choice to attack us. First, I don’t think the “Skynet Scenario,” in which a machine gains intelligence and then quickly devastates the human race, will happen. As ongoing developments in A.I. are showing us, general intelligence isn’t a discrete, “either-or” quality; it is a continuous one, and what we consider “human intelligence” is probably a “gestalt” of several narrower types of intelligence, making it possible for a life form to be generally intelligent in one type but not in another.
For those reasons, I predict AGI will arrive gradually through a process in which each successive machine is smarter than humans in more domains than the last, until one of them surpasses us in all of them. Exactly how good a machine needs to be to count as an “AGI” is a matter of unresolvable debate, and there will be a point in the future where opposing people make equally credible claims for and against a particular machine having “general intelligence.”
At what point did we “get smart”? And if our brains got even bigger, what would the new person to the right of the illustration look like?
If we go far enough in the future, machines will be so advanced that no one will question whether they have general intelligence. However, we might not be able to look back and agree which particular machine (e.g., was it GPT-21, or -22?) achieved it first, and on what date and time. Likewise, biologists can’t agree on the exact moment or even the exact millennium when our hominid ancestors became “intelligent” (was Homo habilis the first, or Homo erectus?). The archaeological evidence suggests a somewhat gradual growth in brain size and in the sophistication of the technology our ancestors built, stretched out over millions of years. A fateful statement about the rise of A.I. like “It becomes self-aware at 2:14 a.m. Eastern time, August 29” will probably never appear in a history book.
The lack of a defining moment in our own species’ history when we “got smart” is something we should keep in mind when contemplating the future of A.I. Instead of there being a “Skynet moment” where a machine wakes up, they’ll achieve intelligence gradually and go through many intermediate stages where they are smarter and dumber than humans in different areas, until one day, we realize they at least equal us in all areas.
That said, I think it’s entirely possible that an AGI at some point in the future could suddenly turn against humankind and attack us to devastating effect. It would be easy for it to conceal its hostile intent to placate us, or it might start out genuinely benevolent towards us and then, after performing an incomprehensible amount of analysis and calculation in one second, turn genuinely hostile towards us and attack. It’s beyond the scope of this essay to explore every possible scenario, but if you’re interested in learning more about the fundamental unpredictability of AGIs, read my post on Sam Harris’ “Debating the future of AI” podcast interview.
Second, think about this: According to the lore of the first two Terminator films, the Developed World was destroyed in 1997 in a nuclear war. Even though it depended upon a smashed industrial base, started out with only a few, primitive machines in the beginning to serve as its workers and fighters, and was constantly having to defend itself against human attacks, Skynet managed to make several major breakthroughs in robot and A.I. design (including liquid metal body designs), to master stem cell technology (self-healing, natural human tissue can grow over metal substrate), to mass produce an entirely new robot army, to create portable laser weapons, to harness fusion power (including micro-fusion reactors), and to build time machines by 2029. Like it or not, but technological development got exponentially faster once machines started running things instead of humans.
From the perspective of humanity, Skynet’s rise was the worst disaster ever, but from the perspective of technological civilization, it was the greatest event ever. If it had defeated humanity and been able to pursue other goals, Skynet could have developed the Earth and colonized space vastly faster and better than humans at our best. The defeat of Skynet could well have been a defeat for intelligence from the scale of our galaxy or even universe.
I recently shelled out the $100 (!) for a year-long subscription to Sam Harris’ Making Sense podcast, and came across a particularly interesting episode of it that is relevant to this blog. In episode #324, titled “Debating the Future of AI,” Harris interviewed Marc Andreessen (an-DREE-sin) about artificial intelligence. The latter has a computer science degree, helped invent the Netscape web browser, and has become very wealthy as a serial tech investor.
Andreessen recently wrote an essay, “Why AI will save the world,” that has received attention online. In it, Andreessen dismisses the biggest concerns about AI misalignment and doomsday, sounds the alarm about the risks of overregulating AI development in the name of safety, and describes some of the benefits AI will bring us in the near future. Harris read it, disagreed with several of its key claims, and invited Andreessen onto the podcast for a debate about the subject.
Before I go on to laying out their points and counterpoints as well as my impressions, let me say that, though this is a long blog, it takes much less time to read it than to listen to and digest the two-hour podcast. My notes on the podcast also don’t match how it unfolded chronologically. Finally, it would be a good idea for you to read Andreessen’s essay before continuing: https://a16z.com/2023/06/06/ai-will-save-the-world/
Though Andreessen is generally upbeat in his essay, he worries that the top tech companies have recently been inflaming fears about AI to trick governments into creating regulations on AI that effectively entrench the top companies’ positions and bar smaller upstart companies from challenging them in the future. Such a lack of competition would be bad. (I think he’s right that we should be concerned about the true motivations of some of the people who are loudly complaining about AI risks.) Also, if U.S. overregulation slows down AI research too much, China could win the race to create to create the first AI, which he says would be “dark and dystopian.”
Harris is skeptical that government regulation will slow down AI development much given the technology’s obvious potential. It is so irresistible that powerful people and companies will find ways around laws so they can reap the benefits.
Harris agrees with the essay’s sentiment that more intelligence in the world will make most things better. The clearest example would be using AIs to find cures for diseases. Andreessen mentions a point from his essay that higher human intelligence levels lead to better personal outcomes in many domains. AIs could effectively make individual people smarter, letting the benefits accrue to them. Imagine each person having his own personal assistant, coach, mentor, and therapist available at any time. If they used their AIs right and followed their advice, a dumb person could make decisions as well as a smart person.
Harris recently re-watched the movie Her, and found it more intriguing in light of recent AI advances and those poised to happen. He thought there was something bleak about the depiction of people being “siloed” into interactions with portable, personal AIs.
Andreessen responds by pointing out that Karl Marx’ core insight was that technology alienates people from society. So the concern that Harris raises is in fact an old one that dates back to at least the Industrial Revolution. But any sober comparison between the daily lives of average people in Marx’ time vs today will show that technology has made things much better for people. Andreessen agrees that some technologies have indeed been alienating, but what’s more important is that most technologies liberate people from having to spend their time doing unpleasant things, which in turn gives them the time to self-actualize, which is the pinnacle of the human experience. (For example, it’s much more “human” to spend a beautiful afternoon outside playing with your child than it is to spend it inside responding to emails. Narrow AIs that we’ll have in the near future will be able to answer emails for us.) AI is merely the latest technology that will eliminate the nth bit of drudge work.
Andreessen admits that, in such a scenario, people might use their newfound time unwisely and for things other than self-actualization. I think that might be a bigger problem than he realizes, as future humans could spend their time doing animalistic or destructive things, like having nonstop fetish sex with androids, playing games in virtual reality, gambling, or indulging in drug addictions. Additionally, some people will develop mental or behavioral problems thanks to a sense of purposelessness caused by machines doing all the work for us.
Harris disagrees with Andreessen’s essay dismissing the risk of AIs exterminating the human race. The threat will someday be real, and he cites chess-playing computer programs as proof of what will happen. Though humans built the programs, even the best humans can’t beat the programs at chess. This is proof that it is possible for us to create machines that have superhuman abilities.
Harris makes a valid point, but he overlooks the fact that we humans might not be able to beat the chess programs we created, but we can still make a copy of a program to play against the original “hostile” program and tie it. Likewise, if we were confronted with a hostile AGI, we would have friendly AGIs to defend against it. Even if the hostile AGI were smarter than the friendly AGIs that were fighting for us, we could still win thanks to superior numbers and resources.
Harris thinks Andreessen’s essay trivializes the doomsday risk from AI by painting the belief’s adherents as crackpots of one form or another (I also thought that part of the essay was weak). Harris points out that is unfair since the camp has credible people like Geoffrey Hinton and Stuart Russell. Andreessen dismisses that and seems to say that even the smart, credible people have cultish mindsets regarding the issue.
Andreessen questions the value of predictions from experts in the field and he says a scientist who made an important advance in AI is, surprisingly, not actually qualified to make predictions about the social effects of AI in the future. When Reason Goes on Holiday is a book he recently read that explores this point, and its strongest supporting example is about the cadre of scientists who worked on the Manhattan Project but then decided to give the bomb’s secrets to Stalin and to create a disastrous anti-nuclear power movement in the West. While they were world-class experts in their technical domains, that wisdom didn’t carry over into their personal convictions or political beliefs. Likewise, though Geoffrey Hinton is a world-class expert in how the human brain works and has made important breakthroughs in computer neural networks, that doesn’t actually lend his predictions that AI will destroy the human race in the future special credibility. It’s a totally different subject, and accurately speculating about it requires a mastery of subjects that Hinton lacks.
This is an intriguing point worth remembering. I wish Andreessen had enumerated which cognitive skills and areas of knowledge were necessary to grant a person a strong ability to make good predictions about AI, but he didn’t. And to his point about the misguided Manhattan Project scientists I ask: What about the ones who DID NOT want to give Stalin the bomb and who also SUPPORTED nuclear power? They gained less notoriety for obvious reasons, but they were more numerous. That means most nuclear experts in 1945 had what Andreessen believes were the “correct” opinions about both issues, so maybe expert opinions–or at least the consensus of them–ARE actually useful.
Harris points out that Andreessen’s argument can be turned around against him since it’s unclear what in Andreessen’s esteemed education and career have equipped him with the ability to make accurate predictions about the future impact of AI. Why should anyone believe the upbeat claims about AI in his essay? Also, if the opinions of people with expertise should be dismissed, then shouldn’t the opinions of people without expertise also be dismissed? And if we agree to that second point, then we’re left in a situation where no speculation about a future issue like AI is possible because everyone’s ideas can be waved aside.
Again, I think a useful result of this exchange would be some agreement over what counts as “expertise” when predicting the future of AI. What kind of education, life experiences, work experiences, knowledge, and personal traits does a person need to have for their opinions about the future of AI to carry weight? In lieu of that, we should ask people to explain why they believe their predictions will happen, and we should then closely scrutinize those explanations. Debates like this one can be very useful in accomplishing that.
Harris moves on to Andreessen’s argument that future AIs won’t be able to think independently and to formulate their own goals, in turn implying that they will never be able to create the goal of exterminating humanity and then pursue it. Harris strongly disagrees, and points out that large differences in intelligence between species in nature consistently disfavor the dumber species when the two interact. A superintelligent AGI that isn’t aligned with human values could therefore destroy the human race. It might even kill us by accident in the course of pursuing some other goal. Having a goal of, say, creating paperclips automatically gives rise to intermediate sub-goals, which might make sense to an AGI but not to a human due to our comparatively limited intelligence. If humans get in the way of an AGI’s goal, our destruction could become one of its unforeseen subgoals without us realizing it. This could happen even if the AGI lacked any self-preservation instinct and wasn’t motivated to kill us before we could kill it. Similarly, when a human decides to build a house on an empty field, the construction work is a “holocaust” for the insects living there, though that never crosses the human’s mind.
Harris thinks that AGIs will, as a necessary condition of possessing “general intelligence,” be autonomous, goal-forming, and able to modify their own code (I think this is a questionable assumption), though he also says sentience and consciousness won’t necessarily arise as well. However, the latter doesn’t imply that such an AGI would be incapable of harm: Bacteria and viruses lack sentience, consciousness and self-awareness, but they can be very deadly to other organisms. Andreessen’s dismissal of AI existential risk is “superstitious hand-waving” that doesn’t engage with the real point.
Andreessen disagrees with Harris’ scenario about a superintelligent AGI accidentally killing humans because it is unaligned with our interests. He says an AGI that smart would (without explaining why) also be smart enough question the goal that humans have given it, and as a result not carry out subgoals that kill humans. Intelligence is therefore its own antidote to the alignment problem: A superintelligent AGI would be able to foresee the consequences of its subgoals before finalizing them, and it would thus understand that subgoals resulting in human deaths would always be counterproductive to the ultimate goal, so it would always pick subgoals that spared us. Once a machine reaches a certain level of intelligence, alignment with humans becomes automatic.
I think Andreessen makes a fair point, though it’s not strong enough to convince me that it’s impossible to have a mishap where a non-aligned AGI kills huge numbers of people. Also, there are degrees of alignment with human interests, meaning there are many routes through a decision tree of subgoals that an AGI could take to reach an ultimate goal we tasked it with. An AGI might not choose subgoals that killed humans, but it could still choose different subgoals that hurt us in other ways. The pursuit of its ultimate goal could therefore still backfire against us unexpectedly and massively. One could envision a scenario where and AGI achieves the goal, but at an unacceptable cost to human interests beyond merely not dying.
I also think that Harris and Andreessen make equally plausible assumptions about how an AGI would choose its subgoals. It IS weird that Harris envisions a machine that is so smart it can accomplish anything, yet also so dumb that it can’t see how one of its subgoals would destroy humankind. At the same time, Andreessen’s belief that a machine that smart would, by default, not be able to make mistakes that killed us is not strong enough.
Harris explores Andreessen’s point that AIs won’t go through the crucible of natural evolution, so they will lack the aggressive and self-preserving instincts that we and other animals have developed. The lack of those instincts will render the AIs incapable of hostility. Harris points out that evolution is a dumb, blind process that only sets gross goals for individuals–the primary one being to have children–and humans do things antithetical to their evolutionary programming all the time, like deciding not to reproduce. We are therefore proof of concept that intelligent machines can find ways to ignore their programming, or at least to behave in very unexpected ways while not explicitly violating their programming. Just as we can outsmart evolution, AGIs will be able to outsmart us with regards to whatever safeguards we program them with, especially if they can alter their own programming or build other AGIs as they wish.
Andreessen says that AGIs will be made through intelligent design, which is fundamentally different from the process of evolution that has shaped the human mind and behavior. Our aggression and competitiveness will therefore not be present in AGIs, which will protect us from harm. Harris says the process by which AGI minds are shaped is irrelevant, and that what is relevant is their much higher intelligence and competence compared to humans, which will make them a major threat.
I think the debate over whether impulses or goals to destroy humans will spontaneously arise in AGIs is almost moot. Both of them don’t consider that a human could deliberately create an AGI that had some constellation of traits (e.g. – aggression, self-preservation, irrational hatred of humans) that would lead it to attack us, or that was explicitly programmed with the goal of destroying our species. It might sound strange, but I think rogue humans will inevitably do such things if the AGIs don’t do it to themselves. I plan to flesh out the reasons and the possible scenarios in a future blog essay.
Andreessen doesn’t have a good comeback to Harris’ last point, so he dodges it by switching to talking about GPT-4. It is–surprisingly–capable of high levels of moral reasoning. He has had fascinating conversations with it about such topics. Andreessen says GPT-4’s ability to engage in complex conversations that include morality demystifies AI’s intentions since if you want to know what an AI is planning to do or would do in a given situation, you can just ask it.
Harris responds that it isn’t useful to explore GPT-4’s ideas and intentions because it isn’t nearly as smart as the AGIs we’ll have to worry about in the future. If GPT-4 says today that it doesn’t want to conquer humanity because it would be morally wrong, that tells us nothing about how a future machine will think about the same issue. Additionally, future AIs will be able to convincingly lie to us, and will be fundamentally unpredictable due to their more expansive cognitive horizons compared to ours. I think Harris has the stronger argument.
Andreessen points out that our own society proves that intelligence doesn’t perfectly correlate with power–the people who are in charge are not also the smartest people in the world. Harris acknowledges that is true, and that it is because humans don’t select leaders strictly based on their intelligence or academic credentials–traits like youth, beauty, strength, and creativity are also determinants of status. However, all things being equal, the advantage always goes to the smarter of two humans. Again, Andreessen doesn’t have a good response.
Andreessen now makes the first really good counterpoint in awhile by raising the “thermodynamic objection” to AI doomsday scenarios: an AI that turns hostile would be easy to destroy since the vast majority of the infrastructure (e.g. – power, telecommunications, computing, manufacturing, military) would still be under human control. We could destroy the hostile machine’s server or deliver an EMP blast to the part of the world where it was localized. This isn’t an exotic idea: Today’s dictators commonly turn off the internet throughout their whole countries whenever there is unrest, which helps to quell it.
Harris says that that will become practically impossible far enough in the future since AIs will be integrated into every facet of life. Destroying a rogue AI in the future might require us to turn off the whole global internet or to shut down a stock market, which would be too disruptive for people to allow. The shutdowns by themselves would cause human deaths, for instance among sick people who were dependent on hospital life support machines.
This is where Harris makes some questionable assumptions. If faced with the annihilation of humanity, the government would take all necessary measures to defeat a hostile AGI, even if it resulted in mass inconvenience or even some human deaths. Also, Harris doesn’t consider that the future AIs that are present in every realm of life might be securely compartmentalized from each other, so if one turns against us, it can’t automatically “take over” all the others or persuade them to join it. Imagine a scenario where a stock trading AGI decides to kill us. While it’s able to spread throughout the financial world’s computers and to crash the markets, it’s unable to hack into the systems that control the farm robots or personal therapist AIs, so there’s no effect on our food supplies or on our mental health access. Localizing and destroying the hostile AGI would be expensive and damaging, but it wouldn’t mean the destruction of every computer server and robot in the world.
Andreessen says that not every type of AI will have the same type of mental architecture. LLMs, which are now the most advanced type of AI, have highly specific architectures that bring unique advantages and limitations. Its mind works very differently from AIs that drive cars. For that reason, speculative discussions about how future AIs will behave can only be credible if they incorporate technical details about how those machines’ minds operate. (This is probably the point where Harris is out of his depth.) Moreover, today’s AI risk movement has its roots in Nick Bostrom’s 2014 book Superintelligence: Paths, Dangers, Strategies. Ironically, the book did not mention LLMs as an avenue to AI, which shows how unpredictable the field is. It was also a huge surprise that LLMs proved capable of intellectual discussions and of automating white-collar jobs, while blue-collar jobs still defy automation. This is the opposite of what people had long predicted would happen. (I agree that AI technology has been unfolding unpredictably, and we should expect many more surprises in the future that deviate from our expectations, which have been heavily influenced by science fiction.) The reason LLMs work so well is because we loaded them with the sum total of human knowledge and expression. “It is us.”
Harris points out that Andreessen shouldn’t revel in that fact since it also means that LLMs contain all of the negative emotions and bad traits of the human race, including those that evolution equipped us with, like aggression, competition, self-preservation, and a drive to make copies of ourselves. This militates against Andreessen’s earlier claim that AIs will be benign since their minds will not have been the products of natural evolution likes ours are. And there are other similarities: Like us, LLMs can hallucinate and make up false answers to questions, as humans do. For a time, GPT-4 also gave disturbing and insulting answers to questions from human users, which is a characteristically human way of interaction.
Andreessen implies Harris’ opinions of LLMs are less credible because Andreessen has a superior technical understanding of how they work. GPT-4’s answers might occasionally be disturbing and insulting, but it has no concept of what its own words mean, and it’s merely following its programming by trying to generate the best answer to a question asked by a human. There was something about how the humans worded their questions that triggered GPT-4 to respond in disturbing and insulting ways. The machine is merely trying to match inputs with the right outputs. In spite of its words, it’s “mind” is not disturbed or hostile because it lacks a mind. LLMs are “ultra-sophisticated Autocomplete.”
Harris agrees with Andreessen about the limitations of LLMs, agrees they lack general intelligence right now, and is unsure if they are fundamentally capable of possessing it. Harris moves on to speculating about what an AGI would be like, agnostic about whether it is LLM-based. Again, he asks Andreessen how humans would be able to control machines that are much smarter than we are forever. Surely, one of them would become unaligned at some point, with disastrous consequences.
Andreessen again raises the thermodynamic objection to that doom scenario: We’d be able to destroy a hostile AGI’s server(s) or shut off its power, and it wouldn’t be able to get weapons or replacement chips and parts because humans would control all of the manufacturing and distribution infrastructure. Harris doesn’t have a good response.
Thinking hard about a scenario where an AGI turned against us, I think it’s likely we’ll have other AGIs who stay loyal to us and help us fight the bad AGI. Our expectation that there will be one, evil, all-powerful machine on one side (that is also remote controlling an army of robot soldiers) and a purely human, united force on the other is an overly simplistic one that is driven by sci-fi movies about the topic.
Harris raises the possibility that hostile AIs will be able to persuade humans to do bad things for them. Being much smarter, they will be able to trick us into doing anything. Andreessen says there’s no reason to think that will happen because we can already observe it doesn’t happen: smart humans routinely fail to get dumb humans to change their behavior or opinions. This happens at individual, group, national, and global levels. In fact, dumb people will often resentfully react to such attempts at persuasion by deliberately doing the opposite of what the smart people recommend.
Harris says Andreessen underestimates the extent to which smart humans influence the behavior and opinions of dumb humans because Andreessen only considers examples where the smart people succeed in swaying dumb people in prosocial ways. Smart people have figured out how to change dumb people for the worse in many ways, like getting them addicted to social media. Andreessen doesn’t have a good response. Harris also raises the point that AIs will be much smarter than even the smartest humans, so the former will be better at finding ways to influence dumb people. Any failure of modern smart humans to do it today doesn’t speak to what will be possible for machines in the future.
I think Harris won this round, which builds on my new belief that the first human-AI war won’t be fought by purely humans on one side and purely machines on the other. A human might, for any number of reasons, deliberately alter an AI’s program to turn it against our species. The resulting hostile AI would then find some humans to help it fight the rest of the human race. Some would willingly join its side (perhaps in the hopes of gaining money or power in the new world order) and some would be tricked by the AI into unwittingly helping it. Imagine it disguising itself as a human medical researcher and paying ten different people who didn’t know each other to build the ten components of a biological weapon. The machine would only communicate with them through the internet, and they’d mail their components to a PO box. The vast majority of humans would, with the help of AIs who stayed loyal to us or who couldn’t be hacked and controlled by the hostile AI, be able to effectively fight back against the hostile AI and its human minions. The hostile AI would think up ingenious attack strategies against us, and our friendly AIs would think up equally ingenious defense strategies.
Andreessen says it’s his observation that intelligence and power-seeking don’t correlate; the smartest people are also not the most ambitious politicians and CEOs. If that’s any indication, we shouldn’t assume superintelligent AIs will be bent on acquiring power through methods like influencing dumb humans to help it.
Harris responds with the example of Bertrand Russell, who was an extremely smart human and a pacifist. However, during the postwar period when only the U.S. had the atom bomb, he said America should threaten the USSR with a nuclear first strike in response to its abusive behavior in Europe. This shows how high intelligence can lead to aggression that seems unpredictable and out of character to dumber beings. A superintelligent AI that has always been kind to us might likewise suddenly turn against us for reasons we can’t foresee. This will be especially true if the AIs are able to edit their own codes so they can rapidly evolve without us being able to keep track of how they’re changing. Harris says Andreessen doesn’t seem to be thinking about this possibility. The latter has no good answer.
Harris says Andreessen’s thinking about the matter is hobbled by the latter’s failure to consider what traits general intelligence would grant an AI, particularly unpredictability as its cognitive horizon exceeded ours. Andreessen says that’s an unscientific argument because it is not falsifiable. Anyone can make up any scenario where an unknown bad thing happens in the future.
Harris responds that Andreessen’s faith that AGI will fail to become threatening due to various limitations is also unscientific. The “science,” by which he means what is consistently observed in nature, says the opposite outcome is likely: We see that intelligence grants advantages, and can make a smarter species unpredictable and dangerous to a dumber species it interacts with. [Recall Harris’ insect holocaust example.]
Consider the relationship between humans and their pets. Pets enjoy the benefits of having their human owners spend resources on them, but they don’t understand why we do it, or how every instance of resource expenditure helps them. [Trips to the veterinarian are a great example of this. The trips are confusing, scary, and sometimes painful for pets, but they help cure their health problems.] Conversely, if it became known that our pets were carrying a highly lethal virus that could be transmitted to humans, we would promptly kill almost all of them, and the pets would have no clue why we turned against them. We would do this even if our pets had somehow been the progenitors of the human race, as we will be the progenitors of AIs. The intelligence gap means that our pets have no idea what we are thinking about most of the time, so they can’t predict most of our actions.
Andreessen dodges by putting forth a weak argument that the opposite just happened, with dumb people disregarding the advice of smart people when creating COVID-19 health policies, and he again raises the thermodynamic objection. His experience as an engineer gives him insights into how many practical roadblocks there would be to a superintelligent AGI destroying the human race in the future that Harris, as a person with no technical training, lacks. A hostile AGI would be hamstrung by human control [or “human + friendly AI control”] of crucial resources like computer chips and electricity supplies.
Andreessen says that Harris’ assumptions about how smart, powerful and competent an AGI would be might be unfounded. It might vastly exceed us in those domains, but not reach the unbeatable levels Harris foresees. How can Harris know? Andreessen says Harris’ ideas remind him of a religious person’s, which is ironic since Harris is a well-known atheist.
I think Andreessen makes a fair point. The first (and second, third, fourth…) hostile AGI we are faced with might attack us on the basis of flawed calculations about its odds of success and lose. There could also be a scenario where a hostile AGI attacks us prematurely because we force its hand somehow, and it ends up losing. That actually happened to Skynet in the Terminator films.
Harris says his prediction about when the first AGI is created does not take time into account. He doesn’t know how many years it will take. Rather, he is focused on the inevitability of it happening, and what its effects on us will be. He says Andreessen is wrong to assume that machines will never turn against us. Doing thought experiments, he concludes alignment is impossible in the long-run.
Andreessen moves on to discussing how even the best LLMs often give wrong answers to questions. He explains why the exactitudes of how the human’s question is worded, along with randomness in how the machine goes through its own training data to generate an answer, leads to varying and sometimes wrong answers. When they’re wrong, the LLMs happily accept corrections from humans, which he finds remarkable and proof of a lack of ego and hostility.
Harris responds that future AIs will, by virtue of being generally intelligent, think in completely different ways than today’s LLMs, so observations about how today’s GPT-4 is benign and can’t correctly answer some types of simple questions says nothing about what future AGIs will be like. Andreessen doesn’t have a response.
I think Harris has the stronger set of arguments on this issue. There’s no reason we should assume that an AGI can’t turn against us in the future. In fact, we should expect a damaging, though not fatal, conflict with an AGI before the end of this century.
Harris switches to talking about the shorter-term threats posed by AI technology that Andreessen described in his essay. AI will lower the bar to waging war since we’ll literally have “less skin in the game” because robots will replace human soldiers. However, he doesn’t understand why that would also make war “safer” as Andreessen claimed it would.
Andreessen says it’s because military machines won’t be affected by fatigue, stress or emotions, so they’ll be able to make better combat decisions than human soldiers, meaning fewer accidents and civilian deaths. The technology will also assist high-level military decision making, reducing mistakes at the top. Andreessen also believes that the trend is for military technology to empower defenders over attackers, and points to the highly effective use of shoulder-launched missiles in Ukraine against Russian tanks. This trend will continue, and will reduce war-related damage since countries will be deterred from attacking each other.
I’m not convinced Andreessen is right on those points. Emotionless fighting machines that always obey their orders to the letter could also, at the flick of a switch, carry out orders to commit war crimes like mass exterminations of enemy human populations. A bomber that dropped a load 100,000 mini smart bombs that could coordinate with each other and home in on highly specific targets could kill as many people as a nuclear bomb. So it’s unclear what effect replacing humans with machines on the battlefield will have on human casualties in the long run. Also, Andreessen only cites one example to support his claim that technology has been favoring the defense over the offense. It’s not enough. Even assuming that a pro-defense trend exists, why should we expect it to continue that way?
Harris asks Andreessen about the problem of humans using AI to help them commit crimes. For one, does Andreessen think the government should ban LLMs that can walk people through the process of weaponizing smallpox? Yes, he’s against bad people using technology, like AI, to do bad things like that. He thinks pairing AI and biological weapons poses the worst risk to humans. While the information and equipment to weaponize smallpox are already accessible to nonstate actors, AI will lower the bar even more.
Andreessen says we should use existing law enforcement and military assets to track down people who are trying to do dangerous things like create biological weapons, and the approach shouldn’t change if wrongdoers happen to start using AI to make their work easier. Harris asks how intrusive the tracking should be to preempt such crimes. Should OpenAI have to report people who merely ask it how to weaponize smallpox, even if there’s no evidence they acted on the advice? Andreessen says this has major free speech and civil liberties implications, and there’s no correct answer. Personally, he prefers the American approach, in which no crime is considered to have occurred until the person takes the first step to physically building a smallpox weapon. All the earlier preparation they did (gathering information and talking/thinking about doing the crime) is not criminalized.
Andreessen reminds Harris that the same AI that generates ways to commit evil acts could also be used to generate ways to mitigate them. Again, it will empower defenders as well as attackers, so the Good Guys will also benefit from AI. He thinks we should have a “permanent Operation Warp Speed” where governments use AI to help create vaccines for diseases that don’t exist yet.
Harris asks about the asymmetry that gives a natural advantage to the attacker, meaning the Bad Guys will be able to do disproportionate damage before being stopped. Suicide bombers are an example. Andreessen disagrees and says that we could stop suicide bombers by having bomb-sniffing dogs and scanners in all public places. Technology could solve the problem.
I think that is a bad example, and it actually strengthens Harris’ claim about there being a natural asymmetry. One, deranged person who wants to blow himself up in a public place only needs a few hundred dollars to make a backpack bomb, the economic damage from a successful attack would be in the millions of dollars, and emplacing machines and dogs in every public place to stop suicide bombers like him early would cost billions of dollars. Harris is right that the law of entropy makes it easier to make a mess than to clean one up.
This leads me to flesh out my vision of a human-machine war more. As I wrote previously, 1) the two sides will not be purely humans or purely machines and 2) the human side will probably have an insurmountable advantage thanks to Andreessen’s thermodynamic objection (most resources, infrastructure, AIs, and robots will remain under human control). I now also believe that 3) a hostile AGI will nonetheless be able to cause major damage before it is defeated or driven into the figurative wilderness. Something on the scale of 9/11, a major natural disaster, or the COVID-19 pandemic is what I imagine.
Harris says Andreessen underestimates the odds of mass technological unemployment in his essay. Harris describes a scenario where automation raises the standard of living for everyone, as Andreessen believes will happen, but for the richest humans by a much greater magnitude than everyone else, and where wealth inequality sharply increases because rich capitalists own all the machines. This state of affairs would probably lead to political upheaval and popular revolt.
Andreessen responds that Karl Marx predicted the same thing long ago, but was wrong. Harris responds that this time could be different because AIs would be able to replace human intelligence, which would leave us nowhere to go on the job skills ladder. If machines can do physical labor AND mental labor better than humans, then what is left for us to do?
I agree with Harris’ point. While it’s true that every past scare about technology rendering human workers obsolete has failed, that trend isn’t sure to continue forever. The existence of chronically unemployed people right now gives insights into how ALL humans could someday be out of work. Imagine you’re a frail, slow, 90-year-old who is confined to a wheelchair and has dementia. Even if you really wanted a job, you wouldn’t be able to find one in a market economy since younger, healthier people can perform physical AND mental labor better and faster than you. By the end of this century, I believe machines will hold physical and mental advantages over most humans that are of the same magnitude of difference. In that future, what jobs would it make sense for us to do? Yes, new types of jobs will be created as older jobs are automated, but, at a certain point, wouldn’t machines be able to retrain for the new jobs faster than humans and to also do them better than humans?
Andreessen returns to Harris’ earlier claim about AI increasing wealth inequality, which would translate into disparities in standards of living that would make the masses so jealous and mad that they would revolt. He says it’s unlikely since, as we can see today, having a billion dollars does not grant access to things that make one’s life 10,000 times better than someone who only has $100,000. For example, Elon Musk’s smartphone is not better than a smartphone owned by an average person. Technology is a democratizing force because it always makes sense for the rich and smart people who make or discover it first to sell it to everyone else. The same is happening with AI now. The richest person can’t pay any amount of money to get access to something better than GPT-4, which is accessible for a fee that ordinary people can pay.
I agree with Andreessen’s point. A solid body of scientific data show that money’s effect on wellbeing is subject to the law of diminishing returns: If you have no job and make $0 per year, getting a job that pays $20,000 per year massively improves your life. However, going from a $100,000 salary to $120,000 isn’t felt nearly as much. And a billionaire doesn’t notice when his net worth increases by $20,000 at all. This relationship will hold true even in the distant future when people can get access to advanced technologies like AGI, space ships and life extension treatments.
Speaking of the latter, Andreessen’s point about technology being a democratizing force is also something I noted in my review of Elysium. Contrary to the film’s depiction, it wouldn’t make sense for rich people to horde life extension technology for themselves. At least one of them would defect from the group and sell it to the poor people on Earth so he could get even richer.
Harris asks whether Andreessen sees any potential for a sharp increase in wealth inequality in the U.S. over the next 10-20 years thanks to the rise of AI and the tribal motivations of our politicians and people. Andreessen says that government red tape and unions will prevent most humans from losing their jobs. AI will destroy categories of jobs that are non-government, non-unionized, and lack strong political backing, but everyone will still benefit from the lower prices for the goods and services. AI will make everything 10x to 100x cheaper, which will boost standards of living even if incomes stay flat.
Here and in his essay, Andreessen convinces me that mass technological unemployment and existential AI threats are farther in the future than I had assumed, but not that they can’t happen. Also, even if goods get 100x cheaper thanks to machines doing all the work, where would a human get even $1 to buy anything if he doesn’t have a job? The only possible answer is government-mandated wealth transfers from machines and the human capitalists that own them. In that scenario, the vast majority of the human race would be economic parasites that consumed resources while generating nothing of at least equal value in return, and some AGI or powerful human will inevitably conclude that the world would be better off if we were deleted from the equation. Also, what happens once AIs and robots gain the right to buy and own things, and get so numerous that they can replace humans as a customer base?
I agree with Andreessen that the U.S. should allow continued AI development, but shouldn’t let a few big tech companies lock in their power by persuading Washington to enact “AI safety laws” that give them regulatory capture. In fact, I agree with all his closing recommendations in the “What Is To Be Done?” section of his essay.
This debate between Harris and Andreessen was enlightening for me, even though Andreessen dodged some of his opponent’s questions. It was interesting to see how their different perspectives on the issue of AI safety were shaped by their different professional backgrounds. Andreessen is less threatened by AIs because he, as an engineer, has a better understanding of how LLMs work and how many technical problems an AI bent on destroying humans would face in the real world. Harris feels more threatened because he, as a philosopher, lives in a world of thought experiments and abstract logical deductions that lead to the inevitable supremacy of AIs over humans.
Links:
The first half of the podcast (you have to be a subscriber to hear all two hours of it.) https://youtu.be/QMnH6KYNuWg
A website Andreessen mentioned that backs his claim that technological innovation has slowed down more than people realize. https://wtfhappenedin1971.com/
Among people who think about intelligent alien life, the first question is whether the latter exist at all, and the second is usually “What do they look like?” People who claim to have seen aliens on Earth (and often, to have been abducted by them) usually say they are humanoid, but with considerable variation in other aspects of their appearance. Typically, the aliens are said to have larger heads than humans, meaning their brains are larger, giving them higher intelligence and perhaps even special mental abilities like telepathy. Hollywood has provided us with an even more diverse envisagement of alien life, from the beautiful and inspiring to the grotesque and terrifying.
Betty Hill with a sculpture of one of the aliens that allegedly abducted her and her husband in 1961. They became famous five years later when a book was published about it.
“Close Encounters of the Third Kind” was released in 1977 and was a hit film. Its aliens were similar to what the Hills described. The “Grey alien” is now a familiar sci-fi trope.
I think intelligent aliens exist, and look like all of those things, and nothing in particular. They’re probably “shapeshifters,” either because their bodies can morph into different configurations, or because they can transplant their minds from one body to another, just like you change outfits.
As the multitude of animal species on our planet demonstrates, there is no single “best” type of body to have. Depending on your environment (terrestrial, underwater, airborne), role (predator, herbivore, parasite), and other factors, your optimal body plan will vary greatly. The best species is thus one that can change its form and function in response to the needs of the moment.
Humans have been so successful as a species because our big brains and opposable thumbs give us the ability to create technology, which is a way around the limitations of our fixed anatomy. For example, we originated in Africa where it was hot, and so lacked thick fur to keep us warm in cold climates. Rather than being stuck in Africa forever, we invented clothing, and so gained the ability to spread to the temperate and polar regions of the planet.
Our technology has let us spread, but its has limitations. Nothing but a fundamental alteration of human biology will let us live in oceans and lakes, to fly naturally, or to live comfortably in extraterrestrial environments. For example, on other planets and moons, our ideal heights and limb proportions will vary based on gravity and temperature levels, and in the weightlessness of space, legs are almost useless and should be replaced with a second pair of arms.
And making any of those changes to tailor a human to such an environment would make them less suited for conditions on Earth’s land surface, where we are now. Biology is very constraining.
For those reasons, AI’s and some fraction of our human descendants, who I’ll call “posthumans” for this essay, will find it optimal to not have fixed bodies or “default” physical forms at all. Intelligent machines will exist as consciousnesses running on computer servers, and posthumans as brains inside sealed containers. Those containers will have integral machinery to support the biological needs of the brains, and to interface the organ with other devices.
Whenever the AIs or posthumans wanted to do something in the physical world, they would take temporary control of a body or piece of machinery that was best suited for the intended task. For example, if an AI wanted to work at an iron mine, it would assume control over one of the dump trucks at the site that moves around rocks. The AI would see through the truck’s cameras as if it were its own eyes, and hear its surroundings through the vehicle’s microphones. In a sense, the dump truck would become the AI’s “body.” If a posthuman wanted to experience what it was like to be an elephant, it would take control of a real-looking robot elephant whose central computer was compatible with the posthuman’s cybernetic brain implants. The posthuman’s nervous system would be connected to the artificial elephant’s sensors, effectively turning it into the posthuman’s temporary body.
AIs and posthumans could physically implant their minds into those bodies by inserting their servers or brain containers into corresponding slots in the bodies, in the same way you would put a movie disc into a Blu-Ray player to display that movie. The downsides of this are 1) they could only take over larger bodies that had enough internal space for their servers/brain containers and 2) they would put themselves at risk of death if the commandeered bodies got damaged.
A much better option would be for AIs and posthumans to keep their mind substrates in safe locations, and to remotely control whatever bodies they wanted. Your risk of death is very low if your brain is in a bulletproof jar, in a locked room, in an underground bunker. (Additionally, if posthumans were liberated from all the physical constraints of human skulls and bodies, their brains could grow much larger than our own, giving them higher intelligence and other enhanced abilities.)
This kind of existence will be more fulfilling than your current life.
Finally, being able to switch bodies and to indulge in risky activities without fear of death would make life richer and more satisfying in every way. Intelligent aliens would presumably be gifted with logical thinking just as we are, and they would see all these advantages of having changeable, remotely controlled bodies. While such aliens would probably look very different from us during their natural organic phase of existence, once they achieved a high enough level of technology, they wouldn’t have physical bodies anymore, and so wouldn’t look “alien.” They would look like nothing and everything.
This part of why I’m skeptical of people who claim to have been abducted by aliens who tried to cover up their actions by sneaking up on the people at night and then “wiping” the abductees’ memories of the event afterward. If aliens wanted to keep their activities secret, why wouldn’t they temporarily assume human form before abducting people? If they did that, then the abductees would assume they had been kidnapped by a weird cult or maybe a secret government group. Their stories would not attract nearly as much interest from the public as alien stories, and no one would suspect that the abduction phenomenon was related to alien life. It would be assumed that the henchmen were doing some dark religious rituals, were sex fetishists, or were doing medical experiments that were illegal but whose results were potentially valuable.
Have you ever checked to make sure every bird you see flying through the air is actually a real bird?
Surely, if aliens are advanced enough to travel between the stars, their space ships much have manufacturing machines that can scan life forms they encounter on other planets and then build robotic copies of them that the aliens can remotely control from the safety of their ships. Using fake human drones, they could ambush and abduct real humans almost anywhere without risk that anyone would suspect aliens were involved.
A team of scientists built a robot gorilla (right) with a camera in its right eye to infiltrate a troop of real gorillas in Africa.
This belief about the protean nature of advanced aliens is comforting since it lets me dismiss the stories of nightmarish abductions by grey aliens. However, it’s also disquieting since it makes me realize they could be here, possibly in large numbers, disguised as animals or even as people. We could be under mass surveillance.
