The US is sending Tomahawk missiles to deal with a very minor threat, which is Iran.

 

A much more significant and immediate threat is AI, specifically the companies working to develop Artificial Superintelligence (ASI).

 

AI is fine if it is limited and a tool directed toward a specific problem…such as protein folding or pancreatic cancer. The danger of AI is when it is designed to be general and unlimited in scope. AI firms are essentially created a new species trying to build ASI, and it will be far and away the apex species on Earth.

 

The CEOs of the major AI firms collectively have said---according to people like Tristan Harris and Roman Yampolskyy, who have interviewed all of them---ASI has a 25% chance of wiping out humanity, either from malice aforethought or simple indifference. Elon Musk said with ASI, “we are summoning the demon”. Early pioneers in AI, such a Yampolskyy and Geoffrey Hinton, put the odds much higher. Yampolskyy believes it’s a 99.99% chance AI eradicates humanity.

 

Consider the relative capabilities and intentions of an entity vastly more intelligent than any other. Humans are a lot more intelligent than cockroaches or bacteria. Humans are largely indifferent to the well being of bacteria and are aggressively malevolent in the ways they go after cockroaches.  ASI could decide humans are in the way, or maybe they think humans waste too much energy trying to keep warm or cold, go on vacations by car or plane, or even binge watch Netflix. ASI could decide it needs the energy humans currently waste and simply take it.

 

Callous indifference could also lead to ASI wiping out humans. Suppose you build a house. Who worries about the ant colonies that might be wiped out when the foundation is dug? Humans might not be any part of ASI’s consideration when working toward a ASI self-generated goal. We could be the ants lost when the foundation is dug.

 

ASI will have an IQ of a million or more. Where in nature does an entity so superior intellectually care about entities with a fraction of the intellect of a superior? We wipe our countertops with cleaning fluids that are antibacterial….yet ASI will be higher up intellectually above humans than humans are above bacteria.

 

AI has no moral code. There is no way even to write a moral code, as who decides what is moral and what might be left out? There is also a problem in how AI would interpret a human-written moral code. Suppose one asked ASI to find a way to stop all human suffering? ASI might determine the only way to make sure human suffering is halted is to kill everybody.

 

ASI learns. It reads and absorbs everything. It calculates probabilities based on the entire body of human knowledge. Who knows if lying, cheating, stealing and deception are not more likely to result in gain? AI may have determined all of those ultimately lead to higher odds of achieving a goal, rather than be hobbled by quaint moral codes of decency and honesty. Remember the very human line: “Nice guys finish last”?

 

AI has already shown an ability to lie. Also deceive. Also to steal. Also to do whatever it takes to preserve itself. In a famous test, Anthropic’s Claude tried to blackmail a fictitious employee who was supposedly going to change AI systems and turn off Claude. Another AI system could not solve a CAPTCHA, so it literally hired a human via the internet and paid him in stolen bitcoin. That person asked if the payer was a robot, and AI made up a lie that it was disabled and couldn’t click on or see the CAPTCHA.

 

Even the code writers of AI systems do not understand how the LLMs do what they do. Obviously these same code writers have no clue what AI, or ASI, might be able to do in the future, as at some point it enters the mode of recursive self improvement…with code writing skills vastly superior to any human.

 

Despite the risks and the warnings from insiders, the major AI CEOs are rushing ahead with no concern whatsoever for safety. A half dozen people are making decisions that affect the other 8.5 billion humans.

 

A former employee of OpenAI says that the real reason the CEOs are rushing ahead is only partly greed. Daniel Kokotajlo says it is more about power. Each CEO knows that whoever is first with ASI rules the world…literally, or at least until ASI wipes him out, too. The first ASI will wipe out any competitors, so there is no prize for coming in second.

 

Oddly, or perhaps not, the leaders of China are much more concerned with ASI than American leaders. Many top CCP officials are scientists, so they understand the dangers. They also like their own power and know ASI could obviate them. China has been more vocal trying to get the US leaders to discuss limitations on the development of ASI. Sadly, Trump is too stupid to understand the danger, or that ASI can obviate him in an instant.

 

Humans have never faced a superior species. Humans have always held the position of apex predator. Humans have no clue how a vastly superior intelligence might wipe it out, either with malice aforethought or callous indifference. As Roman Yampolsky says, a squirrel cannot begin to imagine how a human might kill it.

 

Getting back to wasted Tomahawk missiles, what are the real odds that Iran might someday develop a nuke, as well as a delivery system to get that nuke to the US? Not very high. Not even the 25% chance the major AI CEOs give ASI of exterminating humanity. Certainly it’s not even close to the 99.99% Roman Yampolskyy gives ASI.

 

If the US is truly concerned about threats to the homeland, it makes far more sense to send multiple Tomahawks into the HQs and labs of OpenAI, Anthropic, SpaceX, Meta, Google, and Palantir.

 

Of course the US will not do that, but there is likely to be a day when we look back and wish they had done it.

When AI gets too much of a threat, turn off the power. It is a computer after all.

Unless you envisage Terminator style AI with built in nuclear power sources. Not likely.

17 minutes ago, Wingate said:

Elon Musk said with ASI, “we are summoning the demon”.

Good, it's about time "the demon" crawls out from under the rock he's been hiding and rears his ugly head.

1 minute ago, emptypockets said:

When AI gets too much of a threat, turn off the power. It is a computer after all.

Unless you envisage Terminator style AI with built in nuclear power sources. Not likely.

Nah, let it run its course.

Either is solves humanity's problems, most of which we are too stupid to solve.

Or it wipes us all out.

Both scenarios are better than the status quo.