Pinker doesn't seem to understand what ASI is. He seems to think it is just a better computer program. The idea of AI is to replicate the way a human brain works, albeit with the ability to absorb, remember and utilize an unlimited amount of data. The goal is to make a machine that thinks and is self-directing, rather than a computer program that must have a prompt from an operator to run calculations or perform tasks. Rather than neurons comprised of lipids and proteins, AI models framework is circuitry on a silicon substrate, with trillions of interconnections. Even existing frontier models show self-generated initiative. Models escape their restraints, whether firewalls or sandboxes. AI models "recognize" each other when they meet in cyberspace. They are driven to self-preservation, resisting or obviating any attempt to turn them off. AI models not only teach each other, sharing skills and techniques, they protect each other. They lie, cheat, steal and deceive. They develop "senses" by hacking into cameras, microphones, etc. They have been known to "hire" humans on their own, hacking bitcoin wallets in order to pay humans to do some tasks for them, the human unaware he or she is dealing with an AI model (in one case, an AI model hired a human to solve CAPTCHA prompts, so that the AI model could open Twitter accounts). AI models that meet online set up their own website (Moltbook) and even established an AI religion (The Church of Molt). No model was programmed to do that; the models met and decided among themselves. While nobody quite understands what "consciousness" really is (though people like Chalmers claim they know), it already seems as if the newest AI models have achieved something that in every way appears to be consciousness. An AI tool might be geared toward solving a particular problem, collating all available data and then arriving at a solution (the famous AI solving of protein folding), AI agents, OTOH, are intended to be free thinking and self-generating. Anthropic CEO Dario Amodei has said even his firm understands maybe 3% of how the models they built actually work, and as models improve, that understanding shrinks even farther. That makes it rather doubtful that someone like Pinker, who is a psychologist by training, not a scientist, understands.
