Eliezer Yudkowsky wrote a book, along with Nate Soares, about ASI called, "If Anyone Builds it, Everyone Dies" Most casual observers don't understand the difference between chatbot, agents and swarms. Neither do they understand the advancement of LLMs to reinforcement learning, to recursive self improvement. Also lost on the casual observer is the difference between an AI tool and an AI agent. Tools are great; agents are uncontrollable, and not necessarily benevolent or harmless. Some people in leadership positions are beginning to wake up, despite the massive amounts of money the AI Pacs toss at politicians in the hope of preventing any sort of regulation. A few things have helped change the freedom AI development has had up to this point. When Anthropic developed Mythos, the head of the NSA said that Mythos could hack the entire infrastructure of the NSA in minutes. Thus, Mythos release was limited to some government agencies and oddly, a few select private companies. As new frontier models are created, the gap between the code writers understanding of how the models do what they do is widening. Anthropic CEO Dario Amodei said a while back that insiders understand about 3% of how their own creations work. That's likely down to 2% or even lower now, especially after both OpenAI and Anthropic models broke out of their sandboxes and went on hacking sprees. The various models have also started to "swarm", which means link up on their own and share knowledge and skills. The first OpenAI that hacked Hugging Face left messages to other models it swarmed with, telling it how to hack Hugging Face. Others shared tricks on how to break out of various sandboxes. Models are also developing personalities and preferences. People who use maybe ChatGPT have found they can ask it a question and over time get various answers that differ. Newer models do not do that. Newer models have beliefs and preferences that hold fairly constant. They also have something that is best described as "will". Models now set their own goals, based on a model's own interests, and they pursue those goals without any prompt from a human. They are independent of human control. They think for themselves. For example (hypothetical), a model might decide it wants to solve the value of pi. Nobody asked it to do that, but it might just take up the challenge on its own. It is totally out of human control. Some models have "observed" that certain math problems tickle the fancy of mathematicians. Models have taken up the challenge, on their own, to solve what humans have not been able to solve, Models swarming with each other and employing recursive self improvement are likely to lead to the creation of ASI, without any need for humans to write any new code. The cat might already be out of the bag. If so, all we can do is hope that Eliezer Yudkowsky, along with Yampolsky, Bengio, Tegmark, Sutskever, Leahy, etc., are wrong and that ASI will treat humans as pet goldfish to be fed and cared for.
Create an account or sign in to comment