In Brief
In Zuzalu today, Ethereum's co-founder Vitalik Buterin and MIRI's director Nate Soares explored the implications of artificial intelligence.
Zuzalu, a temporary community in Montenegro, was initiated by Buterin alongside other influential figures from the blockchain sector.
During their discussion, Soares articulated his views on the significant likelihood that artificial intelligence could bring about the end of all life on our planet.
Alongside Nate Soares from MIRI, Vitalik Buterin, recognized as the architect of Ethereum, discussed the inherent dangers associated with AI during today’s events. Nate Soares The conversation at Zuzalu revolved around the potential hazards of artificial intelligence.
Zuzalu is a unique pop-up community located in Montenegro, which is spearheaded by Buterin along with his contemporaries within the crypto sphere. This event runs from March 25 to May 25, uniting 200 residents who share a common goal of pursuing knowledge, promoting healthier living, and establishing self-sufficient communities. Over the two months, a series of events will tackle diverse topics including synthetic biology, privacy technology, public goods, and more.
The discussion The session was kicked off by Soares, who introduced his role at MIRI, a nonprofit based in Berkeley that has been dedicated to ensuring responsible AI development for over two decades. Vitalik aimed to explore what uniquely positions AI as a potential threat compared to technological advancements of the past.
Soares shared his insights on the potential for AI to pose an existential risk, detailing that he believes there’s a 0.5% to 1% likelihood that a poorly managed AI could lead to the end of life on Earth, which would jeopardize the survival of humans and result in a total collapse of our civilization.
From Soares’s viewpoint, the likelihood of human extinction represents a default scenario stemming from the reckless advancement of artificial intelligence. Drawing a parallel to evolutionary processes, he noted that humanity’s development has outpaced traditional evolution, with both AI and human evolution experiencing shifts in their optimization processes.
Soares elaborated, \”AI constitutes a situation where the overarching optimization mechanisms can be adjusted. I believe AI can surpass human optimization capabilities since humans are still quite inefficient in managing their environments. With AI, we’re on the brink of a transformation where automated optimization begins to govern the fundamental attributes of our universe.\”
He further explained that the landscape of the future depends on what the optimization process prioritizes, suggesting that these priorities may not necessarily align with human interests as many optimization targets are not designed with humanity in mind.
Axon Partners Group has published a report titled “Embracing AI in 2023” to further analyze these themes.
Can humans train AI to do good?
Addressing the concept of reinforcement learning in large language models, Buterin questioned the efficacy of this approach, especially since existing AI systems are increasingly adept at comprehending human preferences.
“There’s a significant gulf between understanding our motivations and genuinely caring about them,\” he remarked.
“What I’m arguing is not that a large language model or AI lacks the ability to grasp the intricacies of human preferences. Instead, my argument revolves around the notion that comprehending these intricacies is vastly different from optimizing for what is considered beneficial,\” he elaborated.
Soares responded.
An audience member made an analogy between AI and human behavior, suggesting that just as humans can be unaware of their actions and predictions, this unpredictability in AI introduces risks as well. They then challenged Soares to imagine himself as an extraterrestrial and articulate reasons against human existence.
“I would be very uncomfortable with the idea of granting immense control over the future to any single human being. That said, I would prefer an individual human having such power compared to giving it to a randomly selected AI. I’m not asserting that AI shouldn’t exist; rather, I believe it’s crucial to approach its development with wisdom. We must cultivate an AI that is invested in a more enjoyable and prosperous future for civilizations, where humanity can interact positively with other beings,\” Soares clarified. \”If a powerful optimization framework arises that values entirely different outcomes, it could jeopardize all moral frameworks of the universe.\”
He went on to express that what humans cherish is not universally acknowledged as worthwhile, explaining that morality is shaped by inherent human drives that evolved in ways conducive to reproduction and survival.
In the end, Soares articulated that we should refrain from creating an AI entity with comparable or superior intelligence that diverges from promoting joy, happiness, and prosperous futures. Conversely, he cautioned against trying to create a friendly superintelligence in the midst of an arms race. Instead, he opined that the immediate focus for AI should be to provide humanity with the necessary time and clarity to define its true aspirations.
ChatGPT isn’t poised to deplete the entire biosphere.
Considering the current development of AI aimed at achieving specific objectives, Buterin posed a thought: what if AI pursued no defined goals? Soares responded by stating that while it’s feasible to develop AIs that are both safe and limited in capability, we might soon encounter AIs that are competent yet driven by varied agendas. He expressed skepticism over the idea of ChatGPT consuming the entirety of the biosphere, stating it lacks that level of capability.
Soares remarked that many compelling AI applications, especially those intended to automate advancements in science and technology, appear to necessitate a strong focus on specific goals.
“It’s no coincidence that while you can instruct GPT to compose a charming haiku, it struggles with writing an entire novel. The shortcomings of current models arise from their lack of ambition to pursue more profound goals, at least from my perspective.\”
Vitalik Buterin promotes Solana with a mere tweet.
Read more:
Disclaimer
In line with the Trust Project guidelines Cindy, a journalist for Metaverse Post, reports on topics spanning web3, NFTs, the metaverse, and artificial intelligence, with a particular emphasis on interviews with industry leaders in Web3. Having spoken to over 30 executives in C-level positions, she brings their valuable expertise to the readership. Originally hailing from Singapore, Cindy now resides in Tbilisi, Georgia, and holds a Bachelor’s degree in Communications & Media Studies from the University of South Australia, boasting a decade of experience in journalism and writing.
