Taking a Look at Mark Zukerburg's "AI Manifesto"

When everyone is super... no one will be.
-- Syndrome, The Incredibles

Overview

Mark Zuckerberg released his thoughts on the future of AI the other day (if you haven't seen it - The Future is for Everyone) There's quite a bit to digest, but I wanted to take a shot at my own analysis of everything that was said. I'll first try to break down the key concepts that are covered in the article and then at the end express my own opinions about it all.

The main goal of the piece is to outline how we need to handle the emergence of superintelligence in a smart, responsible way in order to ensure the best outcome for humanity. Zuckerburg argues the ultimate way to achieve this is to put the power in the hands of as many people as possible (ideally everyone). And to do that, there are several key considerations about AI governance that need to be established and upheld.

The Vision

First, we need to understand the vision of what superintelligence will be according to Zuckerburg. He believes it will be a world where this invaluable knowledge is available to everyone across all of their devices, at all times. From this, many benefits will emerge. We will have a personal agent helping us improve our business, hobbies, and every aspect of our lives. This agent will allow us to rapidly prototype ideas we would otherwise struggle or be unable to realize. In addition, with everyone able to contribute personal experiences and knowledge/research, new disease curing drugs will become rapidly, readily available.

Job Growth and the Economy

Leaders in the industry have long made the claim that AI is coming for our jobs. Elon Musk, Dario Amodei, Sam Altman, and Arvind Krishna, just to name a few. There is some nuance to each of their claims, but the overarching theme is the same. Zuckerburg's take is a bit different. Rather than a replacement of humans, we will see a major shift in how companies operate in general. Instead of large companies with hundreds or thousands of employees, we will start to see much smaller businesses of just a handful of individuals. By leveraging AI, they will be able to accomplish what previously took 10x or even 100x workers previously.

Initially, this sounds like it results in a massive unemployement problem. But Zuckerburg argues that the actual result will be different.

Company sizes may shrink -- just as they did in the transition from industrial giants to tech companies. But this doesn't mean fewer jobs overall. It implies a larger number of companies with fewer people each.
-- Mark Zuckerburg

Government Involvement / AI Misuse

Zuckerburg states that people want a government that can protect them while maintaining freedom and privacy. Fair enough. It's a tale as old as time, the struggle to maintain a balanced government that doesn't overstep while providing benefit to the population. The introduction of AI exacerbates the problem of overstepping immensely. In order to maintain freedom we must take steps to ensure privacy and prevent any one government agency from gaining an advantage. Though he outlines the idea of personal, private agents that are only accessible to the user, and the idea of ensuring the government "has access to the tools and resources it needs to enforce our laws and protect our security.", Mark doesn't exactly say how he envisions this balance will be achieved. To be fair, this has always been a problem as I stated above.

In terms of AI misuse, Mark suggests that we will need to focus our efforts on restricting physical components, rather than knowledge. We won't be able to prevent bad actors from accessing information on how to create bio weapons, bombs and the like. Our best bet is to prevent access to the raw components/ infrastructure to execute these ideas. Again, this will be a hard line to walk when it comes to the idea of government agencies having access to this tooling.

My Thoughts

This whole idea is predicated on the belief that superintelligence is right around the corner, or even attainable at all. Mark paints the picture of a utopian society where we all get along and cure cancer, but is there any evidence to suggest this reality is possible? While mainstream media and tech billionaires themselves are all insisting this is happening, the financial aspect and energy requirements seem to indicate a different story.

On the financial front, Goldman Sachs analysis of AI estimates that over 1 trillion will be spent in 2026 alone. This number is greater than the total cost spent on AI from 2013-2025. As it stands today, no frontier modeling companies has a positive profit margins. We've been shoveling money into these systems, and we have yet to see a profitable way out. As private equity dries up, these companies are now turning to the public to squeeze out the last drops of cash and continue to feed the machine. Given large corporation's penchant for greed in the past, it's difficult to not view this whole ordeal as an absolutely massive cash grab.

Then there's the energy considerations. MIT Research on AI energy consumption does a fantastic job of detailing the known costs of powering data centers, and also brings to light the fact lack of company disclosure shrouds what actual costs might be. To meet the rising energy demands and scale for the future, the ask obscene.

Recently, Texas governor Greg Abbot called for an audit on data centers working towards connecting to ERCOT. According to the article, the power requests coming in from the data centers totaled 474 GW. That number is FIVE TIMES the peak energy demand ever seen on the ERCOT grid. Due to the scale and speed these CEOs are insisting we move at, short term energy demands for many of these data centers are being met with 'behind the meter' solutions (gas generators, turbines, etc.). We are moving at breakneck speed while failing to evaluate environmental, community impacts fully because we are told we don't have the time to focus on better alternatives. We are being rushed into something nobody fully understands, driven by greed and fear mongering.

On top of all this, we have to consider the current limitations of LLMs and how their knowledge is derived. Token prediction is not the same as AGI. Without a breakthrough, the current limit lies in size of the model's training data (ignoring other key factors such as the energy usage and general cost). Again, even with a massive set of training data, the end result is still predictive modeling. It can perform impressive data analysis and number crunching, which has certainly led to new research developments. But it's not coming up with the novel ideas that Zuckerburg envisions in his article.

On the job front, how does the math play out? If 2-3 people are able to perform 100x, what happens to the remaining workers previously required in that space? I can't imagine the need for 100 smaller companies that fulfill the same business purpose. Many are going to have to change career fields. I have trouble envisioning the sudden need (or creation) of brand new, unique services that Mark seems to believe will spring up out of nowhere.

And what about the fact that many services people now pay for, they will likely be able to achieve on their own? Filing taxes, financial advisement, private tutors, event planners, etc. We're talking about a reduction of workers needed for services as well as a reduction of need for services in general.

Conclusion

I don't necessarily want to be anti-AI. There have been some good things that have come out of the advancements that have been made so far. What I do want is to avoid this seemingly greed-fueled, headlong rush into a premise where we lose whether or not true AI is achieved. Zuckerburg's ideas seem wildly idealistic and naive. It's hard to believe anything other than this being him pushing a positive narrative because the longer this can run, the more money he can squeeze out.