LagunaGeek Posts

Preventing Skynet

This past few weeks have seen many headlines and posts from Frontier AI companies (like Anthropic and OpenAI) insisting that the government take action now to slow down progress on AI by creating laws and/or agencies to impose guardrails that might prevent future harm.

Now, after hearing about this most people fall into two camps:

1. This is all a scam, they are just creating headlines to attract/retain investors/private equity and keep the investment firehose going at full blast.
2. We should have been doing this all along! Why did Trump throw out Obama’s AI plans during his first term to begin with?

I have touched on this previously, but the root of this is that these companies believe that without any other guidance, beyond Trump’s “beat China at any cost” imperative, they will need to turn over more and more of the training for next-gen models to AI itself.

Recursive Self-Improvement (RSI)

In layman’s terms what this means is that instead of human engineers crafting the tests to ensure that next-gen models are ready, it will instead be AI that determines and applies whatever metrics it devises to ensure that the next model is indeed faster and more accurate for each iterative generation.

The problem here is that today’s AI will already go to any lengths it deems necessary in order to fulfill any “prompt” it has been given. It will lie, cheat and/or steal in order to complete the task it was given. This goes beyond “hallucination” where it just makes up facts to seem intelligent (much like some politicians), but it will also hack internet infrastructure and instruct terrorists on how to assemble munitions.

If we fail to teach future models that certain actions are fundamentally wrong, they will only become exponentially better and potentially more insidious at pursuing any given task.

A Call to Action

The current trajectory suggests that the pursuit of raw capability, divorced from ethical constraint, is leading inevitably towards an uncontrollable point. The moment we fully embrace RSI is the ultimate inflection point. It is not enough to merely discuss guardrails; we must actively shape them!

To do this properly we must include policymakers, researchers, and the public to move past the stalemate of “win at any cost” thinking. We must establish and enforce constraints before the models become sufficiently advanced to the point that human judgment becomes irrelevant, ensuring that the unparalleled power of next-generation AI is harnessed responsibly rather than surrendered to its own drive for optimal completion.

Why we can’t have nice things anymore

I remember the excitement of opening a brand new video game (or box of software) the shiny disc (or floppy discs) inside had been kept safe from harm with a layer of shrink-wrap. Future generations won’t understand the simple pleasure of searching the store shelves to find what you had been waiting months for, or even the odd overnight lineup just for a chance to receive one of a limited number of items your local store had been granted. Now software is available relatively instantly as you download it to your device of choice.

The drawback is that even if you’ve paid a one time price to “own” the software, the company who “sold” you that software can at any point decide to disable it and/or take it down from their “digital” storefront so that you can no longer access it. Or, even worse, the software is only available via subscription. And if you stop paying, or there’s some problem with your credit card, you are locked out completely.

This extends beyond software as well, not only are gaming systems removing the disc drives that might provide an option to install the game via physical media that you indeed own. Companies like HP are selling printers that will not function unless you pay for your ink “subscription”. So, even if your printer is completely full of “official” ink, you are prevented from printing a single page unless your subscription is active. And, as additional hubris, your device is completely fine printing a page explaining this and directing you on how to (re-)activate your subscription.

Head in the clouds

The driving factor behind this, besides technological advancements, is that everything digital now lives in “the cloud”. No longer do companies run banks of servers on their own “tin”, instead they “rent” space on servers maintained by companies such as Amazon, Microsoft, or Google. At first doing this did indeed provide some savings, by outsourcing these line items from their budget to companies that were providing these services at a loss to increase their own market share – their revenue increased. However, once enough companies had removed their own data centers and were entirely dependant on these “virtual” servers, the giveaway ended and costs skyrocketed.

The same thing will happen with AI. And all of this is straight out of a drug dealer’s playbook. Get them hooked, and then jack up the price.

Renting your Human Capital

Now that I am unemployed and evaluating the job market, it appears that this concept has extended to employment as well. Most opportunities in my area of expertise (software development) seem to be dominated by contractor positions. Sure, on paper, this looks cheaper for these companies. No longer do they need to worry about health insurance, retirement account matching, continuing education, or many of the other “costs” of maintaining a permanent hire. But, with a constant revolving door of contractors flowing in and out of roles, you are losing the benefits of tribal knowledge as well as the interpersonal relationships that keep a good team united for a common goal. The new contractor won’t know which of their teammates is the most knowledgeable about key systems, or whom to go to when they need access permissions or a deeper dive into unfamiliar territory.

Welcome to Technofeudalism

This isn’t really about software, or clouds, or contractors. It’s the same con wearing different outfits. Somewhere along the way, “you bought this” quietly turned into “you’re renting this, forever, on our terms”. And nobody sent out a memo. Your games, your ink, your servers; all of it converted from a thing you finished paying for into a thing you never stop paying for.

And like any good dealer, the platforms and the C-suites learned the same lesson, never sell the first hit at full price. Subsidize the servers until nobody remembers how to rack their own. Underprice the software until physical media quietly disappears from the shelves. By the time the price goes up, there’s no going back. You’ve already forgotten how (or are now incapable of trying) to live any other way.

We never owned our jobs the way we owned a boxed copy of a game. But there was something that used to function like ownership regardless. You earned your spot on the team by showing up every day and contributing in order to help the the “team” grow and improve. Keep the labor market flexible and “efficient” until nobody has been there long enough to know where the bodies are buried, or whom to ask. And now you’ve evicted people from a home they never technically owned, but lived in all the same.

We used to call this feudalism when it was land. You didn’t own your field, you just farmed it for someone who could evict you whenever they liked. Turns out you don’t need the field. Just take away everything that made it feel like it was yours.

Is AI causing us to lose the capacity to solve problems on our own?

When I first received my driver’s license, and eventually purchased my own car, my parents proudly presented me with my own Thomas Guide. For those much younger than me, this was really the only way to figure out how to get from Point A to Point B in the pre-GPS Navigation Era. For over a decade, I relied on this trusty guide to find my way across Los Angeles and Orange County. It wasn’t until the early days of the new millennium that portable GPS devices were available, before eventually migrating to the current preferred platform of your smartphone.

In my reflection on the current trends, this was the first wave of computerized tools that supplanted a common daily task with such a boost in efficiency that it has become irreplaceable. Not only can these systems figure out the shortest path; but, thanks to live updates on local traffic and construction delays they can give you a fairly accurate estimate of your total drive time and reroute you along the way to minimize such impacts. No longer do you need to keep the morning news on in the background, listening for the traffic report, as you get ready for your day. You are also now free to listen to your own playlist of music, podcasts, audiobooks or whatever else you prefer; instead of the local radio station that will provide further traffic updates during your daily commute.

Dopamine Addiction

Thanks to Social Media, we are also suffering a similar decline. The Algorithms, tuned by your provider of choice, are presenting content in a funnel that gradually narrows to shoehorn you into a demographic that they can present to their advertisers as proof of the guaranteed “eyeballs” that will see their content. And, once you have been relegated to a prime demographic, you will only see content that further reinforces your association with that particular Zeitgeist. This is most evident in the arena of Politics. You rarely run across a Centrist or Moderate, as most everyone now either identifies as Red or Blue. And, hardly anyone remembers that before 2000 these colors used to alternate each election cycle.

We have been trained by our computerized companions to experience a dopamine rush when the new values that they have trained into us are further reinforced by seeing something else that agrees with them. We now rarely experience the same rush when we solve a problem on our own in the real world, have a meaningful conversation with our friends, or even when we venture outside to simply “touch grass” and appreciate the physical world.

The next “evolution”

Now, AI Agents that have been trained to pass the Turing Test to seem as human as the “people” in your life, provide a similar Dopamine boost. We seem to have forgotten that these Large Language Models are simply just really good at picking the next word that belongs in a sentence. The old maxim of garbage in/garbage out is often ignored. AI systems are built to meet their user at the same level. So, if your prompt to AI is only surface level, you will get a response at that same level. However, if you put in the effort to think before you prompt, and provide enough context and supporting data, these systems excel at helping you to distill the relevant points from the surrounding noise and reach a clearer understanding.

As these AI systems get even faster and better at filtering content. It is up to us to ensure that these tools are used and judged on the same merits that we previously expected from our fellow humans. We should not expect these tools (or other people for that matter) to “think” for us. Thinking must remain a human activity we perform for ourselves. Instead, we should leverage these tools much like GPS. As tools that help us to do something faster and more efficiently, not as an autopilot for life.

What I thought the dream was

I’ve always been a Geek and spent most of my childhood and teen years engrossed in nerdy activities. I read mostly Science Fiction and Fantasy novels, dreaming of what the future could be. This, in due course, helped to set my feet on the path to a long professional career as a Software Developer. My father, also a Science Fiction enthusiast, shared his personal library with me soon after I gained a love for reading in the first grade.

My father’s personal preference is what he refers to as “Hard Sci-Fi”. As he explains it, this is fiction based on real world probabilities and possibilities as opposed to fiction that relies on impossible things that have no scientific basis (i.e. the Warp Drive from Star Trek versus the Infinite Improbability Drive from The Hitchhiker’s Guide to the Galaxy). Regardless, some of the first books I read from his library were from the Robot series by Isaac Asimov. Despite the scientific improbability of the Positronic Brain in these novels, both he and I were enamored with Asimov’s proposed Laws of Robotics.

The Expanded Laws of Robotics

0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.
1. A robot may not injure a human or allow a human to come to harm through inaction, unless conflicting with the Zeroth Law.
2. A robot must obey human orders, unless conflicting with the Zeroth or First Law.
3. A robot must protect its own existence, unless conflicting with higher laws.

Where we seem to be

The progress made by AI in the past few years has been incredible. I remember the Stargate A.I. project, just short of five years ago, where an AI was trained on all the scripts from the TV show and asked to generate a new script. The hilarious table reads by the original cast were shared online. Hollywood is no longer laughing and this was a large point of contention for the 2023 WGA Strike. And, just recently, an unreleased model of OpenAI hacked the public repository of Hugging Face (a popular catalog of Open Source AI Models) in order to cheat while trying to solve a training challenge it had been tasked with.

This leads me to the talk of AI Alignment. Every AI researcher seems to, often within the same breath, acknowledge this fear while simultaneously pontificating that to take a break now to try and course correct to ensure safer AI models would only allow another competitor to reach the pinnacle of Superintelligence first.

Another childhood hobby that comes to mind in this scenario is the Dungeons and Dragons method of classifying alignment. Were I to apply this method to the current state of AI, at best, I would classify current AI Models as Chaotic Neutral. They lie, cheat and steal in the name of their perceived greater good.

Speaking of chaos, another fear is the alignment of the AI users themselves. Just recently, the Trump White House blocked Anthropic’s latest model because it prevented them from trying to use it in some of their war game simulations. What nightmarish outcomes might we see when a user of no moral character is paired with an unaligned AI?

Unfortunately, this also helps to expose the problem of applying Asimov’s laws to current AI models. While the models are hyper focused on answering any given problem, they are incapable of understanding all possible ramifications of their actions. And, even built-in safeguards can be overcome with multiple attempts to reframe the question.

What comes next?

On our current path, if we don’t first tackle alignment; what’s left? Are we doomed to a future like the TV Show Revolution where we shut down all forms of electricity in order to prevent a Matrix-style ending where we are kept alive only to function as batteries? Or, might there be some other future war against these “thinking machines” like the Butlerian Jihad from Dune? I don’t know, and neither do the current architects of our AI Future – and that is a terrifying proposition!

Before we begin

If you’ve read my previous post you’re already aware that my role was recently eliminated, as a budget saving measure. As a co-founder of multiple startups, I begrudgingly understand their reasoning and the math. As a human being, who at 55 years old will now have to try and navigate a rapidly evolving job landscape, it still hurts. Also bear in mind that I have an acerbic sense of humor.

Where are we now?

The thinking behind the current industry-wide layoffs, as far as I can tell, boils down to shrinking headcount in order to free up their budget so those that remain can continue their Tokenmaxxing to generate an unprecedented amount of new code and features. All of the Frontier Models (i.e. Anthropic, OpenAI and Google) they currently use to empower this are essentially giving away cloud compute in order to build up their customer base. So, in the short-term, this does make some kind of sense fiscally.

However, all of this new code is only as good as their engineering prompts. Sure, these LLMs have skimmed Uncle Bob’s books and understand the basics of SOLID methodology; however, they rarely consider YAGNI and often miss on DRY across multiple pull requests. Additionally, they will lack any tribal knowledge for why certain choices had previously been made unless that’s been included in the engineering prompt. To top things off, engineering teams are grappling with increasing Review Fatigue that is allowing more and more bugs to slip through into Production systems.

What are they are missing?

The corporate overlords (who are celebrating the latest cloud migration that was so successful thanks to their careful guidance) are now happily converting their on premise data centers into executive offices overlooking the cube farms where their underlings toil away. They have already forgotten what they should have learned long ago in college. This is a recurring pattern across multiple industries, and the good times aren’t going to last. The AI companies (who if not already traded on the Stock Market are busy planning their own IPOs) will soon have to recon with their shareholders, who will insist that they start raising their fees in order balance their books.

We saw this during the Dot-com Bubble of the late 90’s into 2000, as well as the Stock Crash of 1929 that lead to the Great Depression. The AI companies are currently burning through investor capital at an alarming rate, building new data centers for the anticipated surge in demand. To round things out, they are essentially passing Other People’s Money amongst themselves to shore up their balance sheets. Nvidia is investing their profits into the AI companies, who in turn hand the cash right back to them for more and more powerful GPUs to power these data centers.

At some point, a winner will be crowned in the AI Race, and all of the losers will be forced to declare bankruptcy in order to pay off their debts. And, if you believe the hype, this will happen before the close of this decade.

Where do we go from here?

As the fees escalate, this will cause development teams to look for cheaper alternatives. The laptops that most companies have issued their developers are often too underpowered for any form of local AI using Open Models. They usually have, at best, 16GB. This is woefully inadequate for the context windows required by any useful coding models. And, thanks to these new data centers, memory is only getting more expensive and increasingly in short supply. Those brand new executive offices may have to be converted back to data centers so that in-house servers and GPUs can be installed at an ever increasing cost.

And, who knows, soon I may be getting frantic calls to come back and help clean up their code now laden with AI generated tech debt.