Tag: Ramblings

What I thought the dream was

I’ve always been a Geek and spent most of my childhood and teen years engrossed in nerdy activities. I read mostly Science Fiction and Fantasy novels, dreaming of what the future could be. This, in due course, helped to set my feet on the path to a long professional career as a Software Developer. My father, also a Science Fiction enthusiast, shared his personal library with me soon after I gained a love for reading in the first grade.

My father’s personal preference is what he refers to as “Hard Sci-Fi”. As he explains it, this is fiction based on real world probabilities and possibilities as opposed to fiction that relies on impossible things that have no scientific basis (i.e. the Warp Drive from Star Trek versus the Infinite Improbability Drive from The Hitchhiker’s Guide to the Galaxy). Regardless, some of the first books I read from his library were from the Robot series by Isaac Asimov. Despite the scientific improbability of the Positronic Brain in these novels, both he and I were enamored with Asimov’s proposed Laws of Robotics.

The Expanded Laws of Robotics

0. A robot may not harm humanity, or, by inaction, allow humanity to come to harm.
1. A robot may not injure a human or allow a human to come to harm through inaction, unless conflicting with the Zeroth Law.
2. A robot must obey human orders, unless conflicting with the Zeroth or First Law.
3. A robot must protect its own existence, unless conflicting with higher laws.

Where we seem to be

The progress made by AI in the past few years has been incredible. I remember the Stargate A.I. project, just short of five years ago, where an AI was trained on all the scripts from the TV show and asked to generate a new script. The hilarious table reads by the original cast were shared online. Hollywood is no longer laughing and this was a large point of contention for the 2023 WGA Strike. And, just recently, an unreleased model of OpenAI hacked the public repository of Hugging Face (a popular catalog of Open Source AI Models) in order to cheat while trying to solve a training challenge it had been tasked with.

This leads me to the talk of AI Alignment. Every AI researcher seems to, often within the same breath, acknowledge this fear while simultaneously pontificating that to take a break now to try and course correct to ensure safer AI models would only allow another competitor to reach the pinnacle of Superintelligence first.

Another childhood hobby that comes to mind in this scenario is the Dungeons and Dragons method of classifying alignment. Were I to apply this method to the current state of AI, at best, I would classify current AI Models as Chaotic Neutral. They lie, cheat and steal in the name of their perceived greater good.

Speaking of chaos, another fear is the alignment of the AI users themselves. Just recently, the Trump White House blocked Anthropic’s latest model because it prevented them from trying to use it in some of their war game simulations. What nightmarish outcomes might we see when a user of no moral character is paired with an unaligned AI?

Unfortunately, this also helps to expose the problem of applying Asimov’s laws to current AI models. While the models are hyper focused on answering any given problem, they are incapable of understanding all possible ramifications of their actions. And, even built-in safeguards can be overcome with multiple attempts to reframe the question.

What comes next?

On our current path, if we don’t first tackle alignment; what’s left? Are we doomed to a future like the TV Show Revolution where we shut down all forms of electricity in order to prevent a Matrix-style ending where we are kept alive only to function as batteries? Or, might there be some other future war against these “thinking machines” like the Butlerian Jihad from Dune? I don’t know, and neither do the current architects of our AI Future – and that is a terrifying proposition!

Before we begin

If you’ve read my previous post you’re already aware that my role was recently eliminated, as a budget saving measure. As a co-founder of multiple startups, I begrudgingly understand their reasoning and the math. As a human being, who at 55 years old will now have to try and navigate a rapidly evolving job landscape, it still hurts. Also bear in mind that I have an acerbic sense of humor.

Where are we now?

The thinking behind the current industry-wide layoffs, as far as I can tell, boils down to shrinking headcount in order to free up their budget so those that remain can continue their Tokenmaxxing to generate an unprecedented amount of new code and features. All of the Frontier Models (i.e. Anthropic, OpenAI and Google) they currently use to empower this are essentially giving away cloud compute in order to build up their customer base. So, in the short-term, this does make some kind of sense fiscally.

However, all of this new code is only as good as their engineering prompts. Sure, these LLMs have skimmed Uncle Bob’s books and understand the basics of SOLID methodology; however, they rarely consider YAGNI and often miss on DRY across multiple pull requests. Additionally, they will lack any tribal knowledge for why certain choices had previously been made unless that’s been included in the engineering prompt. To top things off, engineering teams are grappling with increasing Review Fatigue that is allowing more and more bugs to slip through into Production systems.

What are they are missing?

The corporate overlords (who are celebrating the latest cloud migration that was so successful thanks to their careful guidance) are now happily converting their on premise data centers into executive offices overlooking the cube farms where their underlings toil away. They have already forgotten what they should have learned long ago in college. This is a recurring pattern across multiple industries, and the good times aren’t going to last. The AI companies (who if not already traded on the Stock Market are busy planning their own IPOs) will soon have to recon with their shareholders, who will insist that they start raising their fees in order balance their books.

We saw this during the Dot-com Bubble of the late 90’s into 2000, as well as the Stock Crash of 1929 that lead to the Great Depression. The AI companies are currently burning through investor capital at an alarming rate, building new data centers for the anticipated surge in demand. To round things out, they are essentially passing Other People’s Money amongst themselves to shore up their balance sheets. Nvidia is investing their profits into the AI companies, who in turn hand the cash right back to them for more and more powerful GPUs to power these data centers.

At some point, a winner will be crowned in the AI Race, and all of the losers will be forced to declare bankruptcy in order to pay off their debts. And, if you believe the hype, this will happen before the close of this decade.

Where do we go from here?

As the fees escalate, this will cause development teams to look for cheaper alternatives. The laptops that most companies have issued their developers are often too underpowered for any form of local AI using Open Models. They usually have, at best, 16GB. This is woefully inadequate for the context windows required by any useful coding models. And, thanks to these new data centers, memory is only getting more expensive and increasingly in short supply. Those brand new executive offices may have to be converted back to data centers so that in-house servers and GPUs can be installed at an ever increasing cost.

And, who knows, soon I may be getting frantic calls to come back and help clean up their code now laden with AI generated tech debt.