PECULIARLY PETE

This is, dear reader, not the kind of topic I really enjoy writing about. For all my musings on trains, aeroplanes, military history and related matters, I have not once covered computer science, because, quite frankly, I lack any real expertise or interest. That said, the way things have been going with Artificial Intelligence has become so all-encompassing that it seems pertinent to say something.

We have to start by defining what this “AI” actually is. In very oversimplified terms, it is not like traditional software, which is coded from the bottom up to achieve some aim. Instead, lots of bots are generated, with some fairly random code. These are then set a problem; the ones that get closest to the solution are then copied, and randomised slightly, with the ones that didn’t discarded. The slightly randomised good ones are then set the problem again, and the process repeats, in what is eerily like the process of evolution, but much, much faster. The upshot is, eventually a programme is arrived at which can do the task, but it has been so randomised and tweaked that nobody understands exactly how it does this. With more training data against which the “problem” can be set and the randomised code can draw on, this can be scaled up of course, and used to do things until a few years ago thought impossible.

In particular, this was used to create chatbots, who would, through unknown means, be able to predict which word should come next, and in so doing generate complex, human-sounding answers to questions. These so-called Large Language Models (LLMs) are also very good at following the rules of a programming language, as this is actually a very similar concept, and thus can write pretty decent code. 

The next logical step is to feed the outputs back into the machine. For example, rather than simply asking the chatbot the steps to solve a problem, and having it predict the answer, draw on the internet etc., those answers can be used as prompts to get the LLM to take action. In such a way, an agent can be created, and you may even have used one; these things are moving incredibly fast and adoption has been rapid.

And that, as far as we are concerned dear reader, is where we have to start asking ourselves some pretty serious questions, not just as users of technology, but more fundamentally as humans. There are several future scenarios we need to consider here, and consider very carefully indeed:

Future Scenario #1 – Existential Risk

Because this is the most terrifying, let us deal with it straight away. This is the idea that an AI agent could be set up to improve the AI model; the new AI model is trained to improve on itself, the resulting more intelligent AI then improves itself, and so on and so forth until the resulting model is more intelligent than any human. 

There is a hot debate about whether this is possible at the moment and how far away it might be (I have heard numbers anywhere from absolutely impossible to 10% chance in the next few years), but the fact that it is being entertained as a serious possibility is worrying. Such an intelligent machine may have no use or care for us; it may view us in the same way humans view ants. And that is assuming it does not decide that we are a problem; drones in Ukraine have shown that an awful lot of havoc can be dealt without any human actually going into combat, and expect a super-intelligent AI to be able to do much, much more.

In summary, to write like a true risk assessment, the risk is low, but the consequence is existential.

Future Scenario #2 – AI Accidentally Does a Very Bad Thing

This one is considerably more likely. Anyone carefully reading the news around the Hugging Face attack will have noted that the AI did not hack into Hugging Face to be evil, it merely generated the impression that it needed to do so to steal keys to pretend that it had passed an impossible test. Suffice to say, this is not actually general intelligence (anyone with a functioning brain would realise this is very illegal and not required for the test) or actually very useful (no business is going to want an accidental very large legal bill because its AI didn’t understand the instruction), but it does demonstrate the risk. 

Such accidents could escalate to some catastrophic proportions. Chemical and biological weapons can be made almost entirely by machine, and a military AI may accidentally decide this is a good idea. We have already seen military AIs randomly “decide” that a ship is full of nuclear weapons, for example, and the geopolitical ramifications of such an AI mistake are not hard to imagine. 

Again, in risk assessment terms, the likelihood is moderate, but the consequences could be catastrophic.

Future Scenario #3 – AI Is Instructed To Deliberately Do The Very Bad Thing

A hostile state, or more likely a terror group, could use an AI bot to cause complete chaos, shutting down a country with the stroke of a few keys. Or worse still, could produce biological or chemical weapons, or stage a very convincing false flag. Again, ramifications are not hard to understand. 

Likewise, this is a moderate to high likelihood with potentially catastrophic consequences.

If none of these futures sound very promising to you, you’re not alone. And I am afraid the bad news does not stop there. Those in the pro-AI camp may point out that humans can also be sly, and do catastrophic things; but there are a few reasons why AI is fundamentally different:

• An AI cannot be held accountable. There is no way to “punish” it for wrongdoing. It cannot meaningfully be deprived of liberty or status.
  
• An AI can have considerably more reach than a single human, and could create copies and sub-agents to extend this still further.
  
• There is no way to quantify the risk. Without dissecting the extremely large code base of the model (which remember, we didn’t write, we just evolved towards) we do not know how or why it works. Without knowing this, we cannot know what would make it go “rogue” or do something unexpected.

And all of this is before we even get to the implications of what has already been done. Between them, the world’s AI labs have, at least by my reckoning:

• (allegedly) pirated the work of millions of artists, writers, in fact anyone who has created anything that has been digitised. I say allegedly, because they have used vast amounts of internet data for training models, and our current legal frameworks do not define this well. Rest assured, my view is that this is piracy. There is a world of difference between drawing inspiration from someone else, and using a machine to statistically spit out work that is based on theirs. 

• Acquired botnets they didn’t really understand, put them in poorly designed sandboxes, and allowed them to meddle with various third parties including US Government websites. This one doesn’t come with an “allegedly” because if you or I did this, we would be facing a long, long spell at His Majesty’s Pleasure.

• Used up vast natural resources in the middle of a climate crisis, for no essential purpose. When we speak of data centres in the Gigawatt scale, it is easy to lose sight of what that means. The biggest power station in the UK, Drax, has an installed capacity of around 2.6GW. To do this, it burns thousands of tonnes of biomass (essentially wood pellet) a day. Granted, there are renewable ways to generate this power, but either way, this involves a huge amount of infrastructure, adding a vast amount of pollution even if the electricity production requires no carbon emissions.
  And this is before we even start on water, but the implications are not good on either metric.
  
• By their own admission, developed models that may end up killing everyone. Despite admitting this, they somehow sleep at night and go to work in the morning, continuing to build them. The mind truly boggles.

• Through demand for chips, have inflamed the Taiwan Strait crisis, the biggest potential flash point for another world war. To cut a very long story short, the People’s Republic of China claims that Taiwan is part of their territory, and the Taiwanese government (really the remnant of the old Republic of China) wants to remain an independent, democratic nation. This wouldn’t be a big deal except that Taiwan manufactures the vast majority of the GPUs that end up in these AI data centres (and computer chips generally), and the PRC is currently banned from accessing them. Suffice to say, if these chips get any more essential to the world economy, the PRC may be tempted to try an invasion, bringing the US and China into conflict.

It would do us well to linger on that last point, because AI is often phrased as an issue of US AI labs. This is not the case; one of the reasons we know about their wrongdoing is that the US constitution guarantees the right to freedom of speech, and the press. Neither of these are the case in the People’s Republic, and I suspect that, given their behaviour around Taiwan and Hong Kong, (not to mention the fish of the world’s oceans) they are not using this technology in a responsible, world-friendly way either.

The good news is that we can do things about this, before the world is dragged, against its will, into a new dark age. Even if we just manage to get the attention of the British public, it will not be meaningless on the world stage; this year, Britain has presidency of the G20; we can set the tone at least.

We already have frameworks for how to move forward – we must merely demand that they be implemented. One I would particularly draw your attention to is the Pro-Human AI declaration (The Pro-Human AI Declaration) but there are plenty of other suggestions out there – do explore these at your leisure.

How to close this out? I suppose one ought to aim for some kind of balance, especially given the wide variety and rapidly increasing quality of the output of AI, but to be completely frank with you, dear reader, the risk is far too high for us to be even handed. At present there is not much, other than a few months or years of development, between AI models and catastrophe – we can only trust that we, as a species, act fast enough. God willing we will all get through this, and this article will seem needlessly hyperbolic to future audiences!

Either way, rest assured, Peculiarly Pete, however infrequently posted, will remain mercifully AI free until further notice!