Conjurors of Content

Wizards of Math and Stats

Formative AI technologies optimise processes against rapidly changing variables. They show up almost everywhere: healthcare, business, autonomous vehicles, personalised content.

They can transform practically every sector because they make existing processes dramatically more efficient. Those who master their deployment enjoy strong advantages over competitors. Big Tech eclipsed the rest of the economy in part through first-mover advantages in AI, backed by data, compute, and the engineers who knew what to do with both.

The comparison is electricity and motive power a century ago: power drills and tractors instead of mule-driven ploughs. Every business now needs to recognise what AI is bringing to its sector and plan accordingly.

One of the most immediately applicable areas of machine learning is generative AI. Multiple neural networks compete against each other: some try to produce plausible content, others try to detect fakes. Loop them together and you breed successively more convincing representations of almost anything -- human faces, for example.

Generative techniques turn a simple sketch into a painting in the style of a great artist at the touch of a button. They restore damaged or obscured content, massively upscale low-resolution images and video, and transfer an aesthetic je ne sais quoi from one object onto another. Given a few examples, the model follows underlying patterns and correlations you would be hard-pressed to describe in words or mathematics. It can even transform a winter video into summer, or vice versa.

Generative AI is the closest thing to magic in the world today. It is already widely deployed to restore and upscale pre-HD content in movies, TV, and games, and it powers the video filters in Zoom and Snapchat. The earliest applications focused on visual content, but recent developments are about to unleash something larger.

GPT-3 (Generative Pre-trained Transformer 3) by OpenAI is the latest evolution of these techniques. It builds on previous work by taking it to a massive scale, ingesting almost the entire known internet, with a stupendous number of parameters (the relative strengths of connections between things).

GPT-2 had 1.5 billion parameters; GPT-3 uses 175 billion. To the surprise of many researchers, the massive increase in compute made it dramatically more capable. The same model at 10 billion parameters completes maths problems at a D- level; at 100 billion, B-; at 175 billion, A+. Scale in compute turns out to matter as much as the abundance of data that powered deep learning a decade ago.

GPT-3 is accessible only via API for now, owing to the hardware requirements (it cost OpenAI around $5 million to compute). That is not a deal-breaker for business use; in fact, it makes it easier to start applying these techniques in minutes instead of weeks.

Hardware improvements will make such costs far cheaper for those who want to train their own models. The human brain has an estimated 100 trillion parameters, give or take, and we will see models of comparable complexity achievable for the same $5 million cost before the end of this decade. I expect the same pattern of increasing capability to hold as parameter size grows further. That is as worrying as it is exciting.

Right now, GPT-3 can be applied to a startling range of creative endeavours. The same model translates poetry from Chinese to English, plays chess, solves maths problems, serves as a hilarious dungeon master, works out treatment regimens and medicine dosages -- a massive amount of flexible capability. Bloggers have even used GPT-3 to generate new posts from their existing content. Unbeknownst to their readers, the generated articles proved surprisingly popular.

It is still closer to human intuition than human intelligence per se, but it is adaptable, flexible, and capable within limitations. GPT-3 is a significant step in AI, though not an intelligence panacea. Its multifunctionality is formidable, but it still lacks executive functions or logical reasoning, and it’s restricted to working with text. It’s adaptable, so long as humans clearly define the problem to be solved. One can think of it like a babbling savant genie, but that’s still incredibly valuable. The next version will be even more flexible, so much so that entire creative industries may be made obsolete overnight. I strongly recommend that businesses in all sectors experiment with GPT-3 and similar generative AI technologies, and become familiar with their application. Those who embrace this new wave will be as well-positioned as big tech has been to reap the benefits of the previous wave of deep learning.

ML is definitely one of the hottest careers, and that is likely to increase even further. Deep Learning has emerged in the past ten years or so, enabling amazing new predictive processes that can find patterns within patterns, and make order out of chaos. This has transformed industry, but has had less immediate effect upon the office. That’s about to change, thanks to revolutionary new models such as Large Language Models, Transformers and Diffusion models, sometimes described as Foundation Models. These are very large statistical models that can be dynamically reconfigured to solve for thousands of different problems just with a simple natural language request, typically described as a ‘prompt’.

With this new technology, ML is finally accessible to the masses, as we no longer require much skill beyond asking a simple question to obtain quick and reasonable assistance with almost any digital office task one can imagine. The latest models are even generating computer code, video, 3D models, virtual personalities, and music on demand from nothing more than a description of the desired output. One might think that ML skills will be less needed as a result. However, by making the power of state-of-the-art ML clear to the public, the desire for improved machine learning capabilities to optimise almost any problem we can conceive of will be greatly increased.

Machine learning and Statistics are related disciplines. Statistics is about analysing data, and constructing models to explain and predict phenomena, whereas ML is about creating technical information pipelines of automated data analysis that can construct their own internal models to make a prediction. Statistics is human-focused and easily explainable. ML is machine-focused and less explainable, but may be more powerful in circumstances where there are very complex patterns, perhaps with too many variables for a human being to manage.

Both occupations require a solid grounding in mathematics and statistics, with a procedural focus on statistics (how to clean up data so it can be used, how to analyse, usually coded in R), versus a technical focus in ML on which model to apply, and the computing code required to implement it (usually Python).

However, there is a lot of mixing up of terms, sometimes due to honest confusion, and sometimes due to wilful misrepresentation. Due to the long time that the term has been used, ‘AI’ can refer to anything from a hand-written chess algorithm to a sophisticated transformer that can turn a simple natural language request into a masterfully executed output. Often, rather basic data science is dressed up as ML or AI. On the other hand, companies often apply the jackhammer of AI to surgically split a peanut of a problem when good old fashioned data science would be far cheaper and quicker. Data science holds up the modern economy far more than ML, but it remains an unsung hero. Data Science is still often a prerequisite for ML, as it provides the grounding to help ensure accuracy and robustness in ML models, which can reduce the risks of an unethical outcome, such as disproportionate treatment or other statistical biases.

Already, academia has been plundered by industry for top ML talent, lured away by large salaries and generous research budgets, and new graduates can hardly come soon enough. There will be increasing demand for skills with building and applying Foundation models in particular, as well as in other, more specialised areas such as machine vision to help embedded systems such as robots to understand the physical world with ever-greater precision. New search engines for prompts are emerging also, as the art of constructing new prompts to manifest unseen potential from existing models will be another hot commodity, like sorcerers figuring out the pronunciation of words written in a spellbook. ML is here to stay, and it is only getting stranger.