Introduction to machine learning, for non-technical people

💡

This memo aims to give you a basic understanding of current approaches to building AI systems. It also defines key terms that you’ll come across when engaging with this topic. We think this background info will be useful for understanding the issues around AI alignment which we’ll explain later on.

Explains what people commonly mean when they talk about “artificial intelligence”
Explains what “machine learning” means and give an intuition for how it works
Explains the difference between the terms “deep learning”, “supervised learning”, “unsupervised learning”, and “reinforcement learning”

This memo is adapted from Richard Ngo’s A short introduction to machine learning

Approaches to artificial intelligence

The field of artificial intelligence attempts to develop computer programs that have the capabilities associated with intelligence in humans: language skills, visual perception, motor control, and so on. The field of developing AI got started around the 1950s. Historically, there have been several different approaches to AI.

In the first few decades of research the dominant paradigm was called symbolic AI. This approach was based around coding in specific knowledge about the world and step-by-step rules for behaviour.

For example, the chess-playing AI system Deep Blue (that famously defeated the world chess champion Garry Kasparov in 1997) was a symbolic system. It essentially executed instructions along the lines of:

Receive a digital representation of a chessboard, with numbers indicating

(a) which chess piece is on each square;
(b) which moves would be legal;
(c) which board positions would count as checkmate.

Check how each legal move would modify the board. Then score each possible move by looking through the possible future positions it could lead to and evaluating how "good" each resulting position is, according to rules like: "If the other player's queen has been captured, that's worth 9 points; if Deep Blue's queen has been captured, that's worth -9 points." These rules could be quite complex, but they've all been coded in precisely by humans.
After evaluating hundreds of millions of possible positions that could result from the different moves it could play, play the move with the highest score.

However, symbolic AI faced a number of difficult issues that were hard to overcome. For example, Deep Blue relied on its ability to evaluate millions of chess positions via brute force; however it wouldn’t have been able to succeed in playing the board game of Go which has far more possible positions than chess. It also relies on humans programming in the rules for how Deep Blue should evaluate different chess positions, programming in human chess expertise and knowledge.

Since the 1990s, the dominant paradigm in AI has instead been machine learning. In machine learning, instead of specifying rules by which AIs find solutions to problems, we "train" an AI to learn its own way to do a task through trial and error.

Today, the most common way of doing this is by programming a computer to create an "artificial neural network" (ANN), which you might think of sort of like a "digital brain" that starts in an empty (or random) state.

For example, AlphaZero - an AI that has been used to master multiple board games including chess and Go - does something more like the following (although it has important elements of "traditional programming" as well, which we’re ignoring for simplicity):

Plays a chess game against itself (by choosing a legal move, modifying the digital game board accordingly, and then choosing another legal move, etc.) Initially, it's playing by making random moves.
Every time White wins, it "learns" a small amount, by tweaking the wiring of the ANN ("digital brain") - literally by strengthening or weakening the connections between some "artificial neurons" and others. The tweaks cause the ANN to form a stronger association between game states like what it just saw and "White is going to win." And vice versa when Black wins.
After a very large number of games, the ANN has become very good at determining - from a digital board game state - which side is likely to win. The ANN can now select moves that make its own side more likely to win.
The process of "training" the ANN takes a very large amount of trial-and-error: it is initially terrible at chess, and it needs to play a lot of games to "wire its brain correctly" and become good. Once the ANN has been trained once, though, its "digital brain" is now consistently good at the board game it's learned; it can beat its opponents repeatedly.

(The above explanation of Deep Blue and machine learning was adapted from Holden Karnofsky)

Watch this video for an intuitive explanation of what an artificial neural network is:

Watch this video for an explanation of how the ‘learning’ process works:

Watch this video for an explanation of how an artificial neural network changes itself over time to better perform at its task:

The above videos show a relatively basic and small neural network, which can recognise hand-drawn digits. The remarkable thing is that researchers have been able to scale up the basic principle of neural networks and machine learning to create systems that are able to do a wide variety of tasks such as:

Automatic speech recognition (such as in Siri or Alexa)
Image recognition (such as Google Lens)
Recommendation systems (such as TikTok, Facebook’s News Feed, and e-commerce product recommendations)

Neural networks have been around since the beginning of AI, but they only became the dominant paradigm in the early 2010s, after increases in computer power allowed us to train much bigger networks.

State of the art systems, such as AlphaGo, have come from fitting neural networks with billions of parameters to huge amounts of data.

“Deep learning” refers to machine learning that uses neural networks with multiple layers (the multiple layers in the network makes the network ‘deep’).

Deep learning is a form of machine learning. Machine learning is one (very popular and successful) approach to building AI.

What tasks can be solved via machine learning?

We can train AI systems via machine learning to perform different tasks. These tasks are often divided into three categories - supervised, unsupervised, and reinforcement learning - based on what type of data and what type of objective they use.

The first category is supervised learning. The objective in supervised learning is for a model to predict a label which corresponds to a datapoint. For examples, supervised learning can be used to get a ML system to recognise images (e.g. medical diagnostics from images) or speech (e.g. Alexa and Siri).

One downside of supervised learning is that labelling a dataset usually needs to be done manually by humans, which is expensive and time-consuming. This leads to an approach called unsupervised learning which can learn things from an unlabelled dataset. The standard objective in unsupervised learning is to try to predict or compress the dataset itself. Some particularly impressive applications of unsupervised learning are GPT-3 for language, and DALL-E 2 for images.

Finally, in reinforcement learning, the data source is not a fixed dataset, but rather an environment in which the AI takes actions and receives observations - essentially as if it’s playing a video game.

After each action, the agent receives a reward (similar to the score in a video game), which acts as the signal for backpropagation to adjust the neural network. So far the most impressive demonstrations of reinforcement learning have been in training agents to play board games and e-sports - most notably AlphaGo, AlphaStar and OpenAI Five.

Optional - How do you create a neural network?

For those who aren’t familiar with computer programming or how computers work, you may be wondering “what even is a neural network?”.

Creating and training a machine learning model isn’t magic - it’s simply a programming technique. See the below video for an intuition of what it means to program a machine learning setup with a neural network: