How Does a Bunch of Math Become AI?

I started with a simple question: can we write the simplest possible AI model?

The answer was a few lines of Python. We gave the model four examples:

1 → 2
2 → 4
3 → 6
4 → 8

Its job was to learn the pattern and predict what should happen when we gave it 5.

The model had one rule:

[
\text{prediction} = \text{weight} \times \text{input}
]

The weight started at zero. After training, it became approximately two. The model predicted ten.

That sounds straightforward. But it led me to a much more interesting question: how does a collection of mathematical operations actually learn something?

The First Piece: Learning From Error

We never told the model, “Multiply everything by two.”

Instead, we gave it examples and measured how wrong its predictions were. For each example, we used squared error:

[
\text{loss} = (wx-y)^2
]

Here, (w) is the weight, (x) is the input, and (y) is the correct answer.

The model then needed a way to decide how to change its weight. That is where calculus entered the picture.

The derivative of the loss tells us how the error changes as the weight changes:

[
\frac{dL}{dw} = 2(wx-y)x
]

That expression initially looked like something someone had pulled out of nowhere. But each piece had a reason.

The 2 came from differentiating the square. The final x came from differentiating the expression inside it. Connecting those two steps is the chain rule.

Training repeatedly used that derivative to make small adjustments to the weight. With a suitable step size, the predictions improved.

For this tiny model, we can explain the learning process precisely. There is no mystery about why the weight moves toward two.

So Is AI Just Mathematical Ideas Put Together?

In a broad sense, yes.

People choose a model structure, define an objective, and use mathematical methods to adjust the model’s parameters.

Our example combined algebra, calculus, and optimization. Large language models use those foundations alongside matrix operations, probability, and neural-network structures such as attention.

But there is an essential distinction:

People design the learning process. Training determines the values the model learns.

Writing the instructions for learning is different from writing every rule the finished model will use.

Where the Explanation Gets Harder

Knowing how weights change does not automatically tell us what the resulting weights represent.

Imagine a model trained on sentences like these:

One apple plus one apple makes two apples.
One stone plus one stone makes two stones.
One pencil plus one pencil makes two pencils.

Now ask it:

One button plus one button makes ___ buttons.

Suppose it answers “two.”

What did it learn?

Perhaps it learned a sentence pattern. Perhaps it learned a rule that applies across different objects. Perhaps it developed a representation useful for counting.

That one answer cannot settle the question.

We can test it with different wording:

I had one button. You gave me another.
How many buttons do I have?

Success on varied examples gives us stronger evidence that the model learned something more general. But explaining exactly how it performs that calculation internally takes further investigation.

This is where the gap appears: we can understand the training procedure better than we understand every strategy the trained model develops.

When Does a Matrix Become a Cat?

The conversation then turned to images.

Suppose a cat photograph becomes a 25 × 25 grid of numbers. How do those numbers become “cat”?

First, the pixel values describe brightness or color. They do not automatically carry the meaning “cat.” A dog photograph can have exactly the same dimensions.

A trained model transforms those pixels into internal representations and uses them to make a prediction.

For an oversimplified example, imagine a classifier that receives two measurements:

Pointed ears:       0.9
Whisker visibility: 0.8

Training might teach it that this combination often occurs in pictures labeled “cat.” Real models learn much richer features, and these two clues would certainly be insufficient on their own.

The important idea is that numbers acquire a role through the model’s calculations and training.

There does not have to be one number, or one fixed matrix, that always means “cat.” Different photographs can produce different internal values while still leading to the same classification.

The Needle in the Haystack

Trying to locate the model’s internal representation of “cat” can feel like searching for a needle in a haystack.

Researchers can examine internal activity, change parts of it, and observe what happens to the model’s behavior. If changing a particular component affects recognition of cats, that provides evidence about its role.

But a useful concept may be distributed across many components. A component may also contribute to several different concepts. A complete explanation can therefore be difficult.

Training does not require us to find those representations manually. The error signal guides adjustments across the network.

That is the remarkable part: we can create a process that develops useful internal patterns without explicitly specifying every pattern beforehand.

What Surprised Me Most

The tiny model and the large model share an underlying idea:

Make a prediction. Measure the error. Adjust the parameters. Repeat.

In the tiny model, the result is easy to inspect: one weight becomes two.

In a large model, many interacting parameters can support complicated behavior. We understand substantial parts of the mechanism, while still investigating how particular abilities arise and operate.

What began as a request for simple code became a question about knowledge itself: how can numerical adjustments produce patterns that work across new situations?

The first glimpse of the answer was already there in that small program.

We never wrote “the answer is two.”

We built a process that could learn it.