Part I · Chapter 1 of 18

The Neuron

From if statements to weighted sums — the smallest unit of learning.

You already write neurons. Every time you code a threshold like this:

if (0.6 * temperature + 0.4 * humidity > 75) turnOnFan();

you’ve built one: a weighted sum of inputs compared against a threshold. That is a neuron, and it is the unit this site builds from.

Look at the shape of that if statement. Each input gets a weight, a number saying how much it matters. Temperature matters more than humidity here (0.6 vs 0.4), so it pulls harder on the decision. Add up the pulls and compare the total against a threshold: cross it and the neuron fires.

Rearrange it slightly and the threshold becomes a bias term b on the left-hand side, and the comparison is always against zero:

if (w1 * x1 + w2 * x2 + b > 0) fire();

Why rearrange? Because now the decision has a geometry. The equation w1·x1 + w2·x2 + b = 0 is a line. Everything on one side fires, everything on the other side doesn’t. Choosing weights is choosing where that line sits. Try it yourself:

Build a neuron
w₁·x + w₂·y + b = 0.04 → predicts +1, actually +1w₁·x + w₂·y + b = -0.17 → predicts −1, actually +1w₁·x + w₂·y + b = -0.15 → predicts −1, actually +1w₁·x + w₂·y + b = -0.21 → predicts −1, actually +1w₁·x + w₂·y + b = -0.04 → predicts −1, actually +1w₁·x + w₂·y + b = -0.30 → predicts −1, actually +1w₁·x + w₂·y + b = -0.08 → predicts −1, actually +1w₁·x + w₂·y + b = -0.21 → predicts −1, actually +1w₁·x + w₂·y + b = -0.15 → predicts −1, actually +1w₁·x + w₂·y + b = -0.22 → predicts −1, actually +1w₁·x + w₂·y + b = -0.18 → predicts −1, actually +1w₁·x + w₂·y + b = 0.06 → predicts +1, actually +1w₁·x + w₂·y + b = 0.06 → predicts +1, actually +1w₁·x + w₂·y + b = -0.27 → predicts −1, actually +1w₁·x + w₂·y + b = -0.06 → predicts −1, actually +1w₁·x + w₂·y + b = -1.02 → predicts −1, actually −1w₁·x + w₂·y + b = -1.31 → predicts −1, actually −1w₁·x + w₂·y + b = -1.33 → predicts −1, actually −1w₁·x + w₂·y + b = -1.22 → predicts −1, actually −1w₁·x + w₂·y + b = -1.21 → predicts −1, actually −1w₁·x + w₂·y + b = -1.24 → predicts −1, actually −1w₁·x + w₂·y + b = -1.14 → predicts −1, actually −1w₁·x + w₂·y + b = -1.05 → predicts −1, actually −1w₁·x + w₂·y + b = -1.23 → predicts −1, actually −1w₁·x + w₂·y + b = -1.15 → predicts −1, actually −1w₁·x + w₂·y + b = -1.29 → predicts −1, actually −1w₁·x + w₂·y + b = -1.06 → predicts −1, actually −1w₁·x + w₂·y + b = -1.11 → predicts −1, actually −1w₁·x + w₂·y + b = -1.04 → predicts −1, actually −1w₁·x + w₂·y + b = -1.11 → predicts −1, actually −1

18 / 30 correct

Drag the sliders to move the decision line. Score: how many points end up on their correct side.

With the two clouds, a few slider nudges find a line that separates them perfectly. You just did by hand what learning will soon do automatically: adjust weights until the decisions come out right.

Now click Try XOR.

The pattern itself is simple: same-sign coordinates are one class, opposite-sign coordinates the other. But no line you can draw gets them all correct. A few dozen seconds of slider-dragging makes that clear. This is the XOR wall, and it is not a toy problem. It once stalled neural network research for years.

A bit of history

The hard-threshold neuron you just played with has a famous name: the perceptron, built by Frank Rosenblatt in 1958 as a room-sized machine the New York Times promised would soon “walk, talk, see, write.” In 1969, Marvin Minsky and Seymour Papert used exactly the XOR limitation you just experienced to argue perceptrons were a dead end, and neural network research froze for roughly a decade. You’ll occasionally still see “multi-layer perceptron” (MLP) used as a name for the stacked networks of chapter 6; on this site we just say neuron and network.

The escape route has two parts, and they are the next two ideas in this course: activation functions that bend (so decisions aren’t stuck being straight lines), and layers that stack (so simple decisions compose into complicated ones). A single neuron draws a line. A network of them can carve curved boundaries.

Show the math

A neuron computes y=σ(w⋅x+b)y = \sigma(w \cdot x + b), where ww is the weight vector, bb is the bias, and σ\sigma is an activation function. The figure’s σ\sigma is a hard step: output 11 if w⋅x+b>0w \cdot x + b > 0, else −1-1. The decision boundary w⋅x+b=0w \cdot x + b = 0 is a hyperplane (a line in 2D, a plane in 3D), and ww is its normal vector: it points perpendicular to the boundary, toward the positive class.