HW3 Companion Worksheets

Start here

Four moves, every time

1 · PredictBefore running, write down what you expect to see.

2 · TraceWork through the calculation by hand, using tiny numbers.

3 · TweakChange one thing, predict again, then run.

4 · ExplainClose the AI chat. Explain the idea in two sentences of your own.

A wrong prediction is not a failure. It shows you exactly which line of code to look at again.

HW3 question	Worksheet	Warm-up notebook
Q1.1 · 1.1 Load and split	01	Sections 7–8
Q1.1 · 1.2 and 1.4 Baseline, RMSE, R²	02	Section 4
Q1.1 · 1.3 Visualization	03	Section 9
Q1.2 Overfitting demo	04	Section 11
Q1.3 Decision boundary	05	Sections 3, 5
Bonus Q1.4 Gradient descent	06	Section 10
Part 2 · Chapter 4	07	Section 6

Worksheet 01 · Q1.1 (1.1)

Load and split the data

≈ 10 minutes

Predict · before you prompt

1. In the California Housing dataset, what does one row describe? What is the target, the value we try to predict? (You can look this up.)

2. The dataset has 20,640 rows. With a 60/20/20 split, how many rows go to each part?

train validation test

Trace · after it runs

3. Find the line or lines in your code that make the split. Copy the line that sets the sizes:

4. Many solutions call train_test_split twice: first with test_size=0.4, then with test_size=0.5. Why does 0.5 in the second call produce 20% of the whole dataset?

Tweak

5. Change random_state=42 to another number and rerun. Before you run: will the sizes change? Will the first five rows of the training set change?

Explain

6. Why do we leave the test set alone until the very end?

Check your numbers

Q2: 12,384 / 4,128 / 4,128. Q4: the second call splits the remaining 40% in half, and half of 40% is 20%. Q5: sizes stay the same; which rows land in each part changes. random_state makes the shuffle repeatable, so everyone who uses 42 gets the same split.

Worksheet 02 · Q1.1 (1.2, 1.4)

Baseline, RMSE and R², by hand

≈ 15 minutes · calculator welcome

Four houses, with values in units of $100,000. The baseline always guesses the mean, which is 3. The model makes its own guesses. For each guess, the error is guess − actual.

Trace

house	actual	baseline guess	baseline error	error²	model guess	model error	error²
A	1	3	2	4	1.5	0.5	0.25
B	2	3			2
C	3	3			3.5
D	6	3			5
mean of error² (MSE)					mean of error² (MSE)
RMSE = √MSE					RMSE = √MSE

R² = 1 − (model MSE ÷ baseline MSE) =

Trace · in your code

Find the line in your notebook that computes RMSE. Copy it, and label the part that does each step: error, square, mean, root. (Sometimes a library function does two steps at once.)

Explain

What does the R² value in your notebook say about your linear model compared with always guessing the mean? Which is better, and by how much?

Check your numbers

Baseline errors 2, 1, 0, −3; squares 4, 1, 0, 9; MSE 3.5; RMSE ≈ 1.87. Model errors 0.5, 0, 0.5, −1; squares 0.25, 0, 0.25, 1; MSE 0.375; RMSE ≈ 0.61. R² = 1 − 0.375/3.5 ≈ 0.89. In words: the model's typical miss is about $61,000, compared with about $187,000 for the baseline.

The warm-up notebook, section 4, runs this same example in code.

Worksheet 03 · Q1.1 (1.3)

Match the picture to the code

≈ 10 minutes

Every visual element in your plot comes from a specific line of code. For each element, find its line in your notebook and write down the key part, such as "--" or facecolors="none".

In the picture	The code that draws it
dashed regression line
solid observed points
hollow predicted points
dotted residual lines
axis limits
choosing about 12 evenly spaced points

Tweak

Predict first, then try each change:

Change the 12 points to 4. What will the plot look like?
Make the predicted points filled instead of hollow. Why might hollow be the better choice?

Explain

What does a long residual line mean? What does it mean when the observed point sits above the line?

Worksheet 04 · Q1.2

Read the overfitting curves

≈ 15 minutes

Predict · sketch before you run

Draw what you expect the training loss and validation loss to look like in each run. Use two different pens or line styles.

Run A · normal data, few epochsepoch →

Run B · tiny data, many epochsepoch →

Trace · in your code

Find the two settings that make Run B different from Run A. Write down their variable names and values.

training-set size: number of epochs:

Find the loop that trains the model one epoch at a time and records the losses. Which line records the training loss, and which records the validation loss?

Compare

How did your sketch compare with the real plot? At which epoch would you stop Run B, and why?

Tweak

Give Run B ten times more training examples. Predict the new curves, then run. Did the gap between the two curves grow or shrink?

Explain

Finish this sentence using what your plot shows: "Overfitting is when…"

Worksheet 05 · Q1.3

A decision boundary, by hand

≈ 15 minutes

A classifier computes a score = (each feature × its weight, all added up) + bias. A positive score puts a point on one side of the boundary; a negative score puts it on the other. The boundary itself is wherever the score is exactly 0.

Suppose the quadratic model uses features [x1, x2, x1², x1·x2, x2²], weights [0, 0, −1, 0, −1] and bias 0.45. Most of the weights are 0, so the score simplifies to 0.45 − x1² − x2². (Your notebook may use different numbers; you can redo this table with yours.)

Trace

point (x1, x2)	x1²	x2²	score = 0.45 − x1² − x2²	+ or −?
A (0.2, 0.1)	0.04	0.01	0.40	+
B (0.9, 0.3)
C (0.0, 0.7)

Now use the linear model, with features [x1, x2], weights [1, 0] and bias 0. What is its score for A, B and C? What shape is its boundary?

Think

The quadratic boundary is where x1² + x2² = 0.45. That is a circle. What is its radius? Which bias would make a circle of radius 0.5?

Why can no straight line separate one ring of points from another ring around it?

Tweak

In your notebook, change BIAS_QUADRATIC. Predict first: does the circle grow or shrink?

Check your numbers

B: 0.81, 0.09, score −0.45, so B is outside the circle. C: 0, 0.49, score −0.04, so C is just barely outside. Linear scores equal x1: 0.2, 0.9 and 0.0 (C sits exactly on the boundary), and the boundary is the straight vertical line x1 = 0. The circle's radius is √0.45 ≈ 0.67; bias 0.25 gives radius 0.5. A larger bias makes a larger circle.

Worksheet 06 · Bonus Q1.4

One step of gradient descent

≈ 15 minutes · calculator welcome

The model predicts w × x. The data are x = [1, 2, 3] and y = [2, 4, 6], so the best w is 2. Start at w = 0 with a learning rate of 0.1.

errors   = w * x - y
loss     = mean(errors²)
gradient = 2 * mean(errors * x)
new w    = w - learning_rate * gradient

Trace

step	w	predictions w·x	errors	loss	errors·x	gradient	new w
0	0	0, 0, 0	−2, −4, −6
1

Think

The first gradient is negative. Why does that make w go up?

Redo step 0 with a learning rate of 0.25. Is the new w closer to 2 or farther away than 0 was? What do you think happens if you keep going?

Check in code

Section 10 of the warm-up notebook runs this exact loop. Compare its printout with your table, then try learning rates of 0.2 and 0.25.

Check your numbers

Step 0: loss 18.67; errors·x = −2, −8, −18; gradient −18.67; new w ≈ 1.87. Step 1: predictions 1.87, 3.73, 5.60; errors ≈ −0.13, −0.27, −0.40; loss ≈ 0.08; gradient ≈ −1.24; new w ≈ 1.99. With a learning rate of 0.25, the new w is 4.67, which is 2.67 away from 2 (it started 2 away). Each step overshoots by more, so w bounces farther away every time.

Worksheet 07 · Part 2

Reading dense textbook code

≈ 15 minutes

Chapter 4 packs a lot into each line. Two habits make it readable: unchain a line into one step per line, and rewrite library calls as plain loops. Practice both here on lines that HW3 does not grade, then use them on the Q2 questions.

Unchain

Early in the chapter, the book compares one image of a 3 with the "average 3":

dist_3_abs = (a_3 - mean3).abs().mean()
dist_3_sqr = ((a_3 - mean3)**2).mean().sqrt()

Pretend each image has only three pixels: a_3 = [0, 0.5, 1] and mean3 = [0.2, 0.5, 0.6]. Fill in one row per step.

step	code	what it does	result
1	a_3 - mean3
2a	.abs()
3a	.mean()
2b	**2
3b	.mean()
4b	.sqrt()

You have seen the second line before, under a different name. Which worksheet was it in?

Rewrite as a loop

Chapter 4 often uses torch.where(condition, a, b). Item by item, it means "if the condition holds, take a; otherwise take b". Here is a practice line (not from the book) and its plain-Python version:

labels = torch.where(temps > 20, 1, 0)

# plain Python
labels = []
for t in temps:
    if t > 20:
        labels.append(1)
    else:
        labels.append(0)

Now write the plain-Python version of (temps > 20).float().mean(). What does the result tell you about temps?

Connect

Chapter 4 lists seven steps for training a model (initialize, predict, loss, gradient, step, repeat, stop). Which lines of the Worksheet 06 loop match each step?

Check your numbers

1: [−0.2, 0, 0.4]. 2a: [0.2, 0, 0.4]. 3a: 0.2, the mean absolute difference. 2b: [0.04, 0, 0.16]. 3b: ≈ 0.067. 4b: ≈ 0.26. The second line is RMSE from Worksheet 02, used here to measure how different two images are.

Any question

Using AI beyond "write it for me"

The AI that wrote your code can also help you understand it. Paste your cell and try one of these prompts:

Predict with me"Explain this cell line by line for a beginner. After each line, ask me to predict what it does before you tell me."
Quiz me"Ask me three short questions about this code. Wait for my answers before giving yours."
Show every step"Rewrite this without numpy or sklearn, as plain Python loops, so I can see every step."
What matters"Which single line matters most here? What would happen if I deleted it? Let me guess first."
Find the bug"Put one small bug in this code and let me find it."
Tiny numbers"Run this idea on a dataset of 4 rows so I can check every number by hand."

The explain-back testClose the chat. Write two sentences about what your cell does. Then paste them back and ask, "What did I get wrong or leave out?" If you can explain it without the chat open, you understand it.

Next week, HW4 asks you to add comments to a code file, trace a tiny model by hand, and write a prompt that describes code. Those are the same moves you practiced here.