An optional lesson. Students find where the penguin model fails (an unseen species, impossible measurements, confident wrong answers), add a "not like any penguin" check with kneighbors, test harder questions (sex, island), and write a model card.
Load these into Waypoints Prepare: choose Load URL and paste the copied address, or download the file and import it. In Prepare, the worksheet's Teacher edition shows the answers; the student copy hides them.
Predict and Run 10 min · Investigate 10 min · Modify 15 min · Make 10 min
Optional: not needed for completing the course. A classifier can only choose between the classes in its training data, so an emperor penguin (not in the data) is called Gentoo; impossible measurements still get a 100%-sure answer, because the vote share compares neighbours with each other and ignores how far away they are. model.kneighbors(...) returns the distances, which the first Modify uses to refuse far-away inputs. A fair test shows species is easiest (88% here with this split), sex harder (73%) and island hardest (64%) from bill and flipper length alone. The data comes from three islands near Palmer Station, 2007 to 2009; a model knows only that world. The info question moves to a human context (a school using past records): be ready for discussion about fairness and human review. A model card is a short honest description used in industry.
Separating confidence from correctness. Ask students to argue for and against trusting a 100% answer before they code the check.
| What you will see | A question to ask | The fix |
|---|---|---|
| ValueError: Input contains NaN when testing sex. | Are there penguins with no sex recorded? | pd.read_csv("penguins.csv").dropna(). |
| Every mystery says not like any penguin. | How far is too far, and which distance is checked? | distances[i][0] is the nearest; try a limit of about 10. |
One possible solution for each task. Other correct programs are fine.
import pandas as pd
from sklearn.neighbors import KNeighborsClassifier
penguins = pd.read_csv("penguins.csv")
columns = ["bill_length_mm", "flipper_length_mm"]
model = KNeighborsClassifier(n_neighbors=5)
model.fit(penguins[columns], penguins["species"])
mysteries = pd.DataFrame([[39.0, 190.0], [48.0, 195.0], [80.0, 350.0], [20.0, 120.0]], columns=columns)
guesses = model.predict(mysteries)
chances = model.predict_proba(mysteries)
distances, places = model.kneighbors(mysteries)
for i in range(len(guesses)):
if distances[i][0] > 10:
print(str(list(mysteries.iloc[i])) + ": not like any penguin in the data")
else:
print(str(list(mysteries.iloc[i])) + ": " + guesses[i] + ", " + str(round(max(chances[i]) * 100)) + "% sure")
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
penguins = pd.read_csv("penguins.csv").dropna()
for target in ["species", "sex", "island"]:
X = penguins[["bill_length_mm", "flipper_length_mm"]]
y = penguins[target]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=1)
model = KNeighborsClassifier(n_neighbors=5).fit(X_train, y_train)
print(target + ": " + str(round(model.score(X_test, y_test) * 100)) + "%")
Stretch: Find the island model's mistakes and explain them (some species live on more than one island).
Support: Give students the kneighbors line, and ask them to write the if.
Why can a model be 100% sure and still wrong?
Its confidence only compares the answers it knows (here, the votes of its nearest examples); it does not know when an input is unlike anything it has seen.
The micro:bit and the computer course trains Teachable Machine models; the same limits apply. Discuss large language models with the same three questions: what data, how tested, what it cannot do.