Skip to content
TESA Open Knowledge

Hands-on: capture a balanced dataset on the board, split it on the PC

Module 5 — Training and deploying to several targets · Slides: slides.md · Module overview · Course page

Fill four points in s11_dataset.py to read the IMU, write the CSV, count per class, and steer you toward the class that’s still short, until the board gives a balanced three-class /gestures.csv, then copy it over and split it into train/val/test on the PC, proving with np.bincount that every set holds every class.

Objectives

Section titled “Objectives”

By the end of this lesson, you will:

  1. Fill the four points in practice/s11_dataset.py so that pressing a label button adds 200 new, non-zero lines to /gestures.csv, the balance bars move, and the hint names the class with the fewest samples.
  2. Capture a dataset on the board with all three classes reaching target, then on the PC run load_csv → make_windows → split → normalize in that order, and show an np.bincount for all three sets with every class present.
  3. Deliberately capture an unbalanced dataset, and use the numbers to explain why a stratified split carries the imbalance into every set, and why it must be fixed at capture time.

Before you start

Section titled “Before you start”

You’ve been through lesson 5.1, and understand class balance, stratified splitting, and the no-leakage rule. Have a PC with Python 3, numpy, and dataset_tools.py ready, and a board if you’ll capture real data.

Concepts

Section titled “Concepts”

The whole file reads as one sentence: choose a label → read the IMU in a burst → write it to CSV → count per class → say which class is still short → every class at target means the dataset is ready. The skeleton is still the same four beats (import → create once → loop → ui.poll). What’s new is inside record(). The first two points are carried over from the DAQ logger: (1) ax, ay, az, gx, gy, gz = sensors.bmi270.motion(), giving accel in m/s² (az around 9.81 at rest) and gyro in deg/s (2) f.write("%s,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f\n" % (label, ax, ay, az, gx, gy, gz)), whose column order must match CHANNELS, and whose label must spell exactly like CLASSES, since load_csv() uses CLASSES.index(label) — spelling Idle with a capital already errors out.

The last two points are the brains of dataset engineering: (3) counts[label] += BURST after finishing a burst, which is what makes the balance bars move, and (4) fewest = min(counts, key=counts.get), which loops over every key and returns the class name with the smallest count, letting the program say what to collect next. Record at 20 ms per sample (50 Hz), matching the Motion model — at a different rate, the same motion stretches or shrinks inside a window.

Once you’ve captured enough, copy /gestures.csv off the board (BENTO IDE’s file transfer, or mpremote) to data/gestures.csv on the PC, then call load_csv → make_windows → split → normalize(Xtr, Xva, Xte). Always split before normalize. Finish with a report of np.bincount for all three sets. If any set is missing a class — for example, test showing [20 20 0] — it means shaking wasn’t captured enough, and you need to go back and capture more on the board.

Worked example

Section titled “Worked example”

s11_dataset_full.py adds a red warning when the smallest class drops below 70% of the largest (BALANCE_TOL = 0.30), shows the min/max balance ratio, estimates the window count using the same formula as make_windows(), and has a Clear button. Worth knowing: the counters start at zero every time it runs, even if the existing file already has data in it.

File What this file teaches
examples/s11_dataset_full.py Recording a balanced, train-ready IMU dataset to CSV (full version)

This lesson’s slides also reference files in another lesson and under shared/:

Practice

Section titled “Practice”

The # TODO: comments are at lines 67 (finding fewest), 84 (reading motion()), 88 (f.write), and 93 (counts[label] += BURST). If the CSV is all 0.0, point 84 is still empty. If the bars never move, point 93 is still empty. If the hint always points at idle, point 67 is still empty.

Practice file Topic
practice/s11_dataset.py Recording an IMU dataset that’s “balanced and ready to train” to CSV (the fill-in-the-code version)

Solution

Section titled “Solution”

Open the solution after trying on your own at least once, and read how to use the solutions first.

Solution Pairs with
solution/s11_dataset.py practice/s11_dataset.py

Check your understanding

Section titled “Check your understanding”

The same questions are in quiz.yaml for automated checking.

  1. You press a label button, and the balance bar never moves, even though the file gains lines. Which point is still empty? (single choice · objective 1)

    • a) ax, ay, az, gx, gy, gz = sensors.bmi270.motion()
    • b) f.write(…)
    • c) counts[label] += BURST
    • d) fewest = min(counts, key=counts.get)
    Solution

    c — the bar reads from counts. If BURST is never added to the counter, the file grows but the program has no idea how much was captured — capturing blind.

  2. counts = {‘idle’: 600, ‘circle’: 200, ‘shaking’: 400}. What does min(counts, key=counts.get) return? (single choice · objective 1)

    • a) 200
    • b) ‘circle’
    • c) ‘idle’
    • d) (‘circle’, 200)
    Solution

    b — min loops over the dict’s keys but compares using counts.get, so it returns the class name with the smallest count, not the number. The hint therefore says to capture more circle.

  3. Put the PC-side steps in order so there’s no data leakage (ordering · objective 2)

    • a) split(X, y)
    • b) load_csv(“data/gestures.csv”)
    • c) normalize(Xtr, Xva, Xte)
    • d) make_windows(s, l)
    Solution

    b → d → a → c — read the file → slice into windows → split into sets → normalize using train’s statistics. If normalize ran before split, test’s statistics would leak in.

  4. You capture idle at three times the rate of the other classes, then split stratified. What’s the result? (single choice · objective 3)

    • a) The split automatically balances every set
    • b) Every set gets idle at roughly three times the other classes too — the imbalance follows into every set
    • c) test ends up with no idle at all
    • d) The program errors out
    Solution

    b — a stratified split preserves each class’s original proportion in every set, so it never fixes imbalance. It has to be fixed at capture time.

Lab

Section titled “Lab”

The MVP for lessons 5.1–5.2: a clean, balanced, split dataset from the board — every set (train/val/test) has all three classes in close-to-equal proportion.

  • All four points in the practice file are filled in. Try it on the emulator first, then capture real data on the board until all three classes reach TARGET.
  • Copy gestures.csv to the PC, run split, and print np.bincount for all three sets into your learning log.
  • Capture another file with idle deliberately at three times the other classes, compare the proportions across the three sets, and explain why the imbalance doesn’t disappear on its own.
  • Be able to explain why balance matters, why three sets are needed, and why normalize must fit on train only.

Going further

Section titled “Going further”

In the next pair of lessons (5.3–5.5), we’ll take this dataset and train it into a real model with TensorFlow in Docker, then export it as an int8 .tflite.

Next lesson: lesson 5.3 — Training a model in Docker: one artifact, four targets

Reflect

Section titled “Reflect”
  • How many windows does your dataset hold in total? Do you think that’s enough for a small model, and how would you know?
  • If you had to add a fourth class, such as tap, which files would you need to change, on both the board side and the PC side?

Review questions

Answer on your own first, then open the answer.

  1. You press a label button; the file grows but the balance bar never moves. Which point is still empty? (Objective 1)

    1. ax, ay, az, gx, gy, gz = sensors.bmi270.motion()
    2. f.write(...)
    3. counts[label] += BURST
    4. fewest = min(counts, key=counts.get)
    Show answer

    Answer: C. counts[label] += BURST

    แถบอ่านค่าจาก counts ถ้าไม่บวก BURST เข้าตัวนับ ไฟล์จะโตแต่โปรแกรมไม่รู้ว่าเก็บไปเท่าไร เป็นการเก็บแบบตาบอด

  2. With counts = {'idle': 600, 'circle': 200, 'shaking': 400}, what is min(counts, key=counts.get)? (Objective 1)

    1. 200
    2. 'circle'
    3. 'idle'
    4. ('circle', 200)
    Show answer

    Answer: B. 'circle'

    min วนคีย์ของ dict แต่เทียบด้วย counts.get จึงคืนชื่อคลาสที่ค่าน้อยสุด ไม่ใช่ตัวเลข คำใบ้จึงบอกให้เก็บ circle เพิ่ม

  3. Order the PC steps so that no data leaks. (Objective 2)

    1. split(X, y)
    2. load_csv("data/gestures.csv")
    3. normalize(Xtr, Xva, Xte)
    4. make_windows(s, l)
    Show answer

    Correct order: B. load_csv("data/gestures.csv") → D. make_windows(s, l) → A. split(X, y) → C. normalize(Xtr, Xva, Xte)

    อ่านไฟล์ → ตัดหน้าต่าง → แบ่งกอง → normalize ด้วยสถิติของ train ถ้า normalize ก่อน split สถิติของ test จะรั่วเข้าไป

  4. You capture three times more idle than the other classes and split with stratification. What happens? (Objective 3)

    1. split ปรับให้ทุกกองสมดุลเอง
    2. ทุกกองได้ idle มากกว่าคลาสอื่นราวสามเท่าเหมือนกัน ความไม่สมดุลติดไปทุกกอง
    3. test ไม่มี idle เลย
    4. โปรแกรม error
    Show answer

    Answer: B. ทุกกองได้ idle มากกว่าคลาสอื่นราวสามเท่าเหมือนกัน ความไม่สมดุลติดไปทุกกอง

    stratified รักษาสัดส่วนเดิมของแต่ละคลาสในทุกกอง จึงไม่แก้ความไม่สมดุล ต้องแก้ที่ตอนเก็บ

Cite this lesson

If you teach from this lesson or reuse it in slides or documents, credit it with the text below. If you changed it, add (adapted) after the title.

"Hands-on: capture a balanced dataset on the board, split it on the PC" from TESA Open Knowledge by the Thai Embedded Systems Association (TESA), https://github.com/tesaiot/tesa-qualification-program, licensed under CC BY-NC 4.0

Thai attribution: "ลงมือทำ: เก็บ dataset ที่สมดุลบนบอร์ดแล้วแบ่งบน PC" จาก TESA Open Knowledge โดยสมาคมสมองกลฝังตัวไทย (Thai Embedded Systems Association: TESA) https://github.com/tesaiot/tesa-qualification-program สัญญาอนุญาต CC BY-NC 4.0

Lesson link: https://tesaiot.github.io/tesa-qualification-program/en/courses/edge-ai-developer/m05-training/l02-dataset-lab/

Full guide: how to cite TESA

TESA Open Knowledge · © 2026 สมาคมสมองกลฝังตัวไทย (TESA) · CC BY-NC 4.0

Content is licensed CC BY-NC 4.0. Reuse it non-commercially and credit the Thai Embedded Systems Association (TESA) every time. · How to cite TESA