Skip to content
TESA Open Knowledge

Hands-on: IMU and sound in one file

Module 2 — Collecting sensor data (DAQ) · Slides: slides.md · Module overview · Course page

Fill the four loop steps of s05_multicapture.py — timestamp, read IMU, read sound, write the row — until /multicapture.csv ties the IMU and sound level to the same t_ms on every row, then check the file and the jitter yourself.

Objectives

Section titled “Objectives”

By the end of this lesson, you will:

  1. Fill the four blanks in practice/s05_multicapture.py so one label press adds 200 rows to /multicapture.csv, complete with t_ms, label, ax..gz and db columns.
  2. Check a recorded file: t_ms rises within each burst, about 20 ms apart, db follows real sound, and diagnose which blank is empty from the symptoms in the file.
  3. Explain why the PDM is opened once outside the loop, and released with pdm.deinit() in the finally block.

Before you start

Section titled “Before you start”

You’ve been through lesson 2.3, and understand dBFS, a shared timeline, and jitter. Have a board that can record audio ready, and keep the REPL open to check the file once you’re done recording.

  • Hardware: a TESAIoT Dev Kit board already flashed with BENTO’s MicroPython firmware, or the BENTO Emulator inside BENTO IDE — recording real audio needs a board whose PDM microphone works (the author tested on the PSoC Edge AI Kit; on the TESAIoT Dev Kit, enabling PDM still conflicts with the audio system’s clock). The emulator gives synthetic audio, good for practising the program’s skeleton, but passing on the emulator is not evidence it will pass on the board.
  • Prior lesson: lesson 2.3 — Audio and several sensors on one timeline: 16 kHz PDM, timestamps and jitter

Concepts

Section titled “Concepts”

The whole file’s heart is in record(). Each round of for _ in range(BURST) follows exactly four steps: (1) timestamp — t_ms = time.ticks_diff(time.ticks_ms(), t0) (2) read the IMU — ax, ay, az, gx, gy, gz = sensors.bmi270.motion() (3) read sound — pdm.readinto(buf), then db = dbfs(buf), already given (4) write the row — f.write("%d,%s,%.4f,%.4f,%.4f,%.4f,%.4f,%.4f,%.1f\n" % (t_ms, label, ax, ay, az, gx, gy, gz, db)). All four steps happen in one round, before time.sleep_ms(RATE_MS), so they count as the same instant. Order matters: always read both sensors before writing the row, or the data drifts a round out of alignment.

The top of the file is already given: it opens PDM_PCM(0, sck="P8_5", data="P8_6", sample_rate=16000) once, outside the loop, just like creating a widget once, because opening and closing it repeatedly both costs time and risks conflicts. It prepares a buffer and writes the header if the file doesn’t exist yet. t0 is reset every time a label button is pressed, so each burst starts counting time from its own zero. On exit, the finally block always calls pdm.deinit() to release the microphone — otherwise the next audio program might fail to open PDM because the hardware is still reserved.

Once you’ve recorded, check before trusting: the row count should equal BURST × number of presses, t_ms should rise within each burst, and db should follow real sound. If t_ms is 0 on every row, step 1 is still empty. If db never moves, step 3 is still empty, or the mic isn’t picking up sound. If the file is empty, step 4 is still empty.

Worked example

Section titled “Worked example”

s05_multicapture_full.py records both CSV and a paired raw-audio WAV, measures the real interval per row and the worst-case jitter, then summarizes the row count, peak dBFS and average interval per label into a manifest file. Open it once your practice file is working, to see what a production-grade dataset needs to record on top of this.

File What this file teaches
examples/s05_multicapture_full.py Multi-Sensor Sync Capture (full version)

Practice

Section titled “Practice”

The file has 4 # TODO spots, at lines 78, 83, 88 and 92, matching the four steps in order. Replace pass (and the zero placeholder values) with the calls the hints describe. Then Program to Device, choose a label, perform the motion and make the sound together for about four seconds, watch the row count appear on screen, then open the CSV to check.

Practice file Topic
practice/s05_multicapture.py Recording 2 sensors together on one timeline (the fill-in-the-code version)

Solution

Section titled “Solution”

Open the solution after trying on your own at least once, and read how to use the solutions first.

Solution Pairs with
solution/s05_multicapture.py practice/s05_multicapture.py

Check your understanding

Section titled “Check your understanding”

The same questions are in quiz.yaml for automated checking.

  1. Put the four steps of one round of the record() loop in order (ordering · objective 1)

    • a) pdm.readinto(buf), then db = dbfs(buf)
    • b) t_ms = time.ticks_diff(time.ticks_ms(), t0)
    • c) f.write(…) one row
    • d) ax, ay, az, gx, gy, gz = sensors.bmi270.motion()
    Solution

    b → d → a → c — timestamp → read IMU → read sound → write the row, then sleep_ms before the next round. Every value in the row is therefore the same instant.

  2. You open the CSV and find t_ms is 0 on every row, but the IMU and db values change genuinely. Which blank is still empty? (single choice · objective 2)

    • a) Step 1, timestamp
    • b) Step 2, read the IMU
    • c) Step 3, read sound
    • d) Step 4, write the row
    Solution

    a — the placeholder line t_ms = 0 is still there, so the timeline is missing, and rows can’t be paired by time. ticks_diff needs to be filled in.

  3. Every row’s db column reads −96.0, even when you clap right next to the board. What’s the likely cause? (single choice · objective 2)

    • a) pdm.readinto(buf) hasn’t been filled in yet, so the buffer is all zeros, or the microphone isn’t receiving sound
    • b) RATE_MS is set too high
    • c) Forgot \n at the end of the line
    • d) t0 was never reset
    Solution

    a — dbfs() returns −96.0 when RMS is zero, which happens when the buffer was never filled with real sound.

  4. Which of these explain why pdm.deinit() must be called in finally? (select every correct answer) (multiple choice · objective 3)

    • a) No matter whether you exit via the back button or an error, the microphone is always released
    • b) If it’s not released, the next audio program might fail to open PDM because the hardware is still reserved
    • c) deinit() is what causes the CSV file to be saved
    • d) It’s the same habit as edge_ai.stop() — always leave the machine in a known state
    Solution

    a, b, d — the CSV file is closed and flushed automatically when leaving the with block; deinit() exists to release the microphone hardware.

Lab

Section titled “Lab”

The MVP for lessons 2.3–2.4: a dataset logging at least two sensors on one shared timeline. The /multicapture.csv file has columns t_ms + IMU + db, with real values changing with motion and sound.

  • All four blanks in the practice file are filled in; running on the board produces /multicapture.csv.
  • Record all three labels, each with a different motion and sound, then confirm in the CSV that db and the IMU genuinely change.
  • Check the timeline: are t_ms values close to 20 ms apart? Note in your learning log where you found jitter, then try shrinking CHUNK and compare.
  • Be able to explain where in the code t_ms is stamped, where both sensors are read, and why that must happen in a single round.

Going further

Section titled “Going further”

In the next module (Processing), we’ll convert the raw signals we’ve collected into physical quantities — such as tilt angle, height, and sound level — and show them as live gauges.

Next lesson: lesson 3.1 — From raw numbers to physical quantities: tilt angle, energy, height and dBFS

Reflect

Section titled “Reflect”
  • When was the worst jitter you measured, and which part of the loop do you think caused it?
  • If you had to add a third sensor (such as pressure) into the same file, which parts of the code would you need to change?

Review questions

Answer on your own first, then open the answer.

  1. Order the four steps in one pass of the record() loop. (Objective 1)

    1. pdm.readinto(buf) แล้ว db = dbfs(buf)
    2. t_ms = time.ticks_diff(time.ticks_ms(), t0)
    3. f.write(...) หนึ่งแถว
    4. ax, ay, az, gx, gy, gz = sensors.bmi270.motion()
    Show answer

    Correct order: B. t_ms = time.ticks_diff(time.ticks_ms(), t0) → D. ax, ay, az, gx, gy, gz = sensors.bmi270.motion() → A. pdm.readinto(buf) แล้ว db = dbfs(buf) → C. f.write(...) หนึ่งแถว

    ประทับเวลา → อ่าน IMU → อ่านเสียง → เขียนแถว แล้วค่อย sleep_ms ไปรอบถัดไป ทุกค่าในแถวจึงเป็นเวลาเดียวกัน

  2. In the CSV every t_ms is 0 while IMU and db change properly. Which blank is empty? (Objective 2)

    1. ก้าว 1 ประทับเวลา
    2. ก้าว 2 อ่าน IMU
    3. ก้าว 3 อ่านเสียง
    4. ก้าว 4 เขียนแถว
    Show answer

    Answer: A. ก้าว 1 ประทับเวลา

    บรรทัดสำรอง t_ms = 0 ยังอยู่ เส้นเวลาจึงหาย จับคู่ตามเวลาไม่ได้ ต้องเติม ticks_diff

  3. Every db value is −96.0 even when you clap near the board. What is the likely cause? (Objective 2)

    1. ยังไม่ได้เติม pdm.readinto(buf) buffer จึงเป็นศูนย์ หรือไมโครโฟนไม่ได้รับเสียง
    2. RATE_MS ตั้งสูงเกินไป
    3. ลืม \n ท้ายบรรทัด
    4. t0 ไม่ได้ตั้งใหม่
    Show answer

    Answer: A. ยังไม่ได้เติม pdm.readinto(buf) buffer จึงเป็นศูนย์ หรือไมโครโฟนไม่ได้รับเสียง

    dbfs() คืน −96.0 เมื่อ RMS เป็นศูนย์ ซึ่งเกิดเมื่อ buffer ไม่เคยถูกเติมด้วยเสียงจริง

  4. Which are reasons to call pdm.deinit() in finally? (select all that apply) (Objective 3)

    1. ไม่ว่าจะออกด้วยปุ่ม back หรือ error ไมโครโฟนจะถูกคืนเสมอ
    2. ถ้าไม่คืน โปรแกรมเสียงตัวถัดไปอาจเปิด PDM ไม่ได้เพราะฮาร์ดแวร์ยังถูกจอง
    3. deinit() ทำให้ไฟล์ CSV ถูกบันทึก
    4. เป็นนิสัยเดียวกับ edge_ai.stop() คือทิ้งเครื่องไว้ในสถานะที่รู้แน่
    Show answer

    Answer: A. ไม่ว่าจะออกด้วยปุ่ม back หรือ error ไมโครโฟนจะถูกคืนเสมอ · B. ถ้าไม่คืน โปรแกรมเสียงตัวถัดไปอาจเปิด PDM ไม่ได้เพราะฮาร์ดแวร์ยังถูกจอง · D. เป็นนิสัยเดียวกับ edge_ai.stop() คือทิ้งเครื่องไว้ในสถานะที่รู้แน่

    ไฟล์ CSV ถูกปิดและ flush เองเมื่อออกจาก with ส่วน deinit() มีไว้คืนฮาร์ดแวร์ไมโครโฟน

Cite this lesson

If you teach from this lesson or reuse it in slides or documents, credit it with the text below. If you changed it, add (adapted) after the title.

"Hands-on: IMU and sound in one file" from TESA Open Knowledge by the Thai Embedded Systems Association (TESA), https://github.com/tesaiot/tesa-qualification-program, licensed under CC BY-NC 4.0

Thai attribution: "ลงมือทำ: เก็บ IMU กับเสียงลงไฟล์เดียว" จาก TESA Open Knowledge โดยสมาคมสมองกลฝังตัวไทย (Thai Embedded Systems Association: TESA) https://github.com/tesaiot/tesa-qualification-program สัญญาอนุญาต CC BY-NC 4.0

Lesson link: https://tesaiot.github.io/tesa-qualification-program/en/courses/edge-ai-developer/m02-daq/l04-multicapture-lab/

Full guide: how to cite TESA

TESA Open Knowledge · © 2026 สมาคมสมองกลฝังตัวไทย (TESA) · CC BY-NC 4.0

Content is licensed CC BY-NC 4.0. Reuse it non-commercially and credit the Thai Embedded Systems Association (TESA) every time. · How to cite TESA