Skip to content
TESA Open Knowledge

Audio and several sensors on one timeline: 16 kHz PDM, timestamps and jitter

Module 2 — Collecting sensor data (DAQ) · Slides: slides.md · Module overview · Course page

Add a second sensor to the dataset. Learn how the PDM microphone delivers 16 kHz PCM sound, reduce an audio frame to a single dBFS value, then tie sound and the IMU together onto one timeline with ticks_ms and check for jitter.

Objectives

Section titled “Objectives”

By the end of this lesson, you will:

  1. Explain the path PDM → 16-bit PCM → RMS → dBFS, and compute a frame’s dBFS from its RMS (for example, RMS = 3277 gives about −20 dBFS).
  2. Compute the audio sample spacing Ts = 1/fs and one frame’s length t = N/fs, and state how a frame longer than the row period affects jitter.
  3. Explain why you must “read together, write one row, share one timestamp,” and use time.ticks_diff(time.ticks_ms(), t0) instead of plain subtraction.

Before you start

Section titled “Before you start”

You’ve been through lessons 2.1–2.2, have your own /gestures.csv file, and understand DAQ’s four beats. If you’ll use real audio, make sure you have a board whose PDM microphone works (see the hardware line below).

  • Hardware: a TESAIoT Dev Kit board already flashed with BENTO’s MicroPython firmware, or the BENTO Emulator inside BENTO IDE — real audio must come from the board’s PDM microphone. The 06/07 audio examples state they work on the PSoC Edge AI Kit; on the TESAIoT Dev Kit (which has an audio codec), enabling PDM still conflicts with the audio system’s clock. The emulator has PDM_PCM that delivers synthetic audio, only good for trying the program’s skeleton.
  • Prior lesson: lesson 2.2 — Hands-on: a DAQ logger that writes a CSV dataset

See it work first

Section titled “See it work first”

Run 06_mic_level_meter.py and speak or clap into the mic — the VU bar and the dBFS number follow along, sound turning into real numbers before your eyes. Then look at 07_mic_record_wav.py, which records the whole raw audio block into a .wav file. These two are the raw material we’ll tie to the IMU.

Concepts

Section titled “Concepts”

The board’s microphone is PDM, sending high-speed 0/1 bits. The machine.PDM_PCM hardware converts these into 16-bit PCM audio samples at sample_rate=16000, which we read into array.array("h", ...) with pdm.readinto(buf). The sample spacing is $T_s = 1/f_s = 62.5\ \mu s$, and 16 kHz is enough for speech and environmental sound, whose energy mostly sits below 8 kHz, per the Nyquist rule. We read in chunks of CHUNK = 512 samples, so one frame lasts $t = N/f_s = 512/16000 = 32$ ms.

Recording every single sample to CSV isn’t feasible (16,000 values per second), so we reduce a frame to a single value: find the RMS (the wave’s average magnitude), then compare it against the 16-bit maximum in decibels, $\text{dBFS} = 20\log_{10}(\text{RMS}/32768)$ — 0 is the loudest, and more negative means quieter. The whole raw audio block can still be saved separately as WAV if needed.

The problem with two sensors is syncing: if they’re recorded to separate files on separate schedules, there’s no way to pair which sound goes with which motion. The fix is to read together, write one row, share one timestamp: in a single loop round, read the IMU and the MIC back to back, then write one row where both use the same t_ms. t_ms = time.ticks_diff(time.ticks_ms(), t0) uses ticks_diff because the millisecond counter can wrap around back to zero. The real interval between rows will drift somewhat from 20 ms, since reading audio and writing the file both take time — this is called jitter. Because we keep the real t_ms on every row, we can check for it after the fact, and reduce jitter by shrinking CHUNK, trimming other work in the loop, and not printing to the console too often. The file’s schema is t_ms,label,ax,ay,az,gx,gy,gz,db, with a label on every row, so the dataset is ready to train the moment it leaves the board.

Worked example

Section titled “Worked example”

Predict before running 06_mic_level_meter.py: roughly how different will the dBFS be between silence and a clap? Then run it and compare. 07_mic_record_wav.py produces a /rec.wav file you can pull off with BENTO IDE (file transfer) or mpremote, then listen to on your PC.

File What this file teaches
examples/06_mic_level_meter.py VU Meter: PDM mic -> RMS -> dBFS -> a level bar with peak hold
examples/07_mic_record_wav.py Recording audio to a .wav file, with an on-screen countdown

This lesson’s slides also reference files in other lessons:

Check your understanding

Section titled “Check your understanding”

The same questions are in quiz.yaml for automated checking.

  1. An audio frame has RMS = 32768, equal to the 16-bit maximum. What’s the dBFS? (single choice · objective 1)

    • a) 0 dBFS
    • b) −96 dBFS
    • c) +20 dBFS
    • d) 32768 dBFS
    Solution

    a — 20·log10(32768/32768) = 20·log10(1) = 0 dBFS, the loudest possible. Quieter sound gives a negative value; −96 is the value the code uses to represent complete silence.

  2. Put the audio path in order, from the microphone to the db column in the CSV (ordering · objective 1)

    • a) Compute the frame’s RMS
    • b) High-speed PDM 0/1 bits
    • c) Convert to dBFS and write it to the row
    • d) 16-bit PCM samples in buf from pdm.readinto()
    Solution

    b → d → a → c — PDM → PCM in a buffer → RMS → dBFS as one value per frame, which is the first step before building a spectrogram in module 4.

  3. At fs = 16 kHz, how long is a frame of N = 512 samples? (single choice · objective 2)

    • a) 3.2 ms
    • b) 32 ms
    • c) 62.5 µs
    • d) 512 ms
    Solution

    b — t = N / fs = 512 / 16000 = 0.032 seconds = 32 ms, longer than the 20 ms row period. Reading audio is therefore the main cause of jitter.

  4. Which approach guarantees audio and IMU values in a dataset are correctly paired by time? (single choice · objective 3)

    • a) Record audio and the IMU to separate files, then pair them afterward by line order
    • b) Read both sensors in the same loop round, then write one row sharing one t_ms
    • c) Use time.time() to record a separate timestamp for each file
    • d) Set RATE_MS to 0
    Solution

    b — one round = one row = one t_ms, so every column in a row refers to the same instant. Pairing by order easily breaks when the rates differ.

  5. Why use time.ticks_diff(time.ticks_ms(), t0) instead of time.ticks_ms() - t0? (single choice · objective 3)

    • a) Because ticks_diff is faster
    • b) Because the millisecond counter can wrap back to zero; ticks_diff computes the interval correctly even when the counter wraps
    • c) Because ticks_ms returns a value in seconds
    • d) Because subtraction isn’t possible in MicroPython
    Solution

    b — ticks_ms and ticks_diff are MicroPython’s standard pair for safely measuring short intervals, immune to wrap-around.

Lab

Section titled “Lab”
  • Run the VU meter and note the dBFS values for silence, speaking, and a clap, in your learning log.
  • Compute t = N/fs for CHUNK = 256 and 1024, and say which one risks pushing rows further than 20 ms apart.
  • Write the schema for a multi-sensor file by hand, and say which call each column comes from.

Going further

Section titled “Going further”

In lesson 2.4, we’ll fill in the s05_multicapture.py file so it records the IMU and audio onto one timeline in /multicapture.csv.

Next lesson: lesson 2.4 — Hands-on: recording the IMU and audio into one file

Reflect

Section titled “Reflect”
  • What real-world job needs to look at sound and motion together to make the right call?
  • If you’d recorded audio and the IMU to separate files, could you still recover their timing alignment afterward? What extra data would you need?

Review questions

Answer on your own first, then open the answer.

  1. An audio frame has RMS = 32768, the 16-bit maximum. What is its dBFS? (Objective 1)

    1. 0 dBFS
    2. −96 dBFS
    3. +20 dBFS
    4. 32768 dBFS
    Show answer

    Answer: A. 0 dBFS

    20·log10(32768/32768) = 20·log10(1) = 0 dBFS คือดังสุด เสียงที่เบากว่านี้จะได้ค่าติดลบ ส่วน −96 คือค่าที่โค้ดใช้แทนความเงียบสนิท

  2. Order the path of sound from the microphone to the db column in the CSV. (Objective 1)

    1. หา RMS ของเฟรม
    2. บิต PDM 0/1 ความเร็วสูง
    3. แปลงเป็น dBFS แล้วเขียนลงแถว
    4. ตัวอย่าง PCM 16-bit ใน buf จาก pdm.readinto()
    Show answer

    Correct order: B. บิต PDM 0/1 ความเร็วสูง → D. ตัวอย่าง PCM 16-bit ใน buf จาก pdm.readinto() → A. หา RMS ของเฟรม → C. แปลงเป็น dBFS แล้วเขียนลงแถว

    PDM → PCM ใน buffer → RMS → dBFS เป็นค่าเดียวต่อเฟรม ซึ่งเป็นก้าวแรกก่อนทำ spectrogram ในโมดูล 4

  3. At fs = 16 kHz, how long is a frame of N = 512 samples? (Objective 2)

    1. 3.2 ms
    2. 32 ms
    3. 62.5 µs
    4. 512 ms
    Show answer

    Answer: B. 32 ms

    t = N / fs = 512 / 16000 = 0.032 วินาที = 32 ms ยาวกว่าจังหวะแถว 20 ms การอ่านเสียงจึงเป็นตัวการหลักของ jitter

  4. Which method guarantees that sound and IMU values in the dataset can be paired by time? (Objective 3)

    1. เก็บเสียงกับ IMU คนละไฟล์แล้วจับคู่ตามลำดับบรรทัดทีหลัง
    2. อ่านทั้งสองเซนเซอร์ในรอบเดียวของลูป แล้วเขียนแถวเดียวที่มี t_ms ร่วมกัน
    3. ใช้ time.time() บันทึกเวลาแยกให้แต่ละไฟล์
    4. ตั้ง RATE_MS ให้เท่ากับ 0
    Show answer

    Answer: B. อ่านทั้งสองเซนเซอร์ในรอบเดียวของลูป แล้วเขียนแถวเดียวที่มี t_ms ร่วมกัน

    หนึ่งรอบ = หนึ่งแถว = หนึ่ง t_ms ทุกคอลัมน์ของแถวจึงหมายถึงช่วงเวลาเดียวกัน การจับคู่ตามลำดับพลาดง่ายเมื่ออัตราต่างกัน

  5. Why use time.ticks_diff(time.ticks_ms(), t0) instead of time.ticks_ms() - t0? (Objective 3)

    1. เพราะ ticks_diff เร็วกว่า
    2. เพราะตัวนับมิลลิวินาทีวนกลับศูนย์ได้ (wrap-around) ticks_diff คิดช่วงเวลาให้ถูกแม้ตัวนับวนรอบ
    3. เพราะ ticks_ms คืนค่าเป็นวินาที
    4. เพราะการลบใน MicroPython ทำไม่ได้
    Show answer

    Answer: B. เพราะตัวนับมิลลิวินาทีวนกลับศูนย์ได้ (wrap-around) ticks_diff คิดช่วงเวลาให้ถูกแม้ตัวนับวนรอบ

    ticks_ms กับ ticks_diff คือคู่มาตรฐานของ MicroPython สำหรับวัดช่วงเวลาสั้น ๆ อย่างปลอดภัยจาก wrap-around

Cite this lesson

If you teach from this lesson or reuse it in slides or documents, credit it with the text below. If you changed it, add (adapted) after the title.

"Audio and several sensors on one timeline: 16 kHz PDM, timestamps and jitter" from TESA Open Knowledge by the Thai Embedded Systems Association (TESA), https://github.com/tesaiot/tesa-qualification-program, licensed under CC BY-NC 4.0

Thai attribution: "เสียงและหลายเซนเซอร์บนเส้นเวลาเดียว: PDM 16 kHz ประทับเวลา และ jitter" จาก TESA Open Knowledge โดยสมาคมสมองกลฝังตัวไทย (Thai Embedded Systems Association: TESA) https://github.com/tesaiot/tesa-qualification-program สัญญาอนุญาต CC BY-NC 4.0

Lesson link: https://tesaiot.github.io/tesa-qualification-program/en/courses/edge-ai-developer/m02-daq/l03-audio-and-timeline/

Full guide: how to cite TESA

TESA Open Knowledge · © 2026 สมาคมสมองกลฝังตัวไทย (TESA) · CC BY-NC 4.0

Content is licensed CC BY-NC 4.0. Reuse it non-commercially and credit the Thai Embedded Systems Association (TESA) every time. · How to cite TESA