Skip to content
TESA Open Knowledge

Sensor fusion: the model's verdict with the raw sensor

Module 6 — Edge AI apps · Slides: slides.md · Module overview · Course page

Make decisions more trustworthy with sensor fusion: confirm a model’s verdict (what it is) with a raw sensor (how strong it is). Understand AND-style corroboration and k-of-n majority voting as one weighted formula, and study a three-sensor intruder alarm.

Objectives

Section titled “Objectives”

By the end of this lesson, you will:

  1. Explain why a single model’s verdict isn’t enough, and how a model and a raw sensor answer different questions.
  2. Compute a k-of-n vote and a weighted decision S = Σ wᵢsᵢ ≥ θ, and show that AND is the case k = n.
  3. Write a raw-sensor gate gmag = |gx| + |gy| + |gz| > MOTION_FLOOR and the condition fused = model_hit and raw_ok, with an edge trigger.

Before you start

Section titled “Before you start”

You’ve been through lessons 6.3–6.4, have an action pipeline that resists false positives, and remember the rule classifier from lesson 3.3. Open the 10_motion_alarm.py example in BENTO IDE.

  • Hardware: a TESAIoT Dev Kit board already flashed with BENTO’s MicroPython firmware, or the BENTO Emulator inside BENTO IDE — on the emulator, the HW panel only moves the accelerometer (drag to tilt, and the Shake button); the gyro stays near zero. Example 10, which opens the PDM microphone, doesn’t yet run on the TESAIoT Dev Kit (it works on the emulator and the PSoC Edge AI Kit).
  • Prior lesson: lesson 6.4 — Hands-on: an action pipeline that resists false positives

See it work first

Section titled “See it work first”

Run 10_motion_alarm.py, press the Arm switch, then try shaking one sensor at a time, and several at once. Notice it doesn’t alert every time a single sensor moves — it needs two of three votes before showing !! INTRUDER !! (on the emulator, the Shake button lets the radar vote, but this example’s IMU reads the gyro, which the emulator keeps stuck near zero).

Concepts

Section titled “Concepts”

A model can guess wrong with fairly high confidence — for example, answering shaking at 62% when the board was just set down a bit hard. Sensor fusion combines several sources into one better decision. The model (edge_ai, running on the CM55 with the NPU) answers “what is this?” as a probability, while a raw sensor (sensors, read from Python on the CM33) answers “how strong is it?” in physical units. Each fails in its own way, so corroborating them together resists false positives better — the same way a car uses a camera, radar, and lidar together before braking.

Two flavours of fusion come up often. Corroboration (AND): one primary signal plus one confirming gate, firing only when both pass — suited to “don’t alert unless you’re sure.” And majority vote $\text{fire} = [\sum_{i=1}^{n} s_i \ge k]$, as in 10_motion_alarm.py, which votes two of three among radar, IMU, and microphone, tolerating one sensor missing. Both are the same formula: $S = \sum w_i s_i$ and $\text{fire} = [S \ge \theta]$. When $w_i = 1$ and $\theta = n$, that’s AND; lowering $\theta$ relaxes it into a vote. Raising the $w_i$ of a trusted sensor gives it a louder voice.

Our confirming gate is the rule classifier from lesson 3.3: ax, ay, az, gx, gy, gz = sensors.bmi270.motion() (accel in m/s², gyro in deg/s), then gmag = abs(gx) + abs(gy) + abs(gz) with raw_ok = gmag > MOTION_FLOOR (default 40), combined as fused = model_hit and raw_ok, where model_hit = r['label'] == TARGET_CLASS and r['conf'] >= edge_ai.CONF_FLOOR. The event then fires on the rising edge with a fired flag, so it isn’t sent again every frame. Shake hard but the model answers idle, and it doesn’t fire; the model answers shaking but the gyro is light, and it doesn’t fire either.

Worked example

Section titled “Worked example”

10_motion_alarm.py is a three-sensor intruder alarm (radar via sensors.radar()["presence"], IMU, and a sound level from PDM), with an on-screen Arm switch. It’s a state machine, DISARMED → ARMED → TRIGGERED, voting two of three, and updating the screen only when the state changes. The header notes that on the TESAIoT Dev Kit, opening PDM still clashes with the audio system’s clock, so it only works on the PSoC Edge AI Kit and the emulator.

File What this file teaches
examples/10_motion_alarm.py A 3-sensor intruder alarm + an on-screen arm/disarm switch

This lesson’s slides also reference files in another lesson or in shared/:

Check your understanding

Section titled “Check your understanding”

The same questions are in quiz.yaml for automated checking.

  1. Which statement correctly describes the roles of the model and the raw sensor in fusion? (single choice · objective 1)

    • a) The model says what this is (a probability); the raw sensor says how strong it is (physical units)
    • b) Both say the same thing, so they’re interchangeable
    • c) The raw sensor is always more accurate than the model
    • d) The model reads physical values better than the raw sensor
    Solution

    a — the two sources answer different questions and fail in different ways. Corroborating them together resists false positives.

  2. A two-of-three voting system: radar sees it, IMU is quiet, the mic sees it. What’s the result? (single choice · objective 2)

    • a) No alert, because the IMU is quiet
    • b) Alert, because the total 2 ≥ k = 2
    • c) Undetermined
    • d) Alert only if all three see it
    Solution

    b — Σsᵢ = 1 + 0 + 1 = 2, which meets the threshold k = 2. Voting tolerates one sensor missing; requiring all three would be k = n = AND.

  3. Two signals, equal weights wᵢ = 1. What θ makes this equal to AND? (single choice · objective 2)

    • a) θ = 0
    • b) θ = 1
    • c) θ = 2
    • d) θ = 0.5
    Solution

    c — θ equal to the sum of weights means every signal must pass. If θ = 1, it becomes an OR — passing just one fires it.

  4. The model answers shaking at conf 0.8, but gx, gy, gz = 5, 10, 8, and MOTION_FLOOR = 40. What is fused? (single choice · objective 3)

    • a) True, because the model is confident
    • b) False, because gmag = 23 doesn’t exceed 40 — the raw gate fails
    • c) True, because gmag > 0
    • d) An error
    Solution

    b — fusion uses AND. One stage failing means it doesn’t fire. This is exactly the model’s false positive that fusion filters out.

Lab

Section titled “Lab”
  • Run 10_motion_alarm.py (on the PSoC Edge AI Kit or the emulator), and note which action makes which sensor vote, and which doesn’t trigger an alert.
  • Compute S and the firing result for three cases in your learning log, giving radar weight 2, IMU weight 1, mic weight 1, and θ = 3.
  • Write the fused condition out by hand, and identify one move where the model would likely answer shaking but the gyro gate doesn’t pass.

Going further

Section titled “Going further”

In lesson 6.6, we’ll fill in s17_fusion_iot.py to fuse a verdict with the raw gyro, then send the event to an MQTT broker over WiFi.

Next lesson: lesson 6.6 — Hands-on: publishing a fused event to MQTT

Reflect

Section titled “Reflect”
  • Which systems around you should use AND, and which should use voting, based on which kind of mistake is more costly?
  • If one sensor in a voting system fails permanently, how does the system’s behaviour change?

Review questions

Answer on your own first, then open the answer.

  1. Which statement describes the roles of the model and the raw sensor in fusion correctly? (Objective 1)

    1. โมเดลบอกว่านี่คืออะไร (ความน่าจะเป็น) เซนเซอร์ดิบบอกว่าแรงแค่ไหน (หน่วยฟิสิกส์)
    2. ทั้งคู่บอกสิ่งเดียวกัน จึงใช้แทนกันได้
    3. เซนเซอร์ดิบแม่นกว่าโมเดลเสมอ
    4. โมเดลอ่านค่าฟิสิกส์ได้ดีกว่าเซนเซอร์ดิบ
    Show answer

    Answer: A. โมเดลบอกว่านี่คืออะไร (ความน่าจะเป็น) เซนเซอร์ดิบบอกว่าแรงแค่ไหน (หน่วยฟิสิกส์)

    สองแหล่งตอบคนละคำถามและพลาดคนละแบบ เอามายืนยันกันจึงกัน false positive ได้

  2. A two-of-three vote: radar sees, IMU is quiet, mic sees. What is the result? (Objective 2)

    1. ไม่ปลุก เพราะ IMU เงียบ
    2. ปลุก เพราะผลรวม 2 ≥ k = 2
    3. ไม่แน่นอน
    4. ปลุกก็ต่อเมื่อทั้งสามเห็น
    Show answer

    Answer: B. ปลุก เพราะผลรวม 2 ≥ k = 2

    Σsᵢ = 1 + 0 + 1 = 2 ผ่านเกณฑ์ k = 2 การโหวตยอมให้เซนเซอร์ตัวหนึ่งพลาดได้ ถ้าต้องครบทั้งสามคือ k = n = AND

  3. Two signals with equal weights wᵢ = 1. Which θ makes it an AND? (Objective 2)

    1. θ = 0
    2. θ = 1
    3. θ = 2
    4. θ = 0.5
    Show answer

    Answer: C. θ = 2

    θ เท่าผลรวมน้ำหนักคือต้องผ่านทุกตัว ถ้า θ = 1 จะกลายเป็น OR ผ่านตัวเดียวก็ยิง

  4. The model says shaking at conf 0.8, but gx, gy, gz = 5, 10, 8 and MOTION_FLOOR = 40. What is fused? (Objective 3)

    1. True เพราะโมเดลมั่นใจ
    2. False เพราะ gmag = 23 ไม่เกิน 40 ประตูดิบไม่ผ่าน
    3. True เพราะ gmag > 0
    4. error
    Show answer

    Answer: B. False เพราะ gmag = 23 ไม่เกิน 40 ประตูดิบไม่ผ่าน

    fusion ใช้ AND ด่านเดียวไม่ผ่านก็ไม่ยิง นี่คือ false positive ของโมเดลที่ fusion กรองออก

Cite this lesson

If you teach from this lesson or reuse it in slides or documents, credit it with the text below. If you changed it, add (adapted) after the title.

"Sensor fusion: the model's verdict with the raw sensor" from TESA Open Knowledge by the Thai Embedded Systems Association (TESA), https://github.com/tesaiot/tesa-qualification-program, licensed under CC BY-NC 4.0

Thai attribution: "sensor fusion: verdict ของโมเดลกับเซนเซอร์ดิบ" จาก TESA Open Knowledge โดยสมาคมสมองกลฝังตัวไทย (Thai Embedded Systems Association: TESA) https://github.com/tesaiot/tesa-qualification-program สัญญาอนุญาต CC BY-NC 4.0

Lesson link: https://tesaiot.github.io/tesa-qualification-program/en/courses/edge-ai-developer/m06-apps/l05-sensor-fusion/

Full guide: how to cite TESA

TESA Open Knowledge · © 2026 สมาคมสมองกลฝังตัวไทย (TESA) · CC BY-NC 4.0

Content is licensed CC BY-NC 4.0. Reuse it non-commercially and credit the Thai Embedded Systems Association (TESA) every time. · How to cite TESA