Skip to content
TESA Open Knowledge

Telemetry Pipelines, MQTT on the Twin and Fault Injection

Companion videos

Watch on YouTube (opens in a new tab)

Videos by Asst. Prof. Dr. Santi Nuratch, Department of Control Systems and Instrumentation Engineering, Faculty of Engineering, King Mongkut's University of Technology Thonburi (KMUTT) · The whole series in the playlist AIoT Foundation

Course 2 · Module 5 Suggested time: about 4 hours (classifying data · a pipeline · an MQTT broker · a dashboard · fault injection) Format: a hands-on lesson — shaping data from the Twin/firmware, then passing it on to a broker / dashboard under controlled conditions

Lab · Lab notes · ← Table of Contents · ← M04 · M06 →

Objectives (Learning Outcomes)

Section titled “Objectives (Learning Outcomes)”

By the end of this lesson you should be able to:

  1. Explain the device data set: Telemetry, State, Event
  2. Simulate a Data Pipeline to test data formatting
  3. Send simulated data to a Cloud / external system and check it with a Dashboard
  4. Set up an MQTT Broker in the Twin / Studio environment
  5. Publish / Subscribe between the Twin (or the device) and cloud services
  6. Simulate Lossy Network / Error Injection scenarios and test under controlled conditions

This module builds on M04, where you already proved the firmware ↔ Twin has real I/O — now expand the layer of “data going outside Studio” (a Live Data consumer + MQTT), before integrating the system in M06.

Key phrase The Twin is a practice field for the data pipe — practise topics, payloads, dashboards and reconnecting before spending a whole day on the real cloud.

Read alongside this chapter

Section titled “Read alongside this chapter”
Document Use when
M04 — Co-simulation The already-proven I/O pipe · ex05 as an outer consumer
Course 1 M06 — MQTT Broker / QoS / retain / the firmware’s CM55→CM33 path
Course 1 M05 — Sensors What the values you’ll put into telemetry mean
Bitstream Studio Starting the broker · the telemetry route · the Twin’s MQTT tools
TESAIoT_Hackathon web-app/ — ex08 (route/stale) · ex09–ex15 (MQTT)
TESAIoT Developer Hub Encode / publish examples on the firmware side
HiveMQ MQTT Essentials Further reading on topic / QoS concepts
ternion-3d-assets-free 3D visualization alongside a dashboard (if used)

1. Telemetry, State, and Event

Section titled “1. Telemetry, State, and Event”

Separating data types helps design the topic, the send rate, and the dashboard so a dense stream doesn’t bury an important event.

Type Character Example in a Twin lab
Telemetry A continuous stream over time Temperature every 1 s · a BMI270 sample · DevKit Twin channels
State The system’s current status mode=idle, led=on, the Link connected, route=uart|sim
Event Occasional, when a condition becomes true threshold_exceeded, stale, reconnect, fault_injected

1.1 Design habits

Section titled “1.1 Design habits”
Habit Reason
Telemetry uses a fixed topic/rate A dashboard expects a rhythm
State is sent when it changes (or retains the latest) Reduces spam · a new subscriber knows the status immediately
Events keep a timestamp + a short reason Debugging / tracing back in the M06 E2E
Don’t put everything into one undifferentiated topic Testing and ACLs become hard

An example topic structure in the lab (adjustable to your own kit):

device/<deviceId>/devkit-twin/telemetry     ← telemetry stream
device/<deviceId>/state                      ← current mode / flags
device/<deviceId>/event/<name>               ← sparse events
lab/qos-demo                                 ← a QoS/retain test field (ex14)

A default the Hackathon web-app often uses: device/devkit-twin-01/devkit-twin/telemetry (see §5)

Key phrase Telemetry = breathing · State = the current posture · Event = an occurrence worth recording

2. Data Pipeline (Twin Lab View)

Section titled “2. Data Pipeline (Twin Lab View)”

The general sequence:

[Source]
  Firmware / Simulator / Twin virtual device
        │
        ▼
[Shape]
  JSON fields · units · mask · channels
        │
        ▼
[Transport]
  A) Live Data provider (WS) → Studio panels / web-app ex05–ex08
  B) MQTT broker             → web-app ex09–ex15 / cloud / external tools
        │
        ▼
[Observe]
  Dashboard · subscriber log · gauges

2.1 Two pipes you must not confuse

Section titled “2.1 Two pipes you must not confuse”
Pipe Use when Example consumer
Live Data (the telemetry provider) Watching a sample already decoded from the bridge · route / origin / stale ex05, ex06, ex08
MQTT (broker pub/sub) Simulating the cloud layer / an external system ex09, ex12, ex15

Both pipes may show “the same block of temperature,” but the protocol and the failure point differ.

Symptom Suspected pipe
TelemetryClient disconnected Live Data / the bridge / serving the web-app
MQTT disconnected at ws://127.0.0.1:8883/mqtt Start broker hasn’t been done in Studio yet
Live Data is fine, but MQTT is empty No publisher exists yet on that topic
MQTT has messages, but Live Data is silent Different pipes — it doesn’t mean “the sensor is dead”

2.2 Schema / format drills (pipeline test)

Section titled “2.2 Schema / format drills (pipeline test)”

Useful ways to test the pipe before going to the cloud:

  1. Send a payload with all fields → the dashboard goes green
  2. Remove a field, or rename a key → see where the UI breaks
  3. Send the wrong unit (such as °C entered as milli) → the number is off but still parses
  4. Send broken JSON → does the subscriber show a clear error, or stay silent?

Record the results in telemetry-mqtt-lab-notes.md

3. Walkthrough — ex08 Stale, Route, and Origin

Section titled “3. Walkthrough — ex08 Stale, Route, and Origin”

Before opening a broker, get familiar with the quality of the Live Data stream — continuing from M04, which used ex05 as an orientation consumer.

File: Hackathon → web-app/ex08_stale_and_route.html

3.1 What the page proves

Section titled “3.1 What the page proves”
UI Meaning
The connection route The backend the provider uses (relates to Simulator vs Bitstream)
COM open Whether a serial port is open
The last sample’s origin uart or sim — must match the mode chosen in Studio
Sensor pills + stale Whether each sensor is quiet past staleAfterMs, or still fresh
The event log Connection / sample / stale as traceable events over time

3.2 Lab steps

Section titled “3.2 Lab steps”
  1. Link Studio (Simulator or Bitstream — never mixed)
  2. Serve the web-app/ folder and open ex08
  3. Confirm connected · note the route and the origin of the latest sample
  4. Briefly stop the stream (stop the Simulator / briefly unplug the COM, as your round allows) → the pill should go stale, with an event in the log
  5. Resume streaming → the pill goes fresh again

Passes when: you can explain that stale is a host-side Event when telemetry has a gap — not a sensor value

Use ex08 as evidence of lossy / disconnect at the Live Data layer, before moving to the MQTT layer (§6)

4. MQTT Broker in the Twin Host

Section titled “4. MQTT Broker in the Twin Host”

In Course 2, the main host is Bitstream Studio — it usually has a command roughly like:

Toolbar / Server → Start broker

Then the MQTT web page in Hackathon connects to this default:

ws://127.0.0.1:8883/mqtt

(see DEFAULT_MQTT_WS_PATH in web-app/shared/ex-mqtt.js)

Check Expect
Start broker, then open the web page The broker is ready to accept a client
Open ex09 without starting the broker first It stays connecting / errors
An overridden URL ?mqtt=ws://… if the port changed

4.1 Twin / cloud roles (honest mapping)

Section titled “4.1 Twin / cloud roles (honest mapping)”
Role in the course documentation What you actually do in the lab
MQTT in the Twin Studio’s Start broker + publish from the Twin / Sensor Studio / a host tool
Cloud / an external system The web-app subscriber · or a public/LAN broker (review C1 M06)
A real device publishing Firmware on a board, behind Wi‑Fi (C1) — in M05, focus on the host pipe first if time is short

Don’t assume “Start broker” = telemetry automatically exists — there still needs to be a publisher on the topic being subscribed to.

5. Walkthrough — ex09 MQTT Subscriber (main cloud-sim example)

Section titled “5. Walkthrough — ex09 MQTT Subscriber (main cloud-sim example)”

M05’s main example for the MQTT layer: ex09_mqtt_subscriber.html

5.1 What it does

Section titled “5.1 What it does”
  1. Loads mqtt.js through Serve Web App (/@bitstream/mqtt-live-data.js)
  2. Connects to ws://127.0.0.1:8883/mqtt (or ?mqtt=)
  3. Subscribes to the default topic:
device/devkit-twin-01/devkit-twin/telemetry

Built from devkitTwinTopic(deviceId) — change it with ?device= or ?topic=

  1. Shows the last payload (JSON) · counts messages · counts channels if present

5.2 How to run (evidence path)

Section titled “5.2 How to run (evidence path)”
[Bitstream Studio] Start broker
        │
        ▼
[Publisher]
  DevKit Twin MQTT tab
  or Sensor Studio connectivity nodes
  or another publish tool
        │  topic: device/<id>/devkit-twin/telemetry
        ▼
[Broker ws://127.0.0.1:8883/mqtt]
        │
        ▼
[ex09 browser page]  →  Last payload + message count

Short steps:

  1. Start broker in Studio
  2. Serve web-app/ → open ex09
  3. Wait for the badge connected
  4. Publish from the Twin / a host to the topic the web page subscribes to
  5. Take a screenshot of Last payload, alongside the publish source

Passes when: there is at least one JSON message whose fields match what you intended to send (the units and key names)

5.3 Teaching view of the page code

Section titled “5.3 Teaching view of the page code”

The main concept (see the real file in Hackathon):

connectMqtt(mqtt, MQTT_URL)
  → onConnect → client.subscribe(TOPIC)
  → on("message") → JSON.parse → update payload + chips

wireMqttState manages connected / reconnecting / error / disconnected — use it as evidence of reconnecting in the lossy lab (§6)

5.4 Nearby examples (ladder)

Section titled “5.4 Nearby examples (ladder)”
File Use when
ex09 Subscribing to one topic — the main example in §5
ex10 Publishing from the browser
ex11 Wildcards (+ / #)
ex12 DevKit gauges
ex13 A Live data client mixed with an MQTT approach
ex14 Manually experimenting with QoS + retain
ex15 A combined WS + MQTT dashboard

For M05, make sure of ex08 + ex09, then choose at least one from ex10–ex15, depending on time.

6. Publish / Subscribe Patterns

Section titled “6. Publish / Subscribe Patterns”

A short review from Course 1 M06, then apply it in the Twin:

Pattern Who publishes Who subscribes Used in the lab
Device → Cloud The Twin / the firmware ex09 / a dashboard telemetry
Cloud → Device A host tool / ex10 The Twin, or the firmware a command / setting state
A lab mirror ex14 ex11, on the same topic QoS / retain

6.1 QoS and retain (a quick lab with ex14)

Section titled “6.1 QoS and retain (a quick lab with ex14)”
Concept What to try
QoS 0 Send and move on — suited to a dense telemetry stream
QoS 1 / 2 More guaranteed — watch the behaviour on a lab topic
Retain Publish with retain on → open a new subscriber (such as ex11); it should get the latest message immediately

Don’t turn on retain on a high-frequency telemetry topic without thinking — the broker will keep “the latest value,” which could mislead a newcomer into thinking it’s a live stream.

6.2 Mapping Telemetry / State / Event onto MQTT

Section titled “6.2 Mapping Telemetry / State / Event onto MQTT”
Type Topic approach Retain?
Telemetry …/telemetry or …/sensors/<name> Usually not
State …/state Usually yes (the latest value)
Event …/event/<name> Usually not (kept in a log/DB instead)

7. Lossy Network and Error Injection

Section titled “7. Lossy Network and Error Injection”

The goal is not to be the best at breaking a network, but to control one variable and record how the queue / UI / client handles it.

7.1 Experiments you can run in the lab

Section titled “7.1 Experiments you can run in the lab”
Experiment How, in the Twin lab What to watch
A gap in messages Briefly stop the publisher · or stop the Simulator ex08’s stale · a gap on the graph
A brief disconnect Stop the broker and Start it again · or close the tab and reopen it ex09’s reconnecting → connected
High latency Reduce the publish rate / the Quiet scene The time between message counts
A malformed payload Send JSON with a missing field A dashboard error vs silence
Switching backends at the wrong moment (Don’t do this while capturing evidence) mixing sim+uart Origin confusion — used to teach that it must be XOR

7.2 What to write down

Section titled “7.2 What to write down”

For each case:

  1. What was injected / cut
  2. What was seen on the subscriber / Studio (with a timestamp)
  3. The behaviour: drop · retry · reconnect · stale UI · backoff
  4. Acceptable for the project, or must be fixed before M06

A form: telemetry-mqtt-lab-notes.md

Key phrase Good fault injection = change only one thing per round, and keep a log on both sides.

8. How M05 Feeds M06

Section titled “8. How M05 Feeds M06”
After M05, you have Used next in M06
Topics + example payloads The E2E Smart Environmental Monitor
A reusable dashboard / ex09–ex15 Evidence-based acceptance criteria
Notes on lossy / stale / reconnect The risk-analysis section of the report
The ability to tell Live Data from MQTT Not mixing up evidence from the wrong pipe during a demo

Next Steps

Section titled “Next Steps”
  1. Do the lab: Lab
  2. Fill in telemetry-mqtt-lab-notes.md
  3. When ready, continue to M06 — System Integration and Testing

References and Further Reading

Section titled “References and Further Reading”
  1. M04 Co-simulation · Course 1 M06 MQTT
  2. Bitstream Studio — Start broker / Twin MQTT
  3. TESAIoT_Hackathon — web-app/ex08 … ex15
  4. TESAIoT Developer Hub
  5. HiveMQ MQTT Essentials · mqtt.org
  6. ternion-3d-assets-free
  7. Course 2 TOC

Check your understanding

Section titled “Check your understanding”

Three short questions in quiz.yaml, one per objective of this lesson. Try answering them yourself first, then compare with the answer key and explanations in the file.

Lab

Section titled “Lab”

Continue hands-on at Lab: the telemetry pipe and MQTT on the Twin

Lab · Lab notes · ← Table of Contents · ← M04 · M06 →

Review questions

Answer on your own first, then open the answer.

  1. `mode=idle` หรือ `led=on` เป็นข้อมูลชนิดใด และบทเรียนแนะนำ retain อย่างไร (Objective 1)

    1. Telemetry และไม่ตั้ง retain
    2. State และมักตั้ง retain
    3. Telemetry และมักตั้ง retain
    4. Event และมักตั้ง retain
    Show answer

    Answer: B. State และมักตั้ง retain

    ตารางในหัวข้อ 1 และ 6.2: State คือสถานะปัจจุบัน มัก retain เพื่อให้ผู้มาใหม่ได้ค่าล่าสุด

  2. Live Data ปกติ แต่ MQTT subscriber ว่าง ตารางในหัวข้อ 2.1 ชี้สาเหตุใด (Objective 2)

    1. เซ็นเซอร์เสีย
    2. bridge ระหว่าง webview กับ UART หลุด
    3. ต้องเปิด ex05 ก่อน
    4. ยังไม่มี publisher บน topic นั้น
    Show answer

    Answer: D. ยังไม่มี publisher บน topic นั้น

    หัวข้อ 2.1 และ 4.1: Start broker ไม่ได้แปลว่ามี telemetry อัตโนมัติ ต้องมี publisher บน topic ที่ subscribe

  3. ข้อใดเป็นหลักของ fault injection ที่ดีตามบทเรียน (Objective 3)

    1. ปิด log เพื่อลดภาระของระบบ
    2. เปลี่ยนอย่างเดียวต่อรอบ และมี log ทั้งสองฝั่ง
    3. เปลี่ยนหลายอย่างพร้อมกันเพื่อประหยัดเวลา
    4. ผสม sim กับ uart ตอนเก็บหลักฐาน
    Show answer

    Answer: B. เปลี่ยนอย่างเดียวต่อรอบ และมี log ทั้งสองฝั่ง

    Key phrase ในหัวข้อ 7.2

Cite this lesson

If you teach from this lesson or reuse it in slides or documents, credit it with the text below. If you changed it, add (adapted) after the title.

"Telemetry Pipelines, MQTT on the Twin and Fault Injection" from TESA Open Knowledge by the Thai Embedded Systems Association (TESA), https://github.com/tesaiot/tesa-qualification-program, licensed under CC BY-NC 4.0

Thai attribution: "ท่อ telemetry, MQTT บน Twin และ fault injection" จาก TESA Open Knowledge โดยสมาคมสมองกลฝังตัวไทย (Thai Embedded Systems Association: TESA) https://github.com/tesaiot/tesa-qualification-program สัญญาอนุญาต CC BY-NC 4.0

Lesson link: https://tesaiot.github.io/tesa-qualification-program/en/courses/digital-twin/m05-telemetry-cloud/l01-telemetry-cloud-simulation/

This lesson adapts the source below; keep its credit too.
https://github.com/drsanti/TESAIoT-Courses/blob/287c21814ba8c75f693136616dcd270349a15966/C2/M05/README.md · Original content by Asst. Prof. Dr. Santi Nuratch (ผศ.ดร.สันติ นุราช), KMUTT. Course 2 (C2/) of drsanti/TESAIoT-Courses. TESA funded the work and holds the rights; published here under CC BY-NC 4.0. The upstream repository carries no licence file. Text kept faithful; structure, front matter, quizzes and notes added by TESA Open Knowledge.

Full guide: how to cite TESA

TESA Open Knowledge · © 2026 สมาคมสมองกลฝังตัวไทย (TESA) · CC BY-NC 4.0

Content is licensed CC BY-NC 4.0. Reuse it non-commercially and credit the Thai Embedded Systems Association (TESA) every time. · How to cite TESA