E2 — Parallel sets and result polling

Variant

mtb-mpy and mtb-only

MicroPython examples are mtb-mpy only; the C-side reduction and the on-screen observables apply to both variants. The C fragments here are written against the public API in ai_engine.h; complete examples are template/proj_cm55/examples/edge_ai/03_parallel_set_run.c and 04_set_membership.c.

Note

Credit: the Edge AI models are Infineon's, not TESAIoT's. The motion, audio and radar models this firmware ships (proj_cm55/modules/ai_models/model_*.c) are DEEPCRAFT™ Studio exports, copyright Imagimob AB, an Infineon Technologies company. The Siren, Cough and Factory Alarm models are DEEPCRAFT™ Ready Models by the same author, published by Infineon under the Imagimob AI Model Evaluation License Agreement, and are included in this package. TESAIoT trained none of them and owns none of them — what is ours is the engine around them. The shipped model sources carry a bare "All Rights Reserved" reservation and no grant of any kind, so they are credited here rather than licensed on. Our use is research and teaching, and not commercial; a reader who needs a grant must obtain it from Infineon and Imagimob. Train your own at https://www.infineon.com/design-resources/embedded-software/deepcraft-edge-ai-solutions/deepcraft-studio Full credit and the clause citations: THIRD_PARTY_NOTICES.md §2.2, §2.4 and §4.3.

Learning goal

Run several models at once as a set, read a verdict from the set without losing detections, and recognise the settle period that withholds verdicts for a few seconds after a set starts. The single most important sentence: ai_engine_snapshot() is not valid inside a set.

Firmware sequence

Set constants (ai_engine.h): MIC = 255, ROOM = 254, INTRUDER = 253, ALL = 252, AI_PARALLEL_FIRST = ALL. The legacy 13/14/15 aliases resolve to 253/254/255 (Chapter E1). The engine accepts a set index only in an image built with at least one microphone model; otherwise ai_engine_start() returns false.

Set members. ai_engine_set_models() returns the members in effect: an explicit membership given with ai_engine_set_define() replaces the compiled one; otherwise the compiled membership applies. A model whose init() failed once is not dropped from the set for the boot — the next start of the set tries it again.

Custom membership — edge_ai.set_define(set, [i, j, ...]) resolves the legacy index, sends SET_BEGIN, one SET_ADD per member and SET_COMMIT, then reads the membership back with Q_SET — the read-back is the only thing that catches a dropped ADD. CM55 applies the definition whole on SET_COMMIT. The C side does the same in two calls, and the read-back is the only acknowledgement:

static const uint8_t members[] = { 0u, 2u };      /* registry indices */
if (ai_engine_set_define(AI_PARALLEL_INTRUDER, members, 2u) < 0) {
    /* refused whole: a bad index, 0 or more than 8 members, or AI_PARALLEL_ALL */
}
uint8_t  got[8];
uint32_t n = ai_engine_set_members_defined(AI_PARALLEL_INTRUDER, got, 8u);
/* n == 2 and got[] == members[] is the confirmation */

Result reduction inside a set — the loudest positive member, via ai_engine_set_members() + ai_engine_snapshot_model() + ai_result_top_positive():

uint8_t  idx[8];
uint32_t n = ai_engine_set_members(idx, 8u);
int      best = -1;
uint8_t  best_model = 0u, best_class = 1u;
for (uint32_t i = 0u; i < n; i++) {
    ai_result_t r;
    if (!ai_engine_snapshot_model(idx[i], &r)) {
        continue;                                /* never published yet */
    }
    uint8_t cls;
    int pct = ai_result_top_positive(&r, &cls);  /* 0..100, class 0 excluded */
    if (pct > best) { best = pct; best_model = idx[i]; best_class = cls; }
}

Over the model link each class score is reported as 0..100 (score_pct in the Q_RESULT response); class_count == 0 means "no verdict yet", and edge_ai.result() returns None.

Sensor-rate side effect. Starting a set that contains an IMU model raises the accelerometer to 50 Hz for the session. ai_engine_model() returns NULL for a set pseudo-index, so branch on idx >= AI_PARALLEL_FIRST first:

const int a = ai_engine_active();
if (a >= (int)AI_PARALLEL_FIRST) {
    const char *set_name = ai_engine_set_name((uint32_t)a);        /* a set */
} else if (a >= 0) {
    const ai_model_desc_t *d = ai_engine_model((uint32_t)a);       /* one model */
}

Step by step

Step 1 — Enumerate the sets the page offers (both variants)

On screen, open the model dropdown on the Edge AI page. Rows for sets are added only when the set has at least two members and ai_engine_set_name() returns a non-NULL name.

What you should observe. Set rows (Sound Watch, Room Presence, Intruder Watch, All Models) appear after the compiled single models. A set with fewer than two members in this image is simply absent — that is a build-content fact, not a fault.

Step 2 — Select a set and watch the settle period

>>> edge_ai.select(255)      # MIC set; 15 is the legacy alias
True
>>> edge_ai.result()
>>> edge_ai.result()

What you should observe. For the first seconds result() returns None — class_count == 0 — while the set's inputs settle. On screen the verdict label reads filling windows lu%% with the percentage climbing, and the badge shows Loading. On the C side the calls are ai_engine_mic_settling() / ai_engine_mic_settle_pct():

if (ai_engine_mic_settling()) {
    /* verdicts are withheld: show progress rather than silent zero bars */
    uint32_t pct = ai_engine_mic_settle_pct();   /* 0..100 */
}

Keep diagnostics visible during the settle period too: the Edge AI page's stats line keeps updating through it, so an NPU stall in that period is not hidden. mic_settle_pct() is meaningful only inside a set; it returns 100 otherwise.

Step 3 — Poll results

>>> r = edge_ai.result()
>>> r

What you should observe. Once settled, a result whose fields come from the loudest positive member. Each member row on screen carries its own silence timer, counted from how long that member's seq has not advanced:

ai_result_t r;
if (ai_engine_snapshot_model(i, &r)) {
    if (r.seq != last_seq[i]) { last_seq[i] = r.seq; last_change_ms[i] = now_ms; }
    uint32_t silent_ms = now_ms - last_change_ms[i];
}

ai_engine_snapshot_model() returning false means "never published", which is not the same as "silent"; the index must be a real registry index, never a set pseudo-index.

Step 4 — Define your own membership

>>> edge_ai.set_define(253, [0, 2])
>>> edge_ai.select(253)
True

What you should observe. set_define returns after the Q_SET read-back confirms every member; a dropped ADD shows up as a shorter list, and the whole definition is refused if any index is invalid (all-or-nothing). Membership is 1..8 entries. set_define(252, ...) always fails — AI_PARALLEL_ALL is every registered model by definition and cannot be redefined; only INTRUDER, ROOM and MIC can.

Step 5 — Read active() once per render (C side)

const int a = ai_engine_active();                 /* read ONCE per render */
uint8_t   idx[8];
uint32_t  n = (a >= (int)AI_PARALLEL_FIRST)
            ? ai_engine_set_models((uint32_t)a, idx, 8u) : 0u;
/* size the rows from n and label the layout from a -- the same a */

What you should observe. Nothing visible — this is the discipline that keeps the layout tag and the row count from disagreeing when the model link starts or stops the engine from another task mid-render.

Traps

Warning

The snapshot-in-set hazard. ai_engine_snapshot() returns the most recent verdict from any model. In a set, a cough detected at 0.94 is followed by three "unlabelled" verdicts from the other members before a 10 Hz reader sees it — the detection is not late, it is gone. Use ai_engine_snapshot_model() per member and reduce. A set is any index in the range [AI_PARALLEL_FIRST, AI_PARALLEL_MIC]; test the range, not == AI_PARALLEL_MIC, which covers only one of the four sets.

set_define is task context only. Never call it from an ISR or from the pipe callback.

ai_engine_model_count() leads the rows. ai_engine_model(i) may still return NULL for an index with no row behind it — continue, do not abort:

for (uint32_t i = 0u; i < ai_engine_model_count(); i++) {
    const ai_model_desc_t *d = ai_engine_model(i);
    if (d == NULL) {
        continue;               /* no row here: skip it, keep going */
    }
    /* ... use d ... */
}

Variant box

	mtb-mpy	mtb-only
Select a set	edge_ai.select(252..255)	Dropdown row + Load
Define membership	edge_ai.set_define()	Only via your own CM55 code calling ai_engine_set_define() from a task; read back with ai_engine_set_members_defined()
Settle observable	result() is None; screen filling windows lu%%	Screen filling windows lu%%
Verdict	edge_ai.result() dict	Verdict label + per-row silence timers