Events¶
Use setHandler with keys in this section to register callback handlers for events. The values for these settings refer to runtime instances of code objects and are not serialized by save or dup.
The results and iterators available in these handlers vary by event. See the descriptions below for details on what these are.
^result is the most used event by a large margin.
Page conventions
Settings are grouped by concern. Within each group they're listed alphabetically. A setting that serves more than one concern appears once under its primary group; secondary groups link to it.
The code tabs on each setting show one paste-able call site per language. Adapt the placeholder names (s for the Session, plus value, stream, on_event, or on_item for the operand) to your code. Each example assumes the language's standard prelude:
#include <snsr.h>
import com.sensory.speech.snsr.Snsr;
import com.sensory.speech.snsr.SnsrSession;
import snsr
For the full Session lifecycle (snsrNew, new SnsrSession(), snsr.Session(...)) see Your first program.
Audio I/O¶
^sample-count¶
event handle write-only
snsrSetHandler(s, SNSR_SAMPLES_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.SAMPLES_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.SAMPLES_EVENT, on_event)
Samples available event.
Raised whenever audio samples have been read from the input stream and are about to be processed.
none
VAD & endpointing¶
^begin¶
event handle write-only
snsrSetHandler(s, SNSR_BEGIN_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.BEGIN_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.BEGIN_EVENT, on_event)
Begin point detected VAD event.
Raised when speech has been detected. Use begin-ms or begin-sample to retrieve the start point relative to the beginning of ->audio-pcm.
begin-ms, begin-sample, sample-count
none
^end¶
event handle write-only
snsrSetHandler(s, SNSR_END_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.END_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.END_EVENT, on_event)
Endpoint detected VAD event.
Raised trailing-silence ms after end-of-speech has been detected. Use end-ms or end-sample to retrieve the endpoint relative to the beginning of ->audio-pcm.
begin-ms, begin-sample, end-ms, end-sample, sample-count
none
^limit¶
event handle write-only
snsrSetHandler(s, SNSR_LIMIT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.LIMIT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.LIMIT_EVENT, on_event)
Maximum recording reached VAD event.
Raised when max-recording ms of speech has been processed by the VAD before a trailing-silence endpoint is found. Use end-ms or end-sample to retrieve the endpoint relative to the beginning of ->audio-pcm.
begin-ms, begin-sample, end-ms, end-sample, sample-count
none
^silence¶
event handle write-only
snsrSetHandler(s, SNSR_SILENCE_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.SILENCE_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.SILENCE_EVENT, on_event)
No speech detected event.
Raised if no speech is detected within leading-silence ms from the start of ->audio-pcm, adjusted by skip-to-ms or skip-to-sample
none
Wake word & command set¶
^result¶
event handle write-only
snsrSetHandler(s, SNSR_RESULT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.RESULT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.RESULT_EVENT, on_event)
Recognition result available event.
Raised when a final recognition hypothesis is available.
begin-ms, begin-sample, confidence-score, domain, end-ms, end-sample, id, noise-energy, score, signal-energy, snr, sv-score, text
phone-iterator, phrase-iterator, sample-count, word-iterator
Templates & slots¶
^listen-begin¶
event handle write-only
snsrSetHandler(s, SNSR_LISTEN_BEGIN_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.LISTEN_BEGIN_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.LISTEN_BEGIN_EVENT, on_event)
Sequential task has started listening on second slot.
Raised in a sequential task when audio focus has shifted from the first slot to the second. This typically happens when the spotter n the first slot has detected the phrase.
This event is ignored for tasks that do not feature sequential behavior.
none
^listen-end¶
event handle write-only
snsrSetHandler(s, SNSR_LISTEN_END_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.LISTEN_END_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.LISTEN_END_EVENT, on_event)
Sequential task has stopped listening on second slot.
Raised in a sequential task when audio focus has shifted from the second slot back to the first.
This event is ignored for tasks that do not feature sequential behavior.
none
LVCSR & STT¶
^result¶
event handle write-only
Documented under Wake word & command set (^result). Emitted by every recognizer with a final result.
^result-partial¶
event handle write-only tnl stt
snsrSetHandler(s, SNSR_PARTIAL_RESULT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.PARTIAL_RESULT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.PARTIAL_RESULT_EVENT, on_event)
Partial recognition result available event.
Raised when a preliminary recognition hypothesis is available.
begin-ms, begin-sample, domain, end-ms, end-sample, sample-count, score, text
NLU & SLM¶
^nlu-intent¶
event handle write-only tnl stt
snsrSetHandler(s, SNSR_NLU_INTENT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.NLU_INTENT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.NLU_INTENT_EVENT, on_event)
NLU intent available.
This event is raised when a natural language parse result is available.
Each intent that matched input generates an ^nlu-intent event. Use nlu-entity-iterator in this callback handler to retrieve the names of entities found.
An intent is an action, such as turning on the windshield wipers, or setting a microwave clock.
Intents are exactly the same as the top-level NLU slots reported by ^nlu-slot. The entities available inside the handler for this event, however, are only one level deep. The SDK flattens nested nlu-slot-names by separating each level with a period.
For example: Grammar snippet { date { time {hours} {minutes} } } has intent date with entities time.hours and time.minutes, but slot date has a single child slot time, which in turn has two children, hours and minutes.
See the NLU section in the grammar syntax section for more detail on the {} NLU slot capturing operator.
nlu-intent-name, nlu-intent-score, nlu-intent-value, sample-count
^nlu-slot¶
event handle write-only tnl stt
snsrSetHandler(s, SNSR_NLU_SLOT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.NLU_SLOT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.NLU_SLOT_EVENT, on_event)
NLU result available.
This event is raised when a lightweight natural language parse result is available. Each top-level slot that matched input generates a ^nlu-slot event. Use nlu-slot-iterator in this callback handler to retrieve child slot names and values.
See the NLU section in the grammar syntax section for more detail on the {} NLU slot capturing operator.
nlu-match-count, nlu-slot-count, nlu-slot-name, nlu-slot-score, nlu-slot-value, sample-count
^slm-result¶
event handle write-only stt 7.4.0
snsrSetHandler(s, SNSR_SLM_RESULT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.SLM_RESULT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.SLM_RESULT_EVENT, on_event)
SLM final output available
This event is raised when an optional SLM has a complete result available.
In the handler for this callback text holds the entire SLM response.
none
^slm-result-partial¶
event handle write-only stt 7.4.0
snsrSetHandler(s, SNSR_SLM_PARTIAL_RESULT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.SLM_PARTIAL_RESULT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.SLM_PARTIAL_RESULT_EVENT, on_event)
^slm-start¶
event handle write-only stt 7.4.0
snsrSetHandler(s, SNSR_SLM_START_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.SLM_START_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.SLM_START_EVENT, on_event)
SLM processing is about to start
Return STOP from the callback handler to avoid doing SLM processing.
none
Enrollment & adaptation¶
^adapt-started¶
event handle write-only
snsrSetHandler(s, SNSR_ADAPT_STARTED_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.ADAPT_STARTED_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.ADAPT_STARTED_EVENT, on_event)
Wake word model adaptation thread has started.
This handler is called from a thread started to do model adaptation in continuously adapting spotter models. The handle function should return OK to continue with model adaptation, or SKIP to abort adaptation and ignore this enrollment without raising an error.
The handler function's Session argument will be NULL as it is not safe to access this handle from the enrollment thread.
You can use this handler to adjust the adaptation thread priority, or delay the start of adaptation by blocking on a condition variable.
Note
Session teardown waits on joining the adaptation thread. Any handler function registered for this event must return before the associated Session is released.
none
^adapted¶
event handle write-only
snsrSetHandler(s, SNSR_ADAPTED_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.ADAPTED_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.ADAPTED_EVENT, on_event)
Wake word adaptation complete event.
Adaptation is the estimation of spotter model parameters from a number of enrollments for one or more users.
Example
Add a handler for this event to save adapted enrollment contexts.
snsrSave(session, SNSR_FM_RUNTIME, snsrStreamFromFileName("context.snsr", "w"));
session.save(Snsr.FM_RUNTIME, SnsrStream.fromFileName("context.snsr", "w"));
To load a previously saved enrollment context, do so after loading the model that created it.
^done¶
event handle write-only
snsrSetHandler(s, SNSR_DONE_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.DONE_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.DONE_EVENT, on_event)
Wake word adaptation complete event.
Handler called when adaptation has completed and the model-stream model is available.
This handler typically returns STOP to end processing.
none
^enrolled¶
event handle write-only
snsrSetHandler(s, SNSR_ENROLLED_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.ENROLLED_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.ENROLLED_EVENT, on_event)
Wake word enrollment complete event.
Handler called when enrollment is complete, just before model adaptation starts. Use this handler for the earliest possible access to the enrollment context.
Enrollment is the process of taking individual enrollments (audio files or live audio clips), validating them, and adding them to the model. These enrollments are used during the adaptation stage to estimate spotter model parameters.
Example
Add a handler for this event to save enrollment contexts before they are adapted.
snsrSave(session, SNSR_FM_RUNTIME, snsrStreamFromFileName("enrollments.snsr", "w"));
session.save(Snsr.FM_RUNTIME, SnsrStream.fromFileName("enrollments.snsr", "w"));
none
^fail¶
event handle write-only
snsrSetHandler(s, SNSR_FAIL_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.FAIL_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.FAIL_EVENT, on_event)
Wake word enrollment failed event.
Handler called if an enrollment fails quality checks.
audio-stream, enrollment-id, reason, reason-guidance, reason-pass, reason-threshold, reason-value, sample-count
^new-user¶
event handle write-only
snsrSetHandler(s, SNSR_NEW_USER_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.NEW_USER_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.NEW_USER_EVENT, on_event)
New user detected event.
Handler called when a continuously adapting spotter model has adapted to a new user.
^next¶
event handle write-only
snsrSetHandler(s, SNSR_NEXT_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.NEXT_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.NEXT_EVENT, on_event)
^pass¶
event handle write-only
snsrSetHandler(s, SNSR_PASS_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.PASS_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.PASS_EVENT, on_event)
Wake word enrollment succeeded event.
Handler called when an enrollment passes quality checks.
Return SKIP from this event handler to discard an enrollment.
audio-stream, audio-stream-first, audio-stream-last, begin-sample, end-sample, enrollment-id, sample-count
none
^pause¶
event handle write-only
snsrSetHandler(s, SNSR_PAUSE_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.PAUSE_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.PAUSE_EVENT, on_event)
Input stream pause event.
Handler called when a time-consuming processing step is about to start. Use this handler to pause the input stream when doing interactive enrollment.
none
^progress¶
event handle write-only
snsrSetHandler(s, SNSR_PROG_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.PROG_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.PROG_EVENT, on_event)
Wake word adaptation progress event.
Handler called to report adaptation progress.
none
^resume¶
event handle write-only
snsrSetHandler(s, SNSR_RESUME_EVENT, snsrCallback(on_event, NULL, NULL));
s.setHandler(Snsr.RESUME_EVENT, (session, key) -> { /* ... */ });
s.set_handler(snsr.RESUME_EVENT, on_event)
Input stream resume event.
Handler called when a time-consuming processing step has completed. Use this handler to restart an input stream that was stopped during a ^pause event handler.
none