Built-in Voice Wake-up

Overview

Built on top of the T5 chip/module, Wukong AI Hardware Development Framework supports built-in wake-word algorithms, allowing users to wake devices using specific wake words. The default wake word is “Hey, Tuya”.

Tuya can help you define a personalized wake word. You can contact your Tuya account manager to submit requirements and discuss specific project details and plans.

Hardware design requirements

The built-in wake-word algorithm requires hardware support. Design an audio loopback circuit to feed the speaker’s output into the acoustic echo cancellation (AEC) algorithm, thereby ensuring reliable wake word detection during active audio playback.

Regarding the audio loopback circuit, you can refer to the following hardware solutions:

• T5AI-Box Voice Solution
• T5AI-Bubble Voice Solution

Software design requirements

The built-in wake-word algorithm currently operates exclusively within the Tuya voice subsystem and has not yet been opened to the public. You only need to set the trigger_mode of TY_AI_TOY_CFG_DEFAULT to TY_AI_TRIGGER_MODE_WAKEUP or TY_AI_TRIGGER_MODE_FREE to automatically support this functionality. You do not need to worry about its principle and how to use it.

// Define interaction types
typedef enum {
    TY_AI_TRIGGER_MODE_HOLD,        // Press and hold to trigger
    TY_AI_TRIGGER_MODE_ONE_SHOT,    // Press to trigger,  turn-based dialogue mode‌
    TY_AI_TRIGGER_MODE_WAKEUP,      // Keyword wakeup mode
    TY_AI_TRIGGER_MODE_FREE,        // Keyword wakeup and free dialogue mode
} TY_AI_TRIGGER_MODE_E;

// Set the working mode to keyword wake-up mode
#define TY_AI_TOY_CFG_DEFAULT { \
    .audio_trigger_pin = TUYA_GPIO_NUM_12, \
    .spk_en_pin = TUYA_GPIO_NUM_28, \
    .led_pin = TUYA_GPIO_NUM_1, \
    .trigger_mode = TY_AI_TRIGGER_MODE_WAKEUP, \
    .audio_cfg = TY_AI_AUDIO_CFG_DEF \
}

Replace the wake-word algorithm

The wake-word algorithm currently runs on CPU1. After receiving the data that the VAD algorithm determines to be human voice, it performs wake word recognition and sends the successful recognition to CPU0 through inter-process communication (IPC) to wake up the device for interaction.

Only replace the wake-word algorithm

If you have experience in speech processing and want to use your own wake-word algorithm, you can rewrite the tuya_asr_init function to integrate your custom algorithm.

VOID_T tuya_asr_init(VOID_T)
{
    // IPC between cpu0 and cpu1
    tkl_asr_init(asr_cpu1_event, NULL);

    // Queue for the ASR thread, the ASR should fetch data from asr_msg_queue
    tkl_queue_create_init(&asr_msg_queue, sizeof(asr_msg_data_t), 50);

   // New thread for ASR, below are two example threads
#if TUYA_ENABLE_ASR_WANSON     
    tkl_thread_create_in_psram(&sg_hrd_hdl, "tuya_asr_xxx", 1024*4, 4, xxx_asr_task, NULL);
#else
    tkl_thread_create_in_psram(&sg_hrd_hdl, "tuya_asr_yyy", 1024*4, 4, yyy_asr_task, NULL);
#endif
}

Replace the front-end speech algorithm and wake-word algorithm

If you have extensive speech processing expertise and want to fully utilize your own front-end processing algorithms, you can replace the AEC and VAD algorithms.

Then, modify the tuya_asr_enable function to integrate your custom wake-word algorithm into the speech processing pipeline.

BOOL_T tuya_asr_enable(VOID_T)
{
    return FALSE;
}

When the wake-word algorithm is operational, it needs to output wake-word events externally. Refer to the following code snippet:

 int rs = xxx_ASR_Recog((short*)(mic_data + i * RAW_READ_SIZE), 480, (const char **)&text, &score);
 if (rs == 1) {
     bk_printf("xxx_ASR_Recog -> %s \n", text);
     if (os_strcmp(text, "Hello Tuya") == 0   || {      
        tuya_asr_event(1);    // Send wakeup event to cpu0
     } else {
        //Nothing
     }
  }

Support and help

If you have any problems with TuyaOS development, you can post your questions in the Tuya Developer Forum.