Built-in Voice Wake-up

Overview

Built on top of the T5 chip/module, Wukong AI Hardware Development Framework supports built-in wake-word algorithms, allowing users to wake devices using specific wake words. The default wake word is “Hey, Tuya”.

Tuya can help you define a personalized wake word. You can contact your Tuya account manager to submit requirements and discuss specific project details and plans.

Hardware design requirements

The built-in wake-word algorithm requires hardware support. Design an audio loopback circuit to feed the speaker’s output into the acoustic echo cancellation (AEC) algorithm, thereby ensuring reliable wake word detection during active audio playback.

Regarding the audio loopback circuit, you can refer to the following hardware solutions:

• T5AI-Box Voice Solution
• T5AI-Bubble Voice Solution

Software design requirements

The built-in wake-word algorithm currently operates exclusively within the Tuya voice subsystem and has not yet been opened to the public. You only need to set the trigger_mode of TY_AI_TOY_CFG_DEFAULT to TY_AI_TRIGGER_MODE_WAKEUP or TY_AI_TRIGGER_MODE_FREE to automatically support this functionality. You do not need to worry about its principle and how to use it.

// Define interaction types.
typedef enum {
    TY_AI_TRIGGER_MODE_HOLD,        // Press and hold to trigger.
    TY_AI_TRIGGER_MODE_ONE_SHOT,    // Press to trigger,  turn-based dialogue mode‌.
    TY_AI_TRIGGER_MODE_WAKEUP,      // Keyword wakeup mode.
    TY_AI_TRIGGER_MODE_FREE,        // Keyword wakeup and free dialogue mode.
} TY_AI_TRIGGER_MODE_E;

// Set the working mode to keyword wake-up mode.
#define TY_AI_TOY_CFG_DEFAULT { \
    .audio_trigger_pin = TUYA_GPIO_NUM_12, \
    .spk_en_pin = TUYA_GPIO_NUM_28, \
    .led_pin = TUYA_GPIO_NUM_1, \
    .trigger_mode = TY_AI_TRIGGER_MODE_WAKEUP, \
    .audio_cfg = TY_AI_AUDIO_CFG_DEF \
}

Replace the wake-word algorithm

The wake-word algorithm currently runs on CPU1. After receiving the data that the VAD algorithm determines to be human voice, it performs wake word recognition. It sends the successful recognition to CPU0 through inter-process communication (IPC) to wake up the device for interaction.

Only replace the wake-word algorithm

Perform the following steps if you have experience in speech processing and want to use your own wake-word algorithm.

1. Modify the tuya_asr_enable function and disable Tuya’s wake-word algorithm to integrate your custom wake-word algorithm into the speech processing pipeline.

   BOOL_T tuya_asr_enable(VOID_T)
   {
       return FALSE;
   }
   

2. Implement your user_asr_init entry. The following code is for reference only, and you must modify it according to your specific ASR algorithm requirements.

   typedef struct {
       uint8_t type;
       int len;
       char *data;
   } user_asr_msg_data_t;
   
   // ASR message queue handle.
   static TKL_QUEUE_HANDLE user_asr_msg_queue;
   // Currently, audio data is sent from Core 0 to Core 1. This function receives data from Core 0 and forwards it to the ASR message queue.
   VOID_T user_asr_cpu1_event(VOID *buf, UINT_T len, VOID *args)
   {
       OPERATE_RET  rt;
       struct ipc_msg_s *send_msg = (struct ipc_msg_s *)buf;
       user_asr_msg_data_tmsg_data;
       memcpy(&msg_data, &send_msg->buf[0], sizeof(user_asr_msg_data_t));
   
       if (msg_data.type == 1) {
           void *p_buff = (void *)tkl_system_psram_malloc(msg_data.len);
           if (p_buff != NULL) {
               memset(p_buff, 0, msg_data.len);
               *(void **)msg_data.data = p_buff;
           }
       } else {
           rt = tkl_queue_post(user_asr_msg_queue, &msg_data, 0);
           if (rt != OPRT_OK) {
               bk_printf("tkl_queue_post failed %d\n", rt);
           }
       }
   }
   
   // The wakeup status is sent from Core 1 to Core 0.
   OPERATE_RET tuya_asr_event(uint8_t event)
   {
       struct ipc_msg_s send_msg = {0};
   
       send_msg.type   = TKL_IPC_TYPE_ASR;
       send_msg.buf[0] = event;
   
       return tkl_asr_send((UINT8_T *)&send_msg, sizeof(send_msg));
   }
   
   // ASR processing thread, name can be modified as needed.
   int TUTUClear_asr_main(void)
   {
       // Implement as needed.
       W16  TUTUClear_ret = TUTUClear_Init(pExternallyAllocatedMem, &pTUTUClearObject);
       if (TUTUClear_ret != TUTU_OK) {
           bk_printf("Fail to initial user asr%d.\n", TUTUClear_ret);
           goto __err_exit;
       }
   
       UINT_T timeout = TKL_QUEUE_WAIT_FROEVER;
       while (1)
       {
           int rt = tkl_queue_fetch(user_asr_msg_queue, &msg_data, timeout);
           if (rt != OPRT_OK) {
               bk_printf("")
               continue;
           }
   
           // Organize data and feed it into the ASR library - implement as needed.
           TUTUClear_OneFrame(pTUTUClearObject, //accepting 20ms pcm stream
                   (W16*)(mic_data + i * FRAME_SZ_BYTE),
                   &w32WakeWord);
   
       // Evaluate the detection results - implement as needed.
       if (w32WakeWord != 0) {     //check wakeup
               if (1 == w32WakeWord) {
                   bk_printf("TUTUClear_WakeWord -> heytuya\n");
               } else if (2 == w32WakeWord) {
                   bk_printf("TUTUClear_WakeWord -> Hey, Tuya\n");
               } else if (3 == w32WakeWord) {
                   bk_printf("TUTUClear_WakeWord -> 小智同学\n");
               } else {
                   bk_printf("TUTUClear_WakeWord-> unknown\n");
               }
   
           // Send ASR events.
           mic_data_len = 0;
           timeout = TKL_QUEUE_WAIT_FROEVER;
           tuya_asr_event(1);
   
           // Reset algorithm state - implement as needed.
           TUTUClear_Init(pExternallyAllocatedMem, &pTUTUClearObject);
               break;
       }
   }
   }
   
   // Initialization. Replace with your algorithm thread's main function name if modified.
   VOID user_asr_init(VOID_T)
   {
       tkl_asr_init(user_asr_cpu1_event, NULL);
       tkl_queue_create_init(&user_asr_msg_queue, sizeof(asr_msg_data_t), 500);
       tkl_thread_create_in_psram(&sg_hrd_hdl, "tuya_asr", 1024*4, 4, TUTUClear_asr_main, NULL);
   }
   

Replace the front-end speech algorithm and wake-word algorithm

If you have extensive speech processing expertise and want to fully utilize your own front-end processing algorithms, you can replace the AEC and VAD algorithms.

Support and help

If you have any problems with TuyaOS development, you can post your questions in the Tuya Developer Forum.