Last Updated on : 2025-06-30 11:03:01download
Built on top of the T5 chip/module, Wukong AI Hardware Development Framework supports built-in wake-word algorithms, allowing users to wake devices using specific wake words. The default wake word is “Hey, Tuya”.
Tuya can help you define a personalized wake word. You can contact your Tuya account manager to submit requirements and discuss specific project details and plans.
The built-in wake-word algorithm requires hardware support. Design an audio loopback circuit to feed the speaker’s output into the acoustic echo cancellation (AEC) algorithm, thereby ensuring reliable wake word detection during active audio playback.
Regarding the audio loopback circuit, you can refer to the following hardware solutions:
The built-in wake-word algorithm currently operates exclusively within the Tuya voice subsystem and has not yet been opened to the public. You only need to set the trigger_mode
of TY_AI_TOY_CFG_DEFAULT
to TY_AI_TRIGGER_MODE_WAKEUP
or TY_AI_TRIGGER_MODE_FREE
to automatically support this functionality. You do not need to worry about its principle and how to use it.
// Define interaction types.
typedef enum {
TY_AI_TRIGGER_MODE_HOLD, // Press and hold to trigger.
TY_AI_TRIGGER_MODE_ONE_SHOT, // Press to trigger, turn-based dialogue mode.
TY_AI_TRIGGER_MODE_WAKEUP, // Keyword wakeup mode.
TY_AI_TRIGGER_MODE_FREE, // Keyword wakeup and free dialogue mode.
} TY_AI_TRIGGER_MODE_E;
// Set the working mode to keyword wake-up mode.
#define TY_AI_TOY_CFG_DEFAULT { \
.audio_trigger_pin = TUYA_GPIO_NUM_12, \
.spk_en_pin = TUYA_GPIO_NUM_28, \
.led_pin = TUYA_GPIO_NUM_1, \
.trigger_mode = TY_AI_TRIGGER_MODE_WAKEUP, \
.audio_cfg = TY_AI_AUDIO_CFG_DEF \
}
The wake-word algorithm currently runs on CPU1. After receiving the data that the VAD algorithm determines to be human voice, it performs wake word recognition. It sends the successful recognition to CPU0 through inter-process communication (IPC) to wake up the device for interaction.
Perform the following steps if you have experience in speech processing and want to use your own wake-word algorithm.
Confirm whether custom KWS
is supported by checking the main
function in the file vendor/T5/t5_os/projects/tuya_app/main/app_main.c
. If the current SDK
version does not support it, you can modify this code by yourself.
int main(void)
{
...
extern OPERATE_RET tuya_ipc_init(void);
tuya_ipc_init();
#if (CONFIG_SYS_CPU1)
if (tuya_asr_enable()) {
bk_printf("tuya_asr_init\r\n");
extern VOID_T tkl_system_psram_malloc_force_set(BOOL_T enable);
extern BOOL_T tuya_asr_enable(VOID_T);
extern VOID _tuya_asr_int(VOID_T);
tkl_system_psram_malloc_force_set(TRUE);
// FIXME: set cpu freq to 480M only for asr
bk_pm_module_vote_cpu_freq(PM_DEV_ID_DECODER, PM_CPU_FRQ_480M);
_tuya_asr_init();
} else {
bk_printf("user_asr_init\r\n");
extern VOID_T tkl_system_psram_malloc_force_set(BOOL_T enable);
extern VOID user_asr_int(VOID_T);
tkl_system_psram_malloc_force_set(TRUE);
// FIXME: set cpu freq to 480M only for asr
bk_pm_module_vote_cpu_freq(PM_DEV_ID_DECODER, PM_CPU_FRQ_480M);
user_asr_init();
}
extern void tkl_fs_init(void);
tkl_fs_init();
// xTaskCreate(__fs_test, "fs_test", 4096, NULL, 6, (TaskHandle_t * const )&__fs_test_handle);
#endif
...
}
Modify the tuya_asr_enable
function and disable Tuya’s wake-word algorithm to integrate your custom wake-word algorithm into the speech processing pipeline.
BOOL_T tuya_asr_enable(VOID_T)
{
return FALSE;
}
Add the wake-up library. If your custom wake-up word is integrated in the form of a lib
, you need to modify vendor/T5/tuyaos/tuyaos_adapter/CMakelist.txt
, add two new lines at the end of the file, and link the lib
to the program. Note that the name needs to be modified according to the actual storage location of your library.
add_prebuilt_library(user_asr.a "$ENV{ARMINO_PATH}/../../tuyaos/tuyaos_adapter/include/asr/user/user_asr.a")
target_link_libraries(${COMPONENT_LIB} INTERFACE user_asr.a)
Implement your user_asr_init
entry. The following code is for reference only, and you must modify it according to your specific ASR algorithm requirements.
typedef struct {
uint8_t type;
int len;
char *data;
} user_asr_msg_data_t;
// ASR message queue handle.
static TKL_QUEUE_HANDLE user_asr_msg_queue;
// Currently, audio data is sent from Core 0 to Core 1. This function receives data from Core 0 and forwards it to the ASR message queue.
VOID_T user_asr_cpu1_event(VOID *buf, UINT_T len, VOID *args)
{
OPERATE_RET rt;
struct ipc_msg_s *send_msg = (struct ipc_msg_s *)buf;
user_asr_msg_data_tmsg_data;
memcpy(&msg_data, &send_msg->buf[0], sizeof(user_asr_msg_data_t));
if (msg_data.type == 1) {
void *p_buff = (void *)tkl_system_psram_malloc(msg_data.len);
if (p_buff != NULL) {
memset(p_buff, 0, msg_data.len);
*(void **)msg_data.data = p_buff;
}
} else {
rt = tkl_queue_post(user_asr_msg_queue, &msg_data, 0);
if (rt != OPRT_OK) {
bk_printf("tkl_queue_post failed %d\n", rt);
}
}
}
// The wakeup status is sent from Core 1 to Core 0.
OPERATE_RET tuya_asr_event(uint8_t event)
{
struct ipc_msg_s send_msg = {0};
send_msg.type = TKL_IPC_TYPE_ASR;
send_msg.buf[0] = event;
return tkl_asr_send((UINT8_T *)&send_msg, sizeof(send_msg));
}
// ASR processing thread, name can be modified as needed.
int TUTUClear_asr_main(void)
{
// Implement as needed.
W16 TUTUClear_ret = TUTUClear_Init(pExternallyAllocatedMem, &pTUTUClearObject);
if (TUTUClear_ret != TUTU_OK) {
bk_printf("Fail to initial user asr%d.\n", TUTUClear_ret);
goto __err_exit;
}
UINT_T timeout = TKL_QUEUE_WAIT_FROEVER;
while (1)
{
int rt = tkl_queue_fetch(user_asr_msg_queue, &msg_data, timeout);
if (rt != OPRT_OK) {
bk_printf("")
continue;
}
// Organize data and feed it into the ASR library - implement as needed.
TUTUClear_OneFrame(pTUTUClearObject, //accepting 20ms pcm stream
(W16*)(mic_data + i * FRAME_SZ_BYTE),
&w32WakeWord);
// Evaluate the detection results - implement as needed.
if (w32WakeWord != 0) { //check wakeup
if (1 == w32WakeWord) {
bk_printf("TUTUClear_WakeWord -> heytuya\n");
} else if (2 == w32WakeWord) {
bk_printf("TUTUClear_WakeWord -> Hey, Tuya\n");
} else if (3 == w32WakeWord) {
bk_printf("TUTUClear_WakeWord -> 小智同学\n");
} else {
bk_printf("TUTUClear_WakeWord-> unknown\n");
}
// Send ASR events.
mic_data_len = 0;
timeout = TKL_QUEUE_WAIT_FROEVER;
tuya_asr_event(1);
// Reset algorithm state - implement as needed.
TUTUClear_Init(pExternallyAllocatedMem, &pTUTUClearObject);
break;
}
}
}
// Initialization. Replace with your algorithm thread's main function name if modified.
VOID user_asr_init(VOID_T)
{
tkl_asr_init(user_asr_cpu1_event, NULL);
tkl_queue_create_init(&user_asr_msg_queue, sizeof(asr_msg_data_t), 500);
tkl_thread_create_in_psram(&sg_hrd_hdl, "tuya_asr", 1024*4, 4, TUTUClear_asr_main, NULL);
}
If you have extensive speech processing expertise and want to fully utilize your own front-end processing algorithms, you can replace the AEC and VAD algorithms.
If you have any problems with TuyaOS development, you can post your questions in the Tuya Developer Forum.
Is this page helpful?
YesFeedbackIs this page helpful?
YesFeedback