AI Capabilities Development

Overview

Against the backdrop of rapid advancements in large model technology, the innovation directions for smart devices continue to expand. The integration of “smart devices and AI” offers limitless possibilities, such as AI companion robots, AI dolls, AI pets, and other emerging innovative application scenarios.

To help you efficiently implement AI applications, the Tuya Developer Platform offers diverse support, including standardized AI features for different categories, a rich library of smart agent templates, and convenient panel deployment tools. These resources comprehensively ensure the rapid implementation of AI applications across multiple dimensions.

How to implement AI features

Taking AI dolls as an example, their core functionality is an integration of large models, so as to enable personalized character empowerment and deliver smooth, natural conversational experiences.

• Hardware requirements: The device shall have the capability to uplink multimodal data, including the collection and transmission of audio, video, images, text, and other data. For example, to implement AI conversation features, the device must support the hardware configuration of voice reception (microphone) and voice playback (speaker) to ensure smooth human-computer interaction.

• Software requirements: A smart agent shall be deployed on the device to serve as a bridge for interfacing with the large model. The primary responsibility of this agent is to handle the device’s uplink requests, interact with the large model based on the user’s dialogue information, and return the processing results. This way, achieve smart conversation functionalities.

Product AI business capabilities and general capabilities

In order to quickly implement product AI scenarios, the platform provides a range of AI capabilities that are common to all business categories. Users can select the required AI capabilities based on product requirements, and the platform will recommend suitable agent templates.

Differences between agent deployment terminals

Development procedure

Prerequisites

You have defined product functions and created at least one smart product on the Tuya Developer Platform. For more information, see Create Products.

Agent configuration

1. Log in to the Tuya Developer Platform.

2. In the left-side navigation pane, choose Product > Development. Find the desired product and click Develop in the Operation column.

3. In the Function Definition tab, find Product AI Capabilities and click Add Product AI Capabilities.

   The Product AI Capabilities tab is only visible in solutions that support AI capabilities. If the tab is invisible, it means your solution does not support AI capabilities.

   

4. Add product AI capabilities. Select the AI capabilities your product needs to quickly support advanced AI features. And then, click OK.

   

5. Select AI agents. AI capabilities are managed by one or more AI agents. Click Select AI Agent and configure the agents on panels and devices for your product.

   

6. Choose how to implement your AI agent.

   • Method 1: Select an AI agent under your account: You can directly link an agent that you have created under the current account. Select an agent and click > to bind the agent, and then click OK. If no agent is available, go to the AI Agent and create an agent.

   • Method 2: Create an AI agent based on a template: The platform recommends an agent template for you based on the AI ​​capabilities you have selected. Select a template as required and click OK.

     

7. Configure device audio parameters. If the device supports audio functionality and the agent has been configured, you can configure the audio parameters. After adding the agent on devices, click Configure Audio.

   

   Select Audio Format and Timbre to configure device tones, speech speed, and other parameters. And then, click OK.

   

Deploy agents on the panel secondary display

If the panel is bound with an agent, you can manage the agent deployment interactions on the panel’s secondary display in a no-code way. The following figure shows what the agent looks like after it is deployed.
For more information, see Configure Smart AI Secondary Display.

Develop hardware

On top of TuyaOS, Tuya provides you with Wukong, a hardware development framework built in the field of AI hardware. The AI ​​development framework v1.0 has been released. You can download the Wukong AI development framework through Tuya Wind IDE and develop your own AI products. For more information, see TuyaOS development documentation.

If an SDK with AI features is used in the development firmware, the product will automatically be labeled with AI.