On-App AI Audio Processing: ANC Noise Reduction + AEC Echo Cancellation + VAD Voice Detection On-Device Solution

In today’s fast-paced lifestyle, pets often develop loneliness, depression, anxiety, and irritability due to their owners’ frequent outings and lack of interaction and emotional companionship. To improve pets’ sense of security, Tuya launches the Pet Voice Chat Template built on On-App AI, designed to achieve remote pet companionship through powerful voice technology and smart devices — strengthening the emotional bond between owners and pets and effectively alleviating pet anxiety. Beyond pet scenarios, this template can also be extended to smart home appliances, baby monitoring, and other fields, using audio capabilities as a supplement to build more human-centered remote monitoring systems.

1. What Features Does the Tuya Pet Voice Chat Template Offer?

1. Pet Behavior Recognition and Emotional Soothing

Combined with Tuya’s powerful voice recognition and pet behavior analysis technology, the system can identify abnormal behaviors such as frequent howling and whimpering, judge the pet’s emotional state, and quickly take soothing measures. For example, when pet agitation is detected, the template can automatically play a soothing voice package (pre-recorded by the owner, or using a system-designated voice package), or trigger the interaction module to link with other smart devices (such as pet teasers) to soothe the pet in time.

2. Remote Voice Companionship and Two-Way Interaction

Owners can also use Tuya’s remote voice intercom to directly communicate with their pets — "away from home, but never absent in love" — effectively relieving the pet’s loneliness.

Initiating a Voice Call to the Smart Device

2. What Are Tuya’s Core Technical Highlights?

Based on the On-App AI technology architecture, Tuya combines local audio capture, real-time intercom, and file recording functions, further introducing AI noise reduction and AI Voice Activity Detection (VAD) capabilities. Through deep modeling and intelligent separation of human voices and noise in complex environments, the solution ultimately achieves a smarter, more human-like voice interaction experience between pet devices and users.

Tuya On-App AI Overall Architecture Diagram

1. Multi-Format Audio Codec Compatibility

Tuya supports uploading and exporting mainstream audio encoding formats including Opus, PCM, WAV, MP3, G.711A, and mSBC. This ensures high-fidelity, low-latency voice data transmission and collaboration across different devices and transmission protocols — improving system integration flexibility for local storage, server uploads, voice recognition input, and other scenarios.

2. Real-time Intelligent Optimization of Intercom Audio Quality

The Tuya Pet Voice Chat Template integrates multiple advanced audio processing technologies, including Adaptive Active Noise Cancellation (ANC), Acoustic Echo Cancellation (AEC), Automatic Gain Control (AGC), and Voice Activity Detection (VAD). In real-time voice communication across multiple device endpoints, these technologies comprehensively improve voice quality and deliver a clearer, more stable, and more efficient voice interaction experience.

• Adaptive Noise Cancellation (ANC): Tuya deeply integrates deep learning and traditional signal processing algorithms to intelligently identify and distinguish human voices from environmental noise — achieving clear audio pickup even in noisy environments such as subways, streets, and windy conditions.
• Acoustic Echo Cancellation (AEC): Precisely identifies and removes speaker feedback sound in the echo path, preventing sound loops at the microphone end from disrupting voice transmission — greatly optimizing the two-way voice call experience.
• Automatic Gain Control (AGC): Dynamically adjusts volume to ensure consistent loudness from near and far sound sources, avoiding the uneven audio levels caused by varying recording distances and volumes — intelligently adapting to different distances and speaking habits.
• Voice Activity Detection (VAD): Precisely identifies voice segments, noise segments, and silence segments, intelligently filtering out irrelevant noise and silence in real time — reducing system power consumption while improving response speed and processing efficiency.

3. Audio Technology Processing Solutions for Different Scenarios

1. Local Audio Processing

Tuya supports owners uploading their own recorded audio files, automatically optimizing the recordings to ensure clear, smooth, lag-free sound — genuinely soothing pet emotions.

Local Audio Processing Flow Diagram

• Clear, stable recording audible from any distance: The built-in AGC technology automatically adjusts recording gain based on input signal strength, dynamically adapting to the speaker’s distance and volume variations within 0.3–1.5 meters — keeping output volume relatively stable and effectively avoiding uneven audio caused by varying recording distances or volume changes.

• Intelligent voice detection, automatic filtering of invalid audio: The VAD module identifies human voices and silence/noise segments locally in real time, effectively reducing approximately 40% of invalid audio data collection and processing. Tuya supports adjustable sensitivity — higher sensitivity means greater noise robustness and fewer false detections, enabling flexible optimization for actual scenarios. Combined with large AI models, it further improves voice detection accuracy and adaptability in complex environments.

• Multi-format support for flexible output: Tuya is compatible with multiple mainstream audio encoding formats including MP3, G.711A, G.711U, Opus, AAC, and mSBC — meeting storage and playback needs across diverse platforms and application scenarios.

2. Real-time Voice Intercom Processing

In real-time voice intercom scenarios, Tuya supports the Pet Voice Chat Template automatically applying intelligent processing to call audio — ensuring both parties hear clear, noise-free sound with smooth, lag-free communication.

Real-time Voice Intercom Processing Flow Diagram

• Intelligent voice detection, automatic filtering of invalid audio: The VAD module identifies human voices and silence/noise segments locally in real time, effectively reducing approximately 40% of invalid audio data collection and processing.

• AI noise reduction for clarity in complex environments: Tuya’s AI noise reduction technology models voice and noise using deep neural networks. Compared to traditional spectral subtraction algorithms, it improves SNR by 5–15 dB and PESQ by 0.5–1.2 points in complex environments, with latency controlled within 100ms — widely applicable to headphones, voice terminals, and other scenarios with high voice clarity requirements. Combined with AEC and noise reduction, it supports echo delay path compensation of up to 200ms.

• Millisecond low-latency transmission for high-fidelity voice intercom: Using efficient audio compression technology such as Opus encoding, it achieves millisecond-level low-latency transmission while maintaining voice quality — making two-way intercom feel like face-to-face communication. Ideal for time-sensitive scenarios such as voice intercom, smart doorbells, and security monitoring.

4. Development Guide and Support

1. Development Tutorial

Full development guide available at:

https://developer.tuya.com/cn/miniapp/solution-ai/ability/voice-solution/petAudioInteraction/overview

2. Technical Support

For any issues during development, please post your questions on the Tuya Developer Forum:

https://www.tuyaos.com/viewforum.php?f=3