On-App AI Pet Image Quality Inspection: Three-Stage Pipeline On-Device Detection and Anomaly Filtering Mechanism

For products like pet feeders and pet water dispensers, building accurate pet profiles is not only critical for correctly identifying and recording abnormal pet behavior, but also the core foundation for achieving personalized, intelligent, and health-conscious feeding. However, at present, due to the lack of an effective image quality filtering mechanism, user-uploaded images commonly suffer from quality issues. If the system fails to effectively intercept them, low-quality pet profiles will bring a catastrophic user experience to the subsequent use of pet devices.

Types of Low-Quality Images Uploaded by Users

01 Tuya Delivers a Qualitative Leap in Pet Image Quality

To help developers overcome challenges such as difficult biometric data collection, low recognition accuracy, multi-pet interference, and privacy security risks — at low barrier and low cost — Tuya provides an efficient pet image quality detection solution based on its powerful On-App AI architecture. By deeply integrating AI technology, the system can simultaneously complete pet image recognition and anomaly filtering within 1 second, achieving fast and precise image quality assessment. Moreover, this solution directly intercepts low-quality images uploaded by users without requiring cloud-side detection and feedback, greatly reducing network transmission time, improving response speed, and delivering a smoother user experience.

1. What Dimensions Does Tuya Use to Analyze Images?

Tuya’s supported image analysis dimensions and high-quality image evaluation criteria:

Image Analysis Dimensions and Evaluation Criteria

2. Image Quality Detection Pipeline

Tuya’s image quality detection process goes through the following main steps:

• Subject Detection: Uses an object detection model to automatically identify the subject in the frame, outputting subject category, position coordinates, and size information;
• Face Detection: Uses a face detection model to quickly locate the pet’s facial region; then uses a facial landmark detection model (eyes, nose, fur, etc.) to detect key points on the pet’s face for rapid image quality screening;

Facial Landmark Detection Diagram

• Algorithm Validation: Calculates the average brightness of the pet region to remove overexposed or underexposed images; uses the pet’s facial landmark positions to calculate face pose angle and detect whether the face is occluded.

Image Detection Pipeline Diagram

3. Building a Full-Pipeline Pet Device Experience

By enriching pet biometric profiles, Tuya combines with smart pet hardware to deliver an intelligent, all-in-one pet service:

• Precise Identity Recognition: Enables multi-modal biometric identification, ensuring device functions are applied to the correct pet subject
• Smart Device Linkage: Automatically matches personalized feeding plans based on recognition results
• Multi-Pet Household Management: Simultaneously manages independent profiles and personalized settings for multiple pets

02 Technical Highlights of Image Quality Detection

This solution is built on the On-App AI architecture. App-side model deployment uses TensorFlow Lite, offering unique advantages of efficient inference, low latency, low power consumption, and local operation — effectively improving user experience and response speed.

On-App AI Overall Architecture

1. More Flexible AI Architecture: Lightweight and Dynamic

Lightweight and Dynamic Runtime Flow Diagram

1.1 Lightweight Detection Models

Tuya uses lightweight object detection models, pet face region models, and pet facial keypoint models specifically optimized for mobile — enabling precise detection of specified subjects (such as pets and people) and pet facial landmarks in images.

1.2 Dynamic Model Management

Uses an on-demand loading dynamic model management mechanism that supports online model download, update, and deployment — always ensuring the optimal model version is in use, while reducing initial installation package size and improving runtime efficiency.

2. Superior User Experience: Real-time and High Efficiency

2.1 Real-time Interactive Processing

After a user selects an image from their album, detection and processing begin immediately on the device. Across low-, mid-, and high-end devices processing images at various resolutions, the processing speed completes within 1 second — including anomalous image interception and anomaly notification.

Performance benchmark data across different phone models and image resolutions:

Device Performance Benchmark Data

2.2 Efficient Image Quality Filtering

• Species Feature Extraction: The model identifies whether only one cat is present in the image, filtering out multi-cat frames or other interfering species;
• Subject Proportion Analysis: Uses image segmentation to identify subject size, filtering images where the subject occupies too small a proportion of the frame;
• High-Precision Angle Analysis: Performs anomaly filtering on pet face images with excessive tilt or offset angles, improving the accuracy and efficiency of feature extraction.

3. More Secure Computing: Low Cost and Privacy Protection

On-Device Secure Computing Architecture

3.1 Reduced Processing Cost

Compared to cloud-based solutions, mobile-side AI processing significantly reduces computing resource demands and bandwidth consumption, providing a more secure and efficient computing environment for applications.

3.2 Privacy Protection

The system detects and filters human faces or figures appearing in images — especially in cases where people and pets appear together. Tuya applies image filtering to ensure user privacy and security.

03 Development Guide and Support

1. Development Tutorial

Full development tutorial available at:

https://developer.tuya.com/cn/miniapp/solution-ai/ability/picture-solution/petImageQualityAssessment/ability-set/cloud

2. Technical Support

For any issues during development, please post your questions on the Tuya Developer Forum:

https://www.tuyaos.com/viewforum.php?f=3