On-App AI Oral Detection: Multi-Level Classification Model + Heatmap Generation for On-Device Medical Imaging

Oral health is a vital component of overall health, listed by the World Health Organization as one of the ten standards of human health. Oral diseases such as tartar, cavities, and gingivitis, if not detected early and treated promptly, allow pathogenic bacteria and toxins to enter the bloodstream — promoting atherosclerosis and increasing the risk of myocardial infarction.

Modern oral disease examination relies not only on imaging but also on clinical techniques such as visual inspection, probing, percussion, and palpation. While diagnostic accuracy is high, these methods have significant drawbacks: strong dependence on professional equipment, high consultation costs, and time inconvenience. Many patients choose to visit a medical institution only when the toothache becomes unbearable — not only delaying optimal treatment time but also dramatically increasing the cost of care.

01 AI + Dental Mirror = 24-Hour All-Around Dentist

Tuya has been exploring how to enable users to perform daily oral screening and early warning at home, at minimal cost. This time, Tuya launches the On-App AI Dental Mirror Solution — applicable to dental mirrors, smart dental flossers, saliva analyzers, and other examination devices, and extensible to ear scopes and other products with micro cameras. Users simply upload photos or videos of their oral cavity for automatic on-device local AI analysis, quickly screening for six types of oral diseases: tartar, cavities, gingivitis, tooth discoloration, developmental defects, and more. The system also provides relief suggestions and medical referral reminders, achieving "early detection, early intervention." The solution is ready out of the box — low cost, faster to deploy and mass-produce.

Examination Results Displayed Synchronously in the App

Application scenarios:

• Daily home screening: Snap oral images after morning and evening brushing to quickly screen for plaque, gingivitis, and other common issues, building a personal oral health profile.
• Remote health consultation: Before an online consultation, patients can complete self-screening to provide doctors with visual references, improving consultation efficiency.
• Children’s oral care: Through an engaging detection interface, help parents promptly identify childhood cavities, abnormal tooth development, and other issues.
• Post-operative recovery tracking: Regularly record oral conditions, compare historical data, and track recovery progress.

02 Four-Stage Technical Pipeline for Accurate Oral Reports

On-App AI Dental Mirror Solution technical pipeline diagram:

Technical Pipeline Diagram

1. Preliminary Filtering

When an image is detected, the system first uses a binary classification model to screen the input image, filtering out images unrelated to the oral cavity to ensure the validity of subsequent analysis.

2. Fine-Grained Screening

The oral image is further classified to identify normal, cavity, tartar, gingivitis, tooth discoloration, misalignment, ulcer, and other conditions. Effective classification algorithms ensure judgment accuracy.

3. Visual Analysis

After model prediction, the model’s convolutional layers generate heatmaps of regions of interest. Tuya also provides three different filter modes — daylight mode, periodontal mode, and cavity mode — for more comprehensive analysis and display of detection results.

Three Filter Mode Effect Comparison

4. Result Output

Finally, the system returns overall prediction metrics along with rendered filter images, providing users with clear and intuitive analysis results.

03 Powerful AI Models Delivering Differentiated Features

Tuya’s On-App AI Dental Mirror Solution uses TensorFlow Lite for mobile model deployment. This framework offers efficient inference, low latency, and low power consumption advantages — supporting local offline operation on devices to rapidly improve user experience and response speed.

On-App AI Overall Architecture Diagram

1. Offline Local Operation: Oral Detection On-Call 24/7

Traditional dental mirrors typically stop functioning when Wi-Fi is offline or the connection is unstable, as they cannot reach cloud detection in real time. To solve this, Tuya innovatively introduces a local AI model, perfectly resolving this challenge. This not only improves the user experience but also reduces dependency on external networks, effectively ensuring the system’s independence and real-time responsiveness.

2. Oral Disease Prediction: Boosting Market Competitiveness

The newly added oral disease prediction feature helps oral examination devices quickly improve market competitiveness. All prediction inference runs locally, balancing information security and compliance. Local processing also eliminates reliance on cloud computing, enabling higher real-time prediction efficiency at lower operational cost.

3. Flexible AI Architecture: Lighter and More Dynamic

Lightweight and Dynamic Architecture Diagram

3.1 Lightweight Detection Model

We use a mobile-optimized lightweight detection model that first screens oral images, then feeds them through an advanced disease detection model for fine-grained classification. Combined with multiple filter effects that highlight disease regions, results are presented in the form of an examination report — helping users more intuitively identify and understand oral problems and guiding decisions on whether medical treatment is necessary.

3.2 Dynamic Model Management

Tuya uses an on-demand loading dynamic model management mechanism, supporting online download, iterative updates, and deployment — ensuring the app always automatically runs on the optimal version. This reduces initial installation package size, improves runtime efficiency, and continuously optimizes the user experience.

4. Ultra-Fast Experience: Zero-Delay Oral Detection

To validate the processing speed of the On-App AI Dental Mirror Solution, Tuya conducted tests on phones of different price ranges and brands. Results show that after uploading an image, the entire processing time from oral image detection to report output is within 1 second (including heatmap rendering time) — users experience no perceptible delay.

04 Development Guide and Support

1. Development Tutorial

Full development tutorial available at:

https://developer.tuya.com/en/miniapp/solution-ai/ability/oral-solution/overview

2. Technical Support

For any issues during development, please post your questions on the Tuya Developer Forum:

https://www.tuyaos.com/viewforum.php?f=3