AI Text-to-ImageOn-App AI

#Function Introduction

The AI ​​Pixel Screen Image Generation Module generates pixel images locally using an On-App AI Edge Model, eliminating the need for cloud-based inference capabilities. Users simply select a tag to generate a pixel-style image in real-time within 1-2 seconds, supporting collection and Bluetooth delivery to the device for display.

This module is suitable for various pixel screen devices (car screens, clock screens, doorbell screens, custom pixel screens, etc.), meeting users' needs for creating personalized pixel patterns.

#Function Description

1. Tag-Based Image Generation

After the user clicks on a tag (such as "maple tree," "bird," "emoji," etc.), the client will call the local AI model to generate a pixel image of the corresponding size.

2. Collecting Images

Clicking the "Collect" button on the generated image saves it to the local gallery for easy reuse.

3. Sending Images to Devices (Bluetooth Transparent Transmission)

Clicking the "Send Preview" button on the image will send the generated pixel image to the device for real-time display via Bluetooth Big Data Transmission.

#Interaction Flow

sequenceDiagram
    participant U as User
    participant UI as Frontend Page
    participant AI as On-Device AI Model
    participant Dev as Device

    U->>UI: Enter AI Drawing Page
    UI->>UI: Check if model is initialized
    alt Not Initialized
        UI->>AI: Call pixelImageInit to initialize model
        AI-->>UI: Initialization success
        UI->>AI: fetchPixelImageCategoryInfo to get label data
        AI-->>UI: Return label list
    else Already Initialized
        UI->>AI: fetchPixelImageCategoryInfo to get label data
    end

    U->>UI: Select a label
    UI->>UI: Update message flow (user message + placeholder for generation)
    UI->>AI: generationPixelImage(label)
    AI-->>UI: Return generated image path
    UI->>UI: Update message flow to display the image

    U->>UI: Click the Favorite button
    UI->>UI: Save to local gallery

    U->>UI: Click the Send Preview button
    UI->>Dev: Send image to device via BLE transparent transmission
    Dev-->>UI: Display confirmation

#Dynamic datasets

flowchart LR
    %% --- Business Definition Module ---
    subgraph BD[Business Definition]
        A[Define Category] --> B[Define Label]
        B --> C[Reference Image]
    end

    %% --- T2I Generation Module ---
    subgraph TG[Text-to-Image Generation]
        C --> D[Image Preprocessing]
        D --> E[Prompt Inversion]
        E --> F[Image Generation]
    end

    %% --- Dataset Output Module ---
    subgraph DS[Dataset]
        F --> G["Image Dataset\n(Image Data + Image Labels)"]
    end

Custom Pixel Image Dataset Process Description:

• Define Categories: e.g., Fruits, Emoticons.

• Define Tags: e.g., Avocado.

• Reference Image: Select a pixel-style image as a reference image.

• Image Preprocessing: Image cutout, preserving the main subject.

• Hint Word Derivation: Derive hint words from the image to obtain hint words for the text-to-image format.

• Image Generation: Batch generate images using the hint words.

• Image Dataset: Includes image data and image tags.

#Module Working Principle

The core of the module is based on the following logic:

#1. Local AI Model Initialization

Upon entering the page:

if (!hasModelInit) {
  pixelImageInit(); // Initialize local AI model
  fetchPixelImageCategoryInfo(); // Fetch label list
}

#2. Retrieving the Tag List

After the local model initialization is complete, it will return the available tags:

const labelInfo = await fetchPixelImageCategoryInfo();
dispatch(updateLabelAsync(labelInfo));

#3. Generate Pixel Image

After clicking the tag, an image is generated from the client's local model:

generationPixelImage({
  deviceId: devInfo.devId,
  label,
  imageWidth: 462,
  imageHeight: 462,
  outImagePath: localPath,
});

#4. Update the Dialogue Message Flow

The module displays the generation process in a chat format:

• Add user message

• Add "Generating" placeholder message

• Replace with real image content

updateMessages([...messages, newMsg]);

#5. Add to Favorites and Send

Add to Favorites: Adds the image path to your local gallery.

Send to Device: Transmit via Bluetooth:

sendImageToDevice(path);