TuyaOSIPC Development FrameworkType Development KitCapability Map

Capability Map

Last Updated on : 2023-10-26 07:54:13download

This topic describes a set of IPC capabilities, including basic capabilities, audio and video processing, live preview and two-way talk, recording storage, event alerts, stream to third-party platforms, and low power keep-alive. It covers pairing and activation, network connection, device control, OTA firmware update, ring buffer, circular overwrite and frame skipping, P2P technology, two-way talk, and recording control, ensuring a stable IPC connection solution.

Basic capabilities

The basic capabilities include pairing and activation, network connection, device control, and OTA firmware update. For more information, see Capability Map of Networked Product Development Framework.

Audio and video ring buffer

Capabilities	Description
Audio and video buffer	Buffer different types of audio and video data by channels, including SD and HD video as well as audio. This capability applies to all streaming services.
Read/write synchronization mechanism	The internal lock-free synchronization mechanism ensures the producer and consumer that operate on the buffer can work in an independent manner.
Circular overwrite and frame skipping mechanism	The data producer will overwrite the oldest data when the buffer is full. The frame skipping policy will activate automatically to prevent data and encoding errors if network jitter causes blocking while the data consumer is reading.
Configurable buffer size	You can set varying buffer sizes based on the memory capacity of different devices.

Live preview and two-way talk (P2P)

Capabilities	Description
P2P	With Tuya’s proprietary peer-to-peer (P2P) technology, a device and a mobile app can establish a secure P2P connection for data transfer.
HD and SD output	Switch between HD and SD output on the mobile app based on video resolution.
Two-way talk	The IPC sends the audio captured from the mic to the mobile or web client. The IPC receives audio from the client and plays it through the speaker.
Status notification	With the P2P event callbacks, you will get notified of live preview status, such as streaming status and resolution switching. You can define peripheral activities accordingly. For example, turning the monitoring light on or off.

Live preview and two-way talk (WebRTC)

Capabilities	Description
WebRTC	The IPC can establish a connection with WebRTC-compatible browsers and speakers to transfer audio and video in real time.
HD and SD output	Video resolution can be changed on the browser.
Two-way talk	The browser captures the sound and sends the audio to the IPC via a WebRTC connection. The IPC will forward the received audio data to you through a callback. You can play the audio on the specified platform.
Status notification	With the WebRTC event callbacks, you will get notified of live preview status, such as streaming status and resolution switching. You can define peripheral activities accordingly. For example, turning the monitoring light on or off.

Cloud storage

Capability	Description
Order management	Cloud storage is activated automatically after the end user purchases this service. For events that require video recording, the IPC needs to call the control API to initiate recording. After the subscription expires, cloud storage is not available, which cannot be controlled via API calls.
Event storage	After the end user purchases the event storage service, the IPC can call the API to start or stop recording.
Continuous storage	After the end user purchases the continuous storage service, the IPC will start continuous recording 24 hours a day, without an API call.
Recording control	The IPC can call the API to start or stop cloud storage and therefore control video recording.
Silent video	Enabling the silence option allows IPC to record video without audio.

Local storage

Capability	Description
Event storage	The IPC can call the API to start or stop recording.
Continuous storage	The IPC enables continuous recording 24 hours a day, without an API call.
Recording control	The IPC can call the API to start or stop SD card storage and therefore control video recording.
Silent video	Enabling the silence option allows IPC to record video without audio.
Search by month	Check which days of the month have footage. You need to construct reply messages in a fixed format.
Search by day	Check the available footage recorded in a day. You need to construct reply messages in a fixed format.
Search by point in time	The user selects a point in time on the mobile app. The IPC reads the footage accordingly and sends it to the mobile app.
Playback speed	Support up to 32x playback speed.

Event alert

Capability	Description
Extensible event types	Support multiple types of events, such as motion detection, human shape detection, and doorbell calls. You need to configure the required event types for your product on the Tuya IoT Development Platform. Specify them as the request parameters in the data reporting interface on the firmware.
Image message	The IPC sends an event alert with an image via the alert reporting interface. The user will receive the image message in the mobile app’s message center.
Video message	Visitors can leave a video message if a call is not answered. The IPC records a 10-second video and sends it to the cloud. The user will receive the video message in the mobile app’s message center.

Stream to third-party platforms

Capability	Description
Preview	WebRTC and cloud-forward technology enable live video streaming on platforms like Echo Show and Chromecast without requiring additional integration. You can simply purchase the related service to get started.
Two-way talk	You can integrate with audio data callback and audio playback to support talking to visitors through platforms such as Echo Show and Chromecast.

Keep-alive and wake-up in low power

Capability	Description
TCP keep-alive	To develop low power products, you need to integrate with the cloud-based TCP keep-alive services including connection, authentication, encryption, and decryption. When the device connects to the TCP keep-alive services, the cloud will automatically close its MQTT connection with the device, without leaving an offline record.
Wake-up	When an operation on the mobile app tries to wake up the device, the cloud will send data to the device over a TCP keep-alive connection. After receiving data, the microcontroller should be woken up and get online to connect to the Tuya MQTT service. The cloud will automatically close the low-power TCP keep-alive connection, without leaving an offline record.

Prev DocQuick Start

Next DocPairing and Binding