Cloud-Based Video Talk

Cloud-based video talk is implemented to connect an app with a cloud-based video intercom that supports video surveillance, access control, and more capabilities. This module relies on Tuya Smart Life IPC SDK and enables multiple features, such as video query, voice talk, and remote unlocking.

Feature overview

Integrate video surveillance

Use the cloud-based video surveillance management class to load configurations. After a successful callback, initialize a Powered by Tuya (PBT) IP camera (IPC) and connect to the video streaming channels to enable video previewing.

Integrate access control

• Preview videos:

  Similar to the process of video surveillance, use the cloud-based video surveillance management class to load configurations. After a successful callback, initialize a PBT IPC and connect to the video streaming channels to enable video previewing.

• Accept video calls:

  1. Register a device communication listener.
  2. When the device receives a request, the listener triggers a callback. The communication type and data included in the callback indicate information about the device, and device configurations are loaded. After a successful callback, connect to the video streaming channels of the PBT IPC and enable video previewing.
  3. With video previewing enabled, connect to the audio channels to enable voice talk.

Integrate the IPC SDK

The IPC SDK must be integrated. The following IPC SDK version is recommended:

implementation 'com.tuya.smart:tuyasmart-ipcsdk:3.27.5'

Cloud-based video talk management class

This class can be used to manage cloud-based video talk features. For example, load device configurations, register a device communication listener, query video surveillance device management, and query video access control devices.

Example

ITuyaCommunityVisualSpeak visualSpeak = TuyaCommunitySDK.getCommunityVisualSpeakInstance();

Load configurations

Loads device configurations. After the success callback, the subsequent process can continue and the audio and video communication between the app and the device can be started.

API description

void loadConfig(@NonNull String deviceId, @Nullable String gatewayId, ITuyaCommunityCallback callback);

Request parameter

Parameter	Description
deviceId	The device ID, required.
gatewayId	The ID of a gateway to which the device is connected. Otherwise, set the value to null.
callback	The callback.

Example

if (visualSpeak != null) {
    visualSpeak.loadConfig(deviceId, null, new ITuyaCommunityCallback() {
        @Override
        public void onFailure(String s, String s1) {
            L.e(TAG, "[loadConfig] onFailure:" + s1);
        }

        @Override
        public void onSuccess() {
            L.d(TAG, "[loadConfig] onSuccess");
            Intent intent = new Intent(VisualSpeakMainActivity.this, VisualSpeakVideoActivity.class);
            startActivity(intent);
        }
    });
}

Register a device communication listener

Registers a device communication listener. When the device sends a request to the app, the listener triggers a callback to indicate events. For example, the device calls the app, or the device hangs up the call.

API description

// Registers a listener.
void registerVisualSpeakCommunicationListener(IOnVisualSpeakDeviceCommunicationListener listener);

// Removes a listener.
void unRegisterVisualSpeakCommunicationListener(IOnVisualSpeakDeviceCommunicationListener listener);

Example

visualSpeak.registerVisualSpeakCommunicationListener(new IOnVisualSpeakDeviceCommunicationListener() {
    @Override
    public void receiveDeviceMessage(TYVisualSpeakCommunicationType tyVisualSpeakCommunicationType, TYDeviceMsgDataBean tyDeviceMsgDataBean) {
        switch (tyVisualSpeakCommunicationType) {
            case DEVICE_CALLING:
                break;
            case DEVICE_CALL_OTHER_APP_ANSWER:
                break;
            case DEVICE_CALL_CANCEL:
                break;
            case DEVICE_CALL_TIMEOUT:
                break;
            case DEVICE_CALL_HANGUP:
                break;
            case TALKING_TIMEOUT:
                break;
            case UNKNOWN:
            default:
                break;
        }
    }
});

TYVisualSpeakCommunicationType

Type	Description
DEVICE_CALLING	The device is calling the app.
DEVICE_CALL_OTHER_APP_ANSWER	The device calls the app and another mobile phone with the app accepts the call.
DEVICE_CALL_CANCEL	The device cancels the call.
DEVICE_CALL_TIMEOUT	The device call timed out.
DEVICE_CALL_HANGUP	The device hangs up the call.
TALKING_TIMEOUT	The call automatically hangs up due to timeout.
UNKNOWN	An unknown error has occurred.

TYDeviceMsgDataBean

Field	Description
deviceId	The device ID.
reqType	The type of call.
sn	The ID of the call.

Query devices for cloud-based video talk

Returns a list of video access control devices to enable remote unlocking, returns the device status, and stores call records.

API description

ITuyaCommunityAccessControl getAccessControlManager();

Query video surveillance devices

Returns a list of video surveillance devices and other required information.

API description

ITuyaCommunityPublicMonitor getPublicMonitorManager();

Video surveillance device in public areas

Users can check video surveillance devices in public areas by using the app.

Query a list of video surveillance devices in public areas

Returns a list of video surveillance devices in public areas.

API description

void getPublicMonitorList(String communityId,
                   String roomId,
                   ITuyaCommunityResultCallback<ArrayList<TYMonitorDeviceBean>> callback);

Request parameter

Parameter	Description
communityId	The community ID.
roomId	The room ID.
callback	The callback.

Data types of TYMonitorDeviceBean

Field name	Field type	Description
gatewayId	String	The gateway ID.
deviceId	String	The device ID.
deviceStatus	Int	The current device status. Valid values: 1: online 0: offline.
deviceName	String	The name of the device.
deviceThumbnailsUrl	String	The thumbnails of videos from the device.

Video access control

Each video access control device supports the following features:

• Preview videos: Users can preview access control videos and unlock a door by tapping the app.
• Accept device calls: When the device calls a house, the app receives the call. Users can unlock a door or talk to the device through the app.

Query a list of access control devices

Returns a list of access control devices assigned to a user.

API description

void getAccessControlList(String communityId,
                   String roomId,
                   ITuyaCommunityResultCallback<List<TYVisualSpeakDeviceBean>> callback);

Request parameter

Parameter	Description
communityId	The community ID.
roomId	The room ID.
callback	The callback.

Data types of TYVisualSpeakDeviceBean

Field name	Field type	Description
deviceId	String	The device ID.
roomId	String	The room ID.
deviceName	String	The name of the device.

Query the upper limit of access control video previews

Returns the maximum number of times the access control videos can be previewed. You can set this upper limit on the platform. The return value of this API method is an integer that indicates the daily preview quota.

API description

void getAccessControlViewableTimes(String communityId,
                            String roomId,
                            String deviceId,
                            ITuyaCommunityResultCallback<Integer> callback);

Request parameter

Parameter	Description
communityId	The community ID.
deviceId	The device ID.
roomId	The room ID.
callback	The callback.

Query access control status

Indicates the access control status after the app receives a call from the device. The talk channel is built between the app and the device only when the device is in the calling state. Then, users can answer the call on the app.

API description

void getAccessControlStatus(String communityId,
                     String roomId,
                     String deviceId,
                     String sn,
                     ITuyaCommunityResultCallback<TYVisualSpeakCommunicationType> callback);

Request parameter

Parameter	Description
communityId	The community ID.
deviceId	The device ID.
roomId	The room ID.
sn	The ID of the call.
callback	The callback.

Unlock a door

Sends an unlocking command to the device.

API description

void openDoor(String communityId, String roomId, String deviceId, final ISuccessFailureCallback callback)

Request parameter

Parameter	Description
communityId	The community ID.
deviceId	The device ID.
roomId	The room ID.
callback	The callback.

Answer a call

Notifies the device that the app has answered a call. An audio and video streaming channel is built between the app and the device after users tap the app to answer the call.

API description

void accept(String deviceId);

Request parameter

Parameter	Description
deviceId	The device ID.

Reject a call

Rejects a call after the device calls the app.

API description

void reject(Map<String, String> params, final ISuccessFailureCallback callback);

Request parameter

Parameter	Description
params	The Map object that carries data.
callback	The callback.

params

Parameter	Description
device_id	The device ID.
community_id	The community ID.
room_id	The room ID.
sn	The ID of the call.

Example

if (communityAccessControl != null) {
    Map<String, String> map = new HashMap<>();
    map.put("device_id",deviceId);
    map.put("community_id",communityId);
    map.put("room_id",roomId);
    map.put("sn",sn);
    communityAccessControl.reject(map, new ISuccessFailureCallback() {
        @Override
        public void onSuccess() {

        }

        @Override
        public void onFailure(String s, String s1) {

        }
    });
}

Record a call duration

Records the duration of a call after a user hangs up or receives the hangup notification from the device.

API description

void hangUp(Map<String, String> params, final ISuccessFailureCallback callback);

Request parameter

Parameter	Description
params	The Map object that carries data.

params

Parameter	Description
device_id	The device ID.
community_id	The community ID.
room_id	The room ID.
call_time	The start timestamp of the call.
talk_time	The duration of the call. Unit: seconds.
direction	The direction of the call. The value is 1.

Example

long startTime= System.currentTimeMillis();
int talkTime= 20;
if (communityAccessControl != null) {
    Map<String, String> map = new HashMap<>();
    map.put("device_id",deviceId);
    map.put("community_id",communityId);
    map.put("room_id",roomId);
    map.put("call_time",startTime);
 		map.put("talk_time",talkTime);
    map.put("direction","1");
    communityAccessControl.hangUp(map, new ISuccessFailureCallback() {
        @Override
        public void onSuccess() {

        }

        @Override
        public void onFailure(String s, String s1) {

        }
    });
}