Cloud-Based Video Talk

Last Updated on : 2021-12-23 06:47:31download

Cloud-based video talk is implemented to connect an app with a cloud-based video intercom that supports video surveillance, access control, and more capabilities. This module relies on Tuya Smart Life IPC SDK and enables multiple features, such as video query, voice talk, and remote unlocking.

Feature overview

Integrate video surveillance

Use the cloud-based video surveillance management class to load configurations. After a successful callback, initialize a Powered by Tuya (PBT) IP camera (IPC) and connect to the video streaming channels to enable video previewing.

Integrate access control

  • Preview videos:

    Similar to the process of video surveillance, use the cloud-based video surveillance management class to load configurations. After a successful callback, initialize a PBT IPC and connect to the video streaming channels to enable video previewing.

  • Accept video calls:

    1. Register a device communication listener.
    2. When the device receives a request, the listener triggers a callback. The communication type and data included in the callback indicate information about the device, and device configurations are loaded. After a successful callback, connect to the video streaming channels of the PBT IPC and enable video previewing.
    3. With video previewing enabled, connect to the audio channels to enable voice talk.

Integrate the IPC SDK

The IPC SDK must be integrated. The following IPC SDK version is recommended:

implementation 'com.tuya.smart:tuyasmart-ipcsdk:3.27.5'

Cloud-based video talk management class

This class can be used to manage cloud-based video talk features. For example, load device configurations, register a device communication listener, query video surveillance device management, and query video access control devices.

Example

ITuyaCommunityVisualSpeak visualSpeak = TuyaCommunitySDK.getCommunityVisualSpeakInstance();

Load configurations

Loads device configurations. After the success callback, the subsequent process can continue and the audio and video communication between the app and the device can be started.

API description

void loadConfig(@NonNull String deviceId, @Nullable String gatewayId, ITuyaCommunityCallback callback);

Request parameter

Parameter Description
deviceId The device ID, required.
gatewayId The ID of a gateway to which the device is connected. Otherwise, set the value to null.
callback The callback.

Example

if (visualSpeak != null) {
    visualSpeak.loadConfig(deviceId, null, new ITuyaCommunityCallback() {
        @Override
        public void onFailure(String s, String s1) {
            L.e(TAG, "[loadConfig] onFailure:" + s1);
        }

        @Override
        public void onSuccess() {
            L.d(TAG, "[loadConfig] onSuccess");
            Intent intent = new Intent(VisualSpeakMainActivity.this, VisualSpeakVideoActivity.class);
            startActivity(intent);
        }
    });
}

Register a device communication listener

Registers a device communication listener. When the device sends a request to the app, the listener triggers a callback to indicate events. For example, the device calls the app, or the device hangs up the call.

API description

// Registers a listener.
void registerVisualSpeakCommunicationListener(IOnVisualSpeakDeviceCommunicationListener listener);

// Removes a listener.
void unRegisterVisualSpeakCommunicationListener(IOnVisualSpeakDeviceCommunicationListener listener);

Example

visualSpeak.registerVisualSpeakCommunicationListener(new IOnVisualSpeakDeviceCommunicationListener() {
    @Override
    public void receiveDeviceMessage(TYVisualSpeakCommunicationType tyVisualSpeakCommunicationType, TYDeviceMsgDataBean tyDeviceMsgDataBean) {
        switch (tyVisualSpeakCommunicationType) {
            case DEVICE_CALLING:
                break;
            case DEVICE_CALL_OTHER_APP_ANSWER:
                break;
            case DEVICE_CALL_CANCEL:
                break;
            case DEVICE_CALL_TIMEOUT:
                break;
            case DEVICE_CALL_HANGUP:
                break;
            case TALKING_TIMEOUT:
                break;
            case UNKNOWN:
            default:
                break;
        }
    }
});

TYVisualSpeakCommunicationType

Type Description
DEVICE_CALLING The device is calling the app.
DEVICE_CALL_OTHER_APP_ANSWER The device calls the app and another mobile phone with the app accepts the call.
DEVICE_CALL_CANCEL The device cancels the call.
DEVICE_CALL_TIMEOUT The device call timed out.
DEVICE_CALL_HANGUP The device hangs up the call.
TALKING_TIMEOUT The call automatically hangs up due to timeout.
UNKNOWN An unknown error has occurred.

TYDeviceMsgDataBean

Field Description
deviceId The device ID.
reqType The type of call.
sn The ID of the call.

Query devices for cloud-based video talk

Returns a list of video access control devices to enable remote unlocking, returns the device status, and stores call records.

API description

ITuyaCommunityAccessControl getAccessControlManager();

Query video surveillance devices

Returns a list of video surveillance devices and other required information.

API description

ITuyaCommunityPublicMonitor getPublicMonitorManager();

Video surveillance device in public areas

Users can check video surveillance devices in public areas by using the app.

Query a list of video surveillance devices in public areas

Returns a list of video surveillance devices in public areas.

API description

void getPublicMonitorList(String communityId,
                   String roomId,
                   ITuyaCommunityResultCallback<ArrayList<TYMonitorDeviceBean>> callback);

Request parameter

Parameter Description
communityId The community ID.
roomId The room ID.
callback The callback.

Data types of TYMonitorDeviceBean

Field name Field type Description
gatewayId String The gateway ID.
deviceId String The device ID.
deviceStatus Int The current device status. Valid values:
1: online
0: offline.
deviceName String The name of the device.
deviceThumbnailsUrl String The thumbnails of videos from the device.

Video access control

Each video access control device supports the following features:

  • Preview videos: Users can preview access control videos and unlock a door by tapping the app.
  • Accept device calls: When the device calls a house, the app receives the call. Users can unlock a door or talk to the device through the app.

Query a list of access control devices

Returns a list of access control devices assigned to a user.

API description

void getAccessControlList(String communityId,
                   String roomId,
                   ITuyaCommunityResultCallback<List<TYVisualSpeakDeviceBean>> callback);

Request parameter

Parameter Description
communityId The community ID.
roomId The room ID.
callback The callback.

Data types of TYVisualSpeakDeviceBean

Field name Field type Description
deviceId String The device ID.
roomId String The room ID.
deviceName String The name of the device.

Query the upper limit of access control video previews

Returns the maximum number of times the access control videos can be previewed. You can set this upper limit on the platform. The return value of this API method is an integer that indicates the daily preview quota.

API description

void getAccessControlViewableTimes(String communityId,
                            String roomId,
                            String deviceId,
                            ITuyaCommunityResultCallback<Integer> callback);

Request parameter

Parameter Description
communityId The community ID.
deviceId The device ID.
roomId The room ID.
callback The callback.

Query access control status

Indicates the access control status after the app receives a call from the device. The talk channel is built between the app and the device only when the device is in the calling state. Then, users can answer the call on the app.

API description

void getAccessControlStatus(String communityId,
                     String roomId,
                     String deviceId,
                     String sn,
                     ITuyaCommunityResultCallback<TYVisualSpeakCommunicationType> callback);

Request parameter

Parameter Description
communityId The community ID.
deviceId The device ID.
roomId The room ID.
sn The ID of the call.
callback The callback.

Unlock a door

Sends an unlocking command to the device.

API description

void openDoor(String communityId, String roomId, String deviceId, final ISuccessFailureCallback callback)

Request parameter

Parameter Description
communityId The community ID.
deviceId The device ID.
roomId The room ID.
callback The callback.

Answer a call

Notifies the device that the app has answered a call. An audio and video streaming channel is built between the app and the device after users tap the app to answer the call.

API description

void accept(String deviceId);

Request parameter

Parameter Description
deviceId The device ID.

Reject a call

Rejects a call after the device calls the app.

API description

void reject(Map<String, String> params, final ISuccessFailureCallback callback);

Request parameter

Parameter Description
params The Map object that carries data.
callback The callback.

params

Parameter Description
device_id The device ID.
community_id The community ID.
room_id The room ID.
sn The ID of the call.

Example

if (communityAccessControl != null) {
    Map<String, String> map = new HashMap<>();
    map.put("device_id",deviceId);
    map.put("community_id",communityId);
    map.put("room_id",roomId);
    map.put("sn",sn);
    communityAccessControl.reject(map, new ISuccessFailureCallback() {
        @Override
        public void onSuccess() {

        }

        @Override
        public void onFailure(String s, String s1) {

        }
    });
}

Record a call duration

Records the duration of a call after a user hangs up or receives the hangup notification from the device.

API description

void hangUp(Map<String, String> params, final ISuccessFailureCallback callback);

Request parameter

Parameter Description
params The Map object that carries data.

params

Parameter Description
device_id The device ID.
community_id The community ID.
room_id The room ID.
call_time The start timestamp of the call.
talk_time The duration of the call. Unit: seconds.
direction The direction of the call. The value is 1.

Example

long startTime= System.currentTimeMillis();
int talkTime= 20;
if (communityAccessControl != null) {
    Map<String, String> map = new HashMap<>();
    map.put("device_id",deviceId);
    map.put("community_id",communityId);
    map.put("room_id",roomId);
    map.put("call_time",startTime);
 		map.put("talk_time",talkTime);
    map.put("direction","1");
    communityAccessControl.hangUp(map, new ISuccessFailureCallback() {
        @Override
        public void onSuccess() {

        }

        @Override
        public void onFailure(String s, String s1) {

        }
    });
}