Quick Start with IPC SDK for Android

Tuya IPC SDK for Android is developed based on Tuya Smart Life App SDK. IoT mobile apps have gained popularity in smart device control. However, the increasing number of device categories has become a great challenge for application development. To simplify your development process, the Smart Life App SDK provides a bunch of extension SDKs. For example, the IPC SDK abstracts useful features from IP cameras (IPCs) and encapsulates APIs for communication with the devices. This SDK saves you a great deal of effort in application development.

This tutorial walks you through a few steps to get started with the IPC SDK and develop a preferred IoT app within one hour. You will learn how to implement the following features:

• Pair an IPC with the app.
• Preview the live video images collected by the IPC.
• Play back video footage stored on an SD card.
• Record the live video images from the IPC by using the app.

You can go to GitHub and download the sample code for the different features mentioned in this tutorial. Then, you can integrate the required sample code into your SDK development on the Tuya IoT Platform.

App SDK Development GitHub Sample

Preview

This tutorial along with specific panel development helps you create the following sample app for Android.

Preparation

Before you start, the following requirements must be met:

1. An account of the Tuya IoT Platform is registered, an app is built on the platform, and the values of AppKey and AppSecret of the SDK are obtained. For more information, see Preparation.
   The IPC SDK depends on the Smart Life App SDK. Therefore, you must implement certain features such as creating an app account and adding a home before you can turn IPC features into reality based on the SDK. For more information, see Quick Start with Smart Life App SDK for Android.

2. A Powered by Tuya IPC product is created. To get the product, visit TuyaGo.
3. Tuya Smart Life App SDK for Android is integrated into your project with Android Studio. For more information, see Integrate IPC SDK for Android App.

Device pairing

Powered by Tuya IPCs support all pairing modes that are enabled by the Smart Life App SDK, for example, Wi-Fi access point (AP) or hotspot pairing and Bluetooth pairing. Especially, Tuya allows users to scan a QR code to pair an IPC. If IPCs cannot scan the QR code, the Wi-Fi Easy Connect (EZ) mode is recommended. This section describes how to implement device pairing in QR code mode.

Request a token

Similar to the Wi-Fi EZ and AP pairing modes, before the QR code pairing process, the SDK must get a pairing token from the cloud in the networked state.

The token is valid for 10 minutes and expires immediately after the device is paired. A new token must be generated if the device needs to be paired again. To get a token, the current value of homeId must be provided. Therefore, during this process, the account must be in the logged-in state and at least one home is created for the account.

ThingHomeSdk.getActivatorInstance().getActivatorToken(homeId,
		new IThingActivatorGetToken() {

			@Override
			public void onSuccess(String token) {

			}

			@Override
			public void onFailure(String s, String s1) {

			}
		});

Generate a QR code

After you get the pairing token and the SSID and password of the current Wi-Fi network, execute the callback of onQRCodeSuccess to initialize the pairing parameters and get a URL string. Then, you can use the string to generate a QR code.

The following dependency is required: zxing(implementation 'com.google.zxing:core:3.2.1').

// Returns the device pairing token.
            ThingHomeSdk.getActivatorInstance().getActivatorToken(homeId,
                    new IThingActivatorGetToken() {
                        @Override
                        public void onSuccess(String token) {
                            // Generates and shows the QR code.
                            ThingCameraActivatorBuilder builder = new ThingCameraActivatorBuilder()
                                    .setToken(token)
                                    .setPassword(wifiPwd)
                                    .setTimeOut(100)
                                    .setContext(QrCodeConfigActivity.this)
                                    .setSsid(wifiSSId)
                                    .setListener(new IThingSmartCameraActivatorListener() {
                                        @Override
                                        public void onQRCodeSuccess(String qrcodeUrl) {
                                            final Bitmap bitmap;
                                            try {
                                                bitmap = QRCodeUtil.createQRCode(qrcodeUrl, 300);
                                                QrCodeConfigActivity.this.runOnUiThread(new Runnable() {
                                                    @Override
                                                    public void run() {
                                                        mIvQr.setImageBitmap(bitmap);
                                                        mLlInputWifi.setVisibility(View.GONE);
                                                        mIvQr.setVisibility(View.VISIBLE);
                                                    }
                                                });
                                            } catch (WriterException e) {
                                                e.printStackTrace();
                                            }
                                        }

                                        @Override
                                        public void onError(String errorCode, String errorMsg) {

                                        }

                                        @Override
                                        public void onActiveSuccess(DeviceBean devResp) {
                                            Toast.makeText(QrCodeConfigActivity.this,"config success!",Toast.LENGTH_LONG).show();
                                        }
                                    });

                        }

Example:

public static Bitmap createQRCode(String url, int widthAndHeight)
            throws WriterException {
        Hashtable hints = new Hashtable();
        hints.put(EncodeHintType.CHARACTER_SET, "utf-8");
        hints.put(EncodeHintType.MARGIN,0);
        BitMatrix matrix = new MultiFormatWriter().encode(url,
                BarcodeFormat.QR_CODE, widthAndHeight, widthAndHeight, hints);

        int width = matrix.getWidth();
        int height = matrix.getHeight();
        int[] pixels = new int[width * height];
        for (int y = 0; y < height; y++) {
            for (int x = 0; x < width; x++) {
                if (matrix.get(x, y)) {
                    pixels[y * width + x] = BLACK; //0xff000000
                }
            }
        }
        Bitmap bitmap = Bitmap.createBitmap(width, height,
                Bitmap.Config.ARGB_8888);
        bitmap.setPixels(pixels, 0, width, 0, 0, width, height);
        return bitmap;
    }

Start pairing

After a QR code is generated, guide users to point it to the IPC. Then, the device will get the QR code information and emit a prompt tone. After the pairing process is started, the SDK continuously broadcasts the pairing data until a device is paired or the process times out.

Call the API method to start pairing. The builder object of ThingCameraActivatorBuilder has been created. You only need to use the following code block to make the API request:

 mThingActivator = ThingHomeSdk.getActivatorInstance().newCameraDevActivator(builder);
        mThingActivator.createQRCode();
        mThingActivator.start();

Listen for the pairing result

When you set pairing parameters, the builder object of ThingCameraActivatorBuilder is used to create a listener. Then, the callback of IThingSmartCameraActivatorListener can be executed to listen for the pairing result.

public interface IThingSmartCameraActivatorListener {
    void onQRCodeSuccess(String qrcodeUrl);

    void onError(String errorCode, String errorMsg);

    void onActiveSuccess(DeviceBean devResp);
}

Stop pairing

To allow users to cancel or complete pairing during the process, you must call the API method to stop pairing.

mThingActivator.stop();

Live video streaming

Tuya IPC SDK for Android provides basic IPC capabilities, such as live video streaming, playing back footage on an SD card, video screenshots, video recording, and live video talk with IPCs.

Step 1: Initialize an object

You must first initialize an IThingSmartCameraP2P object. This object is used to read properties and methods from IPCs.

ThingCameraView is the rendered view provided by the IPC SDK and allows you to quickly build and render video previews.

    // 1. Creates `IThingSmartCameraP2P`.
    IThingSmartCameraP2P mCameraP2P = null;
    ITuyaIPCCore cameraInstance = TuyaIPCSdk.getCameraInstance();
    if (cameraInstance != null) {
        mCameraP2P = cameraInstance.createCameraP2P(devId));
    }
    ThingCameraView mVideoView = findViewById(R.id.camera_video_view);
    // 2. Creates the callback of the view rendering container.
    mVideoView.setViewCallback(new AbsVideoViewCallback() {
        @Override
        public void onCreated(Object view) {
            super.onCreated(view);
            //3. Binds the rendered view with `IThingSmartCameraP2P`.
            if (null != mCameraP2P){
                mCameraP2P.generateCameraView(view);
            }
        }
    });
    // 4. Builds the rendered view.
    mVideoView.createVideoView(p2pType);
    // 5. Registers a P2P listener.
    AbsP2pCameraListener absP2pCameraListener = new AbsP2pCameraListener() {
        @Override
        public void onSessionStatusChanged(Object camera, int sessionId, int sessionStatus) {
                super.onSessionStatusChanged(o, i, i1);
                // when sessionStatus  is -3 (timeout) or -105 (auth failed), t is recommended to initiate a reconnection, but be careful to avoid loops
    }
                }
    };
    if (null != mCameraP2P){
        mCameraP2P.registerP2PCameraListener(absP2pCameraListener);
    }

Step 2: Create a P2P connection

At the end of initialization operations, a peer-to-peer (P2P) connection must be created. The P2P connection status is subject to your maintenance.

To create a P2P connection, you can set the parameter mode to prioritize connections over a local area network (LAN) or the internet. When LAN connections are prioritized, if TCP connections are not created between the app and an IPC over the same LAN, or the IPC does not support prioritized LAN connections, the SDK automatically selects connections over the internet. For more information, see P2P connection.

• Create a specified connection

  void connect(String devId, int mode, OperationDelegateCallBack callBack);
  

• Close a connection

  You can call the following API method to close a P2P connection or leave the current page.

  void disconnect(String devId, OperationDelegateCallBack callBack);
  

• Listen for the connection status

  To listen for the current connection status, use OperationDelegateCallBack to implement the callback.

  
  mCameraP2P.connect(devId, new OperationDelegateCallBack() {
      @Override
      public void onSuccess(int sessionId, int requestId, String data) {
          // A P2P connection is created.
      }
  
      @Override
      public void onFailure(int sessionId, int requestId, int errCode) {
          // Failed to create a P2P connection.
      }
  });
  }
  

Step 3: Preview live videos

After you create a P2P connection, you can implement live video streaming.

• Start previewing

  In the following code block, clarity specifies the definition mode. Valid values:

  • 2: standard definition (SD)
  • 4: high definition (HD)

  void startPreview(int clarity, OperationDelegateCallBack callBack);
  

• Stop previewing

  int stopPreview(OperationDelegateCallBack callBack);
  

• Listen for the connection status

  Implement the following code block to listen for the previewing status. This implementation also applies to stopPreview.

  mCameraP2P.startPreview(new OperationDelegateCallBack() {
      @Override
      public void onSuccess(int sessionId, int requestId, String data) {
          // Live video streaming is started.
      }
  
      @Override
      public void onFailure(int sessionId, int requestId, int errCode) {
          // Failed to start live video streaming.
      }
  });
  

Step 4: Remove the process

If the IPC video features are not required, the P2P listener must be removed and the P2P object must be destroyed.

@Override
public void onDestroy() {
    if (null != mCameraP2P) {
        mCameraP2P.removeOnP2PCameraListener(absP2pCameraListener);
        mCameraP2P.destroyP2P();
    }
}

Video recording and live video talk

During live video streaming, video recording and live video talk are supported. For more information, see Audio and Video Features.

• Start recording or live video talk

  int startRecordLocalMp4(String folderPath, Context context, OperationDelegateCallBack callBack);
  void startAudioTalk(OperationDelegateCallBack callBack);
  

• Stop recording or live video talk

  int stopRecordLocalMp4(OperationDelegateCallBack callBack);
  void stopAudioTalk(OperationDelegateCallBack callBack);
  

• Listen for status

  To listen for the status of recording and live video talk, use OperationDelegateCallBack to implement the callback.

  public interface OperationDelegateCallBack {
      void onSuccess(int sessionId, int requestId, String data);
  
      void onFailure(int sessionId, int requestId, String data);
  }
  

Playback of video footage

After video footage is saved to an SD card, it can be played back on the app based on the IPC SDK.

Similar to live video streaming, a P2P connection must be created before video footage playback.

The IPC SDK supports query and playback of video footage by day. Users can also query the dates on which video footage is available in a certain month. OperationDelegateCallBack returns the query result.

Query a list of video footage

Create an IThingSmartCameraP2P object to build a P2P connection. After a P2P connection is created, you can implement querying information about video footage stored on an SD card.

Step 1: Query the dates on which video footage is available in a certain month

int year = Integer.parseInt(substring[0]);
int mouth = Integer.parseInt(substring[1]);
queryDay = Integer.parseInt(substring[2]);
mCameraP2P.queryRecordDaysByMonth(year, mouth, new OperationDelegateCallBack() {
  @Override
  public void onSuccess(int sessionId, int requestId, String data) {
    // `data` indicates the returned dates of the month.
    MonthDays monthDays = JSONObject.parseObject(data, MonthDays.class);
    mBackDataMonthCache.put(mCameraP2P.getMonthKey(), monthDays.getDataDays());
    mHandler.sendMessage(MessageUtil.getMessage(MSG_DATA_DATE, ARG1_OPERATE_SUCCESS, data));
  }

  @Override
  public void onFailure(int sessionId, int requestId, int errCode) {
    mHandler.sendMessage(MessageUtil.getMessage(MSG_DATA_DATE, ARG1_OPERATE_FAIL));
  }
});

Step 2: Query the list of video footage stored on a certain date

Users can query the list of video footage on the dates returned in the previous step.

int year = Integer.parseInt(substring[0]);
int mouth = Integer.parseInt(substring[1]);
int day = Integer.parseInt(substring[2]);
mCameraP2P.queryRecordTimeSliceByDay(year, mouth, day, new OperationDelegateCallBack() {
  @Override
  public void onSuccess(int sessionId, int requestId, String data) {
    // `data` indicates the returned video footage for the specified date.
    parsePlaybackData(data);
  }

  @Override
  public void onFailure(int sessionId, int requestId, int errCode) {
    mHandler.sendEmptyMessage(MSG_DATA_DATE_BY_DAY_FAIL);
  }
});

private void parsePlaybackData(Object obj) {
        RecordInfoBean recordInfoBean = JSONObject.parseObject(obj.toString(), RecordInfoBean.class);
        if (recordInfoBean.getCount() != 0) {
            List<TimePieceBean> timePieceBeanList = recordInfoBean.getItems();
            if (timePieceBeanList != null && timePieceBeanList.size() != 0) {
                mBackDataDayCache.put(mCameraP2P.getDayKey(), timePieceBeanList);
            }
            mHandler.sendMessage(MessageUtil.getMessage(MSG_DATA_DATE_BY_DAY_SUCC, ARG1_OPERATE_SUCCESS));
        } else {
            mHandler.sendMessage(MessageUtil.getMessage(MSG_DATA_DATE_BY_DAY_FAIL, ARG1_OPERATE_FAIL));
        }
    }

In this code block, the following two entity object classes are used:

• RecordInfoBean data model

  Parameter	Description
  count	The number of video clips.
  List	The list of video clips.

• TimePieceBean data model

  Parameter	Description
  startTime	The start time of a video clip.
  endTime	The end time of a video clip.
  playTime	The start time when playback of a video clip is started.

Then, users can get the video footage available for playback.

Play back video footage

After the target video footage is found, it can be played back.

• Start playback

  mCameraP2P.startPlayBack(timePieceBean.getStartTime(),
                          timePieceBean.getEndTime(),
                          timePieceBean.getStartTime(), new OperationDelegateCallBack() {
                          @Override
                          public void onSuccess(int sessionId, int requestId, String data){
                              isPlayback = true;
                          }
  
                          @Override
                          public void onFailure(int sessionId, int requestId, int errCode){
                              isPlayback = false;
                          }
                          }, new OperationDelegateCallBack() {
                          @Override
                          public void onSuccess(int sessionId, int requestId, String data){
                              isPlayback = false;
                          }
  
                          @Override
                          public void onFailure(int sessionId, int requestId, int errCode){
                              isPlayback = false;
                          }
                          });
  

  This method includes the following three timestamp parameters:

  • playTime: the time of the initial frame to be played back. It can be set to the same value as startTime.
  • startTime: the start time of playback. It is the value of startTime in the TimePieceBean data model.
  • endTime: the end time of playback. It is the value of endTime in the TimePieceBean data model.

• Pause the playback

  void pausePlayBack(OperationDelegateCallBack callBack);
  

• Resume the playback

  void resumePlayBack(OperationDelegateCallBack callBack);
  

• Stop playback

  void stopPlayBack(OperationDelegateCallBack callBack);
  

• Listen for the callback of OperationDelegateCallBack

  public interface OperationDelegateCallBack {
      void onSuccess(int sessionId, int requestId, String data);
  
      void onFailure(int sessionId, int requestId, String data);
  }
  

  Both pausePlayBack and stopPlayBack can be used to stop playback. When they are called, these rules must be followed:

  • After stopPlayBack is called, resumePlayback cannot be called to resume playback. To resume playback from the time when the playback is last stopped, the timestamp of the last stopped video frame and the duration of the video footage must be saved. This way, the startPlayBack method can be called to resume playback.
  • After the dates on which video footage is available are queried, startPlayBack can be called to resume playback of a video clip when another video clip is being played back or paused. In this case, you do not need to call stopPlayBack to stop the ongoing playback.
    :::

Concepts

The video recording types supported by Powered by Tuya IPCs are classified into continuous recording and event recording.

• Continuous recording: The duration of each video clip is 10 minutes and all video clips proceed continuously. If video recording is paused in the middle of the process, a time interval might exist between video clips in continuous recording mode.

  If all video clips on a certain date proceed continuously, a video clip is automatically followed by the next video clip during the playback. Therefore, after the playback start method is called with the first time point of the first video clip on that date, the playback will continue until the last video frame on that date. Then, the delegate callback of video playback is executed.

• Event recording: The duration of each video clip can be different and the interval between video clips can vary.

  The video clips on a certain date can also be interrupted. For example, an interval exists between Clips A and B. In this case, playback is automatically stopped at the last video frame of Clip A.

Next step

More features are available with the IPC SDK. You can build an app based on the sample project and instructions in the IPC SDK documents.