Live Video Streaming

Tuya IPC SDK for iOS provides basic IPC capabilities, such as live video streaming, playing back footage on an SD card, video screenshots, video recording, and live video talk with IPCs. YUV data is available after video frames are decoded. You can further process raw video stream data based on the YUV data.

Process

After the ThingSmartCameraType instance object is created, live video streaming can be started. Video data transmission and command interaction are implemented through peer-to-peer (P2P) connections. The following figure shows the streaming process.

Create a P2P connection

Before live video streaming, a P2P connection must be created. The P2P connection status is subject to your maintenance. The IPC SDK can only send commands to and receive responses from IPCs.

API description

To create a P2P connection, you can set the parameter mode to prioritize connections over a local area network (LAN) or the internet. When LAN connections are prioritized, if TCP connections are not created between the app and an IPC over the same LAN, or the IPC does not support prioritized LAN connections, the SDK automatically selects connections over the internet.

- (void)connectWithMode:(ThingSmartCameraConnectMode)mode;

Parameters

ThingSmartCameraConnectMode

API description

Close a P2P connection

- (void)disConnect;

Delegate callback

• Indicates the failed SDK initialization.

  - (void)cameraInitFailed:(ThingSmartCameraErrorCode)errorCode;
  

• Indicates that a P2P connection is created.

  - (void)cameraDidConnected:(id<ThingSmartCameraType>)camera;
  

• Indicates that a P2P connection is closed. This method is called only in the case of a passive disconnection. For example, the connection is closed in poor network conditions or the IPC stops the connection. For more information, see Error Codes.

  - (void)cameraDisconnected:(id<ThingSmartCameraType>)camera specificErrorCode:(NSInteger)errorCode
  

Start live video streaming

After a P2P connection is created, live video streaming can be started.

API description

• Starts live video streaming.

  - (void)startPreview;
  

• Starts live video streaming with specified definition.

  - (void)startPreviewWithDefinition:(ThingSmartCameraDefinition)definition;
  

• Stops live video streaming.

  - (void)stopPreview;
  

• Returns a video rendering view.

  - (UIView<ThingSmartVideoViewType> *)videoView;
  

Delegate callback

• Indicates that live video streaming is started.

  - (void)cameraDidBeginPreview:(id<ThingSmartCameraType>)camera;
  

• Indicates that live video streaming is stopped.

  - (void)cameraDidStopPreview:(id<ThingSmartCameraType>)camera;
  

Video rendering

After live video streaming is started, the IPC SDK automatically renders the received video streams. You can call the method - (UIView<ThingSmartVideoViewType> *)videoView of the camera object to get the rendering view. Then, you can add the view to the video screen and set the layout properties.

Protocol description

API description

• The image zoom properties.

  • Default value: NO. In this case, if the aspect ratio of the view is different from that of the video image, black bars will appear on the top and bottom or left and right of the image.
  • If the value is set to YES, the image will stretch to fill the entire view and might cause image distortion.
  The IPC SDK currently does not support the full-screen playing capability. If you set scaleToFill to YES to enable the full-screen mode, the screen orientations and layout are subject to your maintenance.

  @property (nonatomic, assign) BOOL scaleToFill;
  

• Clears the current image and cached video frames.

  - (void)thing_clear;
  

• Captures a screenshot of the current rendered image.

  - (UIImage *)screenshot;
  

Failure callback

The IPC delegate provides the callback to return the error messages that have occurred during IPC operations.

API description

- (void)camera:(id<ThingSmartCameraType>)camera didOccurredErrorAtStep:(ThingCameraErrorCode)errStepCode specificErrorCode:(NSInteger)errorCode;

Parameters

Enum values of ThingCameraErrorCode

Example

Objective-C:

#define kThingSmartIPCConfigAPI @"thing.m.ipc.config.get"
#define kThingSmartIPCConfigAPIVersion @"2.0"

- (void)startStream {
    if (self.connected) {
        [self.camera startPreview];
        return;
    }
    id p2pType = [self.deviceModel.skills objectForKey:@"p2pType"];
    [[ThingSmartRequest new] requestWithApiName:kThingSmartIPCConfigAPI postData:@{@"devId": self.devId} version:kThingSmartIPCConfigAPIVersion success:^(id result) {
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            ThingSmartCameraConfig *config = [ThingSmartCameraFactory ipcConfigWithUid:[ThingSmartUser sharedInstance].uid localKey:self.deviceModel.localKey configData:result];
            self.camera = [ThingSmartCameraFactory cameraWithP2PType:p2pType config:config delegate:self];
            [self.camera connect];
        });
    } failure:^(NSError *error) {
                // Failed to get the configurations.
    }];
}

#pragma mark - ThingSmartCameraDelegate

- (void)cameraDidConnected:(id<ThingSmartCameraType>)camera {
    self.connected = YES;
    // Starts previewing only after a P2P connection is created.
        [camera startPreview];
}

- (void)cameraDisconnected:(id<ThingSmartCameraType>)camera specificErrorCode:(NSInteger)errorCode {
    // A P2P connection is closed due to unstable network conditions.
    self.connected = NO;
    self.previewing = NO;
}

- (void)cameraDidBeginPreview:(id<ThingSmartCameraType>)camera {
        // Starts live video streaming.
    self.previewing = YES;
    // Adds a rendered video view to a video screen.
        [self.view addSubview:camera.videoView];
}

- (void)cameraDidStopPreview:(id<ThingSmartCameraType>)camera {
        // Stops live video streaming.
    self.previewing = NO;
}

// The failure callback.
- (void)camera:(id<ThingSmartCameraType>)camera didOccurredErrorAtStep:(ThingCameraErrorCode)errStepCode specificErrorCode:(NSInteger)errorCode {
        if (errStepCode == Thing_ERROR_CONNECT_FAILED) {
        // Failed to create a P2P connection.
        self.connected = NO;
    }
    else if (errStepCode == Thing_ERROR_START_PREVIEW_FAILED) {
        // Failed to stream live videos.
        self.previewing = NO;
    }
}

Swift:

func startStream() {
    if self.isConnected {
        self.camera.startPreview()
        return
    }
    let p2pType = self.deviceModel.skills["p2pType"]!
    ThingSmartRequest().request(withApiName: kThingSmartIPCConfigAPI, postData: ["devId": self.devId], version: kThingSmartIPCConfigAPIVersion, success: { result in
        guard let responder = result as? [AnyHashable:Any] else {
            return;
        }
        DispatchQueue.global().async {
            let config = ThingSmartCameraFactory.ipcConfig(withUid: ThingSmartUser.sharedInstance().uid, localKey: self.deviceModel.localKey, configData: responder)
            self.camera = ThingSmartCameraFactory.camera(withP2PType: p2pType, config: config, delegate: self)
            self.camera.connect()
        }
    }) { _ in
        // Failed to get the configurations.
    }
}

func cameraDidConnected(_ camera: ThingSmartCameraType!) {
    self.isConnected = true
    // Starts previewing only after a P2P connection is created.
    camera.startPreview()
}

func cameraDisconnected(_ camera: ThingSmartCameraType!, specificErrorCode: Int) {
    // A P2P connection is closed due to unstable network conditions.
    self.isConnected = false
    self.isPreviewing = false
}

func cameraDidBeginPreview(_ camera: ThingSmartCameraType!) {
    // Starts live video streaming.
    self.isPreviewing = true;
    // Adds a rendered video view to a video screen.
    self.view.addSubview(camera.videoView())
}

func cameraDidStopPreview(_ camera: ThingSmartCameraType!) {
    // Stops live video streaming.
    self.isPreviewing = false
}

func camera(_ camera: ThingSmartCameraType!, didOccurredErrorAtStep errStepCode: ThingCameraErrorCode, specificErrorCode errorCode: Int) {
    if errStepCode == Thing_ERROR_CONNECT_FAILED {
        // Failed to create a P2P connection.
        self.isConnected = false
    }else if errStepCode == Thing_ERROR_START_PREVIEW_FAILED {
        // Failed to stream live videos.
        self.isPreviewing = false
    }
}