Live Streaming

Last Updated on : 2024-06-27 09:40:24download

The IPC SDK for iOS provides basic IPC capabilities, such as live streaming, playing back footage on an SD card, video screenshots, video recording, and video talk. YUV data is available after video frames are decoded. You can further process raw video stream data based on the YUV data.

Process

After the ThingSmartCameraType instance object is created, live streaming can be started. Video data transmission and command interaction are implemented through peer-to-peer (P2P) connections. The following figure shows the streaming process.

Live Streaming

Create a P2P connection

Before live streaming, a P2P connection must be created. The P2P connection status is subject to your maintenance. The IPC SDK can only send commands to and receive responses from IPCs.

API description

To create a P2P connection, you can set the parameter mode to prioritize connections over a local area network (LAN) or the internet. When LAN connections are prioritized, if TCP connections are not created between the app and an IPC over the same LAN, or the IPC does not support prioritized LAN connections, the SDK automatically selects connections over the internet.

- (void)connectWithMode:(ThingSmartCameraConnectMode)mode;

Make sure to call ThingSmartCameraType related methods on the same thread to avoid exceptions.

Parameters

Parameter Description
mode The prioritized connection mode.

ThingSmartCameraConnectMode

Enum value Description
ThingSmartCameraConnectAuto Automatically selected
ThingSmartCameraConnectFromInternet Internet connections prioritized
ThingSmartCameraConnectFromLocal LAN connections prioritized

API description

Close a P2P connection

- (void)disConnect;

Delegate callback

  • SDK initialization fails.

    - (void)cameraInitFailed:(ThingSmartCameraErrorCode)errorCode;
    
  • A P2P connection is created.

    - (void)cameraDidConnected:(id<ThingSmartCameraType>)camera;
    
  • A P2P connection is closed. This method is called only in the case of a passive disconnection. For example, the connection is closed in poor network conditions or the IPC stops the connection. For more information, see Error Codes.

    - (void)cameraDisconnected:(id<ThingSmartCameraType>)camera specificErrorCode:(NSInteger)errorCode
    

Start live streaming

After a P2P connection is created, live streaming can be started.

API description

  • Start live streaming with specified definition.

    - (void)startPreviewWithDefinition:(ThingSmartCameraDefinition)definition;
    
  • Stop live streaming.

    - (void)stopPreview;
    
  • Get the video rendering view. Customization is recommended.

    - (UIView<ThingSmartVideoViewType> *)videoView;
    
  • Get the local video rendering view. Customization is recommended.

    - (UIView<ThingSmartVideoViewType> *)localVideoView;
    
  • Bind with a custom video rendering view.

    - (void)registerVideoRenderView:(UIView<ThingSmartVideoViewType> *)videoView;
    
  • Unbind from a custom video rendering view.

    - (void)uninstallVideoRenderView:(UIView<ThingSmartVideoViewType> *)videoView;
    
  • Bind with a custom local video rendering view.

    - (void)bindLocalVideoView:(UIView<ThingSmartVideoViewType> *)videoView;
    
  • Unbind from a custom local video rendering view.

    - (void)unbindLocalVideoView:(UIView<ThingSmartVideoViewType> *)videoView;
    

The binding and unbinding methods must be used as a pair to prevent functionality issues.

Delegate callback

  • Live streaming has started.

    - (void)cameraDidBeginPreview:(id<ThingSmartCameraType>)camera;
    
  • Live streaming is stopped.

    - (void)cameraDidStopPreview:(id<ThingSmartCameraType>)camera;
    

Render video

After live streaming starts, the IPC SDK automatically renders the received video streams. You can call the method - (UIView<ThingSmartVideoViewType> *)videoView of the camera object to get the rendering view. Then, you can add the view to the video screen and set the layout properties.

Protocol description

Protocol name Description
ThingSmartVideoViewType The video renderer protocol. The video rendering implementation varies depending on different IPC implementation solutions.

API description

  • The image zoom properties.

    • Default value: NO. In this case, if the aspect ratio of the view is different from that of the video image, black bars will appear on the top and bottom or left and right of the image.
    • If the value is set to YES, the image will stretch to fill the entire view and might cause image distortion.

    The IPC SDK currently does not support the full-screen playing capability. If you set scaleToFill to YES to enable the full-screen mode, the screen orientations and layout are subject to your maintenance.

    @property (nonatomic, assign) BOOL scaleToFill;
    
  • Clear the current image and cached video frames.

    - (void)thing_clear;
    
  • Capture a screenshot of the current rendered image.

    - (UIImage *)screenshot;
    

The ThingSmartCameraType object provides the autoRender property. The default value is YES, indicating the IPC SDK automatically renders video images. To disable automatic rendering, you can set the value to NO, call the delegate method to get the YUV data of each video frame, and then develop your preferred video rendering solution. For more information about the required method, see Raw stream data.

Failure callback

The IPC delegate provides the callback to return the error messages that have occurred during IPC operations.

API description

- (void)camera:(id<ThingSmartCameraType>)camera didOccurredErrorAtStep:(ThingCameraErrorCode)errStepCode specificErrorCode:(NSInteger)errorCode;

The following method adds extended error information for more detailed error codes, making it the preferred choice for callbacks.

- (void)camera:(id<ThingSmartCameraType>)camera didOccurredErrorAtStep:(ThingCameraErrorCode)errStepCode specificErrorCode:(NSInteger)errorCode extErrorCodeInfo:(id<ThingSmartCameraExtErrorCodeInfo>)extErrorCodeInfo;

Parameters

Parameter Description
camera The IPC object that causes an error.
errStepCode The operation in which the error has occurred.
errorCode The error code that indicates the cause of the error. For more information, see Error Codes.

Enum values of ThingCameraErrorCode

Enum value Description
Thing_ERROR_NONE None
Thing_ERROR_CONNECT_FAILED Failed to create a P2P connection.
Thing_ERROR_START_PREVIEW_FAILED Failed to stream live videos.
Thing_ERROR_START_PLAYBACK_FAILED Failed to play back footage on an SD card.
Thing_ERROR_PAUSE_PLAYBACK_FAILED Failed to pause playback of footage on an SD card.
Thing_ERROR_RESUME_PLAYBACK_FAILED Failed to resume playback of footage on an SD card.
Thing_ERROR_ENABLE_MUTE_FAILED Failed to unmute or mute the video.
Thing_ERROR_START_TALK_FAILED Failed to start video talk.
Thing_ERROR_RECORD_FAILED Failed to record videos.
Thing_ERROR_ENABLE_HD_FAILED Failed to set video definition.
Thing_ERROR_GET_HD_FAILED Failed to get video definition.
Thing_ERROR_QUERY_RECORD_DAY_FAILED Failed to query the date of footage on an SD card.
Thing_ERROR_QUERY_TIMESLICE_FAILED Failed to query footage on an SD card.

Example

Objective-C:

#define kThingSmartIPCConfigAPI @"thing.m.ipc.config.get"
#define kThingSmartIPCConfigAPIVersion @"2.0"

- (void)startStream {
    if (self.connected) {
        [self.camera startPreview];
        return;
    }
    id p2pType = [self.deviceModel.skills objectForKey:@"p2pType"];
    self.camera = [ThingSmartCameraFactory cameraWithP2PType:p2pType deviceId:self.deviceModel.devId delegate:self];
    [self.camera connectWithMode:ThingSmartCameraConnectAuto];
}

#pragma mark - ThingSmartCameraDelegate

- (void)cameraDidConnected:(id<ThingSmartCameraType>)camera {
    self.connected = YES;
    // Starts previewing only after a P2P connection is created.
        [camera startPreview];
}

- (void)cameraDisconnected:(id<ThingSmartCameraType>)camera specificErrorCode:(NSInteger)errorCode {
    // A P2P connection is closed due to unstable network conditions.
    self.connected = NO;
    self.previewing = NO;
}

- (void)cameraDidBeginPreview:(id<ThingSmartCameraType>)camera {
        // Starts live video streaming.
    self.previewing = YES;
    // Adds a rendered video view to a video screen.
        [self.view addSubview:camera.videoView];
}

- (void)cameraDidStopPreview:(id<ThingSmartCameraType>)camera {
        // Stops live video streaming.
    self.previewing = NO;
}

// The failure callback.
- (void)camera:(id<ThingSmartCameraType>)camera didOccurredErrorAtStep:(ThingCameraErrorCode)errStepCode specificErrorCode:(NSInteger)errorCode extErrorCodeInfo:(id<ThingSmartCameraExtErrorCodeInfo>)extErrorCodeInfo {
        if (errStepCode == Thing_ERROR_CONNECT_FAILED) {
        // Failed to create a P2P connection.
        self.connected = NO;
    }
    else if (errStepCode == Thing_ERROR_START_PREVIEW_FAILED) {
        // Failed to stream live videos.
        self.previewing = NO;
    }
}

Swift:

func startStream() {
    if self.isConnected {
        self.camera.startPreview()
        return
    }
    let p2pType = self.deviceModel.skills["p2pType"]!
    self.camera = ThingSmartCameraFactory.camera(withP2PType: p2pType, deviceId: self.deviceModel.devId, delegate: self)
    self.camera.connect(withMode:ThingSmartCameraConnectAuto)
}

func cameraDidConnected(_ camera: ThingSmartCameraType!) {
    self.isConnected = true
    // Starts previewing only after a P2P connection is created.
    camera.startPreview()
}

func cameraDisconnected(_ camera: ThingSmartCameraType!, specificErrorCode: Int) {
    // A P2P connection is closed due to unstable network conditions.
    self.isConnected = false
    self.isPreviewing = false
}

func cameraDidBeginPreview(_ camera: ThingSmartCameraType!) {
    // Starts live video streaming.
    self.isPreviewing = true;
    // Adds a rendered video view to a video screen.
    self.view.addSubview(camera.videoView())
}

func cameraDidStopPreview(_ camera: ThingSmartCameraType!) {
    // Stops live video streaming.
    self.isPreviewing = false
}

func camera(_ camera: ThingSmartCameraType!, didOccurredErrorAtStep errStepCode: ThingCameraErrorCode, specificErrorCode errorCode: Int, extErrorCodeInfo: ThingSmartCameraExtErrorCodeInfo!) {
    if errStepCode == Thing_ERROR_CONNECT_FAILED {
        // Failed to create a P2P connection.
        self.isConnected = false
    }else if errStepCode == Thing_ERROR_START_PREVIEW_FAILED {
        // Failed to stream live videos.
        self.isPreviewing = false
    }
}

After the app runs in the background, video streaming must be stopped. Video data is processed with hardware decoding and Open Graphics Library (OpenGL) rendering. Therefore, video streaming with the app running in the background might cause the app to crash. In most cases, each Tuya-enabled IPC supports up to five concurrent P2P connections with five mobile phones. We recommend that the app closes the P2P connections after running in the background for a certain period to release resources.