Playback of Video Stored on SD Card

Powered by Tuya (PBT) IP cameras (IPCs) support recording and playback of video footage that is stored on SD cards. You can integrate with the IPC SDK to develop these features for your app. After an SD card is inserted into an IPC, users can view the details and status of the SD card and set the recording switch and mode on the app.

Functional description

After video footage is saved to an SD card, it can be played back on the app based on the IPC SDK. Similar to live video streaming, a P2P connection must be created before video footage playback. Through a P2P connection, you can implement querying information about video footage stored on an SD card and playing back desired video clips.

For more information, see Memory Card Management for IPCs on iOS.

Flowchart

Video clips

The duration of the video footage stored on an SD card ranges from 10 seconds to 10 minutes. The IPC SDK supports query and playback of video footage by day. Users can also query the dates on which video footage is available in a certain month. The query result is returned by the delegate method of ThingSmartCameraDelegate.

Query dates with video footage stored in a month

API description

Returns the dates on which video footage is available in a certain month.

- (void)queryRecordDaysWithYear:(NSUInteger)year month:(NSUInteger)month;

Parameters

Parameter	Description
year	The year, such as 2020.
month	The month, such as 2.

Delegate callback

The callback of the query result.

- (void)camera:(id<ThingSmartCameraType>)camera didReceiveRecordDayQueryData:(NSArray<NSNumber *> *)days;

Parameters

Parameter	Description
camera	The target IPC object.
days	The array of dates on which video footage is available. For example, @[@(1), @(2)] indicates that video footage is available on the first and second days of the month. An empty array is returned if the request failed.

Query all video clips stored on a date

API description

Returns a list of video clips stored on a certain date.

- (void)queryRecordTimeSliceWithYear:(NSUInteger)year month:(NSUInteger)month day:(NSUInteger)day;

Parameters

Parameter	Description
year	The year, such as 2020.
month	The month, such as 2.
day	The day, such as 22.

Delegate callback

The callback of the query result.

- (void)camera:(id<ThingSmartCameraType>)camera didReceiveTimeSliceQueryData:(NSArray<NSDictionary *> *)timeSlices;

Parameters

Parameter	Description
camera	The target IPC object.
timeSlices	The array of time slices in which video clips are queried. An empty array is returned if the request failed.

Data types of timeSlices

timeSlices contains the elements of NSDictionary type.

Field (constant name of SDK)	Type	Description
kThingSmartPlaybackPeriodStartDate	NSDate	The start date of a video clip.
kThingSmartPlaybackPeriodStopDate	NSDate	The end date of a video clip.
kThingSmartPlaybackPeriodStartTime	NSNumber	The start Unix timestamp of a video clip.
kThingSmartPlaybackPeriodStopTime	NSNumber	The end Unix timestamp of a video clip.

Video playback

After the target video clip is found, it can be played back.

Start playback

API description

Starts playback of video footage. Valid values of playTime: [startTime, stopTime).

- (void)startPlayback:(NSInteger)playTime startTime:(NSInteger)startTime stopTime:(NSInteger)stopTime;

Parameters

Parameter	Description
playTime	The Unix timestamp starting from which a video clip is played back.
startTime	The start Unix timestamp of a video clip.
stopTime	The end Unix timestamp of a video clip.

Delegate callback

- (void)cameraDidBeginPlayback:(id<ThingSmartCameraType>)camera;

Pause playback

API description

- (void)pausePlayback;

Delegate callback

Video footage playback is paused.

- (void)cameraDidPausePlayback:(id<ThingSmartCameraType>)camera;

Resume playback

API description

- (void)resumePlayback;

Delegate callback

Video footage playback is resumed.

- (void)cameraDidResumePlayback:(id<ThingSmartCameraType>)camera;

Stop playback

API description

- (void)stopPlayback;

Delegate callback

Video footage playback is stopped.

- (void)cameraDidStopPlayback:(id<ThingSmartCameraType>)camera;

Callback of finished video footage playback

Video footage playback is finished. Status is 1，means the end of the clip, other values mean the end of the whole day

- (void)camera:(id<ThingSmartCameraType>)camera playbackTimeSlice:(NSDictionary *)timeSlice didFinishedWithStatus:(NSInteger)status;

Continuous playback

The video recording types supported by Powered by Tuya IPCs are classified into continuous recording and event recording.

• Continuous recording: The duration of each video clip is 10 minutes and all video clips proceed continuously. If video recording is paused in the middle of the process, a time interval might exist between video clips in continuous recording mode.

  If all video clips on a certain date proceed continuously, a video clip is automatically followed by the next video clip during the playback. Therefore, after the playback start method is called with the first time point of the first video clip on that date, the playback will continue until the last video frame on that date. Then, the delegate callback of video playback is executed.

• Event recording: The duration of each video clip can be different and the interval between video clips can vary.

  The video clips on a certain date can also be interrupted. For example, an interval exists between Clips A and B. In this case, playback is automatically stopped at the last video frame of Clip A. The IPC SDK does not receive the callback of stopped video footage playback.

  However, this feature has been optimized in the latest Tuya IPC SDK. Specifically, in event recording mode, at the end of each video clip, a callback is triggered. Therefore, after the delegate callback of stopped video footage playback, you can implement playback of the next video clip to achieve continuous playback. If the device firmware is not the latest version, you must call the method to return video frame data. The timestamp of the frame indicates whether the current video clip is the last frame and whether playback is stopped.

Pause and stop

Both pausePlayback and stopPlayback can be used to stop playback. When they are called, these rules must be followed:

• After stopPlayback is called, resumePlayback cannot be called to resume playback.
• To resume playback from the time when the playback is last stopped, the timestamp of the last stopped video frame and the duration of the video footage must be saved. This way, the startPlayback method can be called to resume playback.
• After the dates on which video footage is available are queried, startPlayback can be called to resume playback of a video clip when another video clip is being played back or paused. In this case, you do not need to call stopPlayback to stop the ongoing playback.

Set the playback speed

Certain IPCs support playback of video footage stored on an SD card at a specified speed. After the P2P connection is created, you can call an API method of the IPC SDK to query the list of playback speeds supported by the device. Then, the playback speed can be modified during the playback.

API description

Returns a list of playback speeds supported by the IPC. ThingSmartCameraPlayBackSpeed returns an array of enum type. This method is called only after a P2P connection is created.

- (NSArray<NSNumber *> *)getSupportPlaySpeedList;

Enum values of ThingSmartCameraPlayBackSpeed

Enum value	Description
ThingSmartCameraPlayBackSpeed_05TIMES	0.5x
ThingSmartCameraPlayBackSpeed_10TIMES	1x
ThingSmartCameraPlayBackSpeed_15TIMES	1.5x
ThingSmartCameraPlayBackSpeed_20TIMES	2x
ThingSmartCameraPlayBackSpeed_25TIMES	2.5x
ThingSmartCameraPlayBackSpeed_30TIMES	3x
ThingSmartCameraPlayBackSpeed_35TIMES	3.5x
ThingSmartCameraPlayBackSpeed_40TIMES	4x

API description

Sets the speed at which video footage stored on an SD card is played back.

- (void)speedPlayWithPlayBackSpeed:(ThingSmartCameraPlayBackSpeed)playBackSpeed;

Parameters

Parameter	Type	Description
playBackSpeed	ThingSmartCameraPlayBackSpeed	The video footage playback speed that must be supported by the device.

Video download

Certain IPCs support downloading video footage from an SD card to the app.

Query support for deletion

Checks whether the device supports downloading video footage from an SD card to the app. This method is called only after a P2P connection is created.

- (BOOL)isSupportPlaybackDownload;

Start downloading

API description

- (int)downloadPlayBackVideoWithRange:(NSRange)timeRange filePath:(NSString *)videoPath success:(void(^)(NSString *filePath))success progress:(void(^)(NSUInteger progress))progress failure:(void(^)(NSError *error))failure;

Parameters

Parameter	Type	Description
timeRange	NSRange	The period in which video footage is downloaded.
videoPath	NSString	The file path to save video records as MP4 files that are suffixed with .mp4.
success	Block	The success callback.
progress	Block	The callback of the download progress indicated by an integer. Valid values: 0 to 100.
failure	Block	The failure callback. A negative value indicates a failed download.

Pause video downloading

API description

- (int)pausePlayBackDownloadWithResponse:(void (^)(int))callback;

Parameters

Parameter	Type	Description
callback	Block	The callback. A value of 0 indicates that the download process is paused. Otherwise, an error has occurred.

Resume video downloading

API description

- (int)resumePlayBackDownloadWithResponse:(void (^)(int))callback;

Parameters

Parameter	Type	Description
callback	Block	The callback. A value of 0 indicates that the download process is resumed. Otherwise, an error has occurred.

Stop video downloading

API description

- (int)stopPlayBackDownloadWithResponse:(void (^)(int errCode))callback;

Parameters

Parameter	Type	Description
callback	Block	The callback. A value of 0 indicates that the download process is stopped. Otherwise, an error has occurred.

Video deletion

Certain IPCs support deleting video footage from an SD card. The deletion is controlled on the app. Currently, video footage can be deleted only by date.

Query support for deletion

Checks whether the device supports deleting video footage from an SD card. This method is called only after a P2P connection is created.

- (BOOL)isSupportPlaybackDelete;

Delete video stored on a specified date

API description

Deletes video footage stored on a specified date from the SD card.

- (int)deletePlayBackDataWithDay:(NSString *)day onResponse:(void (^)(int errCode))callback onFinish:(void (^)(int errCode))finishedCallBack;

Parameters

Parameter	Type	Description
day	NSString	The date in the format of yyyy-MM-dd.
callback	Block	The callback of the command sending result. A negative value of errCode indicates a failure. Otherwise, the command is sent as expected.
finishedCallBack	Block	The callback of the video deletion result. A negative value of errCode indicates a failure. Otherwise, all video clips on the specified date are deleted.

Example

ObjC:

#define kThingSmartIPCConfigAPI @"thing.m.ipc.config.get"
#define kThingSmartIPCConfigAPIVersion @"2.0"

- (void)startPlayback {
    if (self.connected) {
        [self.camera queryRecordTimeSliceWithYear:2020 month:2 day:12];
        return;
    }
    id p2pType = [self.deviceModel.skills objectForKey:@"p2pType"];
    [[ThingSmartRequest new] requestWithApiName:kThingSmartIPCConfigAPI postData:@{@"devId": self.devId} version:kThingSmartIPCConfigAPIVersion success:^(id result) {
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
            ThingSmartCameraConfig *config = [ThingSmartCameraFactory ipcConfigWithUid:[ThingSmartUser sharedInstance].uid localKey:self.deviceModel.localKey configData:result];
            self.camera = [ThingSmartCameraFactory cameraWithP2PType:p2pType config:config delegate:self];
            [self.camera connect];
        });
    } failure:^(NSError *error) {
        // Failed to get the configurations.
    }];
}

- (void)pausePlayback {
    [self.camera pausePlayback];
}

- (void)resumePlayback {
    [self.camera resumePlayback];
}

- (void)stopPlayback {
    [self.camera stopPlayback];
}

#pragma mark - ThingSmartCameraDelegate

- (void)cameraDidConnected:(id<ThingSmartCameraType>)camera {
    self.connected = YES;
      // Returns video clips stored on a certain date only after a P2P connection is created.
        [camera queryRecordTimeSliceWithYear:2020 month:2 day:12];
}

- (void)cameraDisconnected:(id<ThingSmartCameraType>)camera {
      // A P2P connection is closed due to unstable network conditions.
    self.connected = NO;
    self.playbacking = NO;
}

- (void)camera:(id<ThingSmartCameraType>)camera didReceiveTimeSliceQueryData:(NSArray<NSDictionary *> *)timeSlices {
      // If no video clips are found, no playback occurs.
        if (timeSlices.count == 0) {
        return;
    }
      // Saves a list of video clips and starts playback from the first video clip.
    self.timeSlicesInCurrentDay = [timeSlices copy];
      self.timeSlicesIndex = 0;
    NSDictionary *timeSlice = timeSlices.firstObject;
    NSInteger startTime = [timeSlice[kThingSmartTimeSliceStartTime] integerValue];
    NSInteger stopTime = [timeSlice[kThingSmartTimeSliceStopTime] integerValue];
      // Starts playback from the first second of the first video clip.
    NSInteger playTime = startTime;
    [camera startPlayback:playTime startTime:startTime stopTime:stopTime];
}

- (void)camera:(id<ThingSmartCameraType>)camera thing_didReceiveVideoFrame:(CMSampleBufferRef)sampleBuffer frameInfo:(ThingSmartVideoFrameInfo)frameInfo {
    NSInteger index = self.timeSlicesIndex + 1;
      // Executes a callback if the next video clip does not exist.
    if (index >= self.timeSlicesInCurrentDay.count) {
        return;
    }
    NSDictionary *currentTimeSlice = [self.timeSlicesInCurrentDay objectAtIndex:self.timeSlicesIndex];
    NSInteger stopTime = [currentTimeSlice[kThingSmartTimeSliceStopTime] integerValue];
      // If the timestamp of the current video frame is equal to or later than the end time of the current video clip, the next video clip will be played back.
    if (frameInfo.nTimeStamp >= stopTime) {
        NSDictionary *nextTimeSlice = [self.timeSlicesInCurrentDay objectAtIndex:index];
        NSInteger startTime = [nextTimeSlice[kThingSmartTimeSliceStartTime] integerValue];
            NSInteger stopTime = [nextTimeSlice[kThingSmartTimeSliceStopTime] integerValue];
            NSInteger playTime = startTime;
            [camera startPlayback:playTime startTime:startTime stopTime:stopTime];
    }
}

- (void)cameraDidBeginPlayback:(id<ThingSmartCameraType>)camera {
      // Starts playback of video footage.
      self.playbacking = YES;
    self.playbackPaused = NO;
    // Adds a rendered video view to a video screen.
        [self.view addSubview:camera.videoView];
}

- (void)cameraDidPausePlayback:(id<ThingSmartCameraType>)camera {
      // Playback of video footage is paused.
    self.playbackPaused = YES;
}

- (void)cameraDidResumePlayback:(id<ThingSmartCameraType>)camera {
       // Playback of video footage is resumed.
    self.playbackPaused = NO;
}

- (void)cameraDidStopPlayback:(id<ThingSmartCameraType>)camera {
      // Playback of video footage is stopped.
       self.playbacking = NO;
    self.playbackPaused = NO;
}

- (void)cameraPlaybackDidFinished:(id<ThingSmartCameraType>)camera {
      // Playback of video footage is finished.
    self.playbacking = NO;
    self.playbackPaused = NO;
}

// The failure callback.
- (void)camera:(id<ThingSmartCameraType>)camera didOccurredErrorAtStep:(ThingCameraErrorCode)errStepCode specificErrorCode:(NSInteger)errorCode {
        if (errStepCode == Thing_ERROR_CONNECT_FAILED) {
          // Failed to create a P2P connection.
        self.connected = NO;
    }
    else if (errStepCode == Thing_ERROR_START_PLAYBACK_FAILED) {
          // Failed to play back video footage stored on the SD card.
        self.playbacking = NO;
            self.playbackPaused = NO;
    }
      else if (errStepCode == Thing_ERROR_PAUSE_PLAYBACK_FAILED) {
                // Failed to pause playback.
    }
    else if (errStepCode == Thing_ERROR_RESUME_PLAYBACK_FAILED) {
                // Failed to resume playback.
    }
}

Swift:

func startPlayback() {
    if self.isConnected {
        self.camera.queryRecordTimeSlice(withYear: 2020, month: 2, day: 12)
        return
    }
    let p2pType = self.deviceModel.skills["p2pType"]!
        ThingSmartRequest().request(withApiName: kThingSmartIPCConfigAPI, postData: ["devId": self.devId], version: kThingSmartIPCConfigAPIVersion, success: { result in
        guard let responder = result as? [AnyHashable:Any] else {
            return;
        }
        DispatchQueue.global().async {
            let config = ThingSmartCameraFactory.ipcConfig(withUid: ThingSmartUser.sharedInstance().uid, localKey: self.deviceModel.localKey, configData: responder)
            self.camera = ThingSmartCameraFactory.camera(withP2PType: p2pType, config: config, delegate: self)
            self.camera.connect()
        }
    }) { _ in
        // Failed to get the configurations.
    }
}

func pausePlayback() {
    self.camera.pausePlayback()
}

func resumePlayback() {
    self.camera.resumePlayback()
}

func stopPlayback() {
    self.camera.stopPlayback()
}

func cameraDidConnected(_ camera: ThingSmartCameraType!) {
    self.isConnected = true
    // Returns video clips stored on a certain date only after a P2P connection is created.
    camera.queryRecordTimeSlice(withYear: 2020, month: 2, day: 12)
}

func cameraDisconnected(_ camera: ThingSmartCameraType!) {
    // A P2P connection is closed due to unstable network conditions.
    self.isConnected = false
    self.isPlaybacking = false
}

func camera(_ camera: ThingSmartCameraType!, didReceiveTimeSliceQueryData timeSlices: [[AnyHashable : Any]]!) {
    // If no video clips are found, no playback occurs.
    guard timeSlices.count > 0 else {
        return;
    }
    // Saves a list of video clips and starts playback from the first video clip.
    self.timeSlices = timeSlices
    self.timesliceIndex = 0
    let video = timeSlices.first!
    let startTime = video[kThingSmartTimeSliceStartTime] as! Int
    let stopTime = video[kThingSmartTimeSliceStopTime] as! Int
    // Starts playback from the first second of the first video clip.
    let playTime = startTime
    camera.startPlayback(playTime, startTime: startTime, stopTime: stopTime)
}

func camera(_ camera: ThingSmartCameraType!, thing_didReceiveVideoFrame sampleBuffer: CMSampleBuffer!, frameInfo: ThingSmartVideoFrameInfo) {
    let index = self.timesliceIndex + 1
    // Executes a callback if the next video clip does not exist.
    guard index < self.timeSlices.count else {
        return
    }
    let currentTimeSlice = timeSlices[self.timesliceIndex]
    let endTime = currentTimeSlice[kThingSmartTimeSliceStopTime] as! Int
    guard frameInfo.nTimeStamp >= endTime else {
        return
    }
    // If the timestamp of the current video frame is equal to or later than the end time of the current video clip, the next video clip will be played back.

    let nextTimeSlice = timeSlices.first!
    let startTime = nextTimeSlice[kThingSmartTimeSliceStartTime] as! Int
    let stopTime = nextTimeSlice[kThingSmartTimeSliceStopTime] as! Int
    let playTime = startTime
    camera.startPlayback(playTime, startTime: startTime, stopTime: stopTime)
}

func cameraDidBeginPlayback(_ camera: ThingSmartCameraType!) {
    // Starts playback of video footage.
    self.isPlaybacking = true
    self.isPlaybackPaused = false
    // Adds a rendered video view to a video screen.
    self.view.addSubview(camera.videoView())
}

func cameraDidPausePlayback(_ camera: ThingSmartCameraType!) {
    // Playback of video footage is paused.
    self.isPlaybackPaused = true
}

func cameraDidResumePlayback(_ camera: ThingSmartCameraType!) {
    // Playback of video footage is resumed.
    self.isPlaybackPaused = false
}

func cameraDidStopPlayback(_ camera: ThingSmartCameraType!) {
    // Playback of video footage is stopped.
    self.isPlaybacking = false
    self.isPlaybackPaused = false
}

func cameraPlaybackDidFinished(_ camera: ThingSmartCameraType!) {
    // Playback of video footage is finished.
    self.isPlaybacking = false
    self.isPlaybackPaused = false
}

func camera(_ camera: ThingSmartCameraType!, didOccurredErrorAtStep errStepCode: ThingCameraErrorCode, specificErrorCode errorCode: Int) {
    if errStepCode == Thing_ERROR_CONNECT_FAILED  {
        // Failed to create a P2P connection.
        self.isConnected = false
    }else if errStepCode == Thing_ERROR_START_PLAYBACK_FAILED {
        // Failed to play back video footage stored on the SD card.
        self.isPlaybacking = false
        self.isPlaybackPaused = false
    }else if errStepCode == Thing_ERROR_PAUSE_PLAYBACK_FAILED {
        // Failed to pause playback.
    }else if errStepCode == Thing_ERROR_RESUME_PLAYBACK_FAILED {
        // Failed to resume playback.
    }
}