Last Updated on : 2024-06-25 10:03:15download
Integrating with streaming components allows you to add streaming media capability to your IoT products. The real-time streaming protocol (RTSP) is used to establish and control the connection between the streaming server and the client (such as a mobile app) for real-time media data transfer.
The RTSP service is used for two purposes:
This solution uses the BK7231U module as the microcontroller. The BK7231U comes with Bluetooth 5.1 and Wi-Fi 802.11n combo connectivity. The streaming component on the module is connected to the mobile app through the RTSP protocol.
tuya_svc_lan_rtsp.c
and tuya_svc_lan_rtsp.h
, on which your project will be dependent.The buffer size depends on the frame format. For example, VGA quality (640 x 480 pixel) video output requires a buffer of more than 30 KB.
Function name | tuya_svc_lan_reg_rtsp_stream_src |
---|---|
Description | After initialization is completed, the device calls this function to register the streaming control callback in the rtsp_stream_src_t . This way, real-time streaming can be implemented after the RTSP server is connected. |
Parameters | The pointer to the struct rtsp_stream_src_t |
Return value | See tuya_error_code.h . |
#include "tuya_svc_lan_rtsp.h"
/**
* @brief register stream src to rtsp
*
* @param[in] src stream info
*
* @return OPRT_OK when success. For other errors, please refer to tuya_error_code.h
*/
OPERATE_RET tuya_svc_lan_reg_rtsp_stream_src(rtsp_stream_src_t src);
Variables | Description |
---|---|
get_sample_rate | Get the sampling rate. |
get_frame_fragment | Transmit image frames to the SDK. |
get_codec | Get the type of the image frame. |
start/stop | Start or stop streaming. |
get_name | Get the name of the streaming, which is stream_0 generally. |
typedef struct {
int (*get_sample_rate)(RTSP_MEDIA_TYPE_E type);
/* `get_frame` is deprecated. Make sure to use the interface `get_frame_fragment`. */
int (*get_frame)(int user_id, RTSP_MEDIA_TYPE_E type, char** buf, int *plen, uint64_t *pts); // Returns 0 on success. The I-frame (the first frame) is required.
int (*get_frame_fragment)(int user_id, rtsp_frame_fragment_t* pframe);
RTP_CODEC_E (*get_codec)(RTSP_MEDIA_TYPE_E type);
int (*start)(); // Returns `user_id`.
int (*stop)(int user_id);
int (*get_name)(char* buf, int* buf_len);
} rtsp_stream_src_t;
Enumerate types of RTSP streaming media
typedef enum {
RTSP_MEDIA_TYPE_INVALID = -1,
RTSP_MEDIA_TYPE_VIDEO = 0,
RTSP_MEDIA_TYPE_AUDIO,
RTSP_MEDIA_TYPE_NUM,
} RTSP_MEDIA_TYPE_E;
Frame format of streaming media
typedef struct {
RTSP_MEDIA_TYPE_E type;
char* buf;
int len;
uint64_t pts;
uint8_t is_eof;
uint8_t fragment_id;
} rtsp_frame_fragment_t;
The following code provides an example of implementing the RTSP media streaming feature on the adaptation layer. You can port the RTSP code provided by the BK7231U platform. The code for the camera feature is for reference only, which should be implemented based on your choice of camera.
// The libraries included.
#include "ty_video_lib.h" // Camera-related file, which can be changed based on your choice of camera.
#include "tuya_hal_network.h" // Networking APIs.
#include "tuya_hal_semaphore.h" // The semaphore.
#include "tuya_svc_lan_rtsp.h" // Contains the RTSP, INIT, UNINIT, and REGISTER APIs.
#include "rtsp_server.h" /* Contains the RTSP callback struct, frame struct, and RTSP APIs, which can be used to create a local server when debugging with VLC. Do not use the APIs in this file when you debug with the mobile app. */
typedef struct {
int pos[RTSP_MEDIA_TYPE_NUM];
uint64_t last_pts[RTSP_MEDIA_TYPE_NUM];
} user_info_s;
user_info_s* users[5] = {0}; // The RTSP component can save up to 5 streams.
SEM_HANDLE sem_handle; // The semaphore for reading images from the camera.
SEM_HANDLE sem_handle_rtsp; // The semaphore for RTSP streaming.
struct video_pkg *vb_pkg; // The data structure used to control camera.
UCHAR_T test_tmp_buf[16384]; // Saves full images. Each image is 16 KB in size.
struct video_pkg{
BYTE_T *pkg_bufs; /* The data */
UINT_T actual_len; /* The actual length of the data */
BYTE_T pkg_id; /* The serial number of an image */
};
// Get the file format.
RTP_CODEC_E get_codec(RTSP_MEDIA_TYPE_E type)
{
switch (type){
case RTSP_MEDIA_TYPE_VIDEO:
return RTP_CODEC_JPEG;
case RTSP_MEDIA_TYPE_AUDIO:
return RTP_CODEC_INVALID;
default:
return 0;
}
}
// The sampling rate differs depending on video types.
int get_sample_rate(RTSP_MEDIA_TYPE_E type)
{
switch (type){
case RTSP_MEDIA_TYPE_VIDEO:
return 90000;
case RTSP_MEDIA_TYPE_AUDIO:
return 8000;
default:
return 8000;
}
}
// Initialize the user_id. RTSP supports multi streams.
int start()
{
int i;
for(i = 0; i < 5; i++){
if (NULL == users[i]){
users[i] = malloc(sizeof(user_info_s));
users[i]->last_pts[RTSP_MEDIA_TYPE_VIDEO] = 0;
users[i]->last_pts[RTSP_MEDIA_TYPE_AUDIO] = 0;
users[i]->pos[RTSP_MEDIA_TYPE_VIDEO] = 0;
users[i]->pos[RTSP_MEDIA_TYPE_AUDIO] = 0;
return i;
}
}
return -1;
}
// Free the user_id.
int stop(int user_id)
{
if (users[user_id]){
free(users[user_id]);
users[user_id] = NULL;
}
return 0;
}
Get the stream name that is used by the RTSP to pull streams.
int get_name(char* buf, int* buf_len)
{
char name[] = "stream_0";
memcpy(buf, name, sizeof(name));
*buf_len = sizeof(name);
return 0;
}
/* Get the image frame.
* IN: user_id, pframe->type
* OUT: pframe->buf
*/
STATIC INT_T __get_frame_fragment_jpeg(INT_T user_id, rtsp_frame_fragment_t* pframe)
{
user_info_s* pinfo = users[user_id];
RTSP_MEDIA_TYPE_E type = pframe->type;
INT_T pos = pinfo->pos[type];
INT_T start = -1, end = -1;
INT_T media_len;
CONST INT_T frame_rate[2] = {28, 50};
UINT_T time_now;
time_now = rtos_get_time();//uni_time_get_posix_ms();
if (time_now - pinfo->last_pts[type] < 1000 / frame_rate[type]) {
return -1;
}
if (pframe == NULL || user_id >=5 || user_id < 0) {
PR_ERR("param invalid");
return -1;
}
tuya_os_adapt_semaphore_wait(sem_handle_rtsp);
memset(test_tmp_buf, 0, sizeof(test_tmp_buf));
memcpy(test_tmp_buf, vb_pkg->pkg_bufs, vb_pkg->actual_len - 5);
media_len = vb_pkg->actual_len - 5;
tuya_os_adapt_semaphore_post(sem_handle);
if (type == RTSP_MEDIA_TYPE_VIDEO){
PR_DEBUG("frame len:%d", media_len);
pframe->buf = test_tmp_buf;
}
else {
return -1;
}
pframe->len = media_len;
pframe->pts = time_now;
pframe->is_eof = 1;
pinfo->last_pts[type] = time_now;
pinfo->pos[type] = media_len;
if (pinfo->pos[type] > 0){
pinfo->pos[type] = 0;
}
return 0;
}
/* Initialize the RTSP application, which can be called after device_init. */
VOID user_rtsp_init()
{
CHAR_T url[256] = {0};
int url_len = 256;
rtsp_stream_src_t src = {0};
/* Register the RTSP control function. */
src.get_codec = __get_codec;
//src.get_frame = __get_frame_jpeg; // This interface is deprecated.
src.get_frame_fragment = __get_frame_fragment_jpeg;
src.get_sample_rate = __get_sample_rate;
src.start = __start;
src.stop = __stop;
src.get_name = __get_name;
tuya_svc_lan_reg_rtsp_stream_src(src);
/* Camera feature APIs, implemented as needed. */
video_device *soc_video_device = NULL;
struct video_pix_format v_format;
struct vbq_config video_buff_q_config;
/* Configure the tuya_config.h file. Then, call tuya_soc_camera_init to build the system and create nodes. */
tuya_soc_camera_init();
soc_video_device = tuya_video_dev_find("ty_video");
if(soc_video_device == NULL){
PR_DEBUG("soc_video_device is null");
return OPRT_COM_ERROR;
}
/* Initialize video_device. */
tuya_video_dev_open(soc_video_device);
/* Set the format. */
v_format.p_type = QVGA_320_240;
v_format.pixelformat = V4L2_PIX_FMT_JPEG;
v_format.field = V4L2_FIELD_ANY;
v_format.colorspace = V4L2_COLORSPACE_JPEG;
tuya_video_dev_ctl(soc_video_device,CMD_SET_FMT,&v_format);
/* Set the buffer. */
video_buff_q_config.video_pkg_buffer_len = 16 * 1024;
video_buff_q_config.video_pkg_num = 2;
tuya_video_dev_ctl(soc_video_device,CMD_STREAM_SET_BUFF,&video_buff_q_config);
/* Start */
tuya_video_dev_ctl(soc_video_device,CMD_STREAM_START,NULL);
tuya_os_adapt_semaphore_create_init(&sem_handle, 0, 1);
tuya_os_adapt_semaphore_create_init(&sem_handle_rtsp, 0, 1);
while(1)
{
/* Read images from camera and wait for the RTSP to fetch them. */
tuya_video_dev_read(soc_video_device,&vb_pkg);
tuya_os_adapt_semaphore_post(sem_handle_rtsp);
/* Free the buffer that has been read. */
tuya_os_adapt_semaphore_wait(sem_handle);
tuya_video_dev_ctl(soc_video_device, CMD_STREAM_DEQUEEU_BUFF,NULL);
}
}
This SDK is not generally available. Contact the project manager and request the SDK.
+-- software
¦ +-- IoTOS2.3.4_ty_iot_sdk_2.3.4_bk7231u_1.1.0 # TuyaOS IoT SDK
¦ ¦ +-- apps # The developer guides with samples.
¦ ¦ +-- sdk # The header files and libraries.
¦ ¦ +-- platforms # The library from the chip platform and toolkit.
¦ ¦ +-- CHANGELOG.md # The version history.
¦ ¦ +-- README.md # The introduction to developing with the SDK.
¦ ¦ +-- build_app.sh # The script for compilation.
¦ +-- Tuya IoTOS IoT SDK x.x.x Release Notes .pdf # The release notes of the SDK.
+-- pc
¦ +-- tools # The development toolkit.
+-- hardware
¦ +-- board # The materials of the module and open source hardware.
¦ +-- chip # The materials of the chip.
After the initialization with device_init
, call user_rtsp_init()
to register the RTSP application callback.
Problem description | Troubleshooting |
---|---|
platforms\bk7231t\bk7231t_os\beken378\driver\uart\uart.bk.c\bk_printf . Insufficient buffer size and no checks for string length result in a buffer overflow. |
Change the buffer size to 256 or 1024 bytes. Change the string copy function to vsnprintf(string, sizeof(string) - 1, fmt, ap); . |
Failed to create threads after updating the compiler toolchain. | Update CFLAGS. |
Failed to convert the SSID to BSSIDs, which causes device pairing in EZ mode to fail. | Make sure you use the legal channels (1 to 13) on the adaptation layer. The BSSID cannot be empty. |
The interface tuya_adapter_wifi_station_get_status is used to return the network status. Its values of network status enum might be different from those defined by the BK7231U platform. |
Check the obtained IP address. |
Failed to pair the device. | Check if you have uploaded the firmware file to the Tuya Developer Platform. If not, upload it and perform a firmware update on the mobile app. |
Key-value (KV) database operation failed. | Bluetooth is initialized at 1f4000 that is intended to be assigned to Tuya’s KV database. You can turn off the Bluetooth to fix this issue. |
Is this page helpful?
YesFeedbackIs this page helpful?
YesFeedback