Bulk Data Transfer

This topic describes how bulk data transfer works on Bluetooth devices.

Overview

Concepts

The communication between the Bluetooth device and the Tuya-enabled mobile app is typically based on DP data. However, the slow rate of DP data communication is not suitable for transferring large amounts of data. Bulk data transfer enables fast transfer of locally stored data in a specified format to a mobile app.

For example, the smartwatch or fitness tracker will store the measured activity data such as pace, distance, and heart rate to the flash memory if the Bluetooth is disconnected. After the mobile app is connected to the Bluetooth device, it will send the bulk data transfer command to allow the device to transmit the cached data.

Features

The bulk data transfer protocol is designed for communication between the Bluetooth device and the mobile app. The value of the flag determines the data storage structure on the device and how data is parsed on the mobile app. The functioning of the bulk data transfer protocol is independent of the type of data structure used. In other words, the transmission protocol is for data pass-through only, without depending on the data structure. The data structure is used for local storage and app parsing data only.

As shown in the figure above:

When the flag is set to 0, the device stores data in TLD format, which is then parsed by the mobile app in this format. The parsed data is forwarded to the panel or cloud for display.

The Tuya-defined TLD format represents a set of DPs. This format applies to most use cases. The standard TLD format simplifies app panel development and shortens the development cycle. See the table below for details on the TLD format.

For example, smartwatches and fitness trackers store and parse daily data such as steps, distance, and heart rate in the TLD format.

When the flag is set to 1, the device stores data in a custom format. The mobile app sends the raw data it receives to the panel or cloud, which will then parse and display the data.

You can consult your product manager or app panel developer about the custom format. The bulk data transfer protocol does not impose particular restrictions on the custom format. If the TLD format does not meet your needs, you can use a custom format instead. The bulk_type values identify various custom formats.

For example, a remote device stores the light installation records and logs in two different formats.

TLD format:

• Type:
  • 1: Stores all the DP data produced simultaneously. The mobile app parses the data.
  • 2: Stores all the DP data produced simultaneously, including the overhead for data structure alignment. The mobile app parses the data.
  • Others: Reserved
• Len: The length of Data.
• Data: Its length is limited by the maximum length of each packet.

When Type is set to 1, the TLD format is as follows:

No.	Length	Field	Description
1	1	Type	Type = 1
2 to 3	2	Len	The length of data.
4 to 7	4	unix_timestamp	Unix timestamp
8…	The length of DP 1	DP 1	The DP data. See DP Format.
…	…	…	…
…	The length of DP n	DP n	The DP data. See DP Format.

When Type is set to 2, the TLD format is as follows:

No.	Length	Field	Description
1	1	Type	Type = 2
2 to 3	2	Len	The length of data.
4 to 9	6	storage_header	The overhead that is not parsed.
10 to 11	2	valid_data_len	The total length of the Unix timestamp and the DP data, in big-endian format.
12 to 15	4	unix_timestamp	Unix timestamp
16…	The length of DP 1	DP 1	The DP data. See DP Format.
…	…	…	…
…	The length of DP n	DP n	The DP data. See DP Format.
…	Len - 6 - 2 - valid_data_len	Invalid_data	The overhead that is not parsed. Pad it with zeros.

DP format:

No.	Length	Field	Description
1	1	dp_id	The dp_id of a DP defined on the Tuya IoT Development Platform.
2	1	dp_type	#define DT_RAW 0 #define DT_BOOL 1 #define DT_VALUE 2 #define DT_STRING 3 #define DT_ENUM 4 #define DT_BITMAP 5
3 to 4	2	dp_len	The length of the DP data.
5…	dp_len	dp_data	The DP data.

Data structure

TUYA_BLE_BULKDATA_EXTERNAL_PARAM_T

typedef struct {
    UINT8_T type;
    UINT8_T flag;
} TUYA_BLE_BULKDATA_EXTERNAL_PARAM_T;

• type: The type of the bulk data.

  • 0x00: Indicates generic integration, used for Tuya’s module. If you develop with the SDK, leave it as is.
  • Others: If you need the mobile app to parse data, set the value of type to 1. If you do not require the mobile app for data parsing, you can define type as needed, starting with a value of 1. Uniqueness of type definitions among all PIDs is not required, allowing for duplicate values.

• flag: Specifies whether the mobile app parses data.

  • 0x00: The mobile app parses data. The data is parsed in DP format, with the value of type set to 1.
  • 0x01: The mobile app does not parse data.
  • 0x02: The mobile app parses data. The data is parsed based on the type. This is not supported currently.

TUYA_BLE_BULKDATA_INFO_T

typedef struct {
    UINT32_T total_length;
    UINT32_T total_crc32;
    UINT32_T block_length;
} TUYA_BLE_BULKDATA_INFO_T;

• total_length: The total length of the bulk data in the specified type.
• total_crc32: The total CRC32 value of the bulk data in the specified type.
• block_length: The bulk data is read by blocks. The mobile app sends the block ID (starting with 0) to the device to specify the block to be transmitted. A block is divided into packets for transmission.

TUYA_BLE_BULKDATA_CB_T

typedef VOID_T (*TUYA_BLE_BULKDATA_INFO_CB)(TUYA_BLE_BULKDATA_INFO_T* info);
typedef VOID_T (*TUYA_BLE_BULKDATA_REPORT_CB)(UINT8_T* p_block_buf, UINT32_T block_length, UINT32_T block_number);
typedef VOID_T (*TUYA_BLE_BULKDATA_ERASE_CB)(UINT8_T type, UINT8_T* status);

typedef struct {
    TUYA_BLE_BULKDATA_INFO_CB info_cb;
    TUYA_BLE_BULKDATA_REPORT_CB report_cb;
    TUYA_BLE_BULKDATA_ERASE_CB erase_cb;
} TUYA_BLE_BULKDATA_CB_T;

• info_cb: The callback for passing the bulk data information. The application layer sends the total length and CRC32 value of the bulk data, as well as the maximum length of each packet, to the bulk data transfer component.
• report_cb: The callback for bulk data reporting. The application layer sends the locally stored data to the bulk data transfer component.
• erase_cb: The callback for bulk data erasure. The bulk data transfer component informs the application layer to erase the locally stored data and return the result of the operation.

API description

Multi-instance configuration

The bulk data transfer component supports multiple instances, enabling it to transfer data of different types.

Take the remote device as an example. It transfers two types of data: bulk_type 1 for T8 light construction information, and bulk_type 2 for local logs. Therefore, two instances are required to handle the transfer processes. In this case, set TUYA_BLE_BULKDATA_TYPE_NUM to 2 to enable multi-instance for the bulk data transfer component.

TUYA_BLE_BULKDATA_TYPE_NUM defaults to 1.

#ifndef TUYA_BLE_BULKDATA_TYPE_NUM
#define TUYA_BLE_BULKDATA_TYPE_NUM                  (1)
#endif

If modification is necessary, it is recommended to define the value of TUYA_BLE_BULKDATA_TYPE_NUM in app_config.h instead of directly altering the default value.

Component initialization

API description

UINT32_T tuya_ble_bulk_data_init(TUYA_BLE_BULKDATA_EXTERNAL_PARAM_T* param, TUYA_BLE_BULKDATA_CB_T* cb);

Parameter description

Parameter	Description
param	See TUYA_BLE_BULKDATA_EXTERNAL_PARAM_T.
cb	See TUYA_BLE_BULKDATA_CB_T.

Information populating callback

The application layer sends the total length and CRC32 value of the bulk data, as well as the maximum length of each packet, to the bulk data transfer component.

API description

typedef VOID_T (*TUYA_BLE_BULKDATA_INFO_CB)(TUYA_BLE_BULKDATA_INFO_T* info);

Parameter description

Parameter	Description
info	See TUYA_BLE_BULKDATA_INFO_T. total_length and total_crc32 are calculated by the application layer based on the stored data. block_length is set as required, usually to 512, but not exceeding TUYA_BLE_SEND_MAX_DATA_LEN (1490).

Data reporting callback

The application layer sends the locally stored data to the bulk data transfer component.

API description

typedef VOID_T (*TUYA_BLE_BULKDATA_REPORT_CB)(UINT8_T* p_block_buf, UINT32_T block_length, UINT32_T block_number);

Parameter description

Parameter	Description
p_block_buf	The start address of the bulk data stored locally.
block_length	The length of the bulk data to be read.
block_number	The block ID of the bulk data to be read, used to calculate the offset address.

Data erasure callback

The bulk data transfer component informs the application layer to erase the locally stored data and return the result of the operation.

API description

typedef VOID_T (*TUYA_BLE_BULKDATA_ERASE_CB)(UINT8_T type, UINT8_T* status);

Parameter description

Parameter	Description
type	The type of the bulk data to be erased.
status	The result of erasure.

Entry point function for data upload

The handler for the bulk data transfer protocol, which is fully encapsulated into the bulk data transfer component.

API description

VOID_T tuya_ble_bulk_data_demo_handler(tuya_ble_bulk_data_request_t* p_data);

Parameter description

Parameter	Description
p_data	The data of the bulk data transfer protocol.

Generate test data

This API is used for testing only. It generates test data to help you verify the functionality of bulk data transfer.

API description

UINT32_T tuya_ble_bulk_data_generation(UINT32_T timestep, UINT8_T* buf, UINT32_T size);

Parameter description

Parameter	Description
timestep	The timestamp of the test data.
buf	The test data.
size	The length of the test data.

How to use

Communication process

Code development

Enable the bulk data transfer component in app_config.h.

#define TUYA_BLE_FEATURE_BULKDATA_ENABLE 1

Add the following snippet in tuya_sdk_callback.c to implement the basic functionality. If the test function (TUYA_SDK_TEST) is enabled, this code is active. Each API used in the code is described in the API documentation.

#include "tuya_ble_bulkdata_demo.h"

#define TUYA_BLE_BULKDATA_BLOCK_SIZE 512

STATIC TUYA_BLE_BULKDATA_EXTERNAL_PARAM_T tuya_ble_external_param = {
    .type = 1,
    .flag = NEED_PARSING_BY_APP,
};
STATIC TUYA_BLE_BULKDATA_CB_T tuya_ble_bulkdata_cb = {0};

VOID_T tuya_ble_bulkdata_info_cb(TUYA_BLE_BULKDATA_INFO_T* info)
{
//    info->total_length = 0;
//    info->total_crc32 = 0;
    info->block_length = TUYA_BLE_BULKDATA_BLOCK_SIZE;
}

VOID_T tuya_ble_bulkdata_report_cb(UINT8_T* p_block_buf, UINT32_T block_length, UINT32_T block_number)
{
    UINT32_T read_addr = BOARD_FLASH_SDK_TEST_START_ADDR + block_number*TUYA_BLE_BULKDATA_BLOCK_SIZE;
    tuya_ble_nv_read(read_addr, p_block_buf, block_length);
}

VOID_T tuya_ble_bulkdata_erase_cb(UINT8_T type, UINT8_T* status)
{
    // Erase all data by type
    // Return result by status
}

OPERATE_RET tuya_init_last(VOID_T)
{
    …………

    tuya_ble_bulkdata_cb.info_cb = tuya_ble_bulkdata_info_cb;
    tuya_ble_bulkdata_cb.report_cb = tuya_ble_bulkdata_report_cb;
    tuya_ble_bulkdata_cb.erase_cb = tuya_ble_bulkdata_erase_cb;
    tuya_ble_bulk_data_init(&tuya_ble_external_param, &tuya_ble_bulkdata_cb);

    return OPRT_OK;
}

Functional testing

Prerequisites

• You have installed the Tuya Smart app on your phone.
• The device must be in pairing mode.

Procedure

Communicate with the mobile app using a host that simulates a real device.

1. Set the PID to the one that has bulk data transfer enabled. You can search for case TEST_CID_SET_PID in the code.

   

2. Connect the device to the Tuya Smart app.

   

3. The device is bound.

4. Simulate the bulk data to be transferred using the host computer. You can search for case TEST_CID_SET_BULK_DATA in the code.

   

5. Pull down the device panel homepage to trigger big data transfer. Wait for the transfer to complete.

   

6. Select the second bottom tab and tap Steps.

   

7. Check the received data.

   

   If you have any problems with host usage, see Logic Host User Guide.

Support and help

If you have any problems with TuyaOS development, you can post your questions in the Tuya Developer Forum.