English
English
简体中文
Contact Us
Register
Log In
layoutIndex

WebRTC

Last Updated on : 2021-08-20 08:30:55download

Tuya IoT Platform provides access to live audios and videos using the Web Real-Time Communication (WebRTC) protocol for IoT devices that are capable of transmitting audios and videos. This topic describes how to integrate with live videos, using the IP camera (IPC) as an example. For more information about the WebRTC protocol, see Build the backend services needed for a WebRTC app.

Overview

Process diagram

WebRTC

Components

  • Web frontend

    • Provide a WebRTC live stream page for you to view with Chrome.
    • Communicate with the Web backend over WebSocket protocol.
    • Call the Javascript API to generate webRTC offers and candidates.
  • Web backend

    • Host the web page.
    • Visit the Tuya IoT Platform and get different configuration information over the HTTP protocol.
    • Connect to Tuya MQTT service.
  • Tuya IoT Platform

    • Provide HTTP APIs for different open platforms.
  • Tuya MQTT

    • Provide asynchronous data transmission channels.
  • IPC powered by Tuya

    • The IPC powered by Tuya.

Connect to live audio and video streams

Prerequisites

  • You have linked the IPC with the Tuya Smart app.
  • You have created a cloud development project. For more information about project creation, see Create a project.
  • You have linked your Tuya Smart app account with the project. For more information, see Link my app.

Procedure

  1. Clone the webrtc-demo-go project to the local.
    WebRTC

  2. In the root directory of the source code, run go get and then go build.
    WebRTC

  3. Configure the parameters in the webrtc.json file.
    WebRTC

    • clientId: Enter the value of Access ID in the Authorization Key section of the cloud project. For more information, see Cloud project parameters.
    • secret: Enter the value of Access Secret in the Authorization Key section of the cloud project. For more information, see Cloud project parameters.
    • autoMode: Select the easy authorization method. Enter the user ID (UID) of the linked app account in uid. For more information, see Device parameters.
      WebRTC
    • deviceId: the ID of the linked device. For more information, see Device parameters.
  4. Run ./webrtc-demo-go.

  5. Use Chrome to log in to http://localhost:3333.

  6. On the web page, click Call.

Action result:

WebRTC

API

Generate MQTT configuration

API description

Generate the MQTT connection configuration for the users.

API endpoint

POST /v1.0/open-hub/access/config

Request parameter

Parameter Data type Location Description Required
link_id Int BODY The unique flag of the user connection with the maximum length of 8 bits. Yes
uid String BODY The user ID. It is not required in the license mode. No
link_type String BODY The connection mode that supports MQTT. Yes
topics String BODY IPC. The MQTT type of topic for WebRTC. Yes

Response parameter

Parameter Data type Description
result MQTT MQTT login information.
success Boolean The status of the response result.
t Long The timestamp of the response result. Unit: milliseconds.
  • MQTT

    Parameter Data type Description
    url String The address to be connected.
    username String The username used for connection.
    password String The password used for connection.
    client_id String The client_id used for connection.
    source_topic String The subscribed topic.
    sink_topic String The posted topic.
    expire_time Integer The expiration time.

Sample request

{
"uid": "ay1564026880284v****",
"link_id": "123456",
"link_type": "MQTT",
"topics": "ipc"
}

Sample response

{
"result": {
"client_id": "cloud_7ef68bc84629ea3f51152760cdf2****",
"expire_time": 7200,
"password": "0426d6917bfd8b88f037c4a598a0****",
"sink_topic": {
"ipc": "/av/moto/moto_id/u/{device_id}"
},
"source_topic": {
"ipc": "/av/u/d09735be24f4b7eb3583b30bcaa2****"
},
"url": "ssl://m1-cn.wgine.com:8883",
"username": "cloud_d09735be24f4b7eb3583b30bcaa2****"
},
"success": true,
"t": 1600847208953
}

Get the WebRTC configuration

API endpoint

GET /v1.0/users/{uId}/devices/{deviceId}/webrtc-configs

Request parameter

Parameter Data type Location Description Required
uId String URI The user ID. Yes
deviceId String URI The device ID. Yes

Response parameter

Parameter Data type Description
audio_attributes AudioAttributes The attribute of talk.
auth String The information about authorization.
id String The device ID.
moto_id String The ID of the connected instance.
p2p_config P2PConfig Configuration information of the connection service.
skill String The skill.
supports_webrtc Boolean Indicates whether WebRTC is supported.
vedio_clarity Integer The video clarity.
  • AudioAttributes

    Parameter Data type Description
    call_mode Integer The talk mode. Valid values:
    • 1: one-way talk.
    • 2: two-way talk.
    hardware_capability Integer The hardware capability. Valid values:
    • 1: MIC.
    • 2: speaker.
  • P2PConfig

    Parameter Data type Description
    ices Token The list of P2P tokens.
  • Token

    Parameter Data type Description
    urls String The address of the ICE service.
    username String The username of the ICE service.
    credential String The password of the ICE service.
    ttl Integer The valid duration of the ICE service. Unit: seconds.

Sample response

{
"result": {
"audio_attributes": {
"call_mode": [
1,
2
],
"hardware_capability": [
1,
2
]
},
"auth": "h85L4pljbuHFR0a/iTgViwA35xi3yTl3NyMsFQL5****",
"id": "6cf2b6d2b09a2f8597****",
"moto_id": "moto_cnpre002",
"p2p_config": {
"ices": [
{
"urls": "stun:49.234.141.77:3478"
},
{
"urls": "stun:tx1stun.tuyacn.com:3478"
},
{
"urls": "nat:tx1nat.tuyacn.com:3478"
},
{
"urls": "nat:tx2nat.tuyacn.com:3478"
},
{
"credential": "kb/EA2whGCcNSM5FjXV2dxAM1MU=",
"ttl": 36000,
"urls": "turn:49.234.141.77:3478",
"username": "1600883205:6cf2b6d2b09a2f8597****"
},
{
"credential": "kb/EA2whGCcNSM5FjXV2dxAM****",
"ttl": 36000,
"urls": "turn:tx1turn.tuyacn.com:3478",
"username": "1600883205:6cf2b6d2b09a2f8597****"
}
]
},
"skill": "{\"webrtc\":3,\"audios\":[{\"channels\":1,\"dataBit\":16,\"codecType\":101,\"sampleRate\":8000}],\"videos\":[{\"streamType\":2,\"profileId\":\"\",\"width\":1920,\"codecType\":2,\"sampleRate\":90000,\"height\":1080},{\"streamType\":4,\"width\":640,\"codecType\":2,\"height\":360}]}",
"supports_webrtc": true,
"vedio_clarity": 4
},
"success": true,
"t": 1600847205437
}

Connect to WebRTC using signaling packets

Response parameter

Parameter Data type Description
protocol Integer The protocol number of the MQTT message. WebRTC is a live streaming service with the value of 302.
pv String The version of the communication protocol.
t Integer The Unix timestamp. Unit: seconds.
data Data The MQTT message frame.
  • Data

    Parameter Data type Description
    header Header The MQTT message header.
    msg Msg The MQTT message body. It can be offer candidate answer disconnect.
  • Header

    Parameter Data type Description
    type String The MQTT message type. It can be offer candidate answer disconnect.
    from String Enter the ID of the sender. For example, enter msid for the client, and enter device_id for the device.
    to String Enter the ID of the receiver. For example, device_id or msid.
    sub_dev_id String The node_id of a sub-device. It is only used by NVR devices.
    sessionid String It is generated at random with a length of 32 bytes upon every connection. The signaling packets of the same connection share the same sessionid.
    moto_id String You can call /v1.0/users/{uId}/devices/{deviceId}/webrtc-configs to get the value. If the returned result does not include the moto_id field, this parameter is not supported.
  • Msg

    It currently supports three MQTT message types: offer, candidate, and disconnet. The format of Msg varies depending on MQTT message types.

    When the type is offer, the Msg format is as follows.

    Parameter Data type Description
    mode String Connection mode: WebRTC.
    sdp String WebRTC offer generated in Web.
    stream_type Interger Stream type. It is sub-stream 1 by default.
    auth String You can call /v1.0/users/{uId}/devices/{deviceId}/webrtc-configs to get the value. Its field is auth.

Sample response

{
"protocol":302,
"pv":"2.2",
"t":1600820048671,
"data":{
"header":{
"from":"AY1600819753305aHO5Sdj8pQMtLZ68XHMUpHKlRKJ87s",
"to":"6c9a943f2ea6929675ymcq",
"sessionid":"00b00036521743319b4d4c01f1705c48",
"moto_id":"moto_5f685396jK",
"type":"offer"
},
"msg":{
"sdp":"v=0\r\no=- 4529163812828363188 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 1\r\na=msid-semantic: WMS 1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP\r\nm=audio 9 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 110 112 113 126\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:Q93I\r\na=ice-pwd:P58s/ZyBRNVnuIxcrcmEmRG5\r\na=ice-options:trickle\r\na=fingerprint:sha-256 E1:01:E0:B3:F1:97:7F:86:07:61:54:BE:42:5F:56:E8:84:58:76:E3:E4:22:94:F1:33:2A:A3:C2:FC:67:05:3E\r\na=setup:actpass\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=sendrecv\r\na=msid:1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP 1c7d25a4-9948-4165-bf4d-62fc39b8b528\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10;useinbandfec=1\r\na=rtpmap:103 ISAC/16000\r\na=rtpmap:104 ISAC/32000\r\na=rtpmap:9 G722/8000\r\na=rtpmap:0 PCMU/8000\r\na=rtpmap:8 PCMA/8000\r\na=rtpmap:106 CN/32000\r\na=rtpmap:105 CN/16000\r\na=rtpmap:13 CN/8000\r\na=rtpmap:110 telephone-event/48000\r\na=rtpmap:112 telephone-event/32000\r\na=rtpmap:113 telephone-event/16000\r\na=rtpmap:126 telephone-event/8000\r\na=ssrc:724809951 cname:7UznE7uyn6JBJ4PA\r\na=ssrc:724809951 msid:1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP 1c7d25a4-9948-4165-bf4d-62fc39b8b528\r\na=ssrc:724809951 mslabel:1VpYoJaai0xSYjWhYxPHqySybB3PaQ6Y3wXP\r\na=ssrc:724809951 label:1c7d25a4-9948-4165-bf4d-62fc39b8b528\r\nm=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 122 102 120 127 119 125 107 108 109 121 114 115 124 118 123\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:Q93I\r\na=ice-pwd:P58s/ZyBRNVnuIxcrcmEmRG5\r\na=ice-options:trickle\r\na=fingerprint:sha-256 E1:01:E0:B3:F1:97:7F:86:07:61:54:BE:42:5F:56:E8:84:58:76:E3:E4:22:94:F1:33:2A:A3:C2:FC:67:05:3E\r\na=setup:actpass\r\na=mid:1\r\na=extmap:14 urn:ietf:params:rtp-hdrext:toffset\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=extmap:12 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay\r\na=extmap:11 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type\r\na=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing\r\na=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space\r\na=extmap:4 urn:ietf:params:rtp-hdrext:sdes:mid\r\na=extmap:5 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id\r\na=extmap:6 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id\r\na=recvonly\r\na=rtcp-mux\r\na=rtcp-rsize\r\na=rtpmap:96 VP8/90000\r\na=rtcp-fb:96 goog-remb\r\na=rtcp-fb:96 transport-cc\r\na=rtcp-fb:96 ccm fir\r\na=rtcp-fb:96 nack\r\na=rtcp-fb:96 nack pli\r\na=rtpmap:97 rtx/90000\r\na=fmtp:97 apt=96\r\na=rtpmap:98 VP9/90000\r\na=rtcp-fb:98 goog-remb\r\na=rtcp-fb:98 transport-cc\r\na=rtcp-fb:98 ccm fir\r\na=rtcp-fb:98 nack\r\na=rtcp-fb:98 nack pli\r\na=fmtp:98 profile-id=0\r\na=rtpmap:99 rtx/90000\r\na=fmtp:99 apt=98\r\na=rtpmap:100 VP9/90000\r\na=rtcp-fb:100 goog-remb\r\na=rtcp-fb:100 transport-cc\r\na=rtcp-fb:100 ccm fir\r\na=rtcp-fb:100 nack\r\na=rtcp-fb:100 nack pli\r\na=fmtp:100 profile-id=2\r\na=rtpmap:101 rtx/90000\r\na=fmtp:101 apt=100\r\na=rtpmap:122 VP9/90000\r\na=rtcp-fb:122 goog-remb\r\na=rtcp-fb:122 transport-cc\r\na=rtcp-fb:122 ccm fir\r\na=rtcp-fb:122 nack\r\na=rtcp-fb:122 nack pli\r\na=fmtp:122 profile-id=1\r\na=rtpmap:102 H264/90000\r\na=rtcp-fb:102 goog-remb\r\na=rtcp-fb:102 transport-cc\r\na=rtcp-fb:102 ccm fir\r\na=rtcp-fb:102 nack\r\na=rtcp-fb:102 nack pli\r\na=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f\r\na=rtpmap:120 rtx/90000\r\na=fmtp:120 apt=102\r\na=rtpmap:127 H264/90000\r\na=rtcp-fb:127 goog-remb\r\na=rtcp-fb:127 transport-cc\r\na=rtcp-fb:127 ccm fir\r\na=rtcp-fb:127 nack\r\na=rtcp-fb:127 nack pli\r\na=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f\r\na=rtpmap:119 rtx/90000\r\na=fmtp:119 apt=127\r\na=rtpmap:125 H264/90000\r\na=rtcp-fb:125 goog-remb\r\na=rtcp-fb:125 transport-cc\r\na=rtcp-fb:125 ccm fir\r\na=rtcp-fb:125 nack\r\na=rtcp-fb:125 nack pli\r\na=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f\r\na=rtpmap:107 rtx/90000\r\na=fmtp:107 apt=125\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 goog-remb\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=rtpmap:121 H264/90000\r\na=rtcp-fb:121 goog-remb\r\na=rtcp-fb:121 transport-cc\r\na=rtcp-fb:121 ccm fir\r\na=rtcp-fb:121 nack\r\na=rtcp-fb:121 nack pli\r\na=fmtp:121 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=4d0015\r\na=rtpmap:114 H264/90000\r\na=rtcp-fb:114 goog-remb\r\na=rtcp-fb:114 transport-cc\r\na=rtcp-fb:114 ccm fir\r\na=rtcp-fb:114 nack\r\na=rtcp-fb:114 nack pli\r\na=fmtp:114 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=640015\r\na=rtpmap:115 rtx/90000\r\na=fmtp:115 apt=114\r\na=rtpmap:124 red/90000\r\na=rtpmap:118 rtx/90000\r\na=fmtp:118 apt=124\r\na=rtpmap:123 ulpfec/90000\r\n",
"auth":"3iHAObTiJ+P1o/OeX8My208vis9Ar6JQygHSLrBxv5U=",
"mode":"webrtc",
"stream_type":1
}
}
}

When the type candidate, the Msg format is as follows.

Parameter Data type Description
mode String Connection mode: WebRTC.
candidate String The candidate WebRTC addresses connected by both parties.

Sample response

{
"protocol":302,
"pv":"2.2",
"t":1600820048672,
"data":{
"header":{
"from":"AY1600819753305aHO5Sdj8pQMtLZ68XHMUpHKlRKJ87s",
"to":"6c9a943f2ea6929675ymcq",
"sessionid":"00b00036521743319b4d4c01f1705c48",
"moto_id":"moto_5f685396jK",
"type":"candidate"
},
"msg":{
"mode":"webrtc",
"candidate":"a=candidate:512512433 1 udp 2122260223 192.168.0.227 50828 typ host generation 0 ufrag Q93I network-id 1",
}
}
}

When the type is disconnect, the Msg format is as follows.

Parameter Data type Description
mode String Connection mode: WebRTC.

Sample response

{
"protocol":302,
"pv":"2.2",
"t":1600820048679,
"data":{
"header":{
"from":"AY1600819753305aHO5Sdj8pQMtLZ68XHMUpHKlRKJ87s",
"to":"6c9a943f2ea6929675ymcq",
"sessionid":"00b00036521743319b4d4c01f1705c48",
"moto_id":"moto_5f685396jK",
"type":"disconnect"
},
"msg":{
"mode":"webrtc"
}
}
}

FAQs

Q: How to receive messages from Tuya’s MQTT service?
A: After you get the open platform configs, you need to use the string after /av/u/ in JSON field in result.source_topic.ipc as the value of from in the MQTT Header. In this way, you can receive MQTT service messages as expected.