有关WebRTC

WebRTC原理及分析
1. 媒体协议
  1. RTP, 实时传输协议
    1. rfc1889, rfc3550: RTP: A Transport Protocol for Real-Time Applications
    2. rfc1890, rfc3551: RTP Profile for Audio and Video Conferences with Minimal Control
    3. rfc4571: Framing Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) Packets over Connection-Oriented Transport
    4. rfc5761: Multiplexing RTP Data and Control Packets on a Single Port
    5. rfc6051: Rapid Synchronisation of RTP Flows
    6. rfc7160: Support for Multiple Clock Rates in an RTP Session
    7. rfc7164: RTP and Leap Seconds
    8. rfc8083: Multimedia Congestion Control: Circuit Breakers for Unicast RTP Sessions
    9. rfc8108, rfc8860: Sending Multiple RTP Streams in a Single RTP Session
    10. rfc5285, rfc8285: A General Mechanism for RTP Header Extensions
    11. 图例
      1. RTP Format, 请参考http://www.networksorcery.com/enp/protocol/rtp.htm
      2. V：RTP版本号，为“10”
      3. P：填充指示位，P为“1”时表示分组有填充（结尾填充数据）
      4. X：扩展指示位，X为“1”时，则表示固定头部后还有一个扩展头部
      5. CC：CSRC计数，指示固定头部后的CSRC的个数
      6. M：由应用文档解释（如视频帧中，可用于标识一帧的结束（一帧的最后一个包））
      7. PT：RTP分组的净荷类型，如： “0”： G.711μ “8”： G.711A “4”： G.723.1 “18”： G.729 “96~127”：用于动态协商的算法，如iLBC、AMR、SPEEX等
      8. 时间戳：表示RTP分组第一个字节的取样时刻。其初值为随机数，每个采样周期加1。如每次传送20ms的采样频率为8000Hz的数据，则每个传输分组时间戳增加160;
      9. SSRC：同步源标识(Synchronous Source)，表示信号的同步信源（如产生媒体流的麦克风、摄像机、RTP混合器等）。其值应随机选择，以保证同一个RTP会话中任意两个同步源的SSRC标识不同
      10. CSRC：分信源(贡献源)标识(Contributing Source)，识别该数据包中的有效载荷的贡献源 RTP分组的头部最多可以包含15个CSRC标识，其数目由CC字段指明; 通过混合器时；来源的RTP包的SSRC作为CSRC，混合器的SSRC做为新的SSRC;
    12. 参考资料
      1. RTP: Overview, http://www.cs.columbia.edu/~hgs/rtp/
  2. SRTP, 安全实时传输协议
    1. rfc3711: The Secure Real-time Transport Protocol (SRTP)
    2. rfc6904: Encryption of Header Extensions in the Secure Real-time Transport Protocol (SRTP)
    3. 图例
      1. SRTP over UDP
      2. SRTP Format, 请参考http://www.networksorcery.com/enp/protocol/srtp.htm
      3. RTP extension
      4. Payload
      5. Pad
      6. Pad count
      7. MKI
      8. Authentication tag
    4. 参考资料
      1. <<基于SRTP协议的视频传输安全技术的研究>>论文, 卢陈越
  3. RTCP, 实时传输控制协议
    1. rfc5506: Support for Reduced-Size Real-Time Transport Control Protocol (RTCP): Opportunities and Consequences
    2. rfc4585: Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)
    3. rfc6222, rfc7022: Guidelines for Choosing RTP Control Protocol (RTCP) Canonical Names (CNAMEs)
    4. 图例
      1. RTCP Format, http://www.networksorcery.com/enp/protocol/rtcp.htm
      2. Version
      3. P
      4. Count
      5. Type
  4. SRTCP, 安全实时传输控制协议
    1. 图例
      1. SRTCP Format, http://www.networksorcery.com/enp/protocol/srtcp.htm
      2. E
      3. SRTCP index
      4. SRTCP MKI
      5. Authentication tag
  5. SCTP, 流控制传输协议
    1. rfc2960, rfc4960: Stream Control Transmission Protocol
    2. rfc3309: Stream Control Transmission Protocol (SCTP) Checksum Change
    3. SCTP in Go, https://2013.asiabsdcon.org/papers/abc2013-P7A-paper.pdf
    4. 图例
      1. SCTP Format, http://www.networksorcery.com/enp/protocol/sctp.htm
      2. Source port
      3. Destination port
      4. Verification tag
      5. Checksum
      6. Chunk
2. 信令与SDP协商
  1. JSEP, JavaScript会话建立协议
    1. rfc8829: JavaScript Session Establishment Protocol (JSEP)
      1. https://www.w3.org/TR/webrtc/#call-flow-browser-to-browser https://cloudconvert.com/svg-to-png
      2. 术语
      3. 媒体流(MediaStream), 包含0或多个Track
      4. 媒体流轨(MediaStreamTrack)
      5. 媒体流约束(MediaStreamConstraints)
      6. 媒体描述(media description)
      7. 媒体传输(media transport)
      8. Semantics and Syntax
      9. Session Descriptions and State Machine
      10. JSEP State Machine
      11. Signaling Model
      12. JSEP Signaling Model
      13. Session Description Format
      14. Session Description Control
      15. RtpTransceivers, 允许应用程序访问有关发送和接收数据包的ICE传输的信息
      16. RtpSenders, 允许应用控制一个 MediaStreamTrack 如何被编码并被发送至远程对端
      17. RtpReceivers, 允许应用程序监视流媒体轨 MediaStreamTrack 的接收情况
      18. ICE
      19. ICE Gathering Overview
      20. ICE Candidate Trickling
      21. ICE Candidate Format
      22. ICE Candidate Policy
      23. ICE Candidate Pool
      24. ICE Versions
      25. Video Size Negotiation
      26. Creating an imageattr Attribute
      27. Interpreting imageattr Attributes
      28. Simulcast
      29. Interactions with Forking
      30. Sequential Forking
      31. Parallel Forking
      32. Interface
      33. PeerConnection
      34. RTCpeerConnection API
      35. Constructor
      36. addTrack, 将一个被指定媒体流 MediaStream 对象包含的新媒体轨加入 RTCPeerConnection
      37. removeTrack
      38. addTransceiver
      39. onaddtrack Event
      40. createDataChannel
      41. ondatachannel Event
      42. createOffer
      43. createAnswer
      44. SessionDescriptionType
      45. Use of Provisional Answers
      46. Rollback
      47. setLocalDescription
      48. setRemoteDescription
      49. currentLocalDescription
      50. pendingLocalDescription
      51. currentRemoteDescription
      52. pendingRemoteDescription
      53. canTrickleIceCandidates
      54. setConfiguration
      55. addIceCandidate
      56. onicecandidate Event
      57. RtpTransceiver
      58. stop
      59. stopped
      60. setDirection
      61. direction
      62. currentDirection
      63. setCodecPreferences
      64. SDP Interaction Procedures
      65. Profile Names and Interoperability
      66. Constructing an Offer
      67. Initial Offers
      68. Subsequent Offers
      69. Options Handling
      70. IceRestart
      71. VoiceActivityDetection
      72. Generating an Answer
      73. Initial Answers
      74. Subsequent Answers
      75. Options Handling
      76. VoiceActivityDetection
      77. Modifying an Offer or Answer
      78. Processing a Local Description
      79. Processing a Remote Description
      80. Processing a Rollback
      81. Parsing a Session Description
      82. Session-Level Parsing
      83. Media Section Parsing
      84. Semantics Verification
      85. SDP ABNF Syntax
      86. Applying a Local Description
      87. Applying a Remote Description
      88. Applying an Answer
      89. Processing RTP/RTCP
  2. SDP, 描述会话的协议
    1. rfc2327, rfc4566, rfc8866: SDP: Session Description Protocol
    2. rfc3266: Support for IPv6 in Session Description Protocol (SDP)
    3. rfc3264, rfc5888, rfc7941, rfc8843: Negotiating Media Multiplexing Using the Session Description Protocol(SDP)
      1. sdp offer/answer model
      2. session level
      3. media level
    4. rfc3605: Real Time Control Protocol (RTCP) attribute in Session Description Protocol (SDP)
    5. rfc3890: A Transport Independent Bandwidth Modifier for the Session Description Protocol (SDP)
    6. rfc4145: TCP-Based Media Transport in the Session Description Protocol (SDP)
    7. rfc6236: Negotiation of Generic Image Attributes in the Session Description Protocol (SDP)
    8. Anatomy of a WebRTC SDP, https://webrtchacks.com/sdp-anatomy/
    9. rfc4145: TCP-Based Media Transport in the Session Description Protocol (SDP)
    10. 在WebRTC中，SDP主要用来描述： 1. 设备支持的媒体能力，包括编解码器等 2. ICE候选地址 3. 流媒体传输协议
  3. SIP, 会话发起协议
    1. rfc3261: SIP: Session Initiation Protocol
    2. rfc3261, rfc3264: SIP: Session Initiation Protocol
  4. XMPP, 开放式实时通信协议
    1. rfc3921, rfc6121: Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence
  5. HTTP/1.1, 超文本传输协议
    1. 参考资料
      1. HTTP API 认证授权术, https://coolshell.cn/articles/19395.html
      2. HTTP权威指南
      3. <<透视HTTP协议>>-罗剑锋
      4. HTTP History
      5. HTTP 协议始于三十年前蒂姆·伯纳斯 - 李的一篇论文；
      6. HTTP/0.9 是个简单的文本协议，只能获取文本资源；
      7. HTTP/1.0 确立了大部分现在使用的技术，但它不是正式标准；
      8. HTTP/1.1 是目前互联网上使用最广泛的协议，功能也非常完善；
      9. HTTP/2 基于 Google 的 SPDY 协议，注重性能改善，但还未普及；
      10. HTTP/3 基于 Google 的 QUIC 协议，是将来的发展方向。
    2. RFC
      1. rfc2616
      2. rfc7230-7235
      3. 2014年重修修订, rfc2616拆分
    3. 特性
      1. HTTP/1.0 vs HTTP/1.1
      2. 增加了 PUT、DELETE 等新的方法；
      3. 增加了缓存管理和控制；
      4. 明确了连接管理，允许持久连接；
      5. 允许响应数据分块（chunked），利于传输大文件；
      6. 强制要求 Host 头，让互联网主机托管成为可能；
      7. 基本认证授权
      8. HTTP Basic
      9. Digest Access
      10. App Secret Key + HMAC
      11. JWT – JSON Web Tokens
      12. OAuth 1.0 – 3 legged & 2 legged
      13. OAuth 2.0 – Authentication Code & Client Credential
      14. 摘要认证
      15. Ajax, client主动发起请求数据
      16. 短连接(实指Tcp连接, 一次请求后关闭)
      17. XHR, XMLHttpRequest对象
      18. XMLHttpRequest Living Standard: https://xhr.spec.whatwg.org/
      19. 服务器Push技术
      20. Comet, 彗星技术
      21. 基于Ajax的长轮询(需要client和server配合)
      22. 服务器推”与传统的 AJAX?
      23. 服务器端会阻塞请求直到有数据传递或超时才返回
      24. 客户端 JavaScript 响应处理函数会在处理完服务器返回的信息后，再次发出请求，重新建立连接
      25. 当客户端处理接收的数据、重新建立连接时，服务器端可能有新的数据到达；这些信息会被服务器端保存直到客户端重新建立连接，客户端会一次把当前服务器端所有的信息取回
      26. 基于iframe
      27. 基于http流
      28. SSE, 服务器推送事件(单向, 不支持跨域)
      29. 支持短轮询
      30. 支持长轮询
      31. 支持http流
      32. html5标准
      33. WebSocket, 全双工
      34. rfc6455: The WebSocket Protocol
      35. Socket.io, client端WebSocket库
      36. html5标准
      37. 一般都是长连接(Tcp keeplive)
  6. HTTPS, 安全Http HTTPS = HTTP + 加密 + 认证 + 完整性保护
    1. RFC
      1. rfc2818: HTTP Over TLS
    2. 实验
      1. ssl测试, https://www.ssllabs.com/ssltest/index.html
      2. Openssl使用
      3. Let's Encrypt使用, docs, https://letsencrypt.org/zh-cn/docs/
      4. 免费申请证书
      5. nginx配置证书
      6. wireshark分析https
      7. #配置浏览器导出SSL Key export SSLKEYLOGFILE="$HOME/keylog.log" #wireshark中配置SSL Key 首选项->协议->TLS(Master-Secret log filename), 指定SSL Key文件路径
      8. chrome行为分析
      9. wireshark: 过滤http chrome打开: https://http1.akamai.com/demo/h1_demo_frame.html #证实chrome会使用大于6个tcp流访问同域网站 wireshark: 追踪流->tcp流 => http and tcp.stream eq 3 #3测试换成其他数字
      10. tshark分析https
      11. tcpdump分析https
    3. 参考资料
      1. HTTPS流程(使用证书的公钥方式)
      2. 图解SSL/TLS协议, http://www.ruanyifeng.com/blog/2014/09/illustration-ssl.html
      3. SSL/TLS协议运行机制的概述, http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html
      4. SSL/TLS协议运行机制的概述, http://www.ruanyifeng.com/blog/2014/02/ssl_tls.html
      5. SSL/TLS发展历史和SSLv3.0协议详解, http://www.bewindoweb.com/271.html
  7. HTTP2
    1. RFC
      1. rfc7540: Hypertext Transfer Protocol version 2
      2. rfc7541: HPACK - Header Compression for HTTP/2
      3. rfc6797: HPACK - HTTP Strict Transport Security
    2. 特性
      1. 二进制协议，不再是纯文本
      2. 可发起多个请求，废弃了 1.1 里的管道
      3. 使用专用算法压缩头部，减少数据传输量
      4. 允许服务器主动向客户端推送数据
      5. 增强了安全性，“事实上”要求加密通信
    3. 实验
      1. wireshark分析http2
      2. wireshark: 过滤http2 chrome打开: https://http2.akamai.com/demo/h2_demo_frame.html
      3. tshark分析http2
    4. 参考资料
      1. http1.1 vs http2, https://http2.akamai.com/demo
      2. HTTP2详解, https://daniel.haxx.se/http2/
      3. HTTP 2.0 协议详解, https://github.com/zqjflash/http2-protocol
      4. Better HTTP/2 Prioritization for a Faster Web, https://blog.cloudflare.com/better-http-2-prioritization-for-a-faster-web/
      5. http2支持清单, https://github.com/httpwg/http2-spec/wiki/Implementations
      6. NGINX_HTTP2_White_Paper_v4, https://www.nginx.com/wp-content/uploads/2015/09/NGINX_HTTP2_White_Paper_v4.pdf
      7. 使用HTTP/2提升性能的7个建议, https://www.w3ctech.com/topic/1563
      8. HTTP性能极限优化, https://taohui.blog.csdn.net/article/details/103952639
      9. HTTP/2资料汇总, https://imququ.com/post/http2-resource.html
  8. HTTP3
    1. RFC
      1. QUIC RFC: https://tools.ietf.org/html/draft-ietf-quic-transport-24 QUIC with TLS RFC: https://quicwg.org/base-drafts/draft-ietf-quic-tls.html#name-introduction 中文QUIC RFC: http://docs.wxclimb.top/draft-ietf-quic-transport-zh.html#fc-credit
    2. 特性
    3. 参考资料
      1. http1 vs http2 vs http3
      2. HTTP3详解, https://http3-explained.haxx.se/zh/
      3. QUIC & HTTP/3 Support：主流浏览器和服务端对 HTTP/3 的支持情况, https://sysin.org/article/QUIC-HTTP3-Support/
      4. QUIC & HTTP/3 Support：主流浏览器和服务端对 HTTP/3 的支持情况, https://sysin.org/article/QUIC-HTTP3-Support/
      5. HTTP/HTTPS/HTTP2/HTTP3/QUIC/WebSocket协议详解, https://segmentfault.com/a/1190000021390130?utm_source=tag-newest
      6. HTTP/3：起源, https://github.com/xitu/gold-miner/blob/master/TODO1/http-3-from-root-to-tip.md
3. P2P内网穿透
  1. nat, 网络地址转换
  2. stun, NAT会话穿越应用程序
    1. rfc3489: STUN - Simple Traversal of User Datagram Protocol (UDP) hrough Network Address Translators (NATs)
      1. Binding Lifetime Discovery
      2. Test II checks the presence of a NAT or symmetric UDP firewall, and discovers a full cone NAT Test I' discovers a symmetric NAT Test III discovers a restricted con NAT or port restricted cone NAT
      3. Open Internet, 主机具有公网IP，允许主动发起和被动响应两种方式的UDP通信
      4. UDP Blocked, 位于防火墙之后，并且防火墙阻止了UDP通信
      5. Symmetric Firewall, 主机具有公网IP，但位于防火墙之后，且防火墙阻止了外部主机的主动UDP通信
      6. Full Cone NAT, 当内网主机创建一个UDP socket并通过它第一次向外发送UDP数据包时，NAT会为之分配一个固定的公网{IP:端口}。此后，通过这个socket发送的任何UDP数据包都是通过这个公网{IP:端口}发送出去的；同时，任何外部主机都可以使用这个公网{IP:端口}向该socket发送UDP数据包。即是说，NAT维护了一个映射表，内网主机的内网{IP:端口}与公网{IP:端口}是一一对应的关系。一旦这个映射关系建立起来（内部主机向某一外部主机发送一次数据即可），任何外部主机就可以直接向NAT内的这台主机发起UDP通信了，此时NAT透明化了
      7. Full Cone NAT
      8. Restricted Cone NAT, 当内网主机创建一个UDP socket并通过它第一次向外发送UDP数据包时，NAT会为之分配一个公网{IP:端口}。此后，通过这个socket向外发送的任何UDP数据包都是通过这个公网{IP:端口}发送出去的；而任何收到过从这个socket发送来的数据的外部主机（由IP标识），都可以通过这个公网{IP:端口}向该socket发送UDP数据包。即是说，NAT维护了一个内网{IP:端口}到公网{IP:端口}的映射，还维护了一个{外部主机{IP}, 公网{IP:端口}}到内网{IP:端口}的映射。因此，要想外部主机能够主动向该内部主机发起通信，必须先由该内部主机向这个外部发起一次通信
      9. Address Restricted Cone NAT
      10. Port Restricted Cone NAT, 当内网主机创建一个UDP socket并通过它第一次向外发送UDP数据包时，NAT会为之分配一个公网{IP:端口}。此后，通过这个socket向外部发送的任何UDP数据包都是通过这个公网{IP:端口}发送出去的；一旦外部主机在{IP:端口}上收到过从这个socket发送来的数据后，都可以通过这个外部主机{IP:端口}向该socket发送UDP数据包。即是说，NAT维护了一个从内网{IP:端口}到公网{IP:端口}的映射，还维护了一个从{外部主机{IP:端口}, 公网{IP:端口}}到内网{IP:端口}的映射
      11. Port Restricted Cone NAT
      12. Symmetrict NAT, 当内网主机创建一个UDP socket并通过它第一次向外部主机1发送UDP数据包时，NAT为其分配一个公网{IP1:端口1}，以后内网主机发送给外部主机1的所有UDP数据包都是通过公网{IP1:端口1}发送的；当内网主机通过这个socket向外部主机2发送UDP数据包时，NAT为其分配一个公网{IP2:端口2}，以后内网主机发送给外部主机2的所有UDP数据包都是通过公网{IP2:端口2}发送的。公网{IP1:端口1}和公网{IP2:端口2}一定不会完全相同（即要么IP不同，要么端口不同，或者都不同）。这种情况下，外部主机只能在接收到内网主机发来的数据时，才能向内网主机回送数据
      13. Symmetric NAT
    2. rfc5780: NAT Behavior Discovery Using Session Traversal Utilities for NAT (STUN)
      1. NAT隐射方式探测
      2. no NAT
      3. 端点无关的隐射NAT
      4. NAT隐射方式探测
      5. 地址和端口相关的隐射NAT
      6. NAT过滤方式探测
      7. 端点无关的过滤NAT
      8. 地址相关的过滤NAT
      9. 地址和端口相关的过滤NAT
    3. rfc5389, rfc8489: Session Traversal Utilities for NAT (STUN)
    4. rfc5764: URI Scheme for the Session Traversal Utilities for NAT (STUN) Protocol
    5. rfc5769: Test Vectors for Session Traversal Utilities for NAT (STUN)
    6. rfc7350: Datagram Transport Layer Security (DTLS) as Transport for Session Traversal Utilities for NAT (STUN)
    7. rfc7443: Application-Layer Protocol Negotiation (ALPN) Labels for Session Traversal Utilities for NAT (STUN) Usages
    8. rfc7635: Session Traversal Utilities for NAT (STUN) Extension for Third-Party Authorization
    9. 图例
      1. STUN Format, http://www.networksorcery.com/enp/protocol/stun.htm
  3. turn, 使用中继穿透NAT:
    1. rfc5766: Traversal Using Relays around NAT (TURN): Relay Extensions to Session Traversal Utilities for NAT (STUN)
    2. rfc6062: Traversal Using Relays around NAT (TURN) Extensions for TCP Allocations
    3. rfc6156: Traversal Using Relays around NAT (TURN) Extension for IPv6
  4. ice, 交互式连接建立
    1. rfc5245 rfc8445: Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols
      1. http://www.hiastar.com/doc/最新ICE协议-RFC5245-RFC8445-Interactive-Connectivity-Establishment-中文详解.pdf
    2. rfc8863: Interactive Connectivity Establishment Patiently Awaiting Connectivity(ICE PAC)
    3. rfc6336: IANA Registry for Interactive Connectivity Establishment (ICE) Options
  5. trickle ice, 缓慢型ice
    1. rfc8838: Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol
  6. mDNS, 多播DNS
    1. rfc6762: Multicast DNS
  7. Tools
    1. Trickle ICE⬇ https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/
    2. 测试NAT外网端口是否开放⬇ http://canyouseeme.org/
    3. //stun客户端go-stun⬇ go get github.com/ccding/go-stun go-stun -vvv -s stun.stunprotocol.org:3478 ------------------------------------------------ curl -L ip.tool.lu
  8. 参考资料
    1. NAT Behavioral Requirements, as Defined by the IETF (RFC 4787) - Part 1. Mapping Behavior⬇ https://netmanias.com/en/?m=view&id=techdocs&tag=248&no=6058
    2. NAT Behavioral Requirements, as Defined by the IETF (RFC 4787) - Part 2. Filtering Behavior⬇ https://netmanias.com/en/?m=view&id=techdocs&tag=248&no=6062
    3. nat-behavior-discovery-using-stun-rfc-5780⬇ https://www.netmanias.com/en/post/techdocs/6067/nat-network-protocol/nat-behavior-discovery-using-stun-rfc-5780
    4. STUN (RFC 3489) vs. STUN (RFC 5389/5780)⬇ https://netmanias.com/en/?m=view&id=techdocs&no=6065&xtag=nat-network-protocol&xref=stun-rfc-3489-vs-stun-rfc-5389-5780
    5. stun协议解析⬇ https://blog.csdn.net/momo0853/article/details/105387675
4. 传输层安全性协议
  1. SSL/TLS, 安全套接层/传输层安全性协议
    1. rfc2246: The TLS Protocol Version 1.0
    2. rfc4346: The Transport Layer Security (TLS) Protocol Version 1.1
    3. rfc5246: The Transport Layer Security (TLS) Protocol Version 1.2
    4. rfc5746: Transport Layer Security (TLS) Renegotiation Indication Extension
    5. rfc8446: The Transport Layer Security (TLS) Protocol Version 1.3
    6. rfc7507: TLS Fallback Signaling Cipher Suite Value (SCSV) for Preventing Protocol Downgrade Attacks
    7. 图例
    8. 参考资料
      1. TLS协议分析与现代加密通信协议设计, https://blog.helong.info/blog/2015/09/06/tls-protocol-analysis-and-crypto-protocol-design/
      2. TLS 握手优化详解, https://imququ.com/post/optimize-tls-handshake.html
  2. DTLS, 数据包安全性协议
    1. rfc4347: Datagram Transport Layer Security
    2. rfc5763: Framework for Establishing a Secure Real-time Transport Protocol (SRTP) Security Context Using Datagram Transport Layer Security (DTLS)
    3. rfc5764: Datagram Transport Layer Security (DTLS) Extension to Establish Keys for the Secure Real-time Transport Protocol (SRTP)
    4. rfc6347: Datagram Transport Layer Security Version 1.2
    5. handshake over DTLS
5. 编解码
  1. rfc67874: WebRTC Audio Codec and Processing Requirements
  2. rfc6716: Definition of the Opus Audio Codec
  3. rfc6386: VP8 Data Format and Decoding Guide
6. WebRTC安全
  1. rfc7675: Session Traversal Utilities for NAT (STUN) Usage for Consent Freshness
  2. rfc8826: Security Considerations for WebRTC
  3. rfc8827: WebRTC Security Architecture
  4. 参考资料
    1. about-crypto, https://gitee.com/yejinlei/about-crypto
7. 其他
  1. rfc8825: Overview: Real-Time Protocols for Browser-Based Applications
  2. rfc8835: Transports for WebRTC
8. 工具
  1. netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
  2. tshark -i lo -d tcp.port==8085,http
  3. strace -k -fp 6 -e trace=network,close,shutdown,rename
by yejinlei
WebRTC相关资料
1. WebRTC 1.0: Real-Time Communication Between Browsers, https://www.w3.org/TR/webrtc/#intro
2. WebRTC权威指南
3. Get Started with WebRTC, https://www.html5rocks.com/en/tutorials/webrtc/basics/#toc-disruptive
WebRTC Trace
1. https://nashaofu.github.io/webrtc-demo/
WebRTC Next
1. WebRTC 的现状和将来：专访 W3C WebRTC Chair Bernard Aboba, https://webrtchacks.com/webrtc-today-tomorrow-bernard-aboba-qa/
Scheme
1. AppRTC
  1. 知识储备
    1. Ndoejs/npm/express/websocket
    2. JS/Grunt
    3. Python
    4. Golang
    5. Nginx
2. SimpleWebRTC
3. Licode
  1. nuve.js
  2. cloudHandler.js
  3. mdb数据库操作
  4. resoucre
    1. room管理
    2. user管理
    3. service管理
    4. token管理
  5. rpc远程调用, mq通讯
  6. erizo_controller信令服务
    1. erizoController
      1. erizoController.js, 简称EC
      2. ./models/Room.js房间管理
      3. ./models/Client.js客户端管理
      4. ./models/Stream.js流管理
      5. ecCloudHandler.js
      6. nuveProxy.js
      7. rpc远程调用,mq通讯
    2. ROV
      1. rovMetricsServer.js,指标
  7. erizoAgent.js, 简称EA initErizo_agent.sh or /initDockerLicode.sh通过node启动,按Agent调度
    1. erizoJS.js 通过lunch.sh来启动进程1:N(N配置指定)
      1. 加载addon
  8. erizo媒体服务, MCU/SFU实现
    1. class MediaStream
    2. class DtlsTransport
    3. class IceConnection
    4. class NicerConnection //使用lynckia/nICEr
    5. class OneToManyProcessor
    6. class SdpInfo
    7. class SrtpChannel
    8. class Stats
    9. class WebRtcConnection
    10. media
    11. pipeline
    12. rtp
    13. stats
    14. thread
  9. RabbitMQ消息中间件
  10. MongoDB数据库
  11. FAQ
    1. //Nodejs C++插件, 文档http://nodejs.cn/api/addons.html grep -nr 'NODE_MODULE' grep -nr 'addon ='|grep require
  12. spine, WebRTC API封装
    1. Spine.js, JavaScript的MVC框架
  13. Client <= erizoClient
  14. 知识储备
    1. Ndoejs/npm/c++ plug编写/Socker.io/Express/Spine/JS
    2. C++/Conan
    3. MQ消息中间件
    4. MongoDB数据库
4. Jitsi
  1. 演示, https://meet.jit.si/
5. Janus
  1. 演示, https://janus.conf.meetecho.com/videoroomtest.html
6. MediaSoup
  1. 演示, https://v3demo.mediasoup.org/
7. OpenVidu/Kurento
  1. 演示, https://www.egv.me/#/hx2021
FAQ
1. 如果信令服务器为scheme外的独立服务器
  1. 可以通过Nginx实现主备容灾和负载均衡
2. 如果信令服务器作为scheme的一部分,例如licode,由其自己负载实现
3. //带宽测试 iperf3 -s iperf3 -c server_ip
开源项目
1. libnice⬇ https://gitlab.freedesktop.org/libnice/libnice/
2. lynckia/nICEr⬇ https://github.com/lynckia/nicer
3. go-stun⬇ https://github.com/ccding/go-stun
4. pjproject⬇ https://github.com/pjsip/pjproject
5. jitsi/turnserver⬇ https://github.com/jitsi/turnserver
6. coturn/coturn⬇ https://github.com/coturn/coturn
7. gortc/gortcd⬇ https://github.com/gortc/gortcd
8. pion⬇ https://pion.ly/
9. webrtc.rs⬇ https://webrtc.rs/
WebRTC API
1. web api
  1. 参考规范: rfc8829 --- WebRTC 1.0: Real-Time Communication Between Browsers, API doc, https://www.w3.org/TR/webrtc/ --- Web API 接口参考, https://developer.mozilla.org/zh-CN/docs/Web/API --- Demos, https://webrtc.github.io/samples/ --- Source, https://github.com/webrtc/samples
  2. 有关Javsscript, Browser, Nodejs
  3. adapter.js, https://github.com/Temasys/AdapterJS
  4. devices
    1. Choose camera, microphone and speaker
    2. Choose media source and audio output
  5. getusermedia
    1. getUserMedia
    2. Basic getUserMedia demo
      1. const constraints = window.constraints = { audio: false, video: true }; //使用getUserMedia函数,传入constraints约束 navigator.mediaDevices.getUserMedia(constraints); document.querySelector('#showVideo').addEventListener('click', e => init(e));
    3. Use getUserMedia with canvas
      1. const video = document.querySelector('video'); const canvas = window.canvas = document.querySelector('canvas'); canvas.width = 480; canvas.height = 360; const button = document.querySelector('button'); button.onclick = function() { canvas.width = video.videoWidth; canvas.height = video.videoHeight; //使用drawImage函数,传入video对象来截图 canvas.getContext('2d').drawImage(video, 0, 0, canvas.width, canvas.height); }; const constraints = { audio: false, video: true }; function handleSuccess(stream) { window.stream = stream; // make stream available to browser console video.srcObject = stream; } function handleError(error) { console.log('navigator.MediaDevices.getUserMedia error: ', error.message, error.name); } navigator.mediaDevices.getUserMedia(constraints).then(handleSuccess).catch(handleError);;
    4. Use getUserMedia with canvas and CSS filters
      1. video.className = filterSelect.value;
    5. Choose camera resolution
      1. //选择分辨率 const qvgaConstraints = { video: {width: {exact: 320}, height: {exact: 240}} }; navigator.mediaDevices.getUserMedia(qvgaConstraints)
    6. Audio-only getUserMedia() output to local audio element
    7. Audio-only getUserMedia() displaying volume
    8. Record stream
    9. Screensharing with getDisplayMedia
    10. Control camera pan, tilt, and zoom
  6. datachannel
    1. Transmit text
    2. Transfer a file
    3. Transfer data
    4. Messaging
  7. capture
    1. Stream from a video element to a video element
    2. Stream from a video element to a peer connection
    3. Stream from a canvas element to a video element
    4. Stream from a canvas element to a peer connection
    5. Record a stream from a canvas element
    6. Guiding video encoding with content hints
  8. peerconnection
    1. state-definitions, 状态定义
      1. signaling状态
      2. RTCSignalingState
      3. RTCPeerConnection连接状态
      4. ice收集状态
      5. ice连接状态
      6. RTCIceConnectionState
    2. Basic peer connection demo
    3. Peer connection using Perfect Negotiation
    4. Audio-only peer connection demo
    5. Change bandwidth on the fly
    6. Change codecs before the call
    7. Upgrade a call and turn video on
    8. Multiple peer connections at once
    9. Forward the output of one PC into another
    10. Munge SDP parameters
    11. Use pranswer when setting up a peer connection
    12. Constraints and stats
    13. More constraints and stats
    14. Display createOffer output for various scenarios
    15. Use RTCDTMFSender
    16. Display peer connection states
    17. ICE candidate gathering from STUN/TURN servers
    18. Do an ICE restart
    19. Web Audio output as input to peer connection
    20. Peer connection as input to Web Audio
    21. Measure how long renegotation takes
  9. extensions
  10. insertable-streams
    1. End to end encryption using WebRTC Insertable Streams
    2. Video analyzer using WebRTC Insertable Streams
    3. Video processing using MediaStream Insertable Streams
    4. Audio processing using MediaStream Insertable Streams
  11. 其他
    1. signaling getway
2. native api
  1. 架构
  2. VIdeo Capture
    1. VideoCaptureModuleV4L2
    2. v4l2 framework
    3. Video for Linux Two API Specification, http://v4l.videotechnology.com/dwg/v4l2.html
  3. Audio Capture/Render
  4. Network I/O
网址 https://tools.ietf.org/html/[rfc] http://www.networksorcery.com/enp/protocol/[rtp].htm