剑痴乎

  • 首页
  • 文章分类
    • 音视频
    • WebRTC
    • 编程之美
    • Linux
    • Windows
    • 生活点滴
    • 校园生活
  • 参考
    • API参考
    • 实用工具
    • 测试音视频
    • 文档
  • 留言板
  • 关于
剑痴乎
代码为剑,如痴如醉
  1. 首页
  2. WebRTC
  3. 正文

WebRTC研究:FEC之RED封装

2020年8月16日 195点热度 0人点赞 0条评论

首先看下webrtcglossary中的定义。

RED stands for REDundant coding and it is a RTP payload format defined in RFC 2198 for encoding redundant audio or video data.

The primary motivation of sending redundant data is to be able to recover packets lost under lossy network conditions. If a packet is lost then the missing information may be reconstructed at the receiver from the redundant data that arrives in the following packet(s).

The use of RED is negotiated in the SDP as an additional payload type and when used the audio/video RTP packets are packaged using RED format with a 6 bytes header, before the primary and secondary payloads conveying the actual audio/video packet and the redundant information.

RED封装简介

RED指冗余编码,并且依据RFC 2198定义了一种RTP负载格式,用于携带冗余编码的音视频数据。RED封装的数据作为payload跟在RTP固定头(FixedHeader)后面。RED封装的数据有自己的Header。在一个RTP包中,会包含多个RED Header,指定了后面携带的编码数据信息。这些RED Header依次跟在RTP Header后面,在这些RED Header后就是各个RED Header对应的冗余编码数据。

如下是一个携带两个RED Header的RTP包结构:
|RTP header| RED header | RED headr | RED encoding data | RED encoding data|

RED Header结构如下:

1
2
3
4
5
    0                   1                    2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3  4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |F|   block PT  |  timestamp offset         |   block length    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  • F: 1 bit长度,说明后面是否还有其他RED Header跟着。为1说明还有其他RED Header跟着,为0说明这是最后一个RED Header
  • blcok PT:7 bits长度,表示该RED Header对应的冗余编码类型
  • timeoffset: 14 bits长度,表示无符号长度时间戳偏移,该偏移是相对于RTP Header的时间戳。用无符号长度做偏移意味着冗余编码数据必须在发送完原始数据后才能发送
  • block length: 10 bits长度,表示该RED Header对应编码数据块长度,该长度包含RED Header字段

对于RTP包中最后一个RED Header,可以忽略block length以及timestamp。因为它们可以从RTP Fixed Header以及整个RTP包长度计算得到。最后一个RED Header结构如下:

1
2
3
4
                      0 1 2 3 4 5 6 7
                     +-+-+-+-+-+-+-+-+
                     |0|   Block PT  |
                     +-+-+-+-+-+-+-+-+

其中F-bit位始终为0。

如下是一个完整RTP包示例,包含两个编码数据块:DVI4编码块,以及一个LPC冗余编码块。可以看到有两个RED Header跟在RTP Fixed Header后,第一个RED header F-bit为1,第二个RED Header由于是最后一个,所以F-bit为0,不包含时间戳偏移以及数据块长度信息。在RED Headrs后就是对应的编码数据块。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
    0                   1                    2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3  4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X| CC=0  |M|      PT     |   sequence number of primary  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |              timestamp  of primary encoding                   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |1| block PT=7  |  timestamp offset         |   block length    |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |0| block PT=5  |                                               |
   +-+-+-+-+-+-+-+-+                                               +
   |                                                               |
   +                LPC encoded redundant data (PT=7)              +
   |                (14 bytes)                                     |
   +                                               +---------------+
   |                                               |               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               +
   |                                                               |
   +                                                               +
   |                                                               |
   +                                                               +
   |                                                               |
   +                                                               +
   |                DVI4 encoded primary data (PT=5)               |
   +                (84 bytes, not to scale)                       +
   /                                                               /
   +                                                               +
   |                                                               |
   +                                                               +
   |                                                               |
   +                                               +---------------+
   |                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

视频RED封装代码

这里我们以视频FEC为例,在WebRTC中音频的带外FEC也用到了RED编码。

UlpfecGenerator::GetUlpfecPacketsAsRed

WebRTC视频相关RED封装代码位于UlpfecGenerator类中。RED封装主要配合UlpFEC使用。这里我们看下UlpFEC编码数据封装为RED包的过程:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
std::vector<std::unique_ptr<RedPacket>> UlpfecGenerator::GetUlpfecPacketsAsRed(
    int red_payload_type,
    int ulpfec_payload_type,
    uint16_t first_seq_num) {
  std::vector<std::unique_ptr<RedPacket>> red_packets;
  red_packets.reserve(generated_fec_packets_.size());
  RTC_DCHECK(!media_packets_.empty());
  ForwardErrorCorrection::Packet* last_media_packet =
      media_packets_.back().get();
  uint16_t seq_num = first_seq_num;
  for (const auto* fec_packet : generated_fec_packets_) {
    // Wrap FEC packet (including FEC headers) in a RED packet. Since the
    // FEC packets in |generated_fec_packets_| don't have RTP headers, we
    // reuse the header from the last media packet.
    RTC_DCHECK_GT(last_media_packet_rtp_header_length_, 0);
    std::unique_ptr<RedPacket> red_packet(
        new RedPacket(last_media_packet_rtp_header_length_ +
                      kRedForFecHeaderLength + fec_packet->data.size()));
    // 创建头部
    red_packet->CreateHeader(last_media_packet->data.data(),
                             last_media_packet_rtp_header_length_,
                             red_payload_type, ulpfec_payload_type);
    red_packet->SetSeqNum(seq_num++);
    red_packet->ClearMarkerBit();
    // 设置UlpFEC编码数据
    red_packet->AssignPayload(fec_packet->data.data(), fec_packet->data.size());
    red_packets.push_back(std::move(red_packet));
  }
 
  ResetState();
 
  return red_packets;
}

RedPacket::CreateHeader

对于视频,WebRTC中RED HeaderF-bit总是为0,不包含时间戳偏移等信息,也就是说一个RTP包只包含一个RED header:

C++
1
2
3
4
5
6
7
8
9
10
11
12
13
14
void RedPacket::CreateHeader(const uint8_t* rtp_header,
                             size_t header_length,
                             int red_payload_type,
                             int payload_type) {
  RTC_DCHECK_LE(header_length + kRedForFecHeaderLength, length_);
  memcpy(data_.get(), rtp_header, header_length);
  // Replace payload type.
  data_[1] &= 0x80;
  data_[1] += red_payload_type;
  // Add RED header
  // f-bit always 0
  data_[header_length] = static_cast<uint8_t>(payload_type);
  header_length_ = header_length + kRedForFecHeaderLength;
}

视频RED解析代码

相关处理位于UlpfecReceiverImpl类中,我们看下主要的代码:

C++
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
bool UlpfecReceiverImpl::AddReceivedRedPacket(const RtpPacket& rtp_packet,
                                              uint8_t ulpfec_payload_type) {
 
  // Remove RED header of incoming packet and store as a virtual RTP packet.
  auto received_packet =
      std::make_unique<ForwardErrorCorrection::ReceivedPacket>();
  received_packet->pkt = new ForwardErrorCorrection::Packet();
 
  // Get payload type from RED header and sequence number from RTP header.
  uint8_t payload_type = rtp_packet.payload()[0] & 0x7f;
  received_packet->is_fec = payload_type == ulpfec_payload_type;
  received_packet->ssrc = rtp_packet.Ssrc();
  received_packet->seq_num = rtp_packet.SequenceNumber();
 
  // 如前面所说,WebRTC中一个视频RTP包只包含一个RED Header
  if (rtp_packet.payload()[0] & 0x80) {
    // f bit set in RED header, i.e. there are more than one RED header blocks.
    // WebRTC never generates multiple blocks in a RED packet for FEC.
    RTC_LOG(LS_WARNING) << "More than 1 block in RED packet is not supported.";
    return false;
  }
 
  ++packet_counter_.num_packets;
  packet_counter_.num_bytes += rtp_packet.size();
  if (packet_counter_.first_packet_time_ms == -1) {
    packet_counter_.first_packet_time_ms = rtc::TimeMillis();
  }
 
  // 获取RED Header后携带的FEC冗余编码数据
  if (received_packet->is_fec) {
    ++packet_counter_.num_fec_packets;
    // everything behind the RED header
    received_packet->pkt->data =
        rtp_packet.Buffer().Slice(rtp_packet.headers_size() + kRedHeaderLength,
                                  rtp_packet.payload_size() - kRedHeaderLength);
  }
 
  return true;
}

参考

[1] RED (REDundant coding).https://webrtcglossary.com/red/.
[2] RTP Payload for Redundant Audio Data.https://tools.ietf.org/html/rfc2198.

本作品采用 知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议 进行许可
标签: WebRTC
最后更新:2020年10月25日

Jeff Young

代码为剑,如痴如醉

打赏 点赞
< 上一篇
下一篇 >

文章评论

取消回复

我的其它小窝

公众号:码上Play(基本不更新,回答问题用)

近期评论
  • Jeff on WebRTC研究:BBR拥塞控制被移除了研究过了,等后面有时间简单说明下
  • xhcx on WebRTC研究:BBR拥塞控制被移除了楼主,BBR移除的原因最近有研究吗,分享一下
  • Jeff on Windows平台WebRTC编译(持续更新)M79是2019年发布的版本,不适用这篇文章。编译…
  • haige on Windows平台WebRTC编译(持续更新)我编译的m79版本,用VS2019打开会报错, F…
  • 菜菜 on libcef编译使用--使用VS2015是真的鸟
版权声明

为支持原创,创作更好的文章,未经许可,禁止任何形式的转载与抄袭,如需转载请邮件私信!本人保留所有法定权利。违者必究!

目录
  • 1 RED封装简介
  • 2 视频RED封装代码
  • 3 视频RED解析代码
  • 4 参考
相关文章推荐
  • WebRTC研究:Simulcast层数变化
  • WebRTC研究:RTP头部扩展
  • WebRTC研究:音频带内FEC
  • WebRTC研究:BBR拥塞控制被移除了
  • WebRTC安卓编译

COPYRIGHT © 2021 剑痴乎. ALL RIGHTS RESERVED.

THEME KRATOS MADE BY VTROIS