首先看下webrtcglossary中的定义。
RED stands for REDundant coding and it is a RTP payload format defined in RFC 2198 for encoding redundant audio or video data.
The primary motivation of sending redundant data is to be able to recover packets lost under lossy network conditions. If a packet is lost then the missing information may be reconstructed at the receiver from the redundant data that arrives in the following packet(s).
The use of RED is negotiated in the SDP as an additional payload type and when used the audio/video RTP packets are packaged using RED format with a 6 bytes header, before the primary and secondary payloads conveying the actual audio/video packet and the redundant information.
RED封装简介
主要定义了一种封装格式，用于携带冗余编码的数据。RED封装的数据作为payload跟在RTP header后面。RED封装有自己的header。在一个RTP包中，会包含多个RED header，指定了后面携带的编码数据信息。这些RED header依次跟在RTP header后面，在这些RED header后就是各个RED header对应的冗余编码数据。如下是一个携带两个RED header的RTP包结构：
|RTP header| RED header | RED headr | RED encoding data | RED encoding data|
RED header结构如下：
|
1 2 3 4 5 |
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |F| block PT | timestamp offset | block length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
- F： 1 bit长度，说明后面是否还有其他RED header跟着。为1说明还有其他RED header跟着，为0说明这是最后一个header
- blcok PT：7 bits长度，表示该RED header对应的冗余编码类型
- timeoffset： 14 bits长度，表示无符号长度时间戳偏移，该偏移相对于RTP header的时间戳。用无符号长度做偏移意味着冗余编码数据必须在发送完原始数据后才能发送
- block length: 10 bits长度，表示该RED header对应编码数据块长度，该长度包含RED header长度
对于RTP包中最后一个RED header，可以忽略block length以及timestamp。因为它们可以从RTP header以及整个RTP包长度计算得到。最后一个RED header结构如下：
|
1 2 3 4 |
0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ |0| Block PT | +-+-+-+-+-+-+-+-+ |
其中F-bit位始终为0。
如下是一个完整RTP包示例，包含两个编码数据块：DVI4编码块，以及一个LPC冗余编码块。可以看到有两个RED header跟在RTP header后，第一个RED header F-bit为1，第二个RED header F-bit为0。在RED headr后就是对应的编码数据块。
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |V=2|P|X| CC=0 |M| PT | sequence number of primary | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | timestamp of primary encoding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | synchronization source (SSRC) identifier | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| block PT=7 | timestamp offset | block length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| block PT=5 | | +-+-+-+-+-+-+-+-+ + | | + LPC encoded redundant data (PT=7) + | (14 bytes) | + +---------------+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | | + + | | + + | | + + | DVI4 encoded primary data (PT=5) | + (84 bytes, not to scale) + / / + + | | + + | | + +---------------+ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
WebRTC封装代码
- UlpfecGenerator::GetUlpfecPacketsAsRed
WebRTC相关封装代码位于UlpfecGenerator类中。RED封装主要配合UlpFEC使用。这里我们看下UlpFEC编码数据封装为RED包的过程:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 |
std::vector<std::unique_ptr<RedPacket>> UlpfecGenerator::GetUlpfecPacketsAsRed( int red_payload_type, int ulpfec_payload_type, uint16_t first_seq_num) { std::vector<std::unique_ptr<RedPacket>> red_packets; red_packets.reserve(generated_fec_packets_.size()); RTC_DCHECK(!media_packets_.empty()); ForwardErrorCorrection::Packet* last_media_packet = media_packets_.back().get(); uint16_t seq_num = first_seq_num; for (const auto* fec_packet : generated_fec_packets_) { // Wrap FEC packet (including FEC headers) in a RED packet. Since the // FEC packets in |generated_fec_packets_| don't have RTP headers, we // reuse the header from the last media packet. RTC_DCHECK_GT(last_media_packet_rtp_header_length_, 0); std::unique_ptr<RedPacket> red_packet( new RedPacket(last_media_packet_rtp_header_length_ + kRedForFecHeaderLength + fec_packet->data.size())); // 创建头部 red_packet->CreateHeader(last_media_packet->data.data(), last_media_packet_rtp_header_length_, red_payload_type, ulpfec_payload_type); red_packet->SetSeqNum(seq_num++); red_packet->ClearMarkerBit(); // 设置UlpFEC编码数据 red_packet->AssignPayload(fec_packet->data.data(), fec_packet->data.size()); red_packets.push_back(std::move(red_packet)); } ResetState(); return red_packets; } |
- RedPacket::CreateHeader
WebRTC中RED headerF-bit总是为0，不包含时间戳偏移等信息，也就是说一个RTP包只包含一个RED header：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
void RedPacket::CreateHeader(const uint8_t* rtp_header, size_t header_length, int red_payload_type, int payload_type) { RTC_DCHECK_LE(header_length + kRedForFecHeaderLength, length_); memcpy(data_.get(), rtp_header, header_length); // Replace payload type. data_[1] &= 0x80; data_[1] += red_payload_type; // Add RED header // f-bit always 0 data_[header_length] = static_cast<uint8_t>(payload_type); header_length_ = header_length + kRedForFecHeaderLength; } |
WebRTC解析代码
位于UlpfecReceiverImpl类中，我们看下主要的代码：
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
bool UlpfecReceiverImpl::AddReceivedRedPacket(const RtpPacket& rtp_packet, uint8_t ulpfec_payload_type) { // Remove RED header of incoming packet and store as a virtual RTP packet. auto received_packet = std::make_unique<ForwardErrorCorrection::ReceivedPacket>(); received_packet->pkt = new ForwardErrorCorrection::Packet(); // Get payload type from RED header and sequence number from RTP header. uint8_t payload_type = rtp_packet.payload()[0] & 0x7f; received_packet->is_fec = payload_type == ulpfec_payload_type; received_packet->ssrc = rtp_packet.Ssrc(); received_packet->seq_num = rtp_packet.SequenceNumber(); // 就像前面说的，WebRTC中一个RTP包只包含一个RED header if (rtp_packet.payload()[0] & 0x80) { // f bit set in RED header, i.e. there are more than one RED header blocks. // WebRTC never generates multiple blocks in a RED packet for FEC. RTC_LOG(LS_WARNING) << "More than 1 block in RED packet is not supported."; return false; } ++packet_counter_.num_packets; packet_counter_.num_bytes += rtp_packet.size(); if (packet_counter_.first_packet_time_ms == -1) { packet_counter_.first_packet_time_ms = rtc::TimeMillis(); } // 获取RED header后携带的FEC编码数据 if (received_packet->is_fec) { ++packet_counter_.num_fec_packets; // everything behind the RED header received_packet->pkt->data = rtp_packet.Buffer().Slice(rtp_packet.headers_size() + kRedHeaderLength, rtp_packet.payload_size() - kRedHeaderLength); } return true; } |
参考
[1] RED (REDundant coding).https://webrtcglossary.com/red/.
[2] RTP Payload for Redundant Audio Data.https://tools.ietf.org/html/rfc2198.
文章评论