Skip to content

Commit 7fcd406

Browse files
ossrs-aiossrs-ai
authored
AI: WebRTC: Support VP9 codec for WebRTC-to-WebRTC streaming. v7.0.123 (#4548) (#4565)
VP9 is a similar codec to HEVC, but for WebRTC, VP9 works better than AVC/HEVC in some special cases. However, SRS only support VP9 for WebRTC, doesn't support converting it to RTMP, for RTMP only support AVC/HEVC/AV1 and SRS cannot support transcoding. Usage: * Publish with VP9: [http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream&codec=vp9](http://localhost:8080/players/whip.html?codec=vp9) * Play with VP9: [http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream&codec=vp9](http://localhost:8080/players/whep.html?codec=vp9)
1 parent 1a96abc commit 7fcd406

17 files changed

+765
-179
lines changed

‎trunk/3rdparty/srs-docs/doc/webrtc.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -439,6 +439,37 @@ The streams:
439439
* HTTP-FLV：[http://localhost:8080/live/show.flv](http://localhost:8080/players/srs_player.html?autostart=true&stream=show.flv)
440440
* RTMP by VLC：rtmp://localhost/live/show
441441

442+
## AV1 Codec Support
443+
444+
SRS supports AV1 codec for WebRTC-to-WebRTC streaming since v4.0.91 ([#2324](https://github.com/ossrs/srs/pull/2324)).
445+
AV1 is a royalty-free codec that saves 30-50% bandwidth compared to H.264. SRS implements AV1 as relay-only (SFU mode),
446+
accepting AV1 streams via WHIP and forwarding to WHEP players without transcoding. AV1 streams cannot be converted to
447+
RTMP/HLS or recorded to DVR.
448+
449+
To use AV1, add the `codec=av1` query parameter to WHIP/WHEP URLs:
450+
451+
* Publish: `http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream&codec=av1`
452+
* Play: `http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream&codec=av1`
453+
454+
Browser support: Chrome/Edge M90+, Firefox (full support), Safari (decode only). Use H.264 if you need RTMP/HLS conversion,
455+
DVR recording, or maximum compatibility.
456+
457+
## VP9 Codec Support
458+
459+
SRS supports VP9 codec for WebRTC-to-WebRTC streaming since v7.0.0 ([#4548](https://github.com/ossrs/srs/issues/4548)).
460+
VP9 is a royalty-free codec that saves 20-40% bandwidth compared to H.264. VP9 works better than H.264/H.265 with congestion control
461+
in WebRTC, making it ideal for keeping streams live under network fluctuations. SRS implements VP9 as relay-only (SFU mode),
462+
accepting VP9 streams via WHIP and forwarding to WHEP players without transcoding. VP9 streams cannot be converted to
463+
RTMP/HLS or recorded to DVR.
464+
465+
To use VP9, add the `codec=vp9` query parameter to WHIP/WHEP URLs:
466+
467+
* Publish: `http://localhost:1985/rtc/v1/whip/?app=live&stream=livestream&codec=vp9`
468+
* Play: `http://localhost:1985/rtc/v1/whep/?app=live&stream=livestream&codec=vp9`
469+
470+
Browser support: Chrome/Edge M29+, Firefox M28+, Opera M16+. Safari does not support VP9. Use H.264 if you need RTMP/HLS conversion,
471+
DVR recording, or Safari compatibility.
472+
442473
## SFU: One to One
443474

444475
Please use `conf/rtc.conf` as config.

‎trunk/doc/CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ The changelog for SRS.
77
<a name="v7-changes"></a>
88

99
## SRS 7.0 Changelog
10+
* v7.0, 2025-11-08, AI: WebRTC: Support VP9 codec for WebRTC-to-WebRTC streaming. v7.0.123 (#4548)
1011
* v7.0, 2025-11-08, AI: API: Add audio_frames and video_frames to HTTP API. v7.0.122 (#4559)
1112
* v7.0, 2025-11-07, AI: WHIP: Return detailed HTTP error responses with proper status codes. v7.0.121 (#4502)
1213
* v7.0, 2025-11-07, AI: HLS: Support query string in hls_key_url for JWT tokens. v7.0.120 (#4426)

‎trunk/src/app/srs_app_rtc_api.cpp‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -666,16 +666,16 @@ srs_error_t SrsGoApiRtcWhip::serve_http(ISrsHttpResponseWriter *w, ISrsHttpMessa
666666
string code_str;
667667
if (true) {
668668
srs_error_t err = srs_success;
669-
669+
670670
err = serve_http_with(w, r);
671671
if (err == srs_success) {
672672
return err;
673673
}
674674

675675
code = srs_error_code(err);
676676
code_str = srs_error_code_str(err);
677-
srs_warn("WHIP: serve http for %s with err %d:%s, %s",
678-
r->url().c_str(), code, code_str.c_str(), srs_error_desc(err).c_str());
677+
srs_warn("WHIP: serve http for %s with err %d:%s, %s",
678+
r->url().c_str(), code, code_str.c_str(), srs_error_desc(err).c_str());
679679
srs_freep(err);
680680
}
681681

‎trunk/src/app/srs_app_rtc_conn.cpp‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3402,6 +3402,38 @@ srs_error_t SrsRtcPublisherNegotiator::negotiate_publish_capability(SrsRtcUserCo
34023402
}
34033403
}
34043404

3405+
track_desc->type_ = "video";
3406+
track_desc->set_codec_payload((SrsCodecPayload *)video_payload);
3407+
break;
3408+
}
3409+
} else if (remote_media_desc.is_video() && srs_video_codec_str2id(ruc->codec_) == SrsVideoCodecIdVP9) {
3410+
std::vector<SrsMediaPayloadType> payloads = remote_media_desc.find_media_with_encoding_name("VP9");
3411+
if (payloads.empty()) {
3412+
return srs_error_new(ERROR_RTC_SDP_EXCHANGE, "no found valid VP9 payload type");
3413+
}
3414+
3415+
for (int j = 0; j < (int)payloads.size(); j++) {
3416+
const SrsMediaPayloadType &payload = payloads.at(j);
3417+
3418+
// Generate video payload for vp9.
3419+
SrsVideoPayload *video_payload = new SrsVideoPayload(payload.payload_type_, payload.encoding_name_, payload.clock_rate_);
3420+
3421+
// TODO: FIXME: Only support some transport algorithms.
3422+
for (int k = 0; k < (int)payload.rtcp_fb_.size(); ++k) {
3423+
const string &rtcp_fb = payload.rtcp_fb_.at(k);
3424+
3425+
if (nack_enabled) {
3426+
if (rtcp_fb == "nack" || rtcp_fb == "nack pli") {
3427+
video_payload->rtcp_fbs_.push_back(rtcp_fb);
3428+
}
3429+
}
3430+
if (twcc_enabled && remote_twcc_id) {
3431+
if (rtcp_fb == "transport-cc") {
3432+
video_payload->rtcp_fbs_.push_back(rtcp_fb);
3433+
}
3434+
}
3435+
}
3436+
34053437
track_desc->type_ = "video";
34063438
track_desc->set_codec_payload((SrsCodecPayload *)video_payload);
34073439
break;
@@ -3804,6 +3836,14 @@ srs_error_t SrsRtcPlayerNegotiator::negotiate_play_capability(SrsRtcUserConfig *
38043836
// @see https://bugs.chromium.org/p/webrtc/issues/detail?id=13166
38053837
track_descs = source->get_track_desc("video", "AV1X");
38063838
}
3839+
} else if (prefer_codec == SrsVideoCodecIdVP9) {
3840+
std::vector<SrsMediaPayloadType> payloads = remote_media_desc.find_media_with_encoding_name("VP9");
3841+
if (payloads.empty()) {
3842+
return srs_error_new(ERROR_RTC_SDP_EXCHANGE, "no found valid VP9 payload type");
3843+
}
3844+
3845+
remote_payload = payloads.at(0);
3846+
track_descs = source->get_track_desc("video", "VP9");
38073847
} else if (prefer_codec == SrsVideoCodecIdHEVC) {
38083848
std::vector<SrsMediaPayloadType> payloads = remote_media_desc.find_media_with_encoding_name("H265");
38093849
if (payloads.empty()) {

‎trunk/src/app/srs_app_rtc_source.cpp‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3851,7 +3851,7 @@ srs_error_t SrsRtcFormat::on_rtp_packet(SrsRtcRecvTrack *track, bool is_audio)
38513851
SrsAudioChannels channels = (SrsAudioChannels)audio_media->channel_;
38523852

38533853
if ((err = stat_->on_audio_info(req_, codec_id, sample_rate, channels,
3854-
SrsAacObjectTypeReserved)) != srs_success) {
3854+
SrsAacObjectTypeReserved)) != srs_success) {
38553855
return srs_error_wrap(err, "stat audio info");
38563856
}
38573857
srs_trace("RTC: parsed %s codec, sample_rate=%dHz, channels=%d",
@@ -3926,6 +3926,13 @@ srs_error_t SrsRtcFormat::on_rtp_packet(SrsRtcRecvTrack *track, bool is_audio)
39263926
srs_video_codec_id2str(codec_id).c_str(),
39273927
srs_hevc_profile2str(profile).c_str(),
39283928
level_id);
3929+
} else if (codec_id == SrsVideoCodecIdAV1 || codec_id == SrsVideoCodecIdVP9) {
3930+
// AV1 and VP9 are relay-only codecs for WebRTC-to-WebRTC
3931+
// No detailed profile/level parsing needed for relay mode
3932+
if ((err = stat_->on_video_info(req_, codec_id, 0, 0, 0, 0)) != srs_success) {
3933+
return srs_error_wrap(err, "stat video info");
3934+
}
3935+
srs_trace("RTC: parsed %s codec", srs_video_codec_id2str(codec_id).c_str());
39293936
}
39303937
}
39313938

‎trunk/src/app/srs_app_statistic.cpp‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -154,6 +154,12 @@ srs_error_t SrsStatisticStream::dumps(SrsJsonObject *obj)
154154
} else if (vcodec_ == SrsVideoCodecIdHEVC) {
155155
video->set("profile", SrsJsonAny::str(srs_hevc_profile2str(hevc_profile_).c_str()));
156156
video->set("level", SrsJsonAny::str(srs_hevc_level2str(hevc_level_).c_str()));
157+
} else if (vcodec_ == SrsVideoCodecIdAV1) {
158+
video->set("profile", SrsJsonAny::null());
159+
video->set("level", SrsJsonAny::null());
160+
} else if (vcodec_ == SrsVideoCodecIdVP9) {
161+
video->set("profile", SrsJsonAny::null());
162+
video->set("level", SrsJsonAny::null());
157163
} else {
158164
video->set("profile", SrsJsonAny::str("Other"));
159165
video->set("level", SrsJsonAny::str("Other"));

‎trunk/src/core/srs_core_version7.hpp‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,6 @@
99

1010
#define VERSION_MAJOR 7
1111
#define VERSION_MINOR 0
12-
#define VERSION_REVISION 122
12+
#define VERSION_REVISION 123
1313

1414
#endif

‎trunk/src/kernel/srs_kernel_codec.cpp‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,8 @@ string srs_video_codec_id2str(SrsVideoCodecId codec)
8282
return "HEVC";
8383
case SrsVideoCodecIdAV1:
8484
return "AV1";
85+
case SrsVideoCodecIdVP9:
86+
return "VP9";
8587
case SrsVideoCodecIdReserved:
8688
case SrsVideoCodecIdReserved1:
8789
case SrsVideoCodecIdReserved2:
@@ -105,6 +107,8 @@ SrsVideoCodecId srs_video_codec_str2id(const std::string &codec)
105107
return SrsVideoCodecIdHEVC;
106108
} else if (upper_codec == "AV1") {
107109
return SrsVideoCodecIdAV1;
110+
} else if (upper_codec == "VP9") {
111+
return SrsVideoCodecIdVP9;
108112
} else if (upper_codec == "VP6") {
109113
return SrsVideoCodecIdOn2VP6;
110114
} else if (upper_codec == "VP6A") {

‎trunk/src/kernel/srs_kernel_codec.hpp‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,10 @@ enum SrsVideoCodecId {
6161
SrsVideoCodecIdAVC = 7,
6262
// See page 79 at @doc https://github.com/CDN-Union/H265/blob/master/Document/video_file_format_spec_v10_1_ksyun_20170615.doc
6363
SrsVideoCodecIdHEVC = 12,
64-
// https://mp.weixin.qq.com/s/H3qI7zsON5sdf4oDJ9qlkg
64+
// AV1 codec for WebRTC, https://github.com/ossrs/srs/pull/2324
6565
SrsVideoCodecIdAV1 = 13,
66+
// VP9 codec for WebRTC, https://github.com/ossrs/srs/pull/4565
67+
SrsVideoCodecIdVP9 = 14,
6668
};
6769
std::string srs_video_codec_id2str(SrsVideoCodecId codec);
6870
SrsVideoCodecId srs_video_codec_str2id(const std::string &codec);

‎trunk/src/kernel/srs_kernel_error.hpp‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -111,10 +111,9 @@
111111
XX(ERROR_STREAM_DISPOSING, 1098, "StreamDisposing", "Stream is disposing") \
112112
XX(ERROR_NOT_IMPLEMENTED, 1099, "NotImplemented", "Feature is not implemented") \
113113
XX(ERROR_NOT_SUPPORTED, 1100, "NotSupported", "Feature is not supported") \
114-
XX(ERROR_SYSTEM_FILE_UNLINK, 1101, "FileUnlink", "Failed to unlink file") \
114+
XX(ERROR_SYSTEM_FILE_UNLINK, 1101, "FileUnlink", "Failed to unlink file") \
115115
XX(ERROR_SYSTEM_AUTH, 1102, "SystemAuth", "Failed to authenticate stream")
116116

117-
118117
/**************************************************/
119118
/* RTMP protocol error. */
120119
#define SRS_ERRNO_MAP_RTMP(XX) \
@@ -382,7 +381,7 @@
382381
XX(ERROR_RTSP_NO_TRACK, 5039, "RtspNoTrack", "Drop RTSP packet for track not found") \
383382
XX(ERROR_RTSP_TOKEN_NOT_NORMAL, 5040, "RtspToken", "Invalid RTSP token state not normal") \
384383
XX(ERROR_RTSP_REQUEST_HEADER_EOF, 5041, "RtspHeaderEof", "Invalid RTSP request for header EOF") \
385-
XX(ERROR_RTSP_NEED_MORE_DATA, 5042, "RtspNeedMoreData", "Need more data to complete RTCP frame parsing") \
384+
XX(ERROR_RTSP_NEED_MORE_DATA, 5042, "RtspNeedMoreData", "Need more data to complete RTCP frame parsing") \
386385
XX(ERROR_RTC_INVALID_SDP, 5043, "RtcInvalidSdp", "Invalid SDP for RTC")
387386

388387
/**************************************************/
@@ -492,7 +491,7 @@ class SrsCplxError
492491
// }
493492
#define srs_error_new(code, fmt, ...) SrsCplxError::create(__FUNCTION__, __FILE__, __LINE__, code, fmt, ##__VA_ARGS__)
494493

495-
// Wrap an existing error with additional context. The error code is
494+
// Wrap an existing error with additional context. The error code is
496495
// preserved from the wrapped error.
497496
//
498497
// Example:
@@ -501,8 +500,8 @@ class SrsCplxError
501500
// }
502501
#define srs_error_wrap(err, fmt, ...) SrsCplxError::wrap(__FUNCTION__, __FILE__, __LINE__, err, fmt, ##__VA_ARGS__)
503502

504-
// Transform an error by wrapping it and changing its error code. Useful
505-
// for converting internal errors to protocol-specific error codes (e.g.,
503+
// Transform an error by wrapping it and changing its error code. Useful
504+
// for converting internal errors to protocol-specific error codes (e.g.,
506505
// HTTP, RTMP). The wrapped error chain is preserved.
507506
//
508507
// Example:

0 commit comments

Comments
 (0)