RTP Payload Format for ISO/IEC 21122 (JPEG XS)intoPIX S.A.Rue Emile Francqui, 9Mont-Saint-Guibert1435Belgium+32 10 23 84 70t.bruylants@intopix.comhttps://www.intopix.com/Université catholique de Louvainbte L2.03.02Ruelle de la Lanterne Magique, 14Louvain-la-Neuve1348Belgium+32 10 47 27 87antonin.descampe@uclouvain.behttps://uclouvain.be/antonin.descampeintoPIX S.A.Rue Emile Francqui, 9Mont-Saint-Guibert1435Belgium+32 10 23 84 70c.damman@intopix.comhttps://www.intopix.com/Fraunhofer IISAm Wolfsmantel 33Erlangen91048Germany+49 9131 776 5126thomas.richter@iis.fraunhofer.dehttps://www.iis.fraunhofer.de/
General
avtcorevideotransportprotocolJointPhotographicExpertsGroupreal-timestream
This document specifies a Real-Time Transport Protocol (RTP) payload
format to be used for transporting video encoded with JPEG XS (ISO/IEC 21122).
JPEG XS is a low-latency, lightweight image coding system. Compared to an
uncompressed video use case, it allows higher resolutions and video frame
rates while offering visually lossless quality, reduced power
consumption, and encoding-decoding latency confined to a fraction of a video frame.
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force
(IETF). It represents the consensus of the IETF community. It has
received public review and has been approved for publication by
the Internet Engineering Steering Group (IESG). Further
information on Internet Standards is available in Section 2 of
RFC 7841.
Information about the current status of this document, any
errata, and how to provide feedback on it may be obtained at
.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents
() in effect on the date of
publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with
respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License.
Table of Contents
. Introduction
. Conventions, Definitions, and Abbreviations
. Media Format Description
. Image Data Structures
. Codestream
. Video Support Box and Color Specification Box
. JPEG XS Frame
. RTP Payload Format
. RTP Packetization
. RTP Header Usage
. Payload Header Usage
. Payload Data
. Traffic Shaping and Delivery Timing
. Congestion Control Considerations
. Payload Format Parameters
. Media Type Registration
. SDP Parameters
. Mapping of Payload Type Parameters to SDP
. Usage with SDP Offer/Answer Model
. IANA Considerations
. Security Considerations
. References
. Normative References
. Informative References
Acknowledgments
Authors' Addresses
Introduction
This document specifies a payload format for packetization of video
signals encoded with JPEG
XS into the Real-time
Transport Protocol (RTP).
The JPEG XS coding system offers compression and recompression of image
sequences with very moderate computational resources while remaining
robust under multiple compression and decompression cycles as well as mixing of
content sources, e.g., embedding of subtitles, overlays, or logos. Typical
target compression ratios ensuring visually lossless quality are in the
range of 2:1 to 10:1 depending on the nature of the source material. The
latency that is introduced by the encoding-decoding process can be confined
to a fraction of a video frame, typically between a small number of lines
down to below a single line.
Conventions, Definitions, and Abbreviations
The key words "MUST", "MUST NOT",
"REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT",
"RECOMMENDED", "NOT RECOMMENDED",
"MAY", and "OPTIONAL" in this document
are to be interpreted as described in BCP 14 when,
and only when, they appear in all capitals, as shown here.
Application Data Unit (ADU):
The unit of source data provided as payload to the transport layer.
In this RTP payload definition, it corresponds to a single JPEG
XS video frame.
Color Specification (CS) box:
An ISO color specification box defined in (JPEG XS Part 3) that includes color-related
metadata required to correctly display JPEG XS video frames, such
as color primaries, transfer characteristics, and matrix
coefficients.
End of Codestream (EOC) marker:
A marker that consists of the two bytes 0xff11 indicating the end of
a JPEG XS codestream.
JPEG XS codestream:
A sequence of bytes representing a compressed image formatted
according to (JPEG XS Part 1).
JPEG XS codestream header:
A sequence of bytes, starting with an SOC marker, at the beginning of
each JPEG XS codestream encoded in multiple markers and marker
segments that does not carry entropy coded data, but metadata such as
the video frame dimension and component precision.
JPEG XS frame:
In the case of progressive video, a single JPEG XS picture segment. In
the case of interlaced video, the concatenation of two JPEG XS
picture segments.
JPEG XS header segment:
The concatenation of a video support box, a color specification
box, and a JPEG XS codestream header.
JPEG XS picture segment:
The concatenation of a video support box, a color specification
box, and a JPEG XS codestream.
JPEG XS stream:
A sequence of JPEG XS frames.
Marker:
A two-byte functional sequence that is part of a JPEG XS
codestream starting with a 0xff byte and a subsequent byte
defining its function.
Marker segment:
A marker along with a 16-bit marker size and payload data
following the size.
Packetization unit:
A portion of an ADU whose boundaries coincide
with boundaries of RTP packet payloads (excluding payload header),
i.e., the first (or respectively, last) byte of a packetization unit is the
first (or respectively, last) byte of an RTP packet payload (excluding its
payload header).
SLH (SLice Header) marker:
A marker that represents a slice header, as defined in .
Slice:
The smallest independently decodable unit of a JPEG XS codestream,
bearing in mind that it decodes to wavelet coefficients, which
still require inverse wavelet filtering to give an image.
Start of a Codestream (SOC) marker:
A marker that consists of the two bytes 0xff10 indicating the
start of a JPEG XS codestream. The SOC marker is considered an
integral part of the JPEG XS codestream header.
Video Support (VS) box:
An ISO video support box, as defined in ,
that includes metadata required to play back a JPEG XS
stream; such metadata could include its maximum bit rate, its subsampling structure, its
buffer model, and its frame rate.
Media Format Description
This section explains the terminology and concepts used in this memo specific to JPEG XS
as specified in , , and .
Image Data Structures
JPEG XS is a low-latency, lightweight image coding system for coding
continuous-tone grayscale or continuous-tone color digital images.
This coding system provides an efficient representation of image
signals through the mathematical tool of wavelet analysis. The wavelet
filter process separates each component into multiple bands, where
each band consists of multiple coefficients describing the image
signal of a given component within a frequency domain specific to the
wavelet filter type, i.e., the particular filter corresponding to the
band.
Wavelet coefficients are grouped into precincts, where each precinct
includes all coefficients over all bands that contribute to a spatial
region of the image.
One or multiple precincts are furthermore combined into slices
consisting of an integer number of precincts. Precincts do not
cross slice boundaries, and wavelet coefficients in precincts that
are part of different slices can be decoded independently of each
other. However, note that the wavelet transformation runs across
slice boundaries. A slice always extends over the full width of the
image but may only cover parts of its height.
Codestream
A JPEG XS codestream is formed by (in the given order):
a JPEG XS codestream header, which starts with a Start of Codestream (SOC) marker,
one or more slices,
an EOC marker to signal the end of the codestream.
The JPEG XS codestream format, including the definition of all
markers, is further defined in .
It represents sample values of a single image, without any interpretation
relative to a color space.
Video Support Box and Color Specification Box
While the information defined in the codestream is sufficient to
reconstruct the sample values of one image, the interpretation of
the samples remains undefined by the codestream itself. This
interpretation is given by the video support box and the color
specification box, which contain significant information to correctly
play the JPEG XS stream. The layout and syntax of these boxes, together
with their content, are defined in .
The video support box provides information on the maximum bit rate,
the frame rate, the interlaced mode (progressive or interlaced), the
subsampling image format, the informative timecode of the current
JPEG XS frame, the profile, the level/sublevel used, and optionally
the buffer model and the mastering display metadata.
Note that the profile and level/sublevel, specified respectively by
the Ppih and Plev
fields, specify limits on the capabilities needed to decode
the codestream and handle the output. Profiles represent a limit on
the required algorithmic features and parameter ranges used in the
codestream. The combination of level and sublevel defines a lower
bound on the required throughput for a decoder in the
image (or decoded) domain and the codestream (or coded) domain, respectively. The
actual defined profiles and levels/sublevels, along with the
associated values for the Ppih and Plev fields, are defined in .
The color specification box indicates the color primaries, transfer
characteristics, matrix coefficients, and video full range flag needed
to specify the color space of the video stream.
JPEG XS Frame
The concatenation of a video support box, a color specification box,
and a JPEG XS codestream forms a JPEG XS picture segment.
In the case of a progressive video stream, each JPEG XS frame consists of one single
JPEG XS picture segment.
In the case of an interlaced video stream, each JPEG XS frame is made
of two concatenated JPEG XS picture segments. The codestream of each
picture segment corresponds exclusively to one of the two fields of
the interlaced frame. Both picture segments SHALL
contain identical boxes (i.e., the byte sequence that contains the
concatenation of the video support box and the color specification
box is exactly the same in both picture segments of the frame).
Note that the interlaced mode, as signaled by the frat field
in the video support box, indicates either progressive interlaced top-field-first or
interlaced bottom-field-first mode. Thus, in the case of interlaced content, its value
SHALL also be identical in both picture segments.
RTP Payload Format
This section specifies the payload format for JPEG XS streams over the
Real-time Transport Protocol
(RTP).
In order to be transported over RTP, each JPEG XS stream is
transported in a distinct RTP stream, identified by a distinct synchronization source (SSRC).
A JPEG XS stream is divided into Application Data Units (ADUs), each ADU
corresponding to a single JPEG XS frame.
RTP Packetization
An ADU is made of several packetization units. If a packetization unit
is bigger than the maximum size of an RTP packet payload, the unit is
split into multiple RTP packet payloads, as illustrated in . As seen there, each
packet SHALL contain (part of) one, and only one,
packetization unit. A packetization unit may extend over multiple
packets. The payload of every packet SHALL have the
same size (based, e.g., on the Maximum Transfer Unit of the network)
with the possible exception of the last packet of a packetization unit. The
boundaries of a packetization unit SHALL coincide with
the boundaries of the payload of a packet (excluding the payload
header), i.e., the first (or, respectively, last) byte of the
packetization unit SHALL be the first (or, respectively,
last) byte of the payload (excluding its header).
There are two different packetization modes defined for this RTP payload format.
Codestream packetization mode:
In this mode, the packetization unit SHALL be the entire
JPEG XS picture segment (i.e., codestream preceded by boxes). This means that
a progressive frame will have a single packetization unit, while an interlaced
frame will have two. The progressive case is illustrated in .
Slice packetization mode:
In this mode, the packetization unit SHALL be the slice,
i.e., there SHALL be data from no more than one slice per RTP
packet. The first packetization unit SHALL be made of the JPEG
XS header segment (i.e., the concatenation of the VS box, the CS box, and the
JPEG XS codestream header). This first unit is then followed by successive
units, each containing one and only one slice. The packetization unit
containing the last slice of a JPEG XS codestream SHALL also
contain the EOC marker immediately following this last slice. This is
illustrated in . In the
case of an interlaced frame, the JPEG XS header segment of the second field
SHALL be in its own packetization unit.
In a constant bitrate (CBR) scenario of JPEG XS, the codestream packetization
mode guarantees that a JPEG XS RTP stream will produce both a constant number
of bytes per video frame and a constant number of RTP packets per video frame.
However, to provide similar guarantees with JPEG XS in a variable bitrate (VBR)
mode or when using the slice packetization mode (for either CBR or VBR), additional
mechanisms are needed. This can involve a constraint at the rate allocation
stage in the JPEG XS encoder to impose a CBR at the slice level,
the usage of padding data, or the insertion of empty RTP packets (i.e., an RTP
packet whose payload data is empty). But, management of the amount of produced
packets per video frame is application dependent and not a strict requirement of
this RTP payload specification.
RTP Header Usage
The format of the RTP header is specified in and
reprinted in for
convenience. This RTP payload format uses the fields of the header in a
manner consistent with that specification.
The RTP payload (and the settings for some RTP header bits) for
packetization units are specified in .
The version (V), padding (P), extension (X), CSRC count (CC),
sequence number, synchronization source (SSRC), and contributing
source (CSRC) fields follow their respective definitions in
.
The remaining RTP header information to be set according to this RTP
payload format is set as follows:
Marker (M) [1 bit]:
If progressive scan video is being transmitted, the marker bit
denotes the end of a video frame. If interlaced video is being
transmitted, it denotes the end of the field. The marker bit SHALL
be set to 1 for the last packet of the video frame/field. It SHALL
be set to 0 for all other packets.
Payload Type (PT) [7 bits]:
The payload type is a dynamically allocated payload type field that
designates the payload as JPEG XS video.
Timestamp [32 bits]:
The RTP timestamp is set to the sampling timestamp of the content.
A 90 kHz clock rate SHALL be used.
As specified in and
, the RTP timestamp designates the
sampling instant of the first octet of the video frame to which the RTP
packet belongs. Packets SHALL NOT include data from multiple video frames, and
all packets belonging to the same video frame SHALL have the same timestamp.
Several successive RTP packets will consequently have equal timestamps
if they belong to the same video frame (that is until the marker bit is set
to 1, marking the last packet of the video frame), and the timestamp is only
increased when a new video frame begins.
If the sampling instant does not correspond to an integer value of
the clock, the value SHALL be truncated to the next lowest integer,
with no ambiguity.
Payload Header Usage
The first four bytes of the payload of an RTP packet in this RTP
payload format are referred to as the "payload header". illustrates the structure of this
payload header.
The payload header consists of the following fields:
Transmission mode (T) [1 bit]:
The T bit is set to indicate that packets are sent sequentially by the
transmitter. This information allows a receiver to dimension its
input buffer(s) accordingly. If T=0, nothing can be assumed about the
transmission order and packets may be sent out of order by the
transmitter. If T=1, packets SHALL be sent sequentially by the
transmitter. The T-bit value SHALL be identical for all packets of
the RTP stream.
pacKetization mode (K) [1 bit]:
The K bit is set to indicate which packetization mode is used. K=0
indicates codestream packetization mode, while K=1 indicates slice
packetization mode. In the case that the Transmission mode (T) is
set to 0 (out of order), the slice packetization mode SHALL be used
and K SHALL be set to 1. This is required because only the slice
packetization mode supports out-of-order packet transmission. The
K-bit value SHALL be identical for all packets of the RTP stream.
Last (L) [1 bit]:
The L bit is set to indicate the last packet of a packetization unit.
As the end of the video frame also ends the packet containing the last unit
of the video frame, the L bit is set whenever the M bit is set. In
the codestream packetization mode, the L bit and M bit get an equivalent
meaning, so they SHALL have identical values in each packet.
Interlaced information (I) [2 bits]:
These two I bits are used to indicate how the JPEG XS frame is scanned
(progressive or interlaced). In case of an interlaced frame, they also
indicate which JPEG XS picture segment the payload is part of (first
or second).
00:
The payload is progressively scanned.
01:
This value is reserved for future use.
10:
The payload is part of the first JPEG XS picture segment of
an interlaced video frame. The height specified in the included
JPEG XS codestream header is half of the height of the entire
displayed image.
11:
The payload is part of the second JPEG XS picture segment of
an interlaced video frame. The height specified in the included
JPEG XS codestream header is half of the height of the entire
displayed image.
F counter [5 bits]:
The Frame (F) counter identifies the video frame number modulo 32 to which a
packet belongs. Frame numbers are incremented by 1 for each video frame
transmitted. The frame number, in addition to the timestamp, may help
the decoder manage its input buffer and bring packets back into their
natural order.
Slice and Extended Packet (SEP) counter [11 bits]:
The SEP counter is used differently
depending on the packetization mode.
In the case of codestream packetization mode (K=0), this
counter resets whenever the Packet counter resets (see ) and increments by
1 whenever the Packet counter overruns.
In the case of slice packetization mode (K=1), this counter
identifies the slice modulo 2047 to which the packet contributes. If
the data belongs to the JPEG XS header segment, this field SHALL have
its maximal value, namely 2047 = 0x07ff. Otherwise, it is the slice
index modulo 2047. Slice indices are counted from 0 (corresponding to
the top of the video frame).
P counter [11 bits]:
The Packet (P) counter identifies the packet number modulo 2048
within the current packetization unit. It is set to 0 at the start of
the packetization unit and incremented by 1 for every subsequent
packet (if any) belonging to the same unit. Practically, if
codestream packetization mode is enabled, this field counts the
packets within a JPEG XS picture segment and is extended by the SEP
counter when it overruns. If slice packetization mode is enabled,
this field counts the packets within a slice or within the JPEG XS
header segment.
Payload Data
The payload data of a JPEG XS RTP stream consists of a concatenation of
multiple JPEG XS frames. Within the RTP stream, all of the video support boxes
and all of the color specification boxes SHALL retain their respective layouts
for each JPEG XS frame. Thus, each video support box in the RTP stream SHALL
define the same sub boxes. The effective values in the boxes are allowed to
change under the condition that their relative byte offsets SHALL NOT change.
Each JPEG XS frame is the concatenation of one or more packetization
unit(s), as explained in .
depicts this
layout for a progressive video frame in the codestream packetization mode,
depicts this
layout for an interlaced video frame in the codestream packetization mode,
depicts this
layout for a progressive video frame in the slice packetization mode, and
depicts this
layout for an interlaced video frame in the slice packetization mode. The Frame
counter value is not indicated because the value is constant for all
packetization units of a given video frame.
Traffic Shaping and Delivery Timing
In order to facilitate proper synchronization between senders and
receivers, it is RECOMMENDED to implement traffic
shaping and delivery timing in accordance with the Network
Compatibility Model compliance definitions specified in . In such a case, the session
description SHALL signal the compliance with the media
type parameter TP. The actual applied traffic shaping and
timing delivery mechanism is outside the scope of this memo and does
not influence the payload packetization.
Congestion Control Considerations
Congestion control for RTP SHALL be used in accordance with
and with any applicable
RTP profile, e.g., RTP/AVP or
RTP/AVPF.
While JPEG XS is mainly designed to be used in controlled network
environments, it can also be employed in best-effort network
environments, like the Internet. However, in this case, the users of
this payload format SHALL monitor packet loss to ensure
that the packet loss rate is within acceptable parameters. This can be
achieved, for example, by means of RTP Control Protocol (RTCP) Feedback for Congestion
Control.
In addition, is an update to
that defines criteria for
when one is required to stop sending RTP Packet Streams and for when
applications implementing this standard SHALL comply
with it.
provides additional information
on the best practices for applying congestion control to UDP streams.
Payload Format Parameters
This section specifies the required and optional parameters of the payload format and/or
the RTP stream. All parameters are declarative, meaning that the information signaled by
the parameters is also present in the payload data, namely in the payload header (see ) or in the JPEG XS header segment . When provided, their respective values SHALL be consistent with the payload.
Media Type Registration
This registration is done using the template defined in and following .
The receiver SHALL ignore any unrecognized parameter.
Type name:
video
Subtype name:
jxsv
Required parameters:
rate:
The RTP timestamp clock rate. Applications using this payload format SHALL use a value of 90000.
packetmode:
This parameter specifies the configured packetization mode as defined
by the pacKetization mode (K) bit in the payload header of .
This value SHALL be equal to the K-bit value configured in the RTP stream (i.e., 0 for codestream or 1 for slice).
Optional parameters:
transmode:
This parameter specifies the configured transmission mode as
defined by the Transmission mode (T) bit in the payload
header of . If
specified, this value SHALL be equal to the
T-bit value configured in the RTP stream (i.e., 0 for
out-of-order-allowed or 1 for sequential-only). If not
specified, a value 1 (sequential-only) SHALL
be assumed and the T bit SHALL be set to 1.
profile:
The JPEG XS profile in use.
Any white space Unicode character in the profile name SHALL be omitted.
Examples of valid profile names are 'Main444.12' or 'High444.12'.
level:
The JPEG XS level in use.
Any white space Unicode character in the level name SHALL be omitted.
Examples of valid levels are '2k-1' or '4k-2'.
sublevel:
The JPEG XS sublevel in use.
Any white space Unicode character in the sublevel name SHALL be omitted.
Examples of valid sublevels are 'Sublev3bpp' or 'Sublev6bpp'.
depth:
Determines the number of bits per sample. This is an
integer with typical values including 8, 10, 12, and 16.
width:
Determines the number of pixels per line. This is an
integer between 1 and 32767, inclusive.
height:
Determines the number of lines per video frame. This is an
integer between 1 and 32767, inclusive.
exactframerate:
Signals the video frame rate in frames per second.
Integer frame rates SHALL be signaled as a single decimal
number (e.g., "25") whilst non-integer frame rates SHALL be
signaled as a ratio of two integer decimal numbers separated
by a "forward-slash" character (e.g., "30000/1001"), utilizing
the numerically smallest numerator value possible.
interlace:
If this parameter name is present, it indicates
that the video is interlaced, or that the video is
Progressive segmented Frame (PsF). If this parameter name is
not present, the progressive video format SHALL be assumed.
segmented:
If this parameter name is present, and the
interlace parameter name is also present, then the video is a
Progressive segmented Frame (PsF). Signaling of this
parameter without the interlace parameter is forbidden.
sampling:
Signals the color difference signal sub-sampling
structure.
Signals utilizing the non-constant luminance
Y'C'B C'R signal format of ,
, ,
or SHALL use the appropriate one
of the following values for the Media Type Parameter
"sampling":
YCbCr-4:4:4
(4:4:4 sampling)
YCbCr-4:2:2
(4:2:2 sampling)
YCbCr-4:2:0
(4:2:0 sampling)
Signals utilizing the Constant Luminance Y'C C'BC C'RC signal
format of SHALL use the
appropriate one of the following values for the Media Type
Parameter "sampling":
CLYCbCr-4:4:4
(4:4:4 sampling)
CLYCbCr-4:2:2
(4:2:2 sampling)
CLYCbCr-4:2:0
(4:2:0 sampling)
Signals utilizing the constant intensity I CT CP signal format
of SHALL use the appropriate one
of the following values for the Media Type Parameter
"sampling":
ICtCp-4:4:4
(4:4:4 sampling)
ICtCp-4:2:2
(4:2:2 sampling)
ICtCp-4:2:0
(4:2:0 sampling)
Signals utilizing the 4:4:4 R' G' B' or RGB signal format
(such as that of , ,
, ,
, or ) SHALL use the
following value for the Media Type Parameter "sampling":
RGB
(RGB or R' G' B' samples)
Signals utilizing the 4:4:4 X' Y' Z' signal format (such as
defined in ) SHALL use the following value for
the Media Type Parameter "sampling":
XYZ
(X' Y' Z' samples)
Key signals as defined in SHALL use
the value key for the Media Type Parameter "sampling". The key
signal is represented as a single component:
KEY
(Samples of the key signal)
Signals utilizing a color sub-sampling other than what is
defined here SHALL use the following value for
the Media Type Parameter "sampling":
UNSPECIFIED
(Sampling signaled by the payload)
colorimetry:
Specifies the system colorimetry used by the image
samples. Valid values and their specification are the
following:
BT601-5:
.
BT709-2:
.
SMPTE240M:
.
BT601:
.
BT709:
.
BT2020:
.
BT2100:
, Table 2 titled "System colorimetry".
ST2065-1:
Academy Color Encoding Specification (ACES).
ST2065-3:
Academy Density Exchange Encoding (ADX).
XYZ:
, section titled "1931 Observer".
UNSPECIFIED:
Colorimetry is signaled in the payload by the color specification box of
, or it must be manually coordinated between sender
and receiver.
Signals utilizing the colorimetry
SHOULD also signal the representational range using the
optional parameter RANGE defined below. Signals utilizing the
UNSPECIFIED colorimetry might require manual coordination between
the sender and the receiver.
TCS:
Transfer Characteristic System. This parameter specifies
the transfer characteristic system of the image samples.
Valid values and their specification are the following:
SDR:
Standard Dynamic Range video streams that utilize the Optical Electrical
Transfer Function (OETF) of or .
Such streams SHALL be assumed to target the Electro-Optical Transfer
Function (EOTF) specified in .
PQ:
High dynamic range video streams that utilize the Perceptual Quantization system
of .
HLG:
High dynamic range video streams that utilize the Hybrid Log-Gamma system
of .
UNSPECIFIED:
Video streams whose transfer characteristics are signaled by the payload
as specified in , or that must be manually
coordinated between sender and receiver.
RANGE:
This parameter SHOULD be used to signal the encoding
range of the sample values within the stream. When paired with
colorimetry, this parameter has two allowed
values, NARROW and FULL, corresponding to the ranges specified
in TABLE 9 of . In any other context, this
parameter has three allowed values: NARROW, FULLPROTECT, and
FULL, which correspond to the ranges specified in
. In the absence of this parameter, and for all
but the UNSPECIFIED colorimetry, NARROW SHALL be the assumed
value. When paired with the UNSPECIFIED colorimetry, FULL
SHALL be the default assumed value.
Encoding considerations:
This media type is framed in RTP and contains binary data; see
.
Security considerations:
See the Security
Considerations section of RFC 9134.
Interoperability considerations:
None
Published specification:
See the References
section of RFC 9134.
Applications that use this media type:
Any application that transmits video over RTP (like SMPTE ST 2110).
Fragment identifier considerations:
N/A
Additional information:
None
Person & email address to contact for further information:
T. Bruylants <rtp@intopix.com> and T. Richter <jpeg-xs-techsupport@iis.fraunhofer.de>.
Intended usage:
COMMON
Restrictions on usage:
This media type depends on RTP framing; hence, it is only defined for transfer via RTP.
Author:
See the Authors' Addresses section of RFC 9134.
Change controller:
IETF Audio/Video Transport Working Group delegated from the IESG.
SDP Parameters
A mapping of the parameters into the Session Description Protocol (SDP)
is provided for applications that use SDP.
Mapping of Payload Type Parameters to SDP
The media type video/jxsv string is mapped to fields in the Session
Description Protocol (SDP) as follows:
The media type ("video") goes in SDP "m=" as the media name.
The media subtype ("jxsv") goes in SDP "a=rtpmap" as the encoding
name, followed by a slash ("/") and the required parameter "rate"
corresponding to the RTP timestamp clock rate (which for the payload
format defined in this document SHALL be 90000).
The required parameter "packetmode" and any of the additional
optional parameters, as described in , go in the SDP media
format description, being the "a=fmtp" attribute (Format
Parameters), by copying them directly from the media type string
as a semicolon-separated list of parameter=value pairs.
All parameters of the media format SHALL correspond to the
parameters of the payload. In case of discrepancies between
payload parameter values and SDP fields, the values from the
payload data SHALL prevail.
The receiver SHALL ignore any parameter that is not defined in .
An example SDP mapping for JPEG XS video is as follows:
m=video 30000 RTP/AVP 112
a=rtpmap:112 jxsv/90000
a=fmtp:112 packetmode=0;sampling=YCbCr-4:2:2;
width=1920;height=1080;depth=10;
colorimetry=BT709;TCS=SDR;RANGE=FULL;TP=2110TPNL
In this example, a JPEG XS RTP stream is to be sent to UDP destination
port 30000, with an RTP dynamic payload type of 112 and a media clock
rate of 90000 Hz. Note that the "a=fmtp:" line has been wrapped to fit
this page and will be a single long line in the SDP file. This example
includes the TP parameter (as specified in ).
Usage with SDP Offer/Answer Model
When JPEG XS is offered over RTP using SDP in an
offer/answer model for negotiation for unicast usage, the following
limitations and rules apply:
The "a=fmtp" attribute SHALL be present specifying the required
parameter "packetmode" and MAY specify any of the
optional parameters, as described in .
All parameters in the "a=fmtp" attribute indicate sending capabilities (i.e., properties of the payload).
An answerer of the SDP is required to support all parameters and
values of the parameters provided by the offerer; otherwise, the
answerer SHALL reject the session. It falls on the
offerer to use values that are expected to be supported by the
answerer. If the answerer accepts the session, it
SHALL reply with the exact same parameter values
in the "a=fmtp" attribute as they were initially offered.
The same RTP payload type number used in the offer
SHOULD be used in the answer, as specified in
.
IANA Considerations
IANA has registered the media type registration "video/jxsv"
as specified in . The
media type has also been added to the IANA registry for "RTP
Payload Format Media Types" .
Security Considerations
RTP packets using the payload format defined in this memo
are subject to the security considerations discussed in
and in any applicable RTP profile such as RTP/AVP,
RTP/AVPF,
RTP/SAVP, or
RTP/SAVPF. This implies
that confidentiality of the media streams is achieved by encryption.
However, as "Securing the RTP Framework: Why RTP
Does Not Mandate a Single Media Security Solution"
discusses, it is not an RTP payload format's responsibility to
discuss or mandate what solutions are used to meet the basic security
goals like confidentiality, integrity, and source authenticity for
RTP in general. This responsibility lies on anyone using RTP in an
application. They can find guidance on available security mechanisms
and important considerations in
"Options for Securing RTP Sessions".
Applications SHOULD use one or more appropriate strong
security mechanisms.
Implementations of this RTP payload format need to take appropriate
security considerations into account. It is important for the decoder
to be robust against malicious or malformed payloads and ensure that
they do not cause the decoder to overrun its allocated memory or otherwise
misbehave. An overrun in allocated memory could lead to arbitrary code
execution by an attacker. The same applies to the encoder, even though
problems in encoders are typically rarer.
This payload format and the JPEG XS encoding do not exhibit any
substantial non-uniformity, either in output or in complexity to perform
the decoding operation; thus, they are unlikely to pose a denial-of-service
threat due to the receipt of pathological datagrams.
This payload format and the JPEG XS encoding do not contain code that is executable.
It is important to note that high-definition (HD) or ultra-high-definition
(UHD) video that is encoded with JPEG XS can have significant bandwidth
requirements (typically more than 1 Gbps for UHD video, especially if
using high framerate). This is sufficient to cause potential for denial
of service if transmitted onto most currently available Internet paths.
Accordingly, if best-effort service is being used, users of this
payload format SHALL monitor packet loss to ensure that the packet
loss rate is within acceptable parameters. Packet loss is considered
acceptable if a TCP flow across the same network path, and
experiencing the same network conditions, would achieve an average
throughput, measured on a reasonable timescale, that is not less than
the RTP flow is achieving. This condition can be satisfied by
implementing congestion control mechanisms to adapt the transmission
rate (or the number of layers subscribed for a layered multicast
session) or by arranging for a receiver to leave the session if the
loss rate is unacceptably high.
This payload format may also be used in networks that provide
quality-of-service guarantees. If enhanced service is being used,
receivers SHOULD monitor packet loss to ensure that the service that
was requested is actually being delivered. If it is not, then they
SHOULD assume that they are receiving best-effort service and behave
accordingly.
ReferencesNormative ReferencesInformation technology - JPEG XS low-latency lightweight image coding system - Part 1: Core coding systemISO/IECInformation technology - JPEG XS low-latency lightweight image coding system - Part 2: Profiles and buffer modelsISO/IECInformation technology - JPEG XS low-latency lightweight image coding system - Part 3: Transport and container formatsISO/IECKey words for use in RFCs to Indicate Requirement LevelsIn many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements.An Offer/Answer Model with Session Description Protocol (SDP)This document defines a mechanism by which two entities can make use of the Session Description Protocol (SDP) to arrive at a common view of a multimedia session between them. In the model, one participant offers the other a description of the desired session from their perspective, and the other participant answers with the desired session from their perspective. This offer/answer model is most useful in unicast sessions where information from both participants is needed for the complete view of the session. The offer/answer model is used by protocols like the Session Initiation Protocol (SIP). [STANDARDS-TRACK]RTP: A Transport Protocol for Real-Time ApplicationsThis memorandum describes RTP, the real-time transport protocol. RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. RTP does not address resource reservation and does not guarantee quality-of- service for real-time services. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks, and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers. The protocol supports the use of RTP-level translators and mixers. Most of the text in this memorandum is identical to RFC 1889 which it obsoletes. There are no changes in the packet formats on the wire, only changes to the rules and algorithms governing how the protocol is used. The biggest change is an enhancement to the scalable timer algorithm for calculating when to send RTCP packets in order to minimize transmission in excess of the intended rate when many participants join a session simultaneously. [STANDARDS-TRACK]RTP Profile for Audio and Video Conferences with Minimal ControlThis document describes a profile called "RTP/AVP" for the use of the real-time transport protocol (RTP), version 2, and the associated control protocol, RTCP, within audio and video multiparticipant conferences with minimal control. It provides interpretations of generic fields within the RTP specification suitable for audio and video conferences. In particular, this document defines a set of default mappings from payload type numbers to encodings. This document also describes how audio and video data may be carried within RTP. It defines a set of standard encodings and their names when used within RTP. The descriptions provide pointers to reference implementations and the detailed standards. This document is meant as an aid for implementors of audio, video and other real-time multimedia applications. This memorandum obsoletes RFC 1890. It is mostly backwards-compatible except for functions removed because two interoperable implementations were not found. The additions to RFC 1890 codify existing practice in the use of payload formats under this profile and include new payload formats defined since RFC 1890 was published. [STANDARDS-TRACK]Media Type Registration of RTP Payload FormatsThis document specifies the procedure to register RTP payload formats as audio, video, or other media subtype names. This is useful in a text-based format description or control protocol to identify the type of an RTP transmission. [STANDARDS-TRACK]Media Type Specifications and Registration ProceduresThis document defines procedures for the specification and registration of media types for use in HTTP, MIME, and other Internet protocols. This memo documents an Internet Best Current Practice.Multimedia Congestion Control: Circuit Breakers for Unicast RTP SessionsThe Real-time Transport Protocol (RTP) is widely used in telephony, video conferencing, and telepresence applications. Such applications are often run on best-effort UDP/IP networks. If congestion control is not implemented in these applications, then network congestion can lead to uncontrolled packet loss and a resulting deterioration of the user's multimedia experience. The congestion control algorithm acts as a safety measure by stopping RTP flows from using excessive resources and protecting the network from overload. At the time of this writing, however, while there are several proprietary solutions, there is no standard algorithm for congestion control of interactive RTP flows.This document does not propose a congestion control algorithm. It instead defines a minimal set of RTP circuit breakers: conditions under which an RTP sender needs to stop transmitting media data to protect the network from excessive congestion. It is expected that, in the absence of long-lived excessive congestion, RTP applications running on best-effort IP networks will be able to operate without triggering these circuit breakers. To avoid triggering the RTP circuit breaker, any Standards Track congestion control algorithms defined for RTP will need to operate within the envelope set by these RTP circuit breaker algorithms.UDP Usage GuidelinesThe User Datagram Protocol (UDP) provides a minimal message-passing transport that has no inherent congestion control mechanisms. This document provides guidelines on the use of UDP for the designers of applications, tunnels, and other protocols that use UDP. Congestion control guidelines are a primary focus, but the document also provides guidance on other topics, including message sizes, reliability, checksums, middlebox traversal, the use of Explicit Congestion Notification (ECN), Differentiated Services Code Points (DSCPs), and ports.Because congestion control is critical to the stable operation of the Internet, applications and other protocols that choose to use UDP as an Internet transport must employ mechanisms to prevent congestion collapse and to establish some degree of fairness with concurrent traffic. They may also need to implement additional mechanisms, depending on how they use UDP.Some guidance is also applicable to the design of other protocols (e.g., protocols layered directly on IP or via IP-based tunnels), especially when these protocols do not themselves provide congestion control.This document obsoletes RFC 5405 and adds guidelines for multicast UDP usage.Ambiguity of Uppercase vs Lowercase in RFC 2119 Key WordsRFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings.SDP: Session Description ProtocolThis memo defines the Session Description Protocol (SDP). SDP is intended for describing multimedia sessions for the purposes of session announcement, session invitation, and other forms of multimedia session initiation. This document obsoletes RFC 4566.Informative ReferencesReference electro-optical transfer function for flat panel displays used in HDTV studio productionITU-RParameter values for ultra-high definition television systems for production and international programme exchangeITU-RImage parameter values for high dynamic range television for use in production and international programme exchangeITU-RStudio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratiosITU-RStudio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratiosITU-RParameter values for the HDTV standards for production and international programme exchangeITU-RParameter values for the HDTV standards for production and international programme exchangeITU-RColorimetry - Part 1: CIE standard colorimetric observersISO/CIEThe Secure Real-time Transport Protocol (SRTP)This document describes the Secure Real-time Transport Protocol (SRTP), a profile of the Real-time Transport Protocol (RTP), which can provide confidentiality, message authentication, and replay protection to the RTP traffic and to the control traffic for RTP, the Real-time Transport Control Protocol (RTCP). [STANDARDS-TRACK]RTP Payload Format for Uncompressed VideoThis memo specifies a packetization scheme for encapsulating uncompressed video into a payload format for the Real-time Transport Protocol, RTP. It supports a range of standard- and high-definition video formats, including common television formats such as ITU BT.601, and standards from the Society of Motion Picture and Television Engineers (SMPTE), such as SMPTE 274M and SMPTE 296M. The format is designed to be applicable and extensible to new video formats as they are developed. [STANDARDS-TRACK]Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)Real-time media streams that use RTP are, to some degree, resilient against packet losses. Receivers may use the base mechanisms of the Real-time Transport Control Protocol (RTCP) to report packet reception statistics and thus allow a sender to adapt its transmission behavior in the mid-term. This is the sole means for feedback and feedback-based error repair (besides a few codec-specific mechanisms). This document defines an extension to the Audio-visual Profile (AVP) that enables receivers to provide, statistically, more immediate feedback to the senders and thus allows for short-term adaptation and efficient feedback-based repair mechanisms to be implemented. This early feedback profile (AVPF) maintains the AVP bandwidth constraints for RTCP and preserves scalability to large groups. [STANDARDS-TRACK]Extended Secure RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/SAVPF)An RTP profile (SAVP) for secure real-time communications and another profile (AVPF) to provide timely feedback from the receivers to a sender are defined in RFC 3711 and RFC 4585, respectively. This memo specifies the combination of both profiles to enable secure RTP communications with feedback. [STANDARDS-TRACK]Options for Securing RTP SessionsThe Real-time Transport Protocol (RTP) is used in a large number of different application domains and environments. This heterogeneity implies that different security mechanisms are needed to provide services such as confidentiality, integrity, and source authentication of RTP and RTP Control Protocol (RTCP) packets suitable for the various environments. The range of solutions makes it difficult for RTP-based application developers to pick the most suitable mechanism. This document provides an overview of a number of security solutions for RTP and gives guidance for developers on how to choose the appropriate security mechanism.Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Security SolutionThis memo discusses the problem of securing real-time multimedia sessions. It also explains why the Real-time Transport Protocol (RTP) and the associated RTP Control Protocol (RTCP) do not mandate a single media security mechanism. This is relevant for designers and reviewers of future RTP extensions to ensure that appropriate security mechanisms are mandated and that any such mechanisms are specified in a manner that conforms with the RTP architecture.RTP Control Protocol (RTCP) Feedback for Congestion ControlAn effective RTP congestion control algorithm requires more fine-grained feedback on packet loss, timing, and Explicit Congestion Notification (ECN) marks than is provided by the standard RTP Control Protocol (RTCP) Sender Report (SR) and Receiver Report (RR) packets. This document describes an RTCP feedback message intended to enable congestion control for interactive real-time traffic using RTP. The feedback message is designed for use with a sender-based congestion control algorithm, in which the receiver of an RTP flow sends back to the sender RTCP feedback packets containing the information the sender needs to perform congestion control.SMPTE Recommended Practice - Key and Alpha SignalsSMPTESMPTE Standard - Academy Color Encoding Specification (ACES)SMPTESMPTE Standard - Academy Density Exchange Encoding (ADX) - Encoding Academy Printing Density (APD) ValuesSMPTESMPTE Recommended Practice - Full-Range Image MappingSMPTESMPTE Standard - Professional Media Over Managed IP Networks: Traffic Shaping and Delivery Timing for VideoSMPTESMPTE Standard - For Television - 1125-Line High-Definition Production Systems - Signal ParametersSMPTESMPTE Standard - D-Cinema Distribution Master - Image CharacteristicsSMPTEAcknowledgments
The authors would like to thank the following people for their valuable
contributions to this memo: ,
, , , , and .
Authors' AddressesintoPIX S.A.Rue Emile Francqui, 9Mont-Saint-Guibert1435Belgium+32 10 23 84 70t.bruylants@intopix.comhttps://www.intopix.com/Université catholique de Louvainbte L2.03.02Ruelle de la Lanterne Magique, 14Louvain-la-Neuve1348Belgium+32 10 47 27 87antonin.descampe@uclouvain.behttps://uclouvain.be/antonin.descampeintoPIX S.A.Rue Emile Francqui, 9Mont-Saint-Guibert1435Belgium+32 10 23 84 70c.damman@intopix.comhttps://www.intopix.com/Fraunhofer IISAm Wolfsmantel 33Erlangen91048Germany+49 9131 776 5126thomas.richter@iis.fraunhofer.dehttps://www.iis.fraunhofer.de/