DetNet D. Guo Internet Draft S. Xu Intended status: Informational New H3C Technologies Co., Ltd Expires: December 9, 2023 June 9, 2023 Jitter Reduction Mechanism for DetNet draft-guo-detnet-jitter-reduction-mechanism-00 Abstract In large-scale deterministic networks (LDN), App-flows need to span multiple deterministic network domains, and the latency in multiple domains is added together. The jitter will be increased. In order to realize the protection service function, App-flows should be transmitted on multiple paths. The delay difference in data transmission on different paths is no different from jitter in end- to-end services. Jitter generated by various factors needs to be controlled to meet business requirements. This document describes the end-to-end jitter reduction mechanism in an LDN. This mechanism can effectively control the end-to-end jitter to meet specific business needs and make the planning of multiple paths for service protection more flexible. Status of this Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may not be modified, and derivative works of it may not be created, and it may not be published except as an Internet-Draft. This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. This document may not be modified, and derivative works of it may not be created, except to publish it as an RFC and to translate it into languages other than English. This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow Guo, et al. Expires December 9, 2023 [Page 1] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html This Internet-Draft will expire on December 9, 2023. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License. Table of Contents 1. Introduction...................................................3 2. Terminology....................................................4 3. Architecture...................................................6 Guo, et al. Expires December 9, 2023 [Page 2] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 4. Proposal Description...........................................7 4.1. Data-Plane Overview......................................10 4.2. Control-Plane Overview...................................12 5. Security Considerations.......................................13 6. IANA Considerations...........................................13 7. Acknowledgments...............................................13 8. Contributors..................................................13 9. References....................................................14 9.1. Normative References.....................................14 Authors' Addresses...............................................16 1. Introduction /----\ / \ | DetNet3 | \ / / \--------/ \ /-------\/ \/-------\ / \ / \ | DetNet2| | DetNet4 | \ / \ / \-----/ \-----/ | | ---|--- --|--- / \ / \ | DetNet1 | | DetNet5 | \ / \ / \--|--/ \--|--/ | | Talker Listener Figure 1 Multiple domains In deterministic networks, as stated in [I-D.ietf-detnet-scaling- requirements], end-to-end service may across multiple network domains and adopt a variety of different queuing mechanisms within each domain. For end-to-end services spanning multiple domains, jitter exists in various factors: o Scheduling and traffic admission control at domain boundaries may cause jitter; Guo, et al. Expires December 9, 2023 [Page 3] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 o The jitter generated by queuing and forwarding mechanisms in the DetNet domain, such as the [IEEE 802.1 QCR] asynchronous shaping method; o Flow aggregation generates jitter. In [RFC 8938], the DetNet data plane also allows for the aggregation of DetNet flows, which can improve scalability by reducing the per-hop state. When the aggregated flows are scheduled, the jitter of the flows cannot be precisely controlled. At the same time, according to [I-D.ietf-detnet-scaling- requirements], large-scale deterministic networks should support cross-domain asynchronous clocks; in addition, multiple domains may be heterogeneous networks (such as TSN, DetNet IP and 5GS). The factors all determine that it is difficult to reduce jitter between domains through a mechanism similar to CSQF[I-D.chen-detnet-sr- based-bounded-latency] or TCQF[I-D.draft-eckert-detnet-tcqf]. While the jitter generated by various factors in the end-to-end transmission path is accumulated, it may not meet the applications' requirements on jitter. This document describes a jitter reduction mechanism to eliminate jitter introduced by multiple factors in large-scale deterministic networks. 2. Terminology The following terminology is introduced in this document: Actual Delay (ActD): The actual transmission delay of a deterministic data packet passing through a specified network domain is called the Actual Time. Reference Delay (RefD): The maximum delay for packets of DetNet flows to pass through the DetNet domain. Fixed delay (FixD): The approximately constant part of the end-to- end transmission delay of the data packets of the App-flows through the DetNet. Path Reference Delay (PthRefD): The maximum end-to-end transmission delay of data packets of App-flows through the DetNet. Path Actual Delay (PthActD): The end-to-end actual transmission delay of a certain packet of App-flows through the DetNet. Guo, et al. Expires December 9, 2023 [Page 4] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 Compensation Delay (CompD): The delay required to compensate the actual delay according to the reference delay. Clock Source: Used as a clock source for subnet time synchronization. I-Gw: The ingress gateway of the deterministic subnet. E-Gw: The egress gateway of the deterministic subnet. HEAD NODE: The I-Gw of the first DetNet domain accessed by the App- flows in the DetNet forwarding path is HEAD NODE. COMPENSATION NODE: This node performs calculation and compensation for time delay. INGRESS-RELAY NODE: The I-Gw gateway of each domain except the HEAD NODE. EGRESS-RELAY NODE: The E-Gw gateway of each domain except the COMPENSATION NODE. Virtual Clock Reference Plane (VCRP): Provides a frequency synchronization reference for the clock used for DetNet data plane [DDP] timing. Guo, et al. Expires December 9, 2023 [Page 5] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 3. Architecture *------------------------------------------------------------* / / / / / Virtual Clock Reference Plane / / (VCRP) / / / *------+----------+----------------------+------------+------* | | | | | +----+----+ +----+----+ | | | | | | | | | Clock | | Clock | | | | Source2 | | Source3 | | | | | | | | | +---------+ +---------+ | | / \ / \ | | / \ / \ | | +-------+ +-------+ +-------+ +-------+ | | | I-Gw2 +--+ E-Gw2 +---+ I-Gw3 +--+ E-Gw3 | | | +---+---+ +-------+ +-------+ +---+---+ | | | | | | | | | | +-------+ +------+ | | | | | +---+-----+ | | +----+----+ | | | | | | | Clock | | | | Clock | | Source1 | | | | Source4 | | | | | | | +---------+ | | +---------+ / \ | | / \ / \ | | / \ +-------+ +-------+ | | +-------+ +-------+ | I-Gw1 +--+ E-Gw1 +--+ +----+ I-Gw4 +--+ E-Gw4 | +---+---+ +-------+ +-------+ +---+---+ | | | | Talker Listener Figure 2 Architecture Guo, et al. Expires December 9, 2023 [Page 6] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 In Figure 2, the Clock Source is a part of VCRP. I-Gw1 is HEAD NODE. E-Gw1, E-Gw2, E-Gw3 are EGRESS-RELAY NODE. I-Gw2, I-Gw3 are INGRESS- RELAY NODE. E-Gw4 is COMPENSATION NODE. The I-Gw gateway of each domain except the HEAD NODE should extract the packet receiving time, and calculate the actual delay of the previous domain, and carry it into the packet. The E-Gw gateway of each domain except the COMPENSATION NODE should save the sending time information of the domain. It is difficult to improve the accuracy of time synchronization, because VCRP may have a large geographical span. Clock source of VCRP provides time synchronization for each DetNet domain. Time synchronization is required within each DetNet domain, not between DetNet domains. Each domain is frequency-synchronized with the clock source provided by VCRP to avoid excessive deviation caused by each domain due to the influence of the environment. Because the clock sources that provide synchronization references for each DetNet domain in VCRP may not physically connected, it is difficult to achieve time synchronization in VCRP. On the premise that the transmission delay in each domain is not large, the frequency accuracy of the clock used for timing is relatively low. It is relatively easy for VCRP to provide a stable clock source with a certain accuracy for each DetNet domain for time synchronization. 4. Proposal Description Basic idea of the scheme: when establishing a deterministic stream session, obtain the reference delay of the path, obtain the actual delay of the data packet during transmission, calculate the compensation delay according to the reference delay and the actual transmission delay, and compensate the transmission delay at the COMPENSATION NODE connected to the Listener. The implementation principle is described in detail below. Guo, et al. Expires December 9, 2023 [Page 7] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 | | | | +----------T2------------+ +----------T3------------+ | | | | +----------T'2-----------+ +----------T'3-----------+ | | | | | | | | +--------+--------+------+ +--------+--------+------+ | I-Gw2 | DetNet2| E-Gw2|----+ I-Gw2 | DetNet2| E-Gw2+-+ +----+---+--------+------+ +--------+--------+------+ | | | | | | | +--------------------+ +---------------------------+ | | | | | || || | +----------T1------------+| |+----------T4------------+ | || || | +----------T'2-----------+| |+----------T'4-----------+ | || || | | || || | +--------+--------+------+| |+--------+--------+------+ | I-Gw2 | DetNet2| E-Gw2++ ++ I-Gw4 | DetNet4| E-Gw4| +----+---+--------+------+ +--------+--------+---+--+ | | | | | | Talker Listener Figure 3 The timing model of transmission delay Figure 3 describes the abstract model of multi-domain transmission delay segmentation timing, where Tn (n=1~4 in Figure 3) is the reference delay RefDn of each domain, and this value is obtained according to the selection of queuing mechanism combined with engineering applications. For a specific deterministic path, there is a master clock in the same domain, and each node in the path will perform time synchronization with this clock, and finally obtain intra-domain time synchronization. The reference delay of the end-to-end path is: PthRefD = FixD + T1 + T2 + T3 + T4. Guo, et al. Expires December 9, 2023 [Page 8] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 If service protection is realized by sending copies of the same data packet through multiple paths, the reference delay of path i is: PthRefD_i = FixD_i + T1_i + T2_i + T3_i + T4_i. Select the reference delay of the path with the largest one as the common reference delay of all the paths, and the final reference delay is: PthRefD = ceil(PthRefD1, PthRefD2, ..., PthRefDn). Let Tn (n=1~4 in Figure 3) be the actual transmission delay ActD of a certain data packet in each domain, and it depends on the time entering the domain and the time of exiting the domain when the data packet is actually delivered. The residence time in domain can also be obtained after the data packet is transmitted. Therefore, the actual transmission delay of this packet is: PthActD = FixD + T1 + T2 + T3 + T4. The delay of the packet needs to be compensated at CompD: CompD = PthRefD - PthActD = (PthRefD - FixD) - (T1 + T2 + T3+ T4). If service protection is achieved by sending copies of the same data packet through multiple paths, the actual delay of path i is PthActD_i = FixD_i + T1_i + T2_i + T3_i + T4_i. The delay of the packet needs to be compensated after transmission in path i is CompD: CompD = (PthRefD - FixD_i) - (T1_i + T2_i + T3_i + T4_i) The formula can be simplified as: CompD = Cap - (T1 + T2 + T3 + T4) (Formula 1), or CompD = Cap_i - (T1_i + T2_i + T3_i + T4_i) (Formula 2), where (PthRefD - FixD) is Cap, and (PthRefD - FixD_i) is Cap_i. Cap or Cap_i can be obtained directly or indirectly before deploying a Guo, et al. Expires December 9, 2023 [Page 9] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 deterministic streaming session. The specific obtaining method will be added in subsequent updates. Compensation is performed on E-Gw4 according to Formula 1 or Formula 2 when the DetNet data packet is transmitted. 4.1. Data-Plane Overview In the data plane, it is necessary to obtain the reference delay from the HEAD NODE to the COMPENSATION NODE. During actual transmission, collect the actual transmission delay in each domain, and compensate the delay at the COMPENSATION NODE based on the reference delay. For delay collection, two methods provide the relevant information required by each gateway: Method 1: Specify the operation of the subsequent gateway in the HEAD NODE. After the HEAD NODE identifies the flow, it encapsulates the relevant information into a data packet, and the subsequent gateway node performs corresponding operations according to the information in the data packet. The advantage of this method is that it simplifies the subsequent gateway configuration, but the disadvantage is that the overhead is large. Method 2: Configure the relevant information in the edge gateway along the path, and the edge gateway node will identify the DetNet flow and then obtain the information from the configuration for corresponding operations. The advantage of this method is that the overhead is small, and the disadvantage is that subsequent gateways need flow-by-flow configuration. The specific operation will be supplemented in subsequent updates. Guo, et al. Expires December 9, 2023 [Page 10] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 +-------------+ +-------------+ +-------------+ | HEAD +----+EGRESS RELAY +----+INGRESS RELAY+-+ +-------------+ +-------------+ +-------------+ | | +-------------------------------------------------------+ | | +-------------+ +-------------+ +-------------+ +-+EGRESS RELAY +----|INGRESS RELAY+----+COMPENSATION | +-------------+ +-------------+ +-------------+ Figure 4 Data Flow The process required for the different roles of the gateway that DetNet flows pass through is described as follows. When receiving a packet, the HEAD NODE performs the following process: 1. Identify the DetNet flow and obtain the cross-domain information provided by the control plane. This cross-domain information includes the actions of the subsequent gateways, which can be encapsulated in the packet or configured by the control plane to the subsequent gateways. 2. Obtain the time when this node receives the packet. 3. Fill the cross-domain information and the time that this node receives the packet into the packet. When receiving a packet, the INGRESS-RELAY NODE performs the following process: 1. Identify the DetNet flow and obtain the cross-domain information provided by the control plane from the packet or from the configuration. The gateway takes actions according to the information. The main procedure is: a) Obtain the time when this node receives the packet. b) Calculate the residence delay of the previous domain. Guo, et al. Expires December 9, 2023 [Page 11] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 2. Fill the residence delay of the previous domain and the time when this node receives the packet into the packet. When receiving a packet, the EGRESS-RELAY NODE performs the following process: 1. Identify the DetNet flow, obtain the cross-domain information provided by the control plane from the packet, and obtain the required actions from the configuration. The main procedure is: a) Add the time when the packet is sent to the specified position. When receiving a packet, the COMPENSATION NODE performs the following process: 1. Identify the DetNet flow, obtain the cross-domain information provided by the control plane from the packet, and obtain the required actions from the configuration. The main procedure is: a) Obtain the time when this node receives the packet and calculate the residence delay of this domain. b) Accumulate the residence delays of each domain. c) Calculate the compensation delay. 2. Send out the packet after completing the compensation. 4.2. Control-Plane Overview The control plane needs to cooperate with the data plane to complete the following transactions: 1. Define gateway operations along the path and configure data plane gateways. 2. Cooperate with the data plane to complete the measurement and calculate the reference delay. 3. Configure the reference delay information to the HEAD NODE or COMPENSATION NODE. Guo, et al. Expires December 9, 2023 [Page 12] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 4. Maintenance during runtime: periodically collect the reference delay of each domain and calculate the reference delay of the whole path, and refresh the reference delay to the HEAD NODE or COMPENSATION NODE. For delay collection, the control plane has two methods to cooperate with the data plane to supply the relevant information required by each gateway: Method 1: The control plane globally distributes the gateway ID, and configures the ID to each edge gateway. The control plane configures the collected delay information to the HEAD NODE. Method 2: The control plane configures flow-by-flow operations to the domain edge gateways along the path. 5. Security Considerations TBD 6. IANA Considerations TBD 7. Acknowledgments The authors express their appreciation and gratitude to Min Liu, Lei Zhou for the review and helpful comments. 8. Contributors The editor wishes to thank and acknowledge the following contributors for contributing text to this document. Rubing Liu New H3C Technologies Co., Ltd 100094 Email: liurubing@h3c.com Guo, et al. Expires December 9, 2023 [Page 13] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 Ning Pan New H3C Technologies Co., Ltd 100094 Email: panning@h3c.com Xusheng Chen New H3C Technologies Co., Ltd 100094 Email: cxs@h3c.com Wei Wang New H3C Technologies Co., Ltd 100094 Email: david_wang@h3c.com Steven Yoe New H3C Technologies Co., Ltd 100094 Email: yoe@h3c.com 9. References 9.1. Normative References [RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas, "Deterministic Networking Architecture", RFC 8655, DOI 10.17487/RFC8655, October 2019, . [I-D.ietf-detnet-scaling-requirements] Liu, P., Li, Y., Eckert, T., Xiong, Q., and J. Ryoo, "Requirements for Large-Scale Deterministic Networks", draft-liu-detnet-large-scale- requirements-05 (work in progress), October 2022. [IEEE802.1Qcr] IEEE, "IEEE Standard for Local and Metropolitan Area Networks -- Bridges and Bridged Networks - Amendment 34: Asynchronous Traffic Shaping", IEEE 802.1Qcr-2020, DOI 10.1109/IEEESTD.2020.9253013, 6 November 2020, . Guo, et al. Expires December 9, 2023 [Page 14] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 [I-D.chen-detnet-sr-based-bounded-latency] Chen, M., Geng, X., and Z. Li, "Segment Routing (SR) Based Bounded Latency", Work in Progress, Internet-Draft, draft-chen-detnet-sr-based- bounded-latency, 3 March 2023, [I-D.eckert-detnet-tcqf] Eckert, T. , Bryant, S., and A. G. Malis, "Deterministic Networking (DetNet) Data Plane - Tagged Cyclic Queuing and Forwarding (TCQF) for bounded latency with low jitter in large scale DetNets" [IEEE802.1AS] IEEE Time-Sensitive Networking (TSN) Task Group., "IEEE Std 802.1AS-2020: IEEE Standard for Local and Metropolitan Area Networks - Timing and Synchronization for Time Sensitive Applications ", 2020. [I-D.ietf-detnet-mpls-over-ip-preof] Varga, B., Farkas, J., Malis, A., "Deterministic Networking(DetNet): DetNet PREOF via MPLS over UDP/IP", Work in Progress, Internet-Draft, draft-ietf-detnet-mpls-over-ip-preof-02, 6 November 2022, < https://www.ietf.org/archive/id/draft-ietf-detnet-mpls- over-ip-preof-02.txt>. [IEEE802.1Qci] IEEE Time-Sensitive Networking (TSN) Task Group., "IEEE Std 802.1Qci-2017: IEEE Standard for Local and Metropolitan Area Networks - Bridges and Bridged Networks- Amendment 28: Per-Stream Filtering and Policing", 2017. [I-D.ietf-detnet-controller-plane-framework] Malis, A., Geng, X., Chen, M., Qin, F., arga, B., "Deterministic Networking (DetNet) Controller Plane Framework" , Work in Progress, Internet-Draft, draft-ietf-detnet-controller-plane- framework-02, 29 June 2022, . Guo, et al. Expires December 9, 2023 [Page 15] Internet-Draft Jitter Reduction Mechanism for DetNet June 2023 Authors' Addresses Daorong Guo New H3C Technologies Co., Ltd Beijing 100094 China Email: guodaorong@h3c.com Shenchao Xu New H3C Technologies Co., Ltd Hangzhou 310052 China Email: xushenchao@h3c.com Guo, et al. Expires December 9, 2023 [Page 16]