TODO Z. Zheng Internet-Draft Y. Ma Intended status: Experimental Y. Liu Expires: 6 November 2023 Alibaba Inc. M. Kühlewind Ericsson 5 May 2023 QUIC-enabled Service Differentiation for Traffic Engineering draft-zmlk-quic-te-00 Abstract This document defines a method for supporting QUIC-enabled service differentiation for traffic engineering through multipath and QUIC connection identifier (CID) encoding. This approach enables end-host networking stacks and applications to select packet routing paths in a wide area network (WAN), potentially improving the end-to-end performance, cost, and reliability. The proposed method can be used in conjunction with segment routing traffic engineering technologies, such as SRv6 TE. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." This Internet-Draft will expire on 6 November 2023. Copyright Notice Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Zheng, et al. Expires 6 November 2023 [Page 1] Internet-Draft quic-te May 2023 Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1. Conventions and Definitions . . . . . . . . . . . . . . . 3 2. Alternatives to support service differentiation in traffic engineering . . . . . . . . . . . . . . . . . . . . . . . 4 3. Using Multipath QUIC . . . . . . . . . . . . . . . . . . . . 4 3.1. New Transport Parameter . . . . . . . . . . . . . . . . . 4 3.2. QUIC Connection Setup . . . . . . . . . . . . . . . . . . 5 3.3. Paths Setup with More Priorities . . . . . . . . . . . . 5 3.4. Effect of CID rotation . . . . . . . . . . . . . . . . . 6 4. CID Generation with Given Priorities . . . . . . . . . . . . 6 5. CID Parsing in Cloud WAN . . . . . . . . . . . . . . . . . . 7 6. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 7 6.1. Packet Flow from Server to Client . . . . . . . . . . . . 7 6.2. Deployment of CID Parser . . . . . . . . . . . . . . . . 9 7. Security Considerations . . . . . . . . . . . . . . . . . . . 9 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 9. Normative References . . . . . . . . . . . . . . . . . . . . 10 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 11 1. Introduction This draft outlines a method that enables a WAN to manage packets from a single QUIC connection with differentiated traffic engineering policies at packet-level granularity. For instance, a WAN can offer two priority routing paths for packets from the same QUIC connection: (1) it directs first-time transmitted packets on a default-priority, low-cost path, and (2) it routes retransmitted packets and handshake packets on a high-priority (low-latency) but more expensive path. Consequently, this approach achieves cost-efficiency and reduces tail latency simultaneously. While there are existing solutions to support Differentiated Service (DiffServ), such as using Differentiated Services Code Point (DSCP) or opening multiple ports to enable QoS control, they have several limitations. For example, DSCP values may be modified by an ISP or a network service provider, and opening multiple ports rasies security concerns (more details as discussed in Section 2). Zheng, et al. Expires 6 November 2023 [Page 2] Internet-Draft quic-te May 2023 The solution proposed in this draft can overcome these limitations, as it is designed with the following properties: (1) The serivce differentiation encoding is immune to modifications by middleboxes. (2) It provides traffic engineering control at packet-level granularity within a connection. (3) A user-space application can directly control the priority of each packet without special support from the kernel. (4) It does not change the deployment method, as current QUIC applications have. In a nutshell, this proposal establishes multiple routing paths at different priority levels inside a single QUIC connection and encodes a packet's routing priority in its CID. Specifically, this proposal reuses mechanisms from Multipath QUIC [Multipath-QUIC], but makes two major changes: (1) Multiple QUIC paths are allowed to be created between a single-homed client and sever (e.g., the client and server can both have only one (IP, port) address). (2) This proposal leverages different CIDs to describe various priority levels for QUIC packets that belong to the same connection. CID is unencrypted (or weakly encrypted via symmetric encryption [QUIC-LB]), allowing it to be parsed by a network node (e.g., an edge gateway). In this draft, QUIC paths can share the same perceived 4-tuple, but as long as their underlying phyiscal routing paths are not the same, they are treated as different paths. This proposal shares many key similarities to Multipath QUIC: (1) Senders manage per-path congestion status; (2) RTT measurements are per-path. (3) Each path uses a separate packet number space (PNS); (4) A path ID is the sequence number of the CID used to send packets. (5) Once multipath is negotiated, ACK_MP frame is used to acknowledge packets in multiple packet number spaces. The proposed solution is intended to work with edge routers that have segment routing capabilities, such as SRv6 [RFC8986]. Note that the QUIC CID is not intended to be directly used for routing in Cloud WAN, but SHOULD be tranlated to SRv6 SID or an MPLS label by an edge router for traffic engineering purposes. 1.1. Conventions and Definitions The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. This document uses the following terms: Zheng, et al. Expires 6 November 2023 [Page 3] Internet-Draft quic-te May 2023 * Cloud WAN: The backbone network provided and controlled by a Cloud provider, or the backbone network operated by an application service provider. * Edge router: Edge router of a backbone network. * QUIC path: The path described in Multipath QUIC protocol, which is a logically end-to-end transmission path created by the client. * Routing path: The network path that a packet physically traverses, also specified as the Cloud WAN path in this document. 2. Alternatives to support service differentiation in traffic engineering This section describes existing solutions that offer service differentiation in traffic engineering and their limitations when applied to the internet. The most commonly used solution is to use the DSCP packet marking [RFC8837] for traffic engineering. However, such a solution faces several limitations: (1) A DSCP marker is subject to modifications by an ISP or a network service provider. (2) Marking DSCP on mobile devices may require kernel changes. (3) DSCP is marked at the granularity of connection or session, which means packets with the same 4-tuple are treated with the same priority, and hence, it cannot support packet-level QoS control. The second solution is to use multiple ports, either source or destination ports, to differentiate QoS between two endpoints. When this solution is used, packets with different priorities are sent to sockets bound to different ports. However, such a solution also faces several drawbacks. Source ports may be modified by a NAT in the network, which changes the priority. Using destination ports to support QoS raises security concerns, as opening many ports increases the potential attack surface for cyber threats. Desination ports other than some well-known ports (e.g., 80, 433) may also be subject to blocking by firewalls. 3. Using Multipath QUIC 3.1. New Transport Parameter This draft defines a new transport parameter to claim the adoption of priority paths. It is negotiated during QUIC handshake as described in [RFC9000]. The new parameter is defined as PRIORITY_PATHS. This parameter is used to indicate the number of priority levels: Zheng, et al. Expires 6 November 2023 [Page 4] Internet-Draft quic-te May 2023 * 0, 1: Only one path is available, i.e., multiple priority levels are not considered. * n (n > 1): n paths are available, i.e., n priority levels are available for use. When the client supports PRIORITY_PATHS, it sends a value greater than 1 to the server side. After receiving it, the server returns the configured value of PRIORITY_PATHS to the client. The PRIORITY_PATHS value that a client uses MUST be consistent with the returned value from server, i.e., the server determines how many of priority paths can be used. The server and CID parser share the same PRIORITY_PATHS values, which are configured by the control plane. After negotiation, the client obtains the same value as the server. Therefore, for the same CID, the client/CID parser/server can calculate the same priority value (see Section 4). 3.2. QUIC Connection Setup When a QUIC client starts a new connection, it uses a CID generated using the default priority (defined as 0). If PRIORITY_PATHS is supported, a server SHOULD respond with a CID that represents the default priority and indicate to the client that it supports PRIORITY_PATHS. 3.3. Paths Setup with More Priorities After QUIC connection negotiation is completed, endpoints need to exchange CIDs with each other. These CIDs are generated using the method described in Section 4. Figure 1 illustrates an example of this process. The client sends CIDs with priority 1 and priority 2 (C-CID1 and C-CID2), and the server also sends CIDs representing these two priority levels (S-CID1 and S-CID2). Zheng, et al. Expires 6 November 2023 [Page 5] Internet-Draft quic-te May 2023 Client Server (CID exchange) NEW_CONNECTION_ID[C-CID1] NEW_CONNECTION_ID[C-CID2] ---> NEW_CONNECTION_ID[S-CID1] <--- NEW_CONNECTION_ID[S-CID2] (New path init with priority 1) PATH_CHALLENGE[S-CID1] ---> <--- PATH_RESPONSE,PATH_CHALLENGE[C-CID1] PATH_RESPONSE[S_CID1] ---> Figure 1: An example of CID exchange and path setup After the CID exchange is completed, the client selects S-CID1 to establish a new path of priority 1 via PATH_CHALLENGE frame. Even though the packet that uses S-CID1 comes from the same 4-tuple as the default path (i.e., priority 0), the server SHOULD treat it as a new path establishment request and respond with PATH_RESPONSE and PATH_CHALLENGE frames using a CID of the same path priority, i.e., C-CID1. 3.4. Effect of CID rotation When an endpoint rotates to a new CID on an existing path, it will not send PATH_CHALLENGE frame and the new CID does not change the path priority, so the peer will not treat this new CID as an attempt to create a new path. 4. CID Generation with Given Priorities Once transport parameter is negotiated and PRIORITY_PATHS is supported, the endpoint will generate several CIDs that are used to represent the priority and be ready for path creation. All QUIC packets described in this document MUST NOT use zero-length CID. The configuration in QUIC server defines how many priority levels (i.e., n) are adopted. For a given priority level p, where 0 <= p <= n-1. A CID is generated as CID = random_num * n + p. When an endpoint receives CIDs from its peer, it obtains the priority of a CID via the following simple modulo calculation: priority = CID % n. Note that the above addition, multiplication and division operations are in the finite field. Zheng, et al. Expires 6 November 2023 [Page 6] Internet-Draft quic-te May 2023 5. CID Parsing in Cloud WAN This section describes CID parsing in a network node, which is called a CID parser. This paser is usually located at the edge of Cloud WAN. It receives traffic from datacenters or mobile carrier networks. The CID parser stores a configuration file，described below: (QUIC server IP and port) ==> (CID length, priority levels n) This configuration specifies packets from/to which QUIC servers require routing on priority paths, the CID length, and priority levels n. It also records the mapping of priority levels to specific traffic engineering policies. The configuration file is issued by the control plane and can be updated during runtime. In the case of SRv6 TE, such a priority level is mapped to SRv6 SIDs as shown below: (priority 0） ==> (SRv6 SID 0) (priority 1） ==> (SRv6 SID 1) (priority 2） ==> (SRv6 SID 2) When a parser receives a QUIC packet, according to the configuration file, it first parses the source IP (from server to client) or destination IP (from client to server) of the packet to check whether it needs to further parse the CID. If necessary, the parser then uses the corresponding n to calculate the priority via modulo calculation. Based on the calculated result, the parser adds the corresponding SRv6 SID to the packet, so that the packet can be routed on the desired priority path. As these operations only include match, insert new header, and modulo calculation，they can be efficiently offloaded to hardware. 6. Examples 6.1. Packet Flow from Server to Client Figure 2 shows a typical example that how the packets from a server to a client are routed on different Cloud WAN routing paths according to different CIDs (priorities). Zheng, et al. Expires 6 November 2023 [Page 7] Internet-Draft quic-te May 2023 +--------------------+ | Cloud VM | | +----------------+ |API: priority description | | APP server | |(retrans_pkts: high_priority) | +----------------+ |(other_pkts: default_priority) | | Multipath QUIC | | | +----------------+ | | | +-------+----+-------+ | | Path1[C-DCID0] | | Path2[C-DCID1] (other_pkts) | | (retrans_pkts) +-----------------------+----+-------------------------+ | | | Cloud WAN | | +---v----v---+ | | +-+ CID parser +-+ | | | +------------+ | | | C-DCID0 | | C-DCID1 | | | | | | +--v-+ +-v--+ | | | R1 | | R2 | | | +--+-+ +--+-+ | | Regular link | | Premium link | | | | | | | +----+ | | | +-----> R3 <------+ | | +-+--+ | | | | +-------------------------+----------------------------+ | +----v-----+ | ISP edge | ++--------++ C-DCID0 || C-DCID1 +-----v--------v------+ | | | +-----------------+ | | | Multipath QUIC | | | +-----------------+ | | | APP client | | | +-----------------+ | | Mobile device | +---------------------+ Figure 2: An example of packet flow from server to client Zheng, et al. Expires 6 November 2023 [Page 8] Internet-Draft quic-te May 2023 The app first describes its priority requirements through the API to the QUIC layer. As shown in this figure, it describes that retransmitted packets should be routed on a high-priority path, while other packets are routed on a default-priority path. When these QUIC packets arrive at the edge of Cloud WAN, the CID parser parses each packet and extracts the priority encoded in the CIDs as described in Section 5. Based on the priority, packets are sent on different routing paths via added SRv6 SIDs. For example, low-priority packets will be routed on a default low-cost path, while high-priority packets will routed on a preimum low-latency path. Since packets using different CIDs belong to the same QUIC connection, they will be combined into a complete and ordered stream by QUIC and then submitted to the upper-layer application. 6.2. Deployment of CID Parser This section discusses some experiences regarding the deployment of CID parsers in a Cloud WAN. Two issues should be considered in the actual deployment: * Where to deploy? * Translating QUIC CID to SRv6 segment identifiers or MPLS labeling? For the first one, parser is usually deployed on the edge of Cloud WAN. For uplink traffic, it is deployed at the intersection of the ISP network; for downlink traffic, it is deployed at the edge routers of a datacenter region. For the second one, this document recommends using SRv6 because it has better scalability and flexibility. 7. Security Considerations TBD. 8. IANA Considerations The following entry in Table Table 1 should be added to the "QUIC Transport Parameters" registry under the "QUIC Protocol" heading: Zheng, et al. Expires 6 November 2023 [Page 9] Internet-Draft quic-te May 2023 +======================+=================+===============+ | Value | Parameter Name. | Specification | +======================+=================+===============+ | TBD (experiments use | PRIORITY_PATHS | Section 4 | | 0x5052494f52495459) | | | +----------------------+-----------------+---------------+ Table 1: New Transport Parameter 9. Normative References [Multipath-QUIC] Liu, Y., Ma, Y., De Coninck, Q., Bonaventure, O., Huitema, C., and M. Kühlewind, "Multipath Extension for QUIC", Work in Progress, Internet-Draft, draft-ietf-quic-multipath-04, 13 March 2023, . [QUIC-LB] Duke, M., Banks, N., and C. Huitema, "QUIC-LB: Generating Routable QUIC Connection IDs", Work in Progress, Internet- Draft, draft-ietf-quic-load-balancers-15, 24 October 2022, . [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8837] Jones, P., Dhesikan, S., Jennings, C., and D. Druta, "Differentiated Services Code Point (DSCP) Packet Markings for WebRTC QoS", RFC 8837, DOI 10.17487/RFC8837, January 2021, . [RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, February 2021, . [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, May 2021, . Zheng, et al. Expires 6 November 2023 [Page 10] Internet-Draft quic-te May 2023 Authors' Addresses Zhilong Zheng Alibaba Inc. Email: zhiyou.zzl@alibaba-inc.com Yunfei Ma Alibaba Inc. Email: yunfei.ma@alibaba-inc.com Yanmei Liu Alibaba Inc. Email: miaoji.lym@alibaba-inc.com Mirja Kühlewind Ericsson Email: mirja.kuehlewind@ericsson.com Zheng, et al. Expires 6 November 2023 [Page 11]