Exclude TCP traffic of DNS queries from transparent proxy rules (kuma-cni and kuma-init)

[jijiechen]

Description

DNS client will retry querying if UDP response message is truncated. Large DNS UDP messages can be truncated according to the RFC. modern DNS can use EDNS0 to negotiate larger UDP buffer sizes, or clients may retry querying using DNS Transport over TCP.

There are users reporting their large DNS queries are either delayed (on Kuma version 2.10 and older versions) or failed entirely (on Kuma 2.11 and newer versions), so we need to implement the enhancement to support this scenario.

FYI.

Described in RFC 1035:

Messages carried by UDP are restricted to 512 bytes (not counting the IP or UDP headers). Longer messages are truncated and the TC bit is set in the header.

And also in RFC 5966:

In the absence of EDNS0 (Extension Mechanisms for DNS 0), the normal behavior of any DNS server needing to send a UDP response that would exceed the 512-byte limit is for the server to truncate the response so that it fits within that limit and then set the TC flag in the response header. When the client receives such a response, it takes the TC flag as an indication that it should retry over TCP instead.

[jijiechen]

From the team discussion, it may also be achieved by excluding TCP traffic of DNS queries from our traffic capturing.

[lahabana]

@jijiechen can you write out what we discussed with the code pointer to coreDNS.

IMO we shouldn't attempt to fix the coreDNS version because we're deprecating this. However, we should provide a reliable fix for the new embedded DNS. Is exclude DNS TCP traffic from redirects a good option? Can we experiment and if it works make this change happen?

[jijiechen]

In CoreDNS scenario, we can work around this issue by making queries of domains who cause the truncated bypass EnvoyDNS and forward to upstream DNS server directly.

So let's choose a easy path that we don't do anything to older versions (<=2.10.x), because CoreDNS is being deprecated in Kuma.

For new versions, we can solve the issue by excluding TCP traffic of DNS queries from our iptables rules.

Background

In CoreDNS scenario, we are not sure if the retry was sent from CoreDNS silently without returning the UDP truncated response (first result) to the client, or it returns it to the client and the retry is sent from the client. From CoreDNS code, we can see it is also sending retry queries when a UDP upstream is returning truncated. https://github.com/coredns/coredns/blob/v1.12.4/plugin/forward/forward.go#L163-L175

1. Normal queries
   app ---->  CoreDNS ----->  EnvoyDNS 
                        (or)  ----->  Cluster CoreDNS  ---->   Enterprise DNS
2. UDP truncated:
   app ---->  CoreDNS ------>  EnvoyDNS
                      (then) ---x-->  Cluster CoreDNS  -----   Enterprise DNS
3. retry with TCP (assuming by client app)
   app ---->  CoreDNS ---x-->  EnvoyDNS
                      (then) ------>  Cluster CoreDNS  ---->   Enterprise DNS