Fixes #95

I didn't test this enough, but lets see what you have to say about it :) I tried to keep it as simple as possible.

This has to be tested with IPv6. I guess that IPv6 is not supported yet (shouldn't be too hard to add IPv6 support though), but first lets make sure it doesn't break it.

Resolves #1064

Seems to work well. Can be tested with @ghedo's python code and his Go code (Now my code uses the same numbers. I've updated his Go code, below, to work with the new edns-like code:)

package main

import (
	"fmt"
	"github.com/DesWurstes/dns"
	"net"
	"sync"
)

func handleRequest(w dns.ResponseWriter, r *dns.Msg) {
	fmt.Println("GOT REQUEST", r)

	m := new(dns.Msg)
	m.SetReply(r)

	svcb := &dns.SVCB{
		Hdr: dns.RR_Header{
			Name:   r.Question[0].Name,
			Rrtype: dns.TypeSVCB,
			Class:  dns.ClassINET,
			Ttl:    3600,
		},
		Priority: 1,
		Target:   ".",
		Value: []dns.SVCBKeyValue{
			&dns.SVCBAlpn{
				Alpn: []string{"h2", "h3"},
			},
			&dns.SVCBPort{
				Port: 117,
			},
			&dns.SVCBIPv4Hint{
				Hint: []net.IP{net.IPv4(1, 1, 1, 1), net.IPv4(1, 1, 1, 2)},
			},
			&dns.SVCBECHConfig{
				ECH: []byte{1, 2},
			},
		},
	}

	m.Answer = append(m.Answer, svcb)
	err := w.WriteMsg(m)
	if err != nil {
		fmt.Println(err)
	}
}

func main() {
	var wg sync.WaitGroup
	wg.Add(1)

	dns.HandleFunc("svcb.example.", handleRequest)

	server := &dns.Server{Addr: ":53", Net: "udp"}
	go server.ListenAndServe()

	wg.Wait()
}

Would it be possible to add support for GSS-TSIG (RFC 3645)? This would make it possible to perform secure DNS updates to a Windows Active Directory environment, which AFAICT doesn't support normal TSIG updates. I figured maybe https://github.com/jcmturner/gokrb5 could be useful to do the Kerberos side of things.

These are two issues that are interesting:

• https://github.com/coredns/coredns/issues/2593
• https://github.com/coredns/coredns/issues/2624

Both are not reproduced in a minimal setting and I failed reproducing it.

It looks like if you restrain the CPU a process has it can get behind and (in some cases) never recover. One way this can show up in this library is too many worker goroutines.

I took a quick look at the code in server.go, in this comment at the top of the file looked a bit ominous:

// The maximum number of idle workers.
//
// This controls the maximum number of workers that are allowed to stay
// idle waiting for incoming requests before being torn down.
//
// If this limit is reached, the server will just keep spawning new
// workers (goroutines) for each incoming request. In this case, each
// worker will only be used for a single request.
const maxIdleWorkersCount = 10000

esp. the last paragraph.

As said, I'm not sure how best to repro this (just sit on queries for 10 seconds?), but if we can it would help to debug the above issues and would be a good (unit) test in general.

• Ignore TCP server shutdown returns excepted error(use of closed network connection)
• Make sure server re-serve works do after shutdown
• Fix server.inFlight race
• WaitGroup deadlock(=.= hanging

The generic struct pack and unpack routines make heavy use of reflection to correctly translate between bytes and Go objects.

Given most questions and answers are A records, writing a specific A record packer/unpacker makes the fast path quicker and generates significantly less garbage.

The pack and unpack routines for specific data types were extracted from the generic routine so that they may be shared with the optimised routines.

Results of go test -bench \(Unp\|P\)ackRRA

| Benchmark | Iterations | ns/op | B/op | allocs/op | | --- | --- | --- | --- | --- | | BenchmarkPackRRA (before) | 2000000 | 688 ns/op | 112 B/op | 7 allocs/op | | BenchmarkPackRRA (after) | 20000000 | 85.9 ns/op | 0 B/op | 0 allocs/op | | BenchmarkUnpackRRA (before) | 1000000 | 1443 ns/op | 272 B/op | 12 allocs/op | | BenchmarkUnpackRRA (after) | 5000000 | 290 ns/op | 128 B/op | 4 allocs/op |

This library has a nasty habit of interpreting domain names relative to the DNS root when it shouldn't. Per RFC 1035, "A relative name is an error when no origin is available" and we should emit an error token from ParseZone when encountering such a case.

This bug was originally reported with a mistaken assumption of initial origin, which uncovered the above issue:

example.com. 3600 IN SOA ns1.example.com. hostmaster 1 (
	86400
	   60
	86400
	 3600
) ; SOA with relative RNAME

	NS ns1 ; implied-owner NS with implied TTL/CLASS and relative-domain RDATA

ns1 60 A 192.0.2.1 ; relative owner with implied CLASS

$ORIGIN api ; relative origin

@ IN A 192.0.2.2 ; origin-shorthand owner with implied TTL

apac 30 CNAME @ ; relative owner with implied CLASS and origin-shorthand RDATA

$ORIGIN NA ; relative origin relative to existing origin

@ 30 IN CNAME cdn.example ; origin-shorthand owner with relative RDATA
cdn.example.NA.api.example.com. 60 IN A 192.0.2.3

; SOA with relative RNAME
; mismatch, expected:
; example.com.	3600	IN	SOA	ns1.example.com. hostmaster.example.com. 1 86400 60 86400 3600
example.com.	3600	IN	SOA	ns1.example.com. hostmaster. 1 86400 60 86400 3600

; implied-owner NS with implied TTL/CLASS and relative-domain RDATA
; mismatch, expected:
; example.com.	3600	IN	NS	ns1.example.com.
example.com.	3600	IN	NS	ns1.

; relative owner with implied CLASS
; mismatch, expected:
; ns1.example.com.	60	IN	A	192.0.2.1
ns1.	60	IN	A	192.0.2.1

; origin-shorthand owner with implied TTL
; mismatch, expected:
; api.example.com.	60	IN	A	192.0.2.2
api.	3600	IN	A	192.0.2.2

; relative owner with implied CLASS and origin-shorthand RDATA
; mismatch, expected:
; apac.api.example.com.	30	IN	CNAME	api.example.com.
apac.api.	30	IN	CNAME	api.

; origin-shorthand owner with relative RDATA
; mismatch, expected:
; NA.api.example.com.	30	IN	CNAME	cdn.example.NA.api.example.com.
NA.api.	30	IN	CNAME	cdn.example.NA.api.

; match
cdn.example.NA.api.example.com.	60	IN	A	192.0.2.3

--- actual
+++ expected
 ; SOA with relative RNAME
-example.com.	3600	IN	SOA	ns1.example.com. hostmaster. 1 86400 60 86400 3600
+example.com.	3600	IN	SOA	ns1.example.com. hostmaster.example.com. 1 86400 60 86400 3600
 
 ; implied-owner NS with implied TTL/CLASS and relative-domain RDATA
-example.com.	3600	IN	NS	ns1.
+example.com.	3600	IN	NS	ns1.example.com.
 
 ; relative owner with implied CLASS
-ns1.	60	IN	A	192.0.2.1
+ns1.example.com.	60	IN	A	192.0.2.1
 
 ; origin-shorthand owner with implied TTL
-api.	3600	IN	A	192.0.2.2
+api.example.com.	60	IN	A	192.0.2.2
 
 ; relative owner with implied CLASS and origin-shorthand RDATA
-apac.api.	30	IN	CNAME	api.
+apac.api.example.com.	30	IN	CNAME	api.example.com.
 
 ; origin-shorthand owner with relative RDATA
-NA.api.	30	IN	CNAME	cdn.example.NA.api.
+NA.api.example.com.	30	IN	CNAME	cdn.example.NA.api.example.com.
 cdn.example.NA.api.example.com.	60	IN	A	192.0.2.3

This works fine x, xErr := NewRR("www.miek.nl. 3600 IN TXT "first" "second"")

but this doesn't: x, xErr := NewRR("www.miek.nl. 3600 ANY TXT "first" "second"") if xErr != nil { t.Fatalf("failed to parse TXT: %s", xErr) }

it fails with the following error: dns_test.go:461: failed to parse TXT: dns: unknown RR type: "TXT" at line: 1:26

I assume this should work - or am I confused?

Various zone files distributed by ICANN via the CZDS as well as via direct interfaces provided by the respective registry operator, contain malformed or bogus records.

Examples at the time of this writing include .us as distributed by Neustar and .tel as available through the CZDS.

High-performance processing pipelines can benefit from filtering those bogus records on the fly.

The ClearError() method allows these classes of applications to simply have the parser state machine to step through these errors without changing the parser or otherwise affecting its correctness.

I'm using coredns 1.2.2 with the built-in file plugin (https://github.com/coredns/coredns/tree/master/plugin/file) to serve a few RFC 1035 master zone files. coredns uses github.com/miekg/dns. I'm also using the more resent version of Go (1.11.1).

In this configuration, coredns memory usage grows steadily and crashes after about 18 hours with an OOM error with a server that contains 2GB of memory. Here's a snippet from the coredns log file showing the crash:

2018/10/10 13:56:17 [INFO] CoreDNS-1.2.2 2018/10/10 13:56:17 [INFO] linux/amd64, go1.11.1, 8a9c6174 ... 1360 fatal error: runtime: out of memory 1361 1362 runtime stack: 1363 runtime.throw(0x192f945, 0x16) 1364 /home/ubuntu/go1.11/src/runtime/panic.go:608 +0x72 1365 runtime.sysMap(0xc064000000, 0x4000000, 0x29ef978) 1366 /home/ubuntu/go1.11/src/runtime/mem_linux.go:156 +0xc7 1367 runtime.(*mheap).sysAlloc(0x29d59a0, 0x4000000, 0x29d59b8, 0x7fd3ecb43a28) 1368 /home/ubuntu/go1.11/src/runtime/malloc.go:619 +0x1c7 1369 runtime.(*mheap).grow(0x29d59a0, 0x1, 0x0) 1370 /home/ubuntu/go1.11/src/runtime/mheap.go:920 +0x42 1371 runtime.(*mheap).allocSpanLocked(0x29d59a0, 0x1, 0x29ef988, 0x0) 1372 /home/ubuntu/go1.11/src/runtime/mheap.go:848 +0x337 1373 runtime.(*mheap).alloc_m(0x29d59a0, 0x1, 0x430008, 0xc000044500) 1374 /home/ubuntu/go1.11/src/runtime/mheap.go:692 +0x119 1375 runtime.(*mheap).alloc.func1() 1376 /home/ubuntu/go1.11/src/runtime/mheap.go:759 +0x4c 1377 runtime.(*mheap).alloc(0x29d59a0, 0x1, 0xc000010008, 0x432339) 1378 /home/ubuntu/go1.11/src/runtime/mheap.go:758 +0x8a 1379 runtime.(*mcentral).grow(0x29d6ed8, 0x0) 1380 /home/ubuntu/go1.11/src/runtime/mcentral.go:232 +0x94 1381 runtime.(*mcentral).cacheSpan(0x29d6ed8, 0xc063aef0d8) 1382 /home/ubuntu/go1.11/src/runtime/mcentral.go:106 +0x2f8 1383 runtime.(*mcache).refill(0x7fd3f5229000, 0xc0000a7b08) 1384 /home/ubuntu/go1.11/src/runtime/mcache.go:122 +0x95 1385 runtime.(*mcache).nextFree.func1() 1386 /home/ubuntu/go1.11/src/runtime/malloc.go:749 +0x32 1387 runtime.systemstack(0xc00008cd00) 1388 /home/ubuntu/go1.11/src/runtime/asm_amd64.s:351 +0x66 1389 runtime.mstart() 1390 /home/ubuntu/go1.11/src/runtime/proc.go:1229 1392 goroutine 4058 [running]: 1393 runtime.systemstack_switch() 1394 /home/ubuntu/go1.11/src/runtime/asm_amd64.s:311 fp=0xc0620b7508 sp=0xc0620b7500 pc=0x45a340 1395 runtime.(*mcache).nextFree(0x7fd3f5229000, 0x8, 0xc06370d3c0, 0x0, 0xc000146a80) 1396 /home/ubuntu/go1.11/src/runtime/malloc.go:748 +0xb6 fp=0xc0620b7560 sp=0xc0620b7508 pc=0x40b7b6 1397 runtime.mallocgc(0x30, 0x176dfc0, 0x1, 0xc06370d3c0) 1398 /home/ubuntu/go1.11/src/runtime/malloc.go:903 +0x793 fp=0xc0620b7600 sp=0xc0620b7560 pc=0x40c103 1399 runtime.newobject(0x176dfc0, 0x446d56) 1400 /home/ubuntu/go1.11/src/runtime/malloc.go:1032 +0x38 fp=0xc0620b7630 sp=0xc0620b7600 pc=0x40c4e8 1401 github.com/miekg/dns.setCNAME(0xc063ff3770, 0x23, 0x3c00010005, 0x0, 0xc063aef080, 0xc0000a1d86, 0x12, 0xc000092b40, 0x22, 0x23, ...) 1402 /home/ubuntu/go/src/github.com/miekg/dns/scan_rr.go:515 +0x3f fp=0xc0620b7730 sp=0xc0620b7630 pc=0x71d01f 1403 github.com/miekg/dns.setRR(0xc063ff3770, 0x23, 0x3c00010005, 0x0, 0xc063aef080, 0xc0000a1d86, 0x12, 0xc000092b40, 0x22, 0x1b68b80, ...) 1404 /home/ubuntu/go/src/github.com/miekg/dns/scan_rr.go:29 +0xdd fp=0xc0620b78c0 sp=0xc0620b7730 pc=0x71731d 1405 github.com/miekg/dns.parseZone(0x1b44740, 0xc00000cea0, 0xc0000a1d86, 0x12, 0xc000092b40, 0x22, 0xc014998348, 0xc063256180, 0x0) 1406 /home/ubuntu/go/src/github.com/miekg/dns/scan.go:469 +0x1fdc fp=0xc0620b7f98 sp=0xc0620b78c0 pc=0x71292c 1407 runtime.goexit() 1408 /home/ubuntu/go1.11/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc0620b7fa0 sp=0xc0620b7f98 pc=0x45c2a1 1409 created by github.com/miekg/dns.parseZoneHelper 1410 /home/ubuntu/go/src/github.com/miekg/dns/scan.go:162 +0xc3

I enabled profiling in the coredns config file and have been watching the heap. Here are two examples showing this issue. Notice that github.com/miekg/dns.parseZone grows continually by about 100MB each hour:

File: coredns Type: inuse_space Time: Oct 11, 2018 at 3:43pm (EDT) Entering interactive mode (type "help" for commands, "o" for options) (pprof) top5 Showing nodes accounting for 385.64MB, 95.53% of 403.68MB total Dropped 11 nodes (cum <= 2.02MB) Showing top 5 nodes out of 23 flat flat% sum% cum cum% 106MB 26.26% 26.26% 303.02MB 75.06% github.com/miekg/dns.parseZone 98.50MB 24.40% 50.66% 98.50MB 24.40% github.com/miekg/dns.appendOrigin (inline) 98MB 24.28% 74.94% 98MB 24.28% github.com/miekg/dns.setCNAME 68MB 16.85% 91.78% 68MB 16.85% github.com/miekg/dns.zlexer 15.12MB 3.75% 95.53% 15.12MB 3.75% github.com/miekg/dns.parseZoneHelper

File: coredns Type: inuse_space Time: Oct 11, 2018 at 4:58pm (EDT) Entering interactive mode (type "help" for commands, "o" for options) (pprof) top5 Showing nodes accounting for 512.96MB, 96.51% of 531.51MB total Dropped 11 nodes (cum <= 2.66MB) Showing top 5 nodes out of 23 flat flat% sum% cum cum% 138.01MB 25.96% 25.96% 138.01MB 25.96% github.com/miekg/dns.appendOrigin (inline) 137.51MB 25.87% 51.84% 404.02MB 76.01% github.com/miekg/dns.parseZone 127.51MB 23.99% 75.83% 127.51MB 23.99% github.com/miekg/dns.setCNAME 90.50MB 17.03% 92.85% 90.50MB 17.03% github.com/miekg/dns.zlexer

Here is the coredns config file I am using: coredns.conf.txt

The zone files being served have about 100,000 records.

Thanks,

Brad

There is issue I'm observing while trying to create fast moving tool with your library (which is great).

On high levels of concurrency, when I let goroutines run wild I'm getting "dial udp a.b.c.d:53: too many open files" error.

IMO that's due to client's exchange logic that only has single way of doing things ritght now, "Exchange() -> dial()/send()->write()/receive()->read()"

That will essentially create socket in OS for every operation and send/receive on it via "send and recv". Sendto and recvfrom in my situation would be most efficient but syscall lib is very hard to deal with, I'm trying to find simpler solutions...

IMO other version of Exchange could have additional net.Conn parameter for clients who send bunch of requests to same address... As coder I'd expect it this way:

dnsconn := c.Dial(addr)
for _, msg := range queries {
    resp, rtt, err := c.ExchangeWith(dnsconn)
    if err {
         ...
    }
    ...
}

Should not be hard to implement... You might suggest different way of handling that. I understand that retrying logic might require to keep more than one net.Conn inside of that "dnsconn" structure.

Consider below program:

package main

import "github.com/miekg/dns"

func main() {
	_, _ = dns.NextLabel("go.dev", -1)
}

The document of NextLabel is not clear about the behavior when offset is negative.

When providing a deadline context to Client.ExchangeContext(), canceling is only honored during the dial stage but not during the read or write stage.

There seems to be a documentation error in client.go...

// Timeout is a cumulative timeout for dial, write and read, defaults to 0 (disabled) - overrides DialTimeout, ReadTimeout,
// WriteTimeout when non-zero. Can be overridden with net.Dialer.Timeout (see Client.ExchangeWithDialer and
// Client.Dialer) or context.Context.Deadline (see ExchangeContext)

From the code, Client.Timeout only overrides Client.DialTimeout. There is also this documentation for Client.ExchangeContext()...

// ExchangeContext acts like Exchange, but honors the deadline on the provided
// context, if present. If there is both a context deadline and a configured
// timeout on the client, the earliest of the two takes effect.

It's awkward if the context deadline is larger than the default dnsTimeout constant of 2s. In that situation, all the client timeouts must be set to a value equal or larger than the context timeout in order to get the desired behavior.

IMO, implementation should be updated to have Client.Timeout behave as documented and supersede Client.DialTimeout + Client.ReadTimeout + Client.WriteTimeout. Additionally, a context deadline should supersede Client.Timeout (or Client.DialTimeout + Client.ReadTimeout + Client.WriteTimeout).

func dnsServerTest() {
	dns.HandleFunc("docker.shared.", func(w dns.ResponseWriter, m *dns.Msg) {
		r := new(dns.Msg)
		r.SetReply(m)
		r.Compress = false
		r.Authoritative = true
		r.RecursionAvailable = true
		var record = map[string]string{"test.docker.shared.": "192.168.0.1"}

		defer w.WriteMsg(r)

		for _, q := range r.Question {
			ip := record[q.Name]
			if q.Qtype == dns.TypeA && ip != "" {
				r.Answer = append(r.Answer, &dns.A{Hdr: dns.RR_Header{Name: q.Name, Rrtype: q.Qtype, Class: q.Qclass}, A: net.ParseIP(ip)})
				log.Printf("Query: [%v --> %s]\n", w.RemoteAddr(), ip)
			}
		}
	})

	udpServer := &dns.Server{Addr: ":53", Net: "udp"}

	go udpServer.ListenAndServe()
	sigs := make(chan os.Signal, 1)
	signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
	<-sigs
	udpServer.Shutdown()
}

➜  ~ nslookup test.docker.shared
Server:		127.0.0.1
Address:	127.0.0.1#53

Name:	test.docker.shared
Address: 192.168.0.1

➜  ~

INFO[2022-10-10 20:15:40] Query: [127.0.0.1:52731 --> 192.168.0.1]     
INFO[2022-10-10 20:15:41] Query: [127.0.0.1:54235 --> 192.168.0.1] 

--- read here !!!

curl http://test.docker.shared
curl: (6) Could not resolve host: test.docker.shared

Both nslookup and dig can be resolved, but curl cannot resolve it, and the browser cannot resolve it. I have tried to refresh the DNS, but it still doesn't work, and resolv.conf is configured correctly

thank, very much

This introduces a dns.Dialer interface in the Client, rather than expecting a *net.Dialer. It's to allow passing in custom dialers that establish connections other than direct TCP or UDP. It would let me open connections through various types of proxies.

It's still a draft as I didn't add a test yet, and the comment around how Timeout is applied would need to be updated. Just wanted to first see if this is something you'd accept first.