DNS delay on first lookup, Ubuntu 19.10 but not earlier versions

[Chris Green]

On Wed, Apr 01, 2020 at 09:43:13PM +1100, Karl Auer wrote:
> On Wed, 2020-04-01 at 11:16 +0100, Chris Green wrote:
> > No difference (but 'dig' doesn't show a delay), here's a 'dig'
> > followed by a 'host' immediately after reboot on one of the systems
> > that shows the delay:-
> 
> If dig doesn't show a delay, then it is NOT the DNS itself that is
> causing the delay, but the querier (browser, whatever).
> 
Yes, I know that really because it's only two systems (running xubuntu
19.10) that show the delay.  I've always been pretty sure that it's a
client (or something on the client) problem.


> BTW those two queries were for different names, so not a useful
> comparison.
> 
Oops, sorry, I keep putting 'esprimo.co.uk' by mistake.  In reality I
nearly always use short names (i.e. just 'esprimo') because there is a
'search zbmc.eu' in /etc/resolv.conf.  I have just been putting the
FQDN here to make things clearer.


> You could also bypass dnsmasq by doing this:
> 
>    dig esprimo.zbmc.eu @ip-address-of-nameserver
> 
No local dnsmasq on these systems, the 'dnsmasq run by Network Manager
for local DNS cache' was dropped a few Ubuntu versions ago I think.
Now it's systemd that is supposed to do local DNS caching and I
suspect it's something in that which is causing this delay.

Just to prove the problem is local/client here's another sequence of
'host' commands directly after reboot:-

    chris$ time host esprimo 192.168.1.4
    Using domain server:
    Name: 192.168.1.4
    Address: 192.168.1.4#53
    Aliases: 

    esprimo.zbmc.eu has address 192.168.1.3

    real	0m0.023s
    user	0m0.005s
    sys	0m0.009s
    chris$ time host esprimo 
    esprimo.zbmc.eu has address 192.168.1.3

    real	0m10.024s
    user	0m0.001s
    sys	0m0.013s
    chris$ time host esprimo
    esprimo.zbmc.eu has address 192.168.1.3

    real	0m0.016s
    user	0m0.006s
    sys	0m0.005s
    chris$ 

Going direct to my DHCP/DNS server 192.168.1.4 the lookup is fast,
*then* using the local system it's slow once, then subsequent lookup
is fast.  If I could simply turn off the local cache/lookup it would
probably speed things up for me, as the DNS server is on the LAN
and I'm really the only user it's fast enough without a local cache.


> ... but from the sound of it there will be no difference anyway.
> 
> What are you doing that causes the lookups? A browser, something else?
> 
Most things in the 'real world', e.g. 'ssh esprimo.zbmc.eu', 'host
esprimo.zbmc.eu', 'nslookup esprimo.zbmc.eu' all show the delay when
run immediately after boot (and after a long idle time).  I've not
tried browsing a page on esprimo.co.uk (though there are web pages I
could try immediately after boot).


It would be interesting to know what the idle time is before the delay
happens again.  I've tried clearing the systemd-resolvd cache but that
doesn't make the delay happen again.  As it is the only guaranteed way
I have of seeing the delay (other than leaving the system idle for a
* long* indeterminate time) is to reboot it.  Fortunately one of the
* 19.10 systems reboots very quickly so it's not that painful doing
* lots of reboots.

-- 
Chris Green