grep with --extended-regexp much slower on Ubuntu 16.04

[Brian Bird]

Thanks for the various suggestions. Some things to note:

The timings are consistently slower no matter what version of random.txt is used. The example I gave was to make it as easy as possible for anyone to test on any machine, but the problem exists even if "random.txt" is the same in all cases.
The slower time only relates to the grep itself, not the generation of the random.txt file
This happens on a fresh install of a desktop version of Ubuntu 16.04 with no other software installed other than the default settings. It also happens on a minimal server install (with the only extra packages being things like sshd)
This happens when I reinstall 14.04 and 16.04 on the same hardware (I've tried other hardware too - same problem). However, even if the hardware causes some variation something this simple should not be an order of magnitude different in the timing.
The load on the machines is as low as possible when running these tests.  Some variation is caused by a variation in CPU or disk load, but it is consistently a *lot* slower.
I tried single user mode - same effect.
I tried different settings for LC_ALL but it had no effect. Both installations are defaulting to " en_GB"

I've also noticed that if --perl-regexp is used instead of --extended-regexp on Ubuntu 16.04 this is ~10 times faster (but not on Ubuntu 14.04). So the problem only exists with --extended-regexp

Some background - I'm looking into this because after upgrading to Ubuntu 16.04LTS server some other software I use is slower by ~40%. This particular issue is one part of it that appears to be contributing a significant portion of the slowdown.

Since I can reproduce this consistently I would like to know if anyone else can/can't reproduce it on Ubuntu 16.04LTS? As mentioned above, the example command I've given is to make it as easy as possible to test it rather than to be 100% consistent regarding the timing. But since the difference is so huge it doesn't matter if there is a little variance in the time to run the grep. 


Eg. some abbreviated output with the same file:
Ubuntu 14:
09:29:12 up 19:59,  1 user,  load average: 0.01, 0.10, 0.09
time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
real    0m0.599s
# time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
real    0m0.607s
# time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
real    0m0.600s
09:29:20 up 19:59,  1 user,  load average: 0.01, 0.10, 0.09


But on Ubuntu 16:
09:31:42 up 21:22,  2 users,  load average: 0.10, 0.12, 0.09
# time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
real    0m5.585s
# time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
real    0m5.787s
# time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
real    0m5.648s
# time grep --perl-regexp "[0-9]{10}$" /tmp/random.txt
real    0m0.465s
09:34:16 up 21:25,  2 users,  load average: 0.16, 0.11, 0.09



-----Original Message-----
From: ubuntu-users [mailto:ubuntu-users-bounces at lists.ubuntu.com] On Behalf Of Duane Whitty
Sent: 08 February 2018 21:39
To: ubuntu-users at lists.ubuntu.com
Cc: duane at nofroth.com
Subject: Re: grep with --extended-regexp much slower on Ubuntu 16.04

On 18-02-08 12:42 PM, Brian Bird wrote:
> I've noticed Ubuntu 16.04LTS seems to run greps with "-extended-regexp" much slower than Ubuntu 14.04LTS. I can reproduce this on the same hardware simply by re-installing with the relevant operating system.
> 
> Eg. On Ubuntu 14.04
> # openssl rand -out /tmp/random.txt -base64 100000000; time grep --extended-regexp "[0-9]{10}$" /tmp/random.txt
> real    0m0.513s
> user    0m0.474s
> sys     0m0.036s
> 
> On Ubuntu 16.04:
> # openssl rand -out /tmp/random.txt -base64 100000000; time grep 
> --extended-regexp "[0-9]{10}$" /tmp/random.txt
> 
> real    0m20.619s
> user    0m20.587s
> sys     0m0.028s
> 
> Q. Can anyone try the above commands and verify if they have similar problems?
> Q. Can anyone suggest anything that might cause this?
> 
> Any pointers appreciated!
> Thanks
> 
> One of my first questions should have been what kind of OS 
> installation
are you doing, e.g., minimal server install, full blown X with desktop environment, etc?  I believe to really get solid test results you need to do a bare minimum install of the OS and preferably be in maintenance mode before running the test.  To state the obvious, serious testing can get complicated because it can be difficult to get rid of outside influences.

Best Regards,
Duane

--
Duane Whitty
duane at nofroth.com

--
ubuntu-users mailing list
ubuntu-users at lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users