[Bug 1686679] Re: [SRU] Ubuntu16.04 : autofs takes extreamly long with large number of direct maps

Tue Jun 13 18:42:06 UTC 2017

[STS Sponsor Feedbacks]

Both debdiff need some small rework...

* Package versions need to be changed :

Currently both debdiff introduce version "5.1.1-1ubuntu4" in Xenial and Yakkety which is not good.
The reason why they are identical for now, it's probably because no changes has been made between when Xenial package has been copied to Yakkety (when it was a devel release still). 
Last change before this one was on "Thu, 19 Nov 2015 17:31:09"

rmadison :
 autofs | 5.1.1-1ubuntu3   | xenial          | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x
 autofs | 5.1.1-1ubuntu3   | yakkety         | source, amd64, arm64, armhf, i386, powerpc, ppc64el, s390x

The good versionning in this case is :

Xenial -->  "5.1.1-1ubuntu3.1"
Yakkety --> "5.1.1-1ubuntu4"

* Add the LP bug in debian/changelog as follow :

  * Add one upstream patch to improve performance
   - autofs-5.1.1-improve-scalability-of-direct-mount-pat.patch (LP: #1686679)

* For both debdiff(s), the quilt patch doesn't apply cleanly, here's the
details :

# quilt push -a
Applying patch autofs-5.1.1-improve-scalability-of-direct-mount-pat.patch
patching file daemon/automount.c
Hunk #1 succeeded at 95 (offset -3 lines).
Hunk #2 succeeded at 126 (offset -3 lines).
patching file include/automount.h
Hunk #1 succeeded at 70 (offset -5 lines).
patching file include/mounts.h
Hunk #1 succeeded at 94 (offset -7 lines).
patching file lib/mounts.c
Hunk #1 succeeded at 943 (offset 2 lines).

* Any particular reason why "autofs-5.1.1-improve-scalability-of-direct-
mount-pat.patch" is not set to apply on the top ?

* quilt header must be in DEP3 format

# "quilt header --dep3 -e" will provide you the template but here's one
I started for you that you can complete, if not Bug: nor Bug-Debian:,
you can remove the lines.

Description : improve scalability of direct mount path component
 With direct mounts, we want to avoid creating path components on remote file systems 
 unless that file system is the root file system. The check boils down to allowing the 
 mkdir if: 
 1/ If it's the root directory, or 
 2/ If it's not a remote file system, or 
 3/ If it's a remote file system that's the root file system We can do that without 
 parsing the mount table and can improve startup time for all cases.
Author: Jeff Mahoney <jeffm at suse.com>
Origin: https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git/commit/?id=67e7d613
Bug: <upstream_bug>, if any
Bug-Ubuntu: https://bugs.launchpad.net/bugs/988397
Bug-Debian: <debian_bug> if any

Let me know when the new debdiff will be ready, and I'll review/sponsor it.

- Eric

** Description changed:

  [Impact]

  autofs service in xenial takes extremely long time since it parses the
  mount table /etc/mtab in contained_in_local_fs() to determine if the mount
  patch would be created on a local file system, that means startup process will read /etc/mtab for every mount, so the performance is extremely low. The customer is complaining about this point, we need to backport the
  following upstream patch.

  https://git.kernel.org/pub/scm/linux/storage/autofs/autofs.git/commit/?id=67e7d613a4b09eeffc57ab44a7acb52027d897b2

  This fix patch removes contained_in_local_fs, and use fs.f_type in is_remote_fstype() to determine if the mount patch would be created on a
   local file system instead.

  [Test Case]

   * Create an autofs direct mount map with large direct mappings (eg:
  8k), below is the test script I used.

  echo "/-      auto.direct        --timeout 60" >> /etc/auto.master
  END=8000
  for i in $(seq 1 $END); do echo $i; echo "/test/samba/test"$i" -fstype=cifs,rw,username=hua,password=password ://192.168.99.169/share" >> /etc/auto.direct; done

   * Start autofs service to see if it will take too much time, below is my
   result before/after applying the fix patch.

  root at node1:~# time service autofs start
  real	3m5.481s
  user	0m0.256s
  sys	0m0.080s
  root at node1:~# time service autofs start
  real	0m0.833s
  user	0m0.000s
  sys	0m0.004s

  [Regression Potential]

  * We are well aware of the principle of this fix patch - avoiding parsing
  the mount table to improve startup time, so seems infinitely better for
  all cases.

  * This fix from upstream has been backported into Redhat as well, and both
   me and customer have positive test results with automount start timings.

  * What releases to fix

  $ git tag --contains 67e7d613a4b09eeffc57ab44a7acb52027d897b2
  release_5_1_2

  $ rmadison autofs5
- [...]
   autofs5 | 5.1.1-1ubuntu3 | xenial | all
   autofs5 | 5.1.1-1ubuntu3 | yakkety | all
   autofs5 | 5.1.2-1ubuntu1 | zesty | all
   autofs5 | 5.1.2-1ubuntu1 | artful | all

  $ rmadison -u debian autofs5
- autofs5    | 5.1.2-1        | unstable        | all
+  autofs5    | 5.1.2-1        | unstable        | all

- Only Xenial and Yakkety are affected since Zesty and Artful already has
- the upstream fix because it's version 5.1.2.
+ - Ubuntu : Only Xenial and Yakkety are affected.
+ Zesty and Artful already has the upstream fix because they are at version 5.1.2.

- Debian unstable also has the fix already.
+ - Debian : unstable also has the fix already.

  [Other Info]

  * Redhat [1] also already backported the following patch, see: https://bugzilla.redhat.com/show_bug.cgi?id=1440769
  * This mailing list also discussed this problem, see: http://www.spinics.net/lists/autofs/msg01161.html

** Changed in: autofs5 (Ubuntu Xenial)
     Assignee: (unassigned) => Hua Zhang (zhhuabj)

** Changed in: autofs5 (Ubuntu Yakkety)
     Assignee: (unassigned) => Hua Zhang (zhhuabj)

** Changed in: autofs5 (Ubuntu Xenial)
       Status: New => In Progress

** Changed in: autofs5 (Ubuntu Yakkety)
       Status: New => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1686679

Title:
  [SRU] Ubuntu16.04 : autofs takes extreamly long with large number of
  direct maps

Status in autofs5 package in Ubuntu:
  Fix Released
Status in autofs5 source package in Xenial:
  In Progress
Status in autofs5 source package in Yakkety:
  In Progress

Bug description:
  [Impact]