[apparmor] [PATCH] [parser] Fix jobs not scaling up to meet available resources when cpus are brought online during compilation

[John Johansen]

Enable dynamically scaling max jobs if new resources are brought online

BugLink: http://bugs.launchpad.net/bugs/1566490

This patch enables to parser to scale the max jobs if new resources are
being brought online by the scheduler.

It only enables the scaling check if there is a difference between the
maximum number of cpus (CONF) and the number of online (ONLN) cpus.

Instead of checking for more resources regardless, of whether the online
cpu count is increasing it limits its checking to a maximum of
MAX CPUS + 1 - ONLN cpus times. With each check coming after fork spawns a
new work unit, giving the scheduler a chance to bring new cpus online
before the next check.  The +1 ensures the checks will be done at least
once after the scheduling task sleeps waiting for its children giving
the scheduler an extra chance to bring cpus online.

Signed-off-by: John Johansen <john.johansen at canonical.com>

=== modified file 'parser/parser_main.c'
--- a/parser/parser_main.c	2016-01-15 01:26:26 +0000
+++ b/parser/parser_main.c	2016-04-05 07:27:06 +0000
@@ -89,6 +89,12 @@
 long jobs_max = -8;			/* 8 * cpus */
 long jobs = JOBS_AUTO;			/* default: number of processor cores */
 long njobs = 0;
+long jobs_scale = 0;			/* number of chance to resample online
+					 * cpus. This allows jobs spawning to
+					 * scale when scheduling policy is
+					 * taking cpus off line, and brings
+					 * them back with load
+					 */
 bool debug_jobs = false;
 
 struct timespec cache_tstamp, mru_policy_tstamp;
@@ -898,6 +904,16 @@
 		RESULT(WORK);						\
 		break;							\
 	}*/								\
+	if (jobs_scale) {						\
+		long n = sysconf(_SC_NPROCESSORS_ONLN);			\
+		if (n > jobs) {						\
+			/* reset sample chances - potentially reduce to 0 */ \
+			jobs_scale = jobs_max + 1 - n;			\
+			jobs = n;					\
+		} else							\
+			/* reduce scaling chance by 1 */		\
+			jobs_scale--;					\
+	}								\
 	if (njobs == jobs) {						\
 		/* wait for a child */					\
 		if (debug_jobs)						\
@@ -959,11 +975,18 @@
 {
 	/* jobs and paralell_max set by default, config or args */
 	long n = sysconf(_SC_NPROCESSORS_ONLN);
+	long maxn = sysconf(_SC_NPROCESSORS_CONF);
 	if (n == -1)
 		/* unable to determine number of processors, default to 1 */
 		n = 1;
+	if (maxn == -1)
+		/* unable to determine number of processors, default to 1 */
+		maxn = 1;
+	if (n < maxn)
+		/* the bigger the difference the more sample chances given */
+		jobs_scale = maxn + 1 - n;
 	jobs = compute_jobs(n, jobs);
-	jobs_max = compute_jobs(n, jobs_max);
+	jobs_max = compute_jobs(maxn, jobs_max);
 
 	if (jobs > jobs_max) {
 		pwarn("%s: Warning capping number of jobs to %ld * # of cpus == '%ld'",