[SRU][Jammy][lowlatency] Ubuntu: [Config] lowlatency: enhance desktop responsiveness
Dimitri John Ledkov
dimitri.ledkov at canonical.com
Tue Oct 31 12:36:02 UTC 2023
Hi,
On Tue, 31 Oct 2023 at 10:54, Gerald Yang <gerald.yang at canonical.com> wrote:
>
> Hi Andrea,
>
> Thanks for the feedback! please let me explain this in more detail
>
> I sent out this SRU because some of our customers need to use NO_HZ_FULL.
> In their scenario, the CPU time is pretty sensitive, and must avoid as much noise as possible to keep
> most CPUs doing only one single task.
> I found some CPU steal time happen on their VM because RCU callback softirqs are still triggered on CPUs
> with isolcpus, nohz_full and rcu_nocbs configured, and lead me to find NO_HZ_FULL is not built into generic kernel.
>
> I think the use case for NO_HZ_FULL should be different than generic kernel, e.g. performance vs low latency
>
> On generic kernel, we also did some tests with NO_HZ_FULL built-in here:
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1919154
>
> The results show performance degradation on AMD EPYC series machines only
> when NO_HZ_FULL is built-in but not enabled, but this doesn't happen on Intel and arm64 machines.
>
> Because one of the customer is still on Focal, if it's possible to merge this into 5.15 kernel, it would be more convenient
> for them to use HWE lowlatency kernel.
> And if there are users need to use NO_HZ_FULL on Focal, lowlatency kernel could be a proper choice for them,
> because so far there is no kernel with NO_HZ_FULL built-in on Focal.
I am ok having this enabled in focal:lowlatency-hwe-5.15 and
jammy:lowlatency-hwe-6.5
but not change anything in focal:lowlatency (5.4 / ga)
jammy:lowlatency (5.15 / ga).
Would that help you? As an exception to our policy. I really feel
uncomfortable changing default GA flavours, but risk levels of making
hwe kernels more like the future sounds safer.
>
> Thanks,
> Gerald
>
> On Tue, Oct 31, 2023 at 2:28 PM Andrea Righi <andrea.righi at canonical.com> wrote:
>>
>> On Tue, Oct 31, 2023 at 12:57:39AM +0800, Gerald Yang wrote:
>> > BugLink: https://bugs.launchpad.net/bugs/2023007
>> >
>> > [Impact]
>> >
>> > The lowlatency kernel in Ubuntu is specifically designed to prioritize
>> > high responsiveness, making it ideal for multimedia environments like
>> > DAWs and audio processing platforms, as well as soft real-time
>> > environments.
>> >
>> > With the introduction of a real-time kernel, it might be worth
>> > reconsidering the role of the lowlatency kernel and potentially
>> > including it as the default kernel in desktop images, focusing on its
>> > suitability for desktop-oriented usage.
>> >
>> > To achieve this, we can enable additional configuration settings and
>> > make it more focused for a low-latency and highly responsive desktop
>> > environment.
>> >
>> > Optionally (for the future) provide also an additional user-space
>> > package that would enable specific run-time kernel settings focused at
>> > certain preset workload profiles (e.g, web navigation, gaming, audio
>> > processing, etc.).
>> >
>> > [Test case]
>> >
>> > Use linux-lowlatency in a desktop environment and measure responsiveness
>> > of interactive applications.
>> >
>> > [Fix]
>> >
>> > Enable the following additional .config settings to make this kernel
>> > more suitable for a low-latency desktop kernel:
>> >
>> > - CONFIG_NO_HZ_FULL=y: enable access to "Full tickless mode" (shutdown
>> > clock tick when possible across all the enabled CPUs if they are
>> > either idle or running 1 task - reduce kernel jitter of running tasks
>> > due to the periodic clock tick, must be enabled at boot time passing
>> > `nohz_full=<cpu_list>`)
>> >
>> > - CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y: move RCU
>> > callbacks from softirq context to kthread context (reduce time spent
>> > in softirqs with preemption disabled to improve the overall system
>> > responsiveness, at the cost of introducing a potential performance
>> > penalty, because RCU callbacks are not processed by kernel threads)
>> >
>> > [Regression potential]
>> >
>> > Enabling all these settings can introduce a potential performance
>> > regression, but the kernel should result more responsive and make
>> > it more suitable for a desktop/multimedia/gaming/audio processing
>> > context.
>> >
>> > Signed-off-by: Gerald Yang <gerald.yang at canonical.com>
>>
>> It's worth mentioning that we already applied these configs to the
>> lowlatency kernels in mantic and in general I don't see any downside if
>> we apply this.
>>
>> However, I'm not sure if a change like this is proper SRU material, we
>> are basically saying "now lowlatency is a desktop kernel everywhere".
>>
>> Certain workloads may experience regressions (especially with the
>> RCU_NOCB_CPU), while NO_HZ_FULL is a lot safer (since it can be
>> enabled/disabled via boot options).
>>
>> We also need to keep in mind that Jammy will get these changes with the
>> lowlatency-hwe-6.5 kernel, so I'm not still 100% convinced to backport
>> this and apply it everywhere... opinions?
>>
>> -Andrea
>>
>> > ---
>> > debian.lowlatency/config/annotations | 13 +++++++++++++
>> > 1 file changed, 13 insertions(+)
>> >
>> > diff --git a/debian.lowlatency/config/annotations b/debian.lowlatency/config/annotations
>> > index 694ae26b6745..3e1556d60809 100644
>> > --- a/debian.lowlatency/config/annotations
>> > +++ b/debian.lowlatency/config/annotations
>> > @@ -15,12 +15,20 @@ CONFIG_HZ_250 note<'Override default HZ used i
>> > CONFIG_LATENCYTOP policy<{'amd64': 'y', 'arm64': 'y'}>
>> > CONFIG_LATENCYTOP note<'https://lists.ubuntu.com/archives/kernel-team/2014-July/045006.html, LP#1655986'>
>> >
>> > +CONFIG_NO_HZ_FULL policy<{'amd64': 'y', 'arm64': 'y'}>
>> > +CONFIG_NO_HZ_FULL note<'Enable access to "Full tickless mode" (LP: #2023007)'>
>> > +
>> > +CONFIG_NO_HZ_IDLE policy<{'amd64': 'n', 'arm64': 'n'}>
>> > +CONFIG_NO_HZ_IDLE note<'Disabled in favor of CONFIG_NO_HZ_FULL (LP: #2023007)'>
>> > +
>> > CONFIG_PREEMPT policy<{'amd64': 'y', 'arm64': 'y'}>
>> > CONFIG_PREEMPT note<'Enable fully preemptible kernel'>
>> >
>> > CONFIG_PREEMPT_VOLUNTARY policy<{'amd64': 'n', 'arm64': 'n'}>
>> > CONFIG_PREEMPT_VOLUNTARY note<'Disable voluntary preemption model'>
>> >
>> > +CONFIG_RCU_NOCB_CPU policy<{'amd64': 'y', 'arm64': 'y'}>
>> > +CONFIG_RCU_NOCB_CPU note<'Move RCU callbacks from softirq context to kthread context (LP: #2023007)'>
>> >
>> > # ---- Annotations without notes ----
>> >
>> > @@ -55,6 +63,8 @@ CONFIG_CEC_PIN policy<{'amd64': 'y', 'arm64': '
>> > CONFIG_CEC_PIN_ERROR_INJ policy<{'amd64': 'n', 'arm64': 'n'}>
>> > CONFIG_COMEDI_TESTS_EXAMPLE policy<{'amd64': 'n', 'arm64': 'm'}>
>> > CONFIG_COMEDI_TESTS_NI_ROUTES policy<{'amd64': 'n', 'arm64': 'm'}>
>> > +CONFIG_CONTEXT_TRACKING policy<{'amd64': 'y', 'arm64': 'y'}>
>> > +CONFIG_CONTEXT_TRACKING_FORCE policy<{'amd64': 'n', 'arm64': 'n'}>
>> > CONFIG_DEBUG_PREEMPT policy<{'amd64': 'n', 'arm64': 'n'}>
>> > CONFIG_HZ policy<{'amd64': '1000', 'arm64': '1000'}>
>> > CONFIG_INLINE_READ_LOCK policy<{'arm64': '-'}>
>> > @@ -89,4 +99,7 @@ CONFIG_PREEMPT_RCU policy<{'amd64': 'y', 'arm64': '
>> > CONFIG_PREEMPT_TRACER policy<{'amd64': 'n', 'arm64': 'n'}>
>> > CONFIG_TASKS_RCU policy<{'amd64': 'y', 'arm64': 'y'}>
>> > CONFIG_TEST_DIV64 policy<{'amd64': 'm', 'arm64': 'm'}>
>> > +CONFIG_TICK_CPU_ACCOUNTING policy<{'amd64': '-', 'arm64': '-'}>
>> > CONFIG_UNINLINE_SPIN_UNLOCK policy<{'amd64': 'y', 'arm64': 'y'}>
>> > +CONFIG_VIRT_CPU_ACCOUNTING policy<{'amd64': 'y', 'arm64': 'y'}>
>> > +CONFIG_VIRT_CPU_ACCOUNTING_GEN policy<{'amd64': 'y', 'arm64': 'y'}>
>> > --
>> > 2.25.1
>> >
>> >
>> > --
>> > kernel-team mailing list
>> > kernel-team at lists.ubuntu.com
>> > https://lists.ubuntu.com/mailman/listinfo/kernel-team
>
> --
> kernel-team mailing list
> kernel-team at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/kernel-team
--
okurrr,
Dimitri
More information about the kernel-team
mailing list