[SRU][Jammy][lowlatency] Ubuntu: [Config] lowlatency: enhance desktop responsiveness
Gerald Yang
gerald.yang at canonical.com
Wed Nov 1 05:02:10 UTC 2023
Thanks Andrea for the information
In this case, I'd like to confirm if this will be merged into 5.15
lowlatency-hwe kernel on Focal? not GA on Jammy
So I can provide some feedbacks to the customer
Thanks,
Gerald
On Tue, Oct 31, 2023 at 10:57 PM Andrea Righi <andrea.righi at canonical.com>
wrote:
> On Tue, Oct 31, 2023 at 09:48:55PM +0800, Gerald Yang wrote:
> > Hi Dimitri,
> >
> > Thanks for checking this!
> > And yes, that would be very helpful to enable it in focal lowlatency-hwe
> > and jammy lowlatency-hwe kernels,
> > at least users/customers could have a way to use this feature.
> >
> > Another question I'd like to ask is roughly when will 6.5 lowlatency-hwe
> > kernel be landed in Jammy? Should I also submit this to 6.2 lowlatency
> > kernel?
>
> We are working on hwe-6.5/lowlatency-hwe-6.5 right now, we are in the
> process of fixing the usual dkms breakage, reviewing test results,
> etc... at this point I think we need to wait after the sprint to have
> the 6.5's out in jammy.
>
> -Andrea
>
> >
> > Thanks,
> > Gerald
> >
> >
> > On Tue, Oct 31, 2023 at 8:36 PM Dimitri John Ledkov <
> > dimitri.ledkov at canonical.com> wrote:
> >
> > > Hi,
> > >
> > > On Tue, 31 Oct 2023 at 10:54, Gerald Yang <gerald.yang at canonical.com>
> > > wrote:
> > > >
> > > > Hi Andrea,
> > > >
> > > > Thanks for the feedback! please let me explain this in more detail
> > > >
> > > > I sent out this SRU because some of our customers need to use
> NO_HZ_FULL.
> > > > In their scenario, the CPU time is pretty sensitive, and must avoid
> as
> > > much noise as possible to keep
> > > > most CPUs doing only one single task.
> > > > I found some CPU steal time happen on their VM because RCU callback
> > > softirqs are still triggered on CPUs
> > > > with isolcpus, nohz_full and rcu_nocbs configured, and lead me to
> find
> > > NO_HZ_FULL is not built into generic kernel.
> > > >
> > > > I think the use case for NO_HZ_FULL should be different than generic
> > > kernel, e.g. performance vs low latency
> > > >
> > > > On generic kernel, we also did some tests with NO_HZ_FULL built-in
> here:
> > > > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1919154
> > > >
> > > > The results show performance degradation on AMD EPYC series machines
> only
> > > > when NO_HZ_FULL is built-in but not enabled, but this doesn't happen
> on
> > > Intel and arm64 machines.
> > > >
> > > > Because one of the customer is still on Focal, if it's possible to
> merge
> > > this into 5.15 kernel, it would be more convenient
> > > > for them to use HWE lowlatency kernel.
> > > > And if there are users need to use NO_HZ_FULL on Focal, lowlatency
> > > kernel could be a proper choice for them,
> > > > because so far there is no kernel with NO_HZ_FULL built-in on Focal.
> > >
> > > I am ok having this enabled in focal:lowlatency-hwe-5.15 and
> > > jammy:lowlatency-hwe-6.5
> > >
> > > but not change anything in focal:lowlatency (5.4 / ga)
> > > jammy:lowlatency (5.15 / ga).
> > >
> > > Would that help you? As an exception to our policy. I really feel
> > > uncomfortable changing default GA flavours, but risk levels of making
> > > hwe kernels more like the future sounds safer.
> > >
> > > >
> > > > Thanks,
> > > > Gerald
> > > >
> > > > On Tue, Oct 31, 2023 at 2:28 PM Andrea Righi <
> andrea.righi at canonical.com>
> > > wrote:
> > > >>
> > > >> On Tue, Oct 31, 2023 at 12:57:39AM +0800, Gerald Yang wrote:
> > > >> > BugLink: https://bugs.launchpad.net/bugs/2023007
> > > >> >
> > > >> > [Impact]
> > > >> >
> > > >> > The lowlatency kernel in Ubuntu is specifically designed to
> prioritize
> > > >> > high responsiveness, making it ideal for multimedia environments
> like
> > > >> > DAWs and audio processing platforms, as well as soft real-time
> > > >> > environments.
> > > >> >
> > > >> > With the introduction of a real-time kernel, it might be worth
> > > >> > reconsidering the role of the lowlatency kernel and potentially
> > > >> > including it as the default kernel in desktop images, focusing on
> its
> > > >> > suitability for desktop-oriented usage.
> > > >> >
> > > >> > To achieve this, we can enable additional configuration settings
> and
> > > >> > make it more focused for a low-latency and highly responsive
> desktop
> > > >> > environment.
> > > >> >
> > > >> > Optionally (for the future) provide also an additional user-space
> > > >> > package that would enable specific run-time kernel settings
> focused at
> > > >> > certain preset workload profiles (e.g, web navigation, gaming,
> audio
> > > >> > processing, etc.).
> > > >> >
> > > >> > [Test case]
> > > >> >
> > > >> > Use linux-lowlatency in a desktop environment and measure
> > > responsiveness
> > > >> > of interactive applications.
> > > >> >
> > > >> > [Fix]
> > > >> >
> > > >> > Enable the following additional .config settings to make this
> kernel
> > > >> > more suitable for a low-latency desktop kernel:
> > > >> >
> > > >> > - CONFIG_NO_HZ_FULL=y: enable access to "Full tickless mode"
> > > (shutdown
> > > >> > clock tick when possible across all the enabled CPUs if they
> are
> > > >> > either idle or running 1 task - reduce kernel jitter of running
> > > tasks
> > > >> > due to the periodic clock tick, must be enabled at boot time
> > > passing
> > > >> > `nohz_full=<cpu_list>`)
> > > >> >
> > > >> > - CONFIG_RCU_NOCB_CPU=y, CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y: move
> RCU
> > > >> > callbacks from softirq context to kthread context (reduce time
> > > spent
> > > >> > in softirqs with preemption disabled to improve the overall
> system
> > > >> > responsiveness, at the cost of introducing a potential
> performance
> > > >> > penalty, because RCU callbacks are not processed by kernel
> threads)
> > > >> >
> > > >> > [Regression potential]
> > > >> >
> > > >> > Enabling all these settings can introduce a potential
> performance
> > > >> > regression, but the kernel should result more responsive and
> make
> > > >> > it more suitable for a desktop/multimedia/gaming/audio
> processing
> > > >> > context.
> > > >> >
> > > >> > Signed-off-by: Gerald Yang <gerald.yang at canonical.com>
> > > >>
> > > >> It's worth mentioning that we already applied these configs to the
> > > >> lowlatency kernels in mantic and in general I don't see any
> downside if
> > > >> we apply this.
> > > >>
> > > >> However, I'm not sure if a change like this is proper SRU material,
> we
> > > >> are basically saying "now lowlatency is a desktop kernel
> everywhere".
> > > >>
> > > >> Certain workloads may experience regressions (especially with the
> > > >> RCU_NOCB_CPU), while NO_HZ_FULL is a lot safer (since it can be
> > > >> enabled/disabled via boot options).
> > > >>
> > > >> We also need to keep in mind that Jammy will get these changes with
> the
> > > >> lowlatency-hwe-6.5 kernel, so I'm not still 100% convinced to
> backport
> > > >> this and apply it everywhere... opinions?
> > > >>
> > > >> -Andrea
> > > >>
> > > >> > ---
> > > >> > debian.lowlatency/config/annotations | 13 +++++++++++++
> > > >> > 1 file changed, 13 insertions(+)
> > > >> >
> > > >> > diff --git a/debian.lowlatency/config/annotations
> > > b/debian.lowlatency/config/annotations
> > > >> > index 694ae26b6745..3e1556d60809 100644
> > > >> > --- a/debian.lowlatency/config/annotations
> > > >> > +++ b/debian.lowlatency/config/annotations
> > > >> > @@ -15,12 +15,20 @@ CONFIG_HZ_250
> > > note<'Override default HZ used i
> > > >> > CONFIG_LATENCYTOP policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > CONFIG_LATENCYTOP note<'
> > > https://lists.ubuntu.com/archives/kernel-team/2014-July/045006.html,
> > > LP#1655986'>
> > > >> >
> > > >> > +CONFIG_NO_HZ_FULL policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > +CONFIG_NO_HZ_FULL note<'Enable
> access
> > > to "Full tickless mode" (LP: #2023007)'>
> > > >> > +
> > > >> > +CONFIG_NO_HZ_IDLE policy<{'amd64':
> > > 'n', 'arm64': 'n'}>
> > > >> > +CONFIG_NO_HZ_IDLE note<'Disabled in
> > > favor of CONFIG_NO_HZ_FULL (LP: #2023007)'>
> > > >> > +
> > > >> > CONFIG_PREEMPT policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > CONFIG_PREEMPT note<'Enable
> fully
> > > preemptible kernel'>
> > > >> >
> > > >> > CONFIG_PREEMPT_VOLUNTARY policy<{'amd64':
> > > 'n', 'arm64': 'n'}>
> > > >> > CONFIG_PREEMPT_VOLUNTARY note<'Disable
> > > voluntary preemption model'>
> > > >> >
> > > >> > +CONFIG_RCU_NOCB_CPU policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > +CONFIG_RCU_NOCB_CPU note<'Move RCU
> > > callbacks from softirq context to kthread context (LP: #2023007)'>
> > > >> >
> > > >> > # ---- Annotations without notes ----
> > > >> >
> > > >> > @@ -55,6 +63,8 @@ CONFIG_CEC_PIN
> > > policy<{'amd64': 'y', 'arm64': '
> > > >> > CONFIG_CEC_PIN_ERROR_INJ policy<{'amd64':
> > > 'n', 'arm64': 'n'}>
> > > >> > CONFIG_COMEDI_TESTS_EXAMPLE policy<{'amd64':
> > > 'n', 'arm64': 'm'}>
> > > >> > CONFIG_COMEDI_TESTS_NI_ROUTES policy<{'amd64':
> > > 'n', 'arm64': 'm'}>
> > > >> > +CONFIG_CONTEXT_TRACKING policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > +CONFIG_CONTEXT_TRACKING_FORCE policy<{'amd64':
> > > 'n', 'arm64': 'n'}>
> > > >> > CONFIG_DEBUG_PREEMPT policy<{'amd64':
> > > 'n', 'arm64': 'n'}>
> > > >> > CONFIG_HZ policy<{'amd64':
> > > '1000', 'arm64': '1000'}>
> > > >> > CONFIG_INLINE_READ_LOCK policy<{'arm64':
> > > '-'}>
> > > >> > @@ -89,4 +99,7 @@ CONFIG_PREEMPT_RCU
> > > policy<{'amd64': 'y', 'arm64': '
> > > >> > CONFIG_PREEMPT_TRACER policy<{'amd64':
> > > 'n', 'arm64': 'n'}>
> > > >> > CONFIG_TASKS_RCU policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > CONFIG_TEST_DIV64 policy<{'amd64':
> > > 'm', 'arm64': 'm'}>
> > > >> > +CONFIG_TICK_CPU_ACCOUNTING policy<{'amd64':
> > > '-', 'arm64': '-'}>
> > > >> > CONFIG_UNINLINE_SPIN_UNLOCK policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > +CONFIG_VIRT_CPU_ACCOUNTING policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > +CONFIG_VIRT_CPU_ACCOUNTING_GEN policy<{'amd64':
> > > 'y', 'arm64': 'y'}>
> > > >> > --
> > > >> > 2.25.1
> > > >> >
> > > >> >
> > > >> > --
> > > >> > kernel-team mailing list
> > > >> > kernel-team at lists.ubuntu.com
> > > >> > https://lists.ubuntu.com/mailman/listinfo/kernel-team
> > > >
> > > > --
> > > > kernel-team mailing list
> > > > kernel-team at lists.ubuntu.com
> > > > https://lists.ubuntu.com/mailman/listinfo/kernel-team
> > >
> > >
> > >
> > > --
> > > okurrr,
> > >
> > > Dimitri
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/kernel-team/attachments/20231101/7f5d6492/attachment-0001.html>
More information about the kernel-team
mailing list