[Bug 2054872] Re: Cannot change IRQ 70 affinity: Input/output error
Robert Malz
2054872 at bugs.launchpad.net
Wed Mar 20 14:28:31 UTC 2024
irqbalance 1.9.3-2ubuntu4 verified on 6.8.0-11-generic
Issue no longer reproduces.
--
You received this bug notification because you are a member of Ubuntu
Sponsors, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/2054872
Title:
Cannot change IRQ 70 affinity: Input/output error
Status in irqbalance package in Ubuntu:
Fix Committed
Status in irqbalance source package in Noble:
Fix Committed
Bug description:
[ Impact ]
* irqbalance during runtime changes smp_affinity by writing to /proc/irq/<id>/smp_affinity file.
Previous upstream implementation was closing file just after writing to it which could cause random IO erorrs.
* Due to this issue, irqbalance could mark IRQs as unmigratable
* Issue is visible in v1.9.3 irqbalance packges because additional check logic has been added (470a64b190628574c28a266bdcf8960291463191)
* Before v1.9.3 irqbalancer could still fail on fclose but that was not leading to marking IRQ as unmigratable
[ Test Plan ]
* Install and run irqbalance in v1.9.3
* Whenever irqbalance will decide to change IRQ affinity following error can occur:
Mar 14 11:59:55 pre-noble irqbalance[1536]: Cannot change IRQ 27 affinity: Input/output error
Mar 14 11:59:55 pre-noble irqbalance[1536]: IRQ 27 affinity is now unmanaged
[ Where problems could occur ]
* Issue is fixed by adding fflush before closing the file to make sure write operation is finished before closing
* During local tests I have not observed any issues after adding file flushing
[ Other Info ]
* Fix for the issue is already merged into the upstream: https://github.com/Irqbalance/irqbalance/pull/302
* Original description of the case below:
Hello Ubuntu Team,
I notice this today when using Ubuntu 24.04 Noble.
These messages appear quite often in the systemd journal.
My kernel is: 6.8.0-rc4-realtime-rt4
# journalctl -b --no-pager --no-hostname | grep irqbalance
Feb 24 13:21:02 systemd[1]: Started irqbalance.service - irqbalance daemon.
Feb 24 13:21:02 (qbalance)[1209]: irqbalance.service: Referenced but unset environment variable evaluates to an empty string: IRQBALANCE_ARGS
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 73 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 73 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 63 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 63 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 71 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 71 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 61 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 61 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 68 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 68 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 58 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 58 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 66 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 66 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 64 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 64 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 72 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 72 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 62 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 62 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 70 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 70 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 60 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 60 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 69 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 69 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 59 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 59 affinity is now unmanaged
Feb 24 13:21:12 irqbalance[1209]: Cannot change IRQ 67 affinity: Input/output error
Feb 24 13:21:12 irqbalance[1209]: IRQ 67 affinity is now unmanaged
Feb 24 13:22:22 irqbalance[1209]: Cannot change IRQ 65 affinity: Input/output error
Feb 24 13:22:22 irqbalance[1209]: IRQ 65 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 73 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 73 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 63 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 63 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 61 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 61 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 68 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 68 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 66 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 66 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 64 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 64 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 72 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 72 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 62 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 62 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 80 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 80 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 60 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 60 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 79 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 79 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 69 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 69 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 67 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 67 affinity is now unmanaged
Feb 24 13:40:36 irqbalance[1209]: Cannot change IRQ 65 affinity: Input/output error
Feb 24 13:40:36 irqbalance[1209]: IRQ 65 affinity is now unmanaged
Feb 24 13:40:46 irqbalance[1209]: Cannot change IRQ 71 affinity: Input/output error
Feb 24 13:40:46 irqbalance[1209]: IRQ 71 affinity is now unmanaged
Feb 24 14:23:06 irqbalance[1209]: Cannot change IRQ 70 affinity: Input/output error
Feb 24 14:23:06 irqbalance[1209]: IRQ 70 affinity is now unmanaged
My irq's for this system is:
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15
0: 116 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
6: 0 0 5405959 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-IO-APIC 6-edge AMDI0010:03
7: 0 0 0 0 0 0 299947 0 0 0 0 0 0 0 0 0 IR-IO-APIC 7-fasteoi pinctrl_amd
8: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-IO-APIC 8-edge rtc0
9: 0 41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
25: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IOMMU-MSI 368-edge AMD-Vi0-Evt
26: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IOMMU-MSI 376-edge AMD-Vi0-PPR
27: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IOMMU-MSI 384-edge AMD-Vi0-GA
28: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 amd_gpio 0 ACPI:Event
29: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 amd_gpio 44 ACPI:Event
30: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 amd_gpio 58 ACPI:Event
31: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 amd_gpio 59 ACPI:Event
32: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 amd_gpio 18 ACPI:Event
33: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-0000:00:01.1 0-edge PCIe PME, pciehp
34: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-0000:00:02.2 0-edge PCIe PME, pciehp
35: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-0000:00:02.4 0-edge PCIe PME
36: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-0000:00:08.1 0-edge PCIe PME
37: 0 0 0 0 0 0 299946 0 0 0 0 0 0 0 0 0 amd_gpio 9 ELAN1201:00
39: 0 0 0 0 0 0 0 0 0 0 23388 0 95 0 0 0 IR-PCI-MSIX-0000:04:00.3 0-edge xhci_hcd
48: 0 0 0 0 0 0 0 0 0 0 0 0 0 8144 787089 0 IR-PCI-MSIX-0000:04:00.4 0-edge xhci_hcd
57: 0 0 0 0 0 0 0 1302 0 0 0 0 0 0 0 0 IR-PCI-MSI-0000:04:00.6 0-edge snd_hda_intel:card2
58: 0 192 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSI-0000:04:00.1 0-edge snd_hda_intel:card1
59: 0 0 0 101 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 0-edge nvme0q0
60: 22957 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 1-edge nvme0q1
61: 0 20716 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 2-edge nvme0q2
62: 0 0 24443 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 3-edge nvme0q3
63: 0 0 0 25781 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 4-edge nvme0q4
64: 0 0 0 0 26558 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 5-edge nvme0q5
65: 0 0 0 0 0 21966 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 6-edge nvme0q6
66: 0 0 0 0 0 0 23507 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 7-edge nvme0q7
67: 0 0 0 0 0 0 0 19535 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 8-edge nvme0q8
68: 0 0 0 0 0 0 0 0 27158 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 9-edge nvme0q9
69: 0 0 0 0 0 0 0 0 0 24531 0 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 10-edge nvme0q10
70: 0 0 0 0 0 0 0 0 0 0 23135 0 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 11-edge nvme0q11
71: 0 0 0 0 0 0 0 0 0 0 0 21913 0 0 0 0 IR-PCI-MSIX-0000:03:00.0 12-edge nvme0q12
72: 0 0 0 0 0 0 0 0 0 0 0 0 25956 0 0 0 IR-PCI-MSIX-0000:03:00.0 13-edge nvme0q13
73: 0 0 0 0 0 0 0 0 0 0 0 0 0 21804 0 0 IR-PCI-MSIX-0000:03:00.0 14-edge nvme0q14
75: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IR-PCI-MSIX-0000:04:00.2 0-edge psp-1
77: 0 0 182 0 0 0 0 0 0 0 0 0 0 128 0 0 IR-IO-APIC 1-fasteoi snd_hda_intel:card0
79: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 25693 0 IR-PCI-MSIX-0000:03:00.0 15-edge nvme0q15
80: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 22773 IR-PCI-MSIX-0000:03:00.0 16-edge nvme0q16
82: 0 0 0 18139 0 0 628301 0 236532 266 0 467208 0 0 0 0 IR-PCI-MSI-0000:02:00.0 0-edge mt7921e
83: 0 0 0 338579 1458 0 0 0 0 1118391 0 0 0 0 0 0 IR-PCI-MSIX-0000:04:00.0 0-edge amdgpu
NMI: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-maskable interrupts
LOC: 1413160 1341033 1515330 1346117 1486268 1353018 1498988 1334907 1475044 1389584 1499673 1360128 1503094 1356648 1481898 1338550 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Performance monitoring interrupts
IWI: 52036 57306 36597 61733 53803 79504 67600 65707 99768 106673 39212 50191 49983 52515 24180 36440 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 1135734 961961 1102398 869315 1245660 933642 1169006 917816 1359995 763672 1220046 820725 1266840 869766 1094597 834027 Rescheduling interrupts
CAL: 1063814 373555 207657 168710 192237 189824 188438 174955 233311 126152 160532 145524 171051 153168 119414 134045 Function call interrupts
TLB: 94988 98433 115358 96450 118525 98758 124773 94916 123529 90824 119527 95110 120917 98504 107344 96577 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 Machine check polls
ERR: 1
MIS: 0
PIN: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Posted-interrupt notification event
NPI: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Nested posted-interrupt event
PIW: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/2054872/+subscriptions
More information about the Ubuntu-sponsors
mailing list