Public bug reported:
In a dual socket athlon 8 core server we encouyntered performance
problems that were due to assymetrical IRQ distribution between the two
8-core CPUs. After downgrade to package
irqbalance_0.56-1ubuntu4_amd64.deb, the IRQs were evenly distributed and
performance was recovered.
Before downgrade:
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15
0: 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge timer
1: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge i8042
4: 10 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge serial
7: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge
8: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge rtc0
9: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi acpi
12: 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge i8042
14: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge pata_atiixp
15: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge pata_atiixp
16: 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ohci_hcd:usb3, ohci_hcd:usb4
17: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb1
18: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ohci_hcd:usb5, ohci_hcd:usb6, ohci_hcd:usb7
19: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb2
22: 30359 2491 13116 73925 50969 82236 138803 194118 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ahci
72: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME
73: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME
74: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME
75: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
76: 20497 9421 59248 73366 60949 58226 50470 43998 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-0
77: 12102 10776 52868 96605 80078 85429 98212 131783 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-1
78: 7788 4844 49547 55632 53024 42477 35030 20804 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-2
79: 2509 2946 19584 45371 40691 46416 52334 54775 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-3
80: 735444 498332 6668615 9257436 6991144 7155873 7072726 8564589 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-4
81: 6453 4472 35521 55873 44156 40590 35993 37210 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-5
82: 405422 409383 3841245 7702981 6360084 6899018 7550541 4756358 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-6
83: 3367 3163 27441 41470 33758 34822 45838 71412 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-7
84: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1
85: 103975 80217 602817 2353875 1763121 2313317 2533812 1971064 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-0
86: 108258 70762 662005 932886 749726 748751 974490 1195653 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-1
87: 1023423 800978 6347718 10701238 6810298 6432991 5088493 7320927 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-2
88: 84034 93574 738634 1290477 935778 964032 868243 426459 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-3
89: 13550 12789 281565 830762 659694 736436 870863 1142131 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-4
90: 125781 84031 1315842 2181539 1461851 1514477 1278920 857619 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-5
91: 122583 114687 597257 1330975 909789 1129461 1378391 1608125 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-6
92: 331017 190173 1185993 1408777 1024668 967490 918780 1134918 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-7
NMI: 6355 5100 4912 5173 4602 4315 4202 4189 3588 2670 2360 2593 2809 2283 2055 2184 Non-maskable interrupts
LOC: 56601130 62244014 51232014 52536246 51292581 50501563 50051589 50787985 19000443 15806014 14106035 15711434 19935130 18430760 17217088 18142808 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 6355 5100 4912 5173 4602 4315 4202 4189 3588 2670 2360 2593 2809 2283 2055 2184 Performance monitoring interrupts
IWI: 514478 443237 759104 880318 1551795 1540520 1625489 1663108 341814 297451 330367 338492 177561 140676 170250 169506 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 55930926 43511076 41888811 39688213 48807626 48413461 48049191 48198466 10975401 11111421 10781619 10522688 14315623 14585058 14057389 13921442 Rescheduling interrupts
CAL: 2090155 1547436 1484451 1445158 1807163 1488941 1443830 1452754 2922638 2021948 1814708 1986602 1728780 1195234 1131957 1176867 Function call interrupts
TLB: 1392134600 1165552202 1094433212 1077303086 1142578704 1025762620 1004245299 999232945 714468110 432664242 386342715 415549247 1307674845 1077891023 1041347292 1058767706 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 1692 Machine check polls
ERR: 1
MIS: 0
After downgrade:
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 CPU13 CPU14 CPU15
0: 51 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge timer
1: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge i8042
4: 10 0 0 0 23 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge serial
7: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge
8: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge rtc0
9: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi acpi
12: 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge i8042
14: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge pata_atiixp
15: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-edge pata_atiixp
16: 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ohci_hcd:usb3, ohci_hcd:usb4
17: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb1
18: 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ohci_hcd:usb5, ohci_hcd:usb6, ohci_hcd:usb7
19: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ehci_hcd:usb2
22: 35601 2491 13116 73925 50978 82236 138803 194118 0 0 0 0 0 0 0 0 IO-APIC-fasteoi ahci
72: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME
73: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME
74: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge PCIe PME
75: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0
76: 20497 9425 59248 73366 60949 58226 50470 43998 0 0 0 0 0 3503 0 0 PCI-MSI-edge eth0-TxRx-0
77: 12102 10776 52874 96605 80089 90854 98212 131783 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-1
78: 10381 4844 49547 55635 53024 42477 35030 20804 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-2
79: 2509 2946 19584 45371 40694 46416 52334 54775 0 0 0 0 2519 0 0 0 PCI-MSI-edge eth0-TxRx-3
80: 735444 498399 6668615 9257496 7255553 7155873 7072726 8564589 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-4
81: 6458 4472 35524 58395 44156 40590 35993 37210 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-5
82: 559452 409383 3841245 7703024 6360084 6899018 7550541 4756358 0 0 0 0 0 0 0 0 PCI-MSI-edge eth0-TxRx-6
83: 3367 3163 27441 41470 33761 34822 45838 71412 0 0 0 2569 0 0 0 0 PCI-MSI-edge eth0-TxRx-7
84: 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1
85: 234249 80217 602871 2353875 1763121 2313317 2533812 1971064 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-0
86: 156917 70762 662005 932966 749726 748751 974490 1195653 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-1
87: 1023423 800978 6535122 10701238 6810339 6432991 5088493 7320927 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-2
88: 84034 93598 738634 1290477 935778 964032 868243 426459 0 0 47855 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-3
89: 13550 12789 281592 830762 659694 736436 870863 1142131 0 62558 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-4
90: 183521 84031 1315842 2181588 1461851 1514477 1278920 857619 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-5
91: 122583 114687 597257 1330975 909892 1129461 1378391 1608125 68590 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-6
92: 331017 238476 1185993 1408777 1024668 967490 918780 1134918 0 0 0 0 0 0 0 0 PCI-MSI-edge eth1-TxRx-7
NMI: 6429 5165 4976 5232 4628 4329 4213 4202 3644 2729 2420 2647 2849 2317 2085 2226 Non-maskable interrupts
LOC: 57080551 62803571 51672099 52991355 51434853 50601643 50128829 50898585 19473681 16126881 14411864 15996454 20147782 18624202 17394379 18365340 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 6429 5165 4976 5232 4628 4329 4213 4202 3644 2729 2420 2647 2849 2317 2085 2226 Performance monitoring interrupts
IWI: 527429 452351 766685 886076 1554468 1542609 1627637 1665057 351739 309184 338788 353867 179640 142506 171575 170925 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 56301286 43787816 42143044 39911714 48892924 48520728 48144566 48289543 11129864 11301123 10971749 10719671 14397328 14670469 14138365 14004762 Rescheduling interrupts
CAL: 2135396 1577596 1514415 1469382 1830108 1497649 1451176 1458539 2972730 2073099 1863239 2036198 1747059 1207298 1142404 1190299 Function call interrupts
TLB: 1414854795 1182782091 1111896700 1093506690 1150104685 1029721908 1006412535 1001832855 728102533 442975795 397908923 426585818 1322725685 1092037288 1053780203 1071544152 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 1708 Machine check polls
ERR: 1
MIS: 0
** Affects: irqbalance (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to irqbalance in Ubuntu.
https://bugs.launchpad.net/bugs/1348157
Title:
IRQBALANCE in 14.04 only works for CPU in socket0
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1348157/+subscriptions