[Bug 1053057] Re: Client queues up lshw calls if talking to old server
Andreas Hasenack
andreas at canonical.com
Fri Sep 21 14:33:30 UTC 2012
** Description changed:
- If a 12.04.X or 12.05 client is talking to a server which does not
- support the hardware-info messages (like LDS 11.07.X), it will queue
- calls to lshw in memory.
+ This is an SRU request for landscape-client in lucid, natty, oneiric and
+ precise.
- If the server the client is talking to is then upgraded (to LDS 12.09,
- for example), all those lshw calls will happen at once, creating a
- storm of one lshw process per day it was talking to the old server. This
- could be just a few processes, or hundreds, depending on how long
+ Landscpae is *NOT* invoking the SRU exception this time
+ (https://wiki.ubuntu.com/StableReleaseUpdates#Landscape).
+
+
+ Explanation
+ ===========
+ If a 12.04.X or 12.05 client is talking to a server which does not support the
+ hardware-info messages (like LDS 11.07.X), it will queue calls to lshw in
+ memory. That in itself is not a big problem per se.
+
+ But if the server the client is talking to is then upgraded to a version that
+ does support hardware-info (like LDS 12.09), all those lshw calls will happen
+ at once, creating a storm of one lshw process per day it was talking to the old
+ server. This could be just a few processes, or hundreds, depending on how long
landscape-client was running and talking to the old server.
- A restart will wipe that queue and is the recommended workaround:
- restart the clients just before upgrading LDS.
+ A restart will wipe that queue and is the recommended workaround: restart the
+ clients just before upgrading LDS.
+
+
+ Test case
+ =========
+ You will need one precise or lucid machine and access to LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
+
+ - deploy LDS 11.07.X
+ - install landscape-client 12.05 (can be on the same machine or another one with
+ network access to the LDS one)
+ - tweak the client code to invoke the hardware-info plugin every minute instead of
+ once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
+ change run_interval to 60
+ - copy /usr/bin/lshw to /usr/bin/lshw.orig and change /usr/bin/lshw to this script:
+ """
+ #!/bin/bash
+
+ date >> /tmp/lshw.log
+ /usr/bin/lshw.orig $@
+ """
+ - run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log
+ is updated
+ - register the client with LDS. AFTER THIS, DO NOT RESTART THE CLIENT AGAIN FOR ANY REASON.
+ - in this scenario, lshw is not being called because the server does not support that
+ information, but the client is accumulating the calls in memory.
+ - leave the client talking to the server for about 10min. Create a simple script activity
+ to make sure it's working (like, send "ifconfig" to the client)
+ - after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update
+ followed by apt-get dist-upgrade. The upgrade will take a few minutes
+ - keep tailing -f /tmp/lshw.log and /var/log/landscape/broker.log on the client
+ - the broker log will get errors while LDS is down upgrading, that's normal
+ - once the client is able to talk to the server again, it will notice the
+ server version switch and log something like this:
+ """
+ 2012-09-19 13:46:25,288 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message
+ network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-re
+ quired-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucaly
+ ptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
+ """
+ - this is the moment where the *buggy* client will spawn a lot of lshw calls, like this:
+ """
+ root at ubuntu:/var/log/landscape# tail -F /tmp/lshw-run.log
+ tail: cannot open `/tmp/lshw-run.log' for reading: No such file or directory
+ tail: `/tmp/lshw-run.log' has appeared; following end of new file
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:43 UTC 2012
+ Wed Sep 19 19:41:44 UTC 2012
+ Wed Sep 19 19:41:44 UTC 2012
+ Wed Sep 19 19:41:44 UTC 2012
+ Wed Sep 19 19:41:44 UTC 2012
+ Wed Sep 19 19:41:44 UTC 2012
+ Wed Sep 19 19:41:44 UTC 2012
+ Wed Sep 19 19:41:45 UTC 2012
+ Wed Sep 19 19:41:45 UTC 2012
+ """
+ - the *fixed* client will be more well behaved, with no such storm of lshw calls:
+ """
+ root at ubuntu:~# tail -F /tmp/lshw-run.log
+ Wed Sep 19 20:19:23 UTC 2012
+ Wed Sep 19 20:20:21 UTC 2012
+ and one per minute from here on
+ """
+
+
+ Regression potential
+ ====================
+ The fix itself is unit tested, and I tested it in all ubuntu releases with the
+ attached debdiff and the test case above, confirming that the issue is gone.
+ The patch is very specific and affects hardware reporting only.
** Description changed:
This is an SRU request for landscape-client in lucid, natty, oneiric and
precise.
- Landscpae is *NOT* invoking the SRU exception this time
+ Landscape is *NOT* invoking the SRU exception this time
(https://wiki.ubuntu.com/StableReleaseUpdates#Landscape).
-
Explanation
===========
If a 12.04.X or 12.05 client is talking to a server which does not support the
hardware-info messages (like LDS 11.07.X), it will queue calls to lshw in
memory. That in itself is not a big problem per se.
But if the server the client is talking to is then upgraded to a version that
does support hardware-info (like LDS 12.09), all those lshw calls will happen
at once, creating a storm of one lshw process per day it was talking to the old
server. This could be just a few processes, or hundreds, depending on how long
landscape-client was running and talking to the old server.
A restart will wipe that queue and is the recommended workaround: restart the
clients just before upgrading LDS.
-
Test case
=========
You will need one precise or lucid machine and access to LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
- deploy LDS 11.07.X
- install landscape-client 12.05 (can be on the same machine or another one with
- network access to the LDS one)
+ network access to the LDS one)
- tweak the client code to invoke the hardware-info plugin every minute instead of
- once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
- change run_interval to 60
+ once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
+ change run_interval to 60
- copy /usr/bin/lshw to /usr/bin/lshw.orig and change /usr/bin/lshw to this script:
"""
#!/bin/bash
date >> /tmp/lshw.log
/usr/bin/lshw.orig $@
"""
- run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log
- is updated
+ is updated
- register the client with LDS. AFTER THIS, DO NOT RESTART THE CLIENT AGAIN FOR ANY REASON.
- in this scenario, lshw is not being called because the server does not support that
- information, but the client is accumulating the calls in memory.
+ information, but the client is accumulating the calls in memory.
- leave the client talking to the server for about 10min. Create a simple script activity
- to make sure it's working (like, send "ifconfig" to the client)
+ to make sure it's working (like, send "ifconfig" to the client)
- after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update
- followed by apt-get dist-upgrade. The upgrade will take a few minutes
+ followed by apt-get dist-upgrade. The upgrade will take a few minutes
- keep tailing -f /tmp/lshw.log and /var/log/landscape/broker.log on the client
- the broker log will get errors while LDS is down upgrading, that's normal
- once the client is able to talk to the server again, it will notice the
- server version switch and log something like this:
+ server version switch and log something like this:
"""
2012-09-19 13:46:25,288 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message
network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-re
quired-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucaly
ptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
"""
- this is the moment where the *buggy* client will spawn a lot of lshw calls, like this:
"""
root at ubuntu:/var/log/landscape# tail -F /tmp/lshw-run.log
tail: cannot open `/tmp/lshw-run.log' for reading: No such file or directory
tail: `/tmp/lshw-run.log' has appeared; following end of new file
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
"""
- the *fixed* client will be more well behaved, with no such storm of lshw calls:
"""
root at ubuntu:~# tail -F /tmp/lshw-run.log
Wed Sep 19 20:19:23 UTC 2012
Wed Sep 19 20:20:21 UTC 2012
and one per minute from here on
"""
-
Regression potential
====================
The fix itself is unit tested, and I tested it in all ubuntu releases with the
attached debdiff and the test case above, confirming that the issue is gone.
The patch is very specific and affects hardware reporting only.
** Description changed:
This is an SRU request for landscape-client in lucid, natty, oneiric and
precise.
Landscape is *NOT* invoking the SRU exception this time
(https://wiki.ubuntu.com/StableReleaseUpdates#Landscape).
Explanation
===========
If a 12.04.X or 12.05 client is talking to a server which does not support the
- hardware-info messages (like LDS 11.07.X), it will queue calls to lshw in
+ hardware-info messages (like LDS 11.07.X), it will queue calls to /usr/bin/lshw in
memory. That in itself is not a big problem per se.
But if the server the client is talking to is then upgraded to a version that
does support hardware-info (like LDS 12.09), all those lshw calls will happen
at once, creating a storm of one lshw process per day it was talking to the old
server. This could be just a few processes, or hundreds, depending on how long
landscape-client was running and talking to the old server.
A restart will wipe that queue and is the recommended workaround: restart the
clients just before upgrading LDS.
Test case
=========
You will need one precise or lucid machine and access to LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
- deploy LDS 11.07.X
- install landscape-client 12.05 (can be on the same machine or another one with
network access to the LDS one)
- tweak the client code to invoke the hardware-info plugin every minute instead of
once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
change run_interval to 60
- copy /usr/bin/lshw to /usr/bin/lshw.orig and change /usr/bin/lshw to this script:
"""
#!/bin/bash
date >> /tmp/lshw.log
/usr/bin/lshw.orig $@
"""
- run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log
is updated
- register the client with LDS. AFTER THIS, DO NOT RESTART THE CLIENT AGAIN FOR ANY REASON.
- in this scenario, lshw is not being called because the server does not support that
information, but the client is accumulating the calls in memory.
- leave the client talking to the server for about 10min. Create a simple script activity
to make sure it's working (like, send "ifconfig" to the client)
- after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update
followed by apt-get dist-upgrade. The upgrade will take a few minutes
- keep tailing -f /tmp/lshw.log and /var/log/landscape/broker.log on the client
- the broker log will get errors while LDS is down upgrading, that's normal
- once the client is able to talk to the server again, it will notice the
server version switch and log something like this:
"""
2012-09-19 13:46:25,288 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message
network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-re
quired-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucaly
ptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
"""
- this is the moment where the *buggy* client will spawn a lot of lshw calls, like this:
"""
root at ubuntu:/var/log/landscape# tail -F /tmp/lshw-run.log
tail: cannot open `/tmp/lshw-run.log' for reading: No such file or directory
tail: `/tmp/lshw-run.log' has appeared; following end of new file
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
"""
- the *fixed* client will be more well behaved, with no such storm of lshw calls:
"""
root at ubuntu:~# tail -F /tmp/lshw-run.log
Wed Sep 19 20:19:23 UTC 2012
Wed Sep 19 20:20:21 UTC 2012
and one per minute from here on
"""
Regression potential
====================
The fix itself is unit tested, and I tested it in all ubuntu releases with the
attached debdiff and the test case above, confirming that the issue is gone.
The patch is very specific and affects hardware reporting only.
** Description changed:
This is an SRU request for landscape-client in lucid, natty, oneiric and
precise.
Landscape is *NOT* invoking the SRU exception this time
(https://wiki.ubuntu.com/StableReleaseUpdates#Landscape).
Explanation
===========
If a 12.04.X or 12.05 client is talking to a server which does not support the
hardware-info messages (like LDS 11.07.X), it will queue calls to /usr/bin/lshw in
- memory. That in itself is not a big problem per se.
+ memory, one per day. That in itself is not a big problem per se.
But if the server the client is talking to is then upgraded to a version that
does support hardware-info (like LDS 12.09), all those lshw calls will happen
at once, creating a storm of one lshw process per day it was talking to the old
server. This could be just a few processes, or hundreds, depending on how long
landscape-client was running and talking to the old server.
A restart will wipe that queue and is the recommended workaround: restart the
clients just before upgrading LDS.
Test case
=========
You will need one precise or lucid machine and access to LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
- deploy LDS 11.07.X
- install landscape-client 12.05 (can be on the same machine or another one with
network access to the LDS one)
- tweak the client code to invoke the hardware-info plugin every minute instead of
once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
change run_interval to 60
- copy /usr/bin/lshw to /usr/bin/lshw.orig and change /usr/bin/lshw to this script:
"""
#!/bin/bash
date >> /tmp/lshw.log
/usr/bin/lshw.orig $@
"""
- run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log
is updated
- register the client with LDS. AFTER THIS, DO NOT RESTART THE CLIENT AGAIN FOR ANY REASON.
- in this scenario, lshw is not being called because the server does not support that
information, but the client is accumulating the calls in memory.
- leave the client talking to the server for about 10min. Create a simple script activity
to make sure it's working (like, send "ifconfig" to the client)
- after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update
followed by apt-get dist-upgrade. The upgrade will take a few minutes
- keep tailing -f /tmp/lshw.log and /var/log/landscape/broker.log on the client
- the broker log will get errors while LDS is down upgrading, that's normal
- once the client is able to talk to the server again, it will notice the
server version switch and log something like this:
"""
2012-09-19 13:46:25,288 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message
network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-re
quired-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucaly
ptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
"""
- this is the moment where the *buggy* client will spawn a lot of lshw calls, like this:
"""
root at ubuntu:/var/log/landscape# tail -F /tmp/lshw-run.log
tail: cannot open `/tmp/lshw-run.log' for reading: No such file or directory
tail: `/tmp/lshw-run.log' has appeared; following end of new file
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
"""
- the *fixed* client will be more well behaved, with no such storm of lshw calls:
"""
root at ubuntu:~# tail -F /tmp/lshw-run.log
Wed Sep 19 20:19:23 UTC 2012
Wed Sep 19 20:20:21 UTC 2012
and one per minute from here on
"""
Regression potential
====================
The fix itself is unit tested, and I tested it in all ubuntu releases with the
attached debdiff and the test case above, confirming that the issue is gone.
The patch is very specific and affects hardware reporting only.
** Description changed:
This is an SRU request for landscape-client in lucid, natty, oneiric and
precise.
Landscape is *NOT* invoking the SRU exception this time
(https://wiki.ubuntu.com/StableReleaseUpdates#Landscape).
Explanation
===========
If a 12.04.X or 12.05 client is talking to a server which does not support the
hardware-info messages (like LDS 11.07.X), it will queue calls to /usr/bin/lshw in
memory, one per day. That in itself is not a big problem per se.
But if the server the client is talking to is then upgraded to a version that
does support hardware-info (like LDS 12.09), all those lshw calls will happen
at once, creating a storm of one lshw process per day it was talking to the old
server. This could be just a few processes, or hundreds, depending on how long
landscape-client was running and talking to the old server.
A restart will wipe that queue and is the recommended workaround: restart the
clients just before upgrading LDS.
Test case
=========
- You will need one precise or lucid machine and access to LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
+ You will need one lucid machine and access to both LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
- deploy LDS 11.07.X
- install landscape-client 12.05 (can be on the same machine or another one with
network access to the LDS one)
- tweak the client code to invoke the hardware-info plugin every minute instead of
once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
change run_interval to 60
- copy /usr/bin/lshw to /usr/bin/lshw.orig and change /usr/bin/lshw to this script:
"""
#!/bin/bash
date >> /tmp/lshw.log
/usr/bin/lshw.orig $@
"""
- - run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log
- is updated
+ - run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log is updated
- register the client with LDS. AFTER THIS, DO NOT RESTART THE CLIENT AGAIN FOR ANY REASON.
- - in this scenario, lshw is not being called because the server does not support that
- information, but the client is accumulating the calls in memory.
- - leave the client talking to the server for about 10min. Create a simple script activity
- to make sure it's working (like, send "ifconfig" to the client)
- - after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update
- followed by apt-get dist-upgrade. The upgrade will take a few minutes
+ - in this scenario, lshw is not being called because the server does not support that information, but the client is accumulating the calls in memory.
+ - leave the client talking to the server for about 10min. Create a simple script activity to make sure it's working (like, send "ifconfig" to the client)
+ - after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update followed by apt-get dist-upgrade. The upgrade will take a few minutes. Answer any dpkg conf file question with "N", keeping the original file installed.
- keep tailing -f /tmp/lshw.log and /var/log/landscape/broker.log on the client
- the broker log will get errors while LDS is down upgrading, that's normal
- once the client is able to talk to the server again, it will notice the
server version switch and log something like this:
"""
- 2012-09-19 13:46:25,288 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message
- network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-re
- quired-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucaly
- ptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
+ 2012-09-19 19:41:43,139 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-required-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucalyptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
"""
- this is the moment where the *buggy* client will spawn a lot of lshw calls, like this:
"""
root at ubuntu:/var/log/landscape# tail -F /tmp/lshw-run.log
tail: cannot open `/tmp/lshw-run.log' for reading: No such file or directory
tail: `/tmp/lshw-run.log' has appeared; following end of new file
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
"""
- - the *fixed* client will be more well behaved, with no such storm of lshw calls:
+ - the *fixed* client will be more well behaved, with no such storm of lshw calls (note: log taken from a different test run, so timestamps won't match):
"""
root at ubuntu:~# tail -F /tmp/lshw-run.log
Wed Sep 19 20:19:23 UTC 2012
Wed Sep 19 20:20:21 UTC 2012
- and one per minute from here on
+ (and one per minute from here on)
"""
Regression potential
====================
The fix itself is unit tested, and I tested it in all ubuntu releases with the
attached debdiff and the test case above, confirming that the issue is gone.
The patch is very specific and affects hardware reporting only.
--
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1053057
Title:
Client queues up lshw calls if talking to old server
Status in Landscape Client:
Fix Committed
Status in “landscape-client” package in Ubuntu:
Fix Released
Status in “landscape-client” source package in Lucid:
New
Status in “landscape-client” source package in Natty:
New
Status in “landscape-client” source package in Oneiric:
New
Status in “landscape-client” source package in Precise:
New
Bug description:
This is an SRU request for landscape-client in lucid, natty, oneiric
and precise.
Landscape is *NOT* invoking the SRU exception this time
(https://wiki.ubuntu.com/StableReleaseUpdates#Landscape).
Explanation
===========
If a 12.04.X or 12.05 client is talking to a server which does not support the
hardware-info messages (like LDS 11.07.X), it will queue calls to /usr/bin/lshw in
memory, one per day. That in itself is not a big problem per se.
But if the server the client is talking to is then upgraded to a version that
does support hardware-info (like LDS 12.09), all those lshw calls will happen
at once, creating a storm of one lshw process per day it was talking to the old
server. This could be just a few processes, or hundreds, depending on how long
landscape-client was running and talking to the old server.
A restart will wipe that queue and is the recommended workaround: restart the
clients just before upgrading LDS.
Test case
=========
You will need one lucid machine and access to both LDS 11.07.X and 12.09. The test itself is simple, but the preparation requires some work.
- deploy LDS 11.07.X
- install landscape-client 12.05 (can be on the same machine or another one with
network access to the LDS one)
- tweak the client code to invoke the hardware-info plugin every minute instead of
once per day. In the file /usr/share/pyshared/landscape/manager/hardwareinfo.py,
change run_interval to 60
- copy /usr/bin/lshw to /usr/bin/lshw.orig and change /usr/bin/lshw to this script:
"""
#!/bin/bash
date >> /tmp/lshw.log
/usr/bin/lshw.orig $@
"""
- run "lshw -xml" as root once, confirm that you get XML output and that /tmp/lshw.log is updated
- register the client with LDS. AFTER THIS, DO NOT RESTART THE CLIENT AGAIN FOR ANY REASON.
- in this scenario, lshw is not being called because the server does not support that information, but the client is accumulating the calls in memory.
- leave the client talking to the server for about 10min. Create a simple script activity to make sure it's working (like, send "ifconfig" to the client)
- after 10min, adjust the sources.list line to grab LDS 12.09 and issue apt-get update followed by apt-get dist-upgrade. The upgrade will take a few minutes. Answer any dpkg conf file question with "N", keeping the original file installed.
- keep tailing -f /tmp/lshw.log and /var/log/landscape/broker.log on the client
- the broker log will get errors while LDS is down upgrading, that's normal
- once the client is able to talk to the server again, it will notice the
server version switch and log something like this:
"""
2012-09-19 19:41:43,139 INFO [MainThread] Accepted types changed: +hardware-info computer-info operation-result memory-info mount-info text-message network-activity mount-activity custom-graph active-process-info reboot-required change-packages-result temperature apt-preferences test users reboot-required-info processor-info load-average packages unknown-package-hashes add-packages free-space action-info resynchronize package-reporter-result eucalyptus-info network-device distribution-info -hardware-inventory -client-uptime -package-locks -computer-uptime
"""
- this is the moment where the *buggy* client will spawn a lot of lshw calls, like this:
"""
root at ubuntu:/var/log/landscape# tail -F /tmp/lshw-run.log
tail: cannot open `/tmp/lshw-run.log' for reading: No such file or directory
tail: `/tmp/lshw-run.log' has appeared; following end of new file
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:43 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:44 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
Wed Sep 19 19:41:45 UTC 2012
"""
- the *fixed* client will be more well behaved, with no such storm of lshw calls (note: log taken from a different test run, so timestamps won't match):
"""
root at ubuntu:~# tail -F /tmp/lshw-run.log
Wed Sep 19 20:19:23 UTC 2012
Wed Sep 19 20:20:21 UTC 2012
(and one per minute from here on)
"""
Regression potential
====================
The fix itself is unit tested, and I tested it in all ubuntu releases with the
attached debdiff and the test case above, confirming that the issue is gone.
The patch is very specific and affects hardware reporting only.
To manage notifications about this bug go to:
https://bugs.launchpad.net/landscape-client/+bug/1053057/+subscriptions
More information about the Ubuntu-sponsors
mailing list