[Bug 1895106] [NEW] output non reproducible inhibiting verification of changes
Christian Ehrhardt
1895106 at bugs.launchpad.net
Thu Sep 10 09:24:04 UTC 2020
Public bug reported:
Hi,
there are multiple classes of issues that make germinate output hard to compare for "what changed".
1. ordering issues
Even the output of the running program changes. I didn't see subprocess spawning or any other asynchronicity. So I'd assume that there are lists generated/gathered that we process as-is.
Example:
1120 Resolving supported dependencies ...
1121 * Chose dict to satisfy dictd
1122 * Chose probert-storage to satisfy curtin
1123 Rescued python-requests-toolbelt-doc from extra to supported
1124 Rescued libmemcached-dbg from extra to supported
Rerun on same content:
1120 Resolving supported dependencies ...
1121 * Chose dict to satisfy dictd
1122 * Chose probert-storage to satisfy curtin
1123 Rescued default-libmysqlclient-dev from extra to supported
1124 Rescued libmpc-dev from extra to supported
1125 Rescued python-openstackclient-doc from extra to supported
1126 Rescued libgnome-menu-3-dev from extra to supported
Why is that - could we just sort any of the gathered lists before we iterate on them.
That could in turn make many other things in the output reproducible.
2. If a package is depended on by multiple packages or seeds a random one is reported.
Example (run with the same seeds multiple times):
all:
-binutils-multiarch | binutils | binutils-multiarch-dev
+binutils-multiarch | binutils | binutils-multiarch-dbg
It is correct that both packages and seeds depend on them but I'd think it could be much better if we'd either:
- report an ordered full list of dependency sources (could become very long but complete)
or
- report the first element out of a sorted list (as short as today, but reproducible)
We could even get "best of both worlds" if we sort the list of
dependency anchors, then concat the fist X (an arbitrary limit we set)
of them and if some are left append ", ..." to reflect that.
This way we would be reproducible, in many cases even complete, and in
corner cases don't explode the list size.
There could be more sources of non reproducibility that come to mind for experts in germinate (I'm not) and I'd appreciate if there could be an overhaul to get as close as reasonable to reproducible output.
Because then if someone modifies seeds (or checks any follow on output based on it) it would stay the same and indicate the actual impact a change has caused.
Well, maybe this was already discussed but I didn't find anything. If it was and considered undoable please point me to a log of the discussion if possible.
** Affects: germinate (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to germinate in Ubuntu.
https://bugs.launchpad.net/bugs/1895106
Title:
output non reproducible inhibiting verification of changes
Status in germinate package in Ubuntu:
New
Bug description:
Hi,
there are multiple classes of issues that make germinate output hard to compare for "what changed".
1. ordering issues
Even the output of the running program changes. I didn't see subprocess spawning or any other asynchronicity. So I'd assume that there are lists generated/gathered that we process as-is.
Example:
1120 Resolving supported dependencies ...
1121 * Chose dict to satisfy dictd
1122 * Chose probert-storage to satisfy curtin
1123 Rescued python-requests-toolbelt-doc from extra to supported
1124 Rescued libmemcached-dbg from extra to supported
Rerun on same content:
1120 Resolving supported dependencies ...
1121 * Chose dict to satisfy dictd
1122 * Chose probert-storage to satisfy curtin
1123 Rescued default-libmysqlclient-dev from extra to supported
1124 Rescued libmpc-dev from extra to supported
1125 Rescued python-openstackclient-doc from extra to supported
1126 Rescued libgnome-menu-3-dev from extra to supported
Why is that - could we just sort any of the gathered lists before we iterate on them.
That could in turn make many other things in the output reproducible.
2. If a package is depended on by multiple packages or seeds a random one is reported.
Example (run with the same seeds multiple times):
all:
-binutils-multiarch | binutils | binutils-multiarch-dev
+binutils-multiarch | binutils | binutils-multiarch-dbg
It is correct that both packages and seeds depend on them but I'd think it could be much better if we'd either:
- report an ordered full list of dependency sources (could become very long but complete)
or
- report the first element out of a sorted list (as short as today, but reproducible)
We could even get "best of both worlds" if we sort the list of
dependency anchors, then concat the fist X (an arbitrary limit we set)
of them and if some are left append ", ..." to reflect that.
This way we would be reproducible, in many cases even complete, and in
corner cases don't explode the list size.
There could be more sources of non reproducibility that come to mind for experts in germinate (I'm not) and I'd appreciate if there could be an overhaul to get as close as reasonable to reproducible output.
Because then if someone modifies seeds (or checks any follow on output based on it) it would stay the same and indicate the actual impact a change has caused.
Well, maybe this was already discussed but I didn't find anything. If it was and considered undoable please point me to a log of the discussion if possible.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/germinate/+bug/1895106/+subscriptions
More information about the foundations-bugs
mailing list