[Bug 1895106] Re: output non reproducible inhibiting verification of changes

[Christian Ehrhardt ]

We could kind of easily mark all that are "Rescued" as such by only
considering the dependencies of Rescued packages also as Rescued. That
sentence sounds odd, but consider this dependency graph:

src:A
bin:A
bin:A-dev (depends on B-dev)
src:B
bin:B
bin:B-dev

Seed: A & B

1. The seed would pull in src:A and bin:A
2. The seed would pull in src:B and bin:B
3. The processing of includes reaches A-dev and it would get "Rescued from A"
4. A-dev is added with Dependencies, this puts B-dev in _all
5. The processing of includes reaches B-dev and it is skipped being already present
Result:
A-dev  Rescued from A
B-dev  A-dev

But the same with different include processing order
1. The seed would pull in src:A and bin:A
2. The seed would pull in src:B and bin:B
3. The processing of includes reaches B-dev and it would get "Rescued from B"
4. B-dev is added with Dependencies, this puts A-dev in _all
5. The processing of includes reaches A-dev and it is skipped being already present
Result:
B-dev  Rescued from B
A-dev  B-dev

In both cases the same set of packages would be seeded.
But the reporting what dragged things in will be different and that makes debugging seed changes harder than it needs to be.

If a dependency out of the "actual seeding" (not includes) has pulled in
a package it already is in _all. So I think we could make it more clear
that much more package than we think are in due to auto-includes.

But we can't just keep output._all unchanged (which would simplify the
check) as then we'd end up adding packages twice - once for a dependency
from a "Rescue" include and once "itself" being an include.

The "why" argument in
 def _add_package(self, seed, pkg, why,
  self._remember_why(seed._reasons, pkg, why, build_tree, recommends)
  self._remember_why(output._all_reasons, pkg, why, build_tree,

But then it calls into "self._add_dependency_tree" which looses the
reasoning the parent was brought in.

Well, the change I have that sorts it already makes it reproducible which this bug is about.
While later one can consider adding "indirectly rescued" labels somehow that would be too error prone to change atm.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to germinate in Ubuntu.
https://bugs.launchpad.net/bugs/1895106

Title:
  output non reproducible inhibiting verification of changes

Status in germinate package in Ubuntu:
  New

Bug description:
  Hi,
  there are multiple classes of issues that make germinate output hard to compare for "what changed".

  1. ordering issues
  Even the output of the running program changes. I didn't see subprocess spawning or any other asynchronicity. So I'd assume that there are lists generated/gathered that we process as-is.
  Example:
    1120 Resolving supported dependencies ...                                             
    1121 * Chose dict to satisfy dictd                                                    
    1122 * Chose probert-storage to satisfy curtin                                        
    1123   Rescued python-requests-toolbelt-doc from extra to supported                                                                                                                          
    1124   Rescued libmemcached-dbg from extra to supported 
  Rerun on same content:
    1120 Resolving supported dependencies ...                                             
    1121 * Chose dict to satisfy dictd                                                    
    1122 * Chose probert-storage to satisfy curtin                                        
    1123   Rescued default-libmysqlclient-dev from extra to supported                                                                                                                            
    1124   Rescued libmpc-dev from extra to supported                                                                                                                                            
    1125   Rescued python-openstackclient-doc from extra to supported                                                                                                                            
    1126   Rescued libgnome-menu-3-dev from extra to supported 

  Why is that - could we just sort any of the gathered lists before we iterate on them.
  That could in turn make many other things in the output reproducible.

  
  2. If a package is depended on by multiple packages or seeds a random one is reported.

  Example (run with the same seeds multiple times):
  all:
  -binutils-multiarch                          | binutils                             | binutils-multiarch-dev
  +binutils-multiarch                          | binutils                             | binutils-multiarch-dbg

  It is correct that both packages and seeds depend on them but I'd think it could be much better if we'd either:
  - report an ordered full list of dependency sources (could become very long but complete)
  or
  - report the first element out of a sorted list (as short as today, but reproducible)

  We could even get "best of both worlds" if we sort the list of
  dependency anchors, then concat the fist X (an arbitrary limit we set)
  of them and if some are left append ", ..." to reflect that.

  This way we would be reproducible, in many cases even complete, and in
  corner cases don't explode the list size.

  
  There could be more sources of non reproducibility that come to mind for experts in germinate (I'm not) and I'd appreciate if there could be an overhaul to get as close as reasonable to reproducible output.
  Because then if someone modifies seeds (or checks any follow on output based on it) it would stay the same and indicate the actual impact a change has caused.

  
  Well, maybe this was already discussed but I didn't find anything. If it was and considered undoable please point me to a log of the discussion if possible.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/germinate/+bug/1895106/+subscriptions