Enhancing cross-distro collaboration via foreign archive keyring availability

Tue Sep 10 17:12:15 UTC 2024

Hi,

On Wed, Sep 04, 2024 at 10:51:30AM +0100, Luca Boccassi wrote:
> For developers like myself who work across distributions, it is very
> valuable to have a secure, simple and transparent way to bootstrap one
> distro from another without relying on external trust anchors that
> have to be verified manually.

I agree that it makes sense to place a trust anchor in the Ubuntu
archive, assuming that there's someone who commits to maintaining it (as
you've said you will).

What I don't understand is why all of this seems far more complicated
than would seem necessary to me. I think this burdens patch pilots and
the SRU team with unnecessary work for little benefit. Why isn't the
architecture simpler? I go into that into detail below. Sorry the text
is long, but there's no way to present a counterargument for something
that seems unnecessarily complicated without diving into the apparent
unnecessary complications :-/

It seems burdensome to me to include a large set of keys and update them
(presumably) more frequently, when (IMHO) only a single anchor is needed
which would presumably need updating less frequently. It's not just the
person maintaining the package, but also somebody else from a privileged
team is going to have to review changes, whether we conclude that such
changes should go into -updates or -backports.

Further, how is such a reviewer going to validate the changes? Doing so
is is crucially important. If this isn't done, then there's a path to
invalidating all of the trust that is supposed to be achieved by
including these keys in the first place.

If consensus is to include any form of these packages, I think it's
essential to also have consensus on what level of validation is
acceptable, and then have that well documented and tooling provided that
implements it. This is so important for keyring-type packages,
especially where validation isn't obvious because they are coming from
outside the distribution, that I think it's unreasonable to expect the
SRU team to review these uploads without this having been thought about
first. If there's one thing that needs to come out of me asking for
consensus over the general approach first, over piecemeal updates, it's
this.

But if all we're doing is taking the keys from other places and updating
them in Ubuntu, validated by some process that ultimately relies on some
set of people to assert that the keys are correct, then what are we
achieving anyway? Can this not just be automated then, and tooling be
provided in the archive instead, so users can just do that directly when
they need? Then there would be much reduced burden on maintainence,
including for the relevant privileged review teams.

We do gain from there being a root of trust in the archive, but that
only need be a single root of trust to bootstrap trust in live, external
sources. This is what I mean above when I say that IMHO only a single
anchor is needed.

Otherwise we're going to get a proliferation of external keyrings that
need to be maintained, seemingly unbounded. I don't think this
architecture scales.

Here's an alternative, much simpler architecture. Someone could start an
"upstream" project that exists for the purpose of maintaining these
keyrings cross-distro. Let's call that a "cross-distro keyring". The
most trivial way to implement this would be to use HTTPS, with trust
rooted via ca-certificates. When a user needs the latest key for
something external, the tooling would simply fetch the latest keys,
validating it on the fly. Obvious improvements would be certificate
pinning, signing of data at rest, etc, but these are implementation
details that are orthogonal to my point. It would still all work with
only a single root of trust needed in Ubuntu's archive, and just one
upstream maintainer rather than a bunch of maintainers across a bunch of
distributions.

The one difference I can see is that it would be necessary to download
the actual final signing keys, rather than have them available in eg. a
local mirror. However, if you're bootstrapping something outside of
Ubuntu, then you have to download that anyway, by definition. Having to
grab the latest copy of the cross-distro keyring at the same time
doesn't seem unreasonable. Once done, validation would still be able to
use the distribution-shipped trust anchor.

If the single root of trust needs updating, then sure. We might even
need to do it urgently in case of a key revocation. But at least it
scales, because one can expect it to be infrequent.

> Given all of this, the costs appear minor, especially compared to
> other updates that are part of point releases. Is there perhaps some
> angle or detail that I am missing here? I appreciate Robie
> double-checking that this is the right way to do it.

Compared to regular updates that are part of point releases, this seems
to be a) new (to us at least, in our stable releases); and b) affects
roots of trust. I think that warrants special consideration. It would be
reputationally damaging to Ubuntu if we were to allow changes to these
keyrings without having thought about how we're validating them, and the
update path were compromised.

>                                                      I hope I was able
> to explain that the architecture is the one commonly used and would
> expect adding a different Ubuntu-only alternative would likely be a
> costly approach, for little benefit.

I'm not proposing a different Ubuntu-only alternative. I'm proposing
changes that I think would be improvements applicable to the whole
ecosystem, as a bar of quality to make this acceptable to Ubuntu to take
in stable releases rather than de-facto from Debian from autosyncs in
the development release. Unless I'm missing some use case or nuance that
necessitates the current architecture that hasn't been pointed out to me
yet.

>                                      Therefore I wanted to ask if we
> can agree on this kind of sporadic updates being ok? If it is helpful
> we could even draft something to be added to
> https://wiki.ubuntu.com/StableReleaseUpdates#Documentation_for_Special_Cases
> so anyone involved in the future can refer to it more easily.

Thank you for raising this here, and indeed my main concern is just that
we're all agreed on the way forward. The appropriate balance is a
subjective decision. I'd like that to incorporate deliberation on my
points above, but ultimately I don't have a problem if the decision
doesn't follow my opinion.

The "Documentation for Special Cases" would be the right place to
document this, indeed - but only if it's concluded that the SRU process
is the appropriate one to use here, rather than Backports. I think
whether -security, -updates or -backports is used is another decision we
need to make.

I think my question on validation is very important though - whatever we
do, we should define what privileged review teams are supposed to do to
validate keyring changes, make sure that we consider that acceptable
from a trust and security perspective, and do what we can to ensure that
it's not a burden for the privileged review teams. For example, if I
could run a command that performs such validation when reviewing an SRU,
and gives me a simple yes/no answer, then that would be ideal.

In Ubuntu, we de-facto trust Debian to validate regular changes to
packages, but only for the development release. We don't have any agreed
or specific arrangement for stable releases.

For new upstreams releases, I prefer to see uscan cryptographically
validate orig tarballs. For software supply chain reasons, it's on my
list of things to drive that I'd like to move towards this being
mandated in Ubuntu as a condition of taking new upstream releases into
Ubuntu stable releases without individual change review.

I'm not sure that will work here though? The obvious thing would be to
say that the identical keyring must already be in sid, but if that is
compromised then Ubuntu will always lose over Debian because a
compromise to the package in sid would hit Ubuntu users first in
practice, so maybe that's not good enough.

Robie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20240910/f62e11d4/attachment.sig>