+1 maintenance report (dogtag-pki vs 389-ds-base)

[Lukas Märdian]

Hey!

Together with Timo I was debugging this issue a little further, and we were
able to pinpoint the faulty commit, introducing the regression:
https://github.com/389ds/389-ds-base/commit/2ccd0bed4e60e44303d5f1cf96bd30572ffea85b

We reverted that commit and tested dogtag-pki vs custom 389-ds-base in a
PPA, to confirm it passes:
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-hirsute-slyon-testing/hirsute/s390x/d/dogtag-pki/20210122_114408_9ecb1@/log.gz

And Timo got in touch with the upstream developers to raise the issue there:
https://github.com/389ds/389-ds-base/issues/4563

Best,
   Lukas

On Thu, Jan 21, 2021 at 9:25 PM Sergio Durigan Junior <sergiodj at ubuntu.com>
wrote:

> On Thursday, January 21 2021, Timo Aaltonen wrote:
>
> > On 21.1.2021 18.59, Lukas Märdian wrote:
> >> NO - dogtag-pki vs ['389-ds-base/1.4.4.9-1build2',
> >> 'net-snmp/5.9+dfsg-3ubuntu1']
> >>
> >> So I had a closer look into the dogtag-pki failure on s390x. I could
> >> easily reproduce the problem inside a s390x LXD container, but
> >> wasn't able to isolate the root cause... After quite some
> >> investigation I was able to produce a debug trace of the problem,
> >> and to me it looks like the issue is actually not inside this
> >> package, but rather inside the LDAP server (i.e. 389-ds-base), as
> >> the debug log shows an "SEVERE: Unable to modify o=pki-tomcat-CA:
> >> netscape.ldap.LDAPException: error result (1); Operations error",
> >> i.e. "Internal Server Error"
> >> (
> https://docs.oracle.com/cd/E19957-01/816-5618-10/netscape/ldap/LDAPException.html#OPERATION_ERROR
> >> <
> https://docs.oracle.com/cd/E19957-01/816-5618-10/netscape/ldap/LDAPException.html#OPERATION_ERROR>).
> I
> >> do not really understand why the LDAP server would behave
> >> differently on s390x than on all the other architectures, but I
> >> guess this is for another time...
> >>
> >> Debug log: https://paste.ubuntu.com/p/vx9JB6VTjF/
> >> <https://paste.ubuntu.com/p/vx9JB6VTjF/>
> >
> > This seems to go back to at least 389-ds-base 1.4.4.4, probably
> > longer. Would be useful to know where it regressed.
> >
> > As to why it only happens on s390x my guess would be that it's related
> > to endianness (it's big-endian). Upstream tests only on amd64 (LE).
>
> I'm also very interested in solving this, because it's blocking net-snmp
> and a bunch of other packages from migrating.
>
> Last week I did some investigation and pretty much stopped at the same
> point as Lukas did.  I wasn't able to pinpoint exactly what the root
> cause is, but Timo's guess is a good starting point.
>
> I fiddled a bit with autopkgtest.db and confirmed that the failures
> started with 389-ds-base/1.4.4.4-1:
>
>   sqlite> SELECT test.package, result.version, result.triggers,
> result.exitcode FROM result INNER JOIN test ON test.id = result.test_id
> where test.package = 'dogtag-pki' and result.triggers LIKE '%389-ds-base%';
>
> I'll see if I have the time to investigate a bit more.
>
> Thanks,
>
> --
> Sergio
> GPG key ID: E92F D0B3 6B14 F1F4 D8E0  EB2F 106D A1C8 C3CB BF14
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20210122/ad26884f/attachment.html>