[Bug 2165179] Re: d/copyright (and Files-Excluded) seems outdated

[Matthew Ruffell]

uscan warn: No files matched excluded pattern as the last matching glob: src/rapidjson/bin/jsonchecker/*
uscan warn: No files matched excluded pattern as the last matching glob: src/civetweb/src/third_party/duktape*
uscan warn: No files matched excluded pattern as the last matching glob: src/civetweb/examples/_obsolete/websocket_client/ssl/server.key.orig
uscan warn: No files matched excluded pattern as the last matching glob: src/civetweb/resources/cert/client.key.orig
uscan warn: No files matched excluded pattern as the last matching glob: src/civetweb/resources/cert/server.key.orig
uscan warn: No files matched excluded pattern as the last matching glob: src/civetweb/resources/cert/server_bkup.key.orig
uscan warn: No files matched excluded pattern as the last matching glob: src/erasure-code/jerasure/jerasure/Examples/makefile.orig
uscan warn: No files matched excluded pattern as the last matching glob: src/erasure-code/jerasure/jerasure/include/config.h.in~
uscan warn: No files matched excluded pattern as the last matching glob: src/erasure-code/jerasure/jerasure/makefile.orig

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ceph in Ubuntu.
https://bugs.launchpad.net/bugs/2165179

Title:
  d/copyright (and Files-Excluded) seems outdated

Status in ceph package in Ubuntu:
  New

Bug description:
  d/copyright has Files-Excluded set, so the original source can be
  repacked when downloading from upstream.

  Some vendored trees have been relocated/removed in the upstream
  source, so many patterns don't match anything, and some undesired
  files may be part of the shipped source.

  Debian has a different list of excludes, but at first glance it also
  seems to be missing some entries or having outdated paths.

  I think it would be nice to:

  a) run `uscan` to get the latest upstream version, and watch for
  `warn: No files matched excluded pattern as the last matching glob`
  to update stale entries, and

  b) build ceph using the pristine upstream tarball and give lintian a
  pedantic run on it - checking for the usual suspects: license-problem-
  json-evil`, `source-is-missing`, `source-contains-prebuilt-ms-help-
  file`, etc; and considering excluding these too.

  c) optionally, if you like rabbit holes alot, double check d/changelog
  as a whole (: but that deserves a separate bug I guess.

  
  ----------------------------------------------------

  Here is a Claude-analysis of what can be looked at:

  ### Outdated patterns

  `src/rapidjson/bin/jsonchecker/*` no longer matches. Upstream moved rapidjson
  under s3select, so the path is now:

      src/s3select/rapidjson/bin/jsonchecker/

  38 files. These carry the JSON licence, which is not DFSG-free because of the
  clause at `src/s3select/rapidjson/license.txt` line 45:

      The Software shall be used for Good, not Evil.

  This is the `license-problem-json-evil` lintian error, and it is the specific
  issue that LP: #1750848 was filed to resolve in 2018. It has silently regressed.
  The files are present in every currently-supported Ubuntu ceph source package
  (focal, jammy, noble, plucky, questing, resolute, stonking).

  `debian/copyright` also has no `Files:` stanza covering the relocated rapidjson
  at all, so the licence is neither excluded nor documented.

  ### Other stale entries

  `src/civetweb/src/third_party/duktape*` — civetweb is no longer vendored at
  `src/civetweb`. It now lives at:

      src/jaegertracing/opentelemetry-cpp/third_party/prometheus-
  cpp/3rdparty/civetweb/

  The pattern matches nothing. Two duktape copies ship (1.5.2 and 1.8.0), together
  with `.../civetweb/test/ajax/jquery.js`.

  These four entries match nothing at all — the files no longer exist
  upstream:

      src/civetweb/examples/_obsolete/websocket_client/ssl/server.key.orig
      src/civetweb/resources/cert/client.key.orig
      src/civetweb/resources/cert/server.key.orig
      src/civetweb/resources/cert/server_bkup.key.orig
      src/erasure-code/jerasure/jerasure/Examples/makefile.orig
      src/erasure-code/jerasure/jerasure/include/config.h.in~
      src/erasure-code/jerasure/jerasure/makefile.orig

  ### Entries that drifted

  `*/DotZLib.chm` is a glob, so it still works, but the number of copies changed
  without anyone noticing — 2 in 19.2.6, 3 in 20.2.1:

      src/boost/libs/beast/test/extern/zlib-1.2.12/contrib/dotzlib/DotZLib.chm
      src/boost/libs/iostreams/test/zlib-1.2.11/contrib/dotzlib/DotZLib.chm       (new in 20.x)
      src/boost/tools/boost_install/test/iostreams/zlib-1.2.11/contrib/dotzlib/DotZLib.chm

  Likewise `*/jquery.js`. The set of matches has changed across releases — the
  `src/cpp_redis/**` copies are gone in 20.2.1 and a new one appeared at
  `src/nvmeof/gateway/spdk/intel-ipsec-mb/docs/jquery.js`. A glob happened to
  absorb that; an explicit path would not have.

  ### Inconsistency

  `qa/workunits/erasure-code/jquery.flot.js` is excluded, but
  `qa/workunits/erasure-code/jquery.flot.categories.js` sits in the same directory,
  has the same provenance and licence, and is not excluded.

  ### What is actually still excluded in 20.2.1

  Only these patterns still match (243 files in 19.2.6):

  | Pattern | Files |
  |---|---|
  | `src/rocksdb/docs/*` | 236 |
  | `*/jquery.js` | 4 |
  | `*/DotZLib.chm` | 2 (3 in 20.x) |
  | `qa/workunits/erasure-code/jquery.flot.js` | 1 |

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ceph/+bug/2165179/+subscriptions