REVU: Automated Package Checks

[Loïc Martin]

Kjeldgaard Morten wrote:
> I am not sure that more automated package analysis well help much. The  
> uploaders already have Lintian and other tools at their disposal, yet  
> the fact is that many packages have lots of Lintian issues remaining  
> on the binary packages.

Except for linitan errors, on the other hand lots of packages come with
easily to spot by script errors. Just off of my head:
1. debian/control
- wrong target dirsto (actually not always a mistake, since the upload
might sit in REVU for month before first review - if all uploads could
be checked at the start of a development cycle and either modified or an
email sent, it would save one back and forth comment cycle;
- wrong priority;
- maintainer field not assigned to MOTU, packager email need to be moved
to XSBC-Original-Maintainer;
- too many spaces in the file, too many characters on one or more lines
(valid for other files in debian/ too);
- debhelper version don't match compat one.

2. debian/copyright
- no copyright assigned, no licence;
- licence is GPL but the file doesn't direct to the location on the
system (or to a wrong location, not existing directory or GPL instead of
GPL2);
- no copyright/licence for the packaging (not sure it's mandatory).

Then maybe a grep on the source files for "copyright" "©" "(C)" and the
log added to the files so one can check without having to download the
files.

3. debian/changelog
- no bug closed or wrong syntax.

4. other files:
- comment created automatically not removed;
- presence of an INSTALL file;
- no man(s) page(s);
- does REVU check for binaries in the packages? I've seen comments on
IRC that some upload need binaries removed, so if REVU don't do that it
might be nice to implement it (if feasible).

Some of the aforementioned issues might be invalid, some are just
cosmetic and shouldn't block a review or an upload, but they often
appear in reviews.

I agree those problems with an upload are easy to spot, and sometimes
don't take more than a few minutes by experienced MOTU, who might fail
to see the point of automating the process. That would be indeed
pointless if the review came just a few days after a first upload, but
the fact is it can take month, thus restarting the review process
(uploader need to fix those, upload a new package, then wait for a few
weeks/month before the real review, far more time consuming, starts).

Any automated reply is a benefit in the current state of REVU, since the
uploader can fix the package instantly (and go to IRC for questions) and
that would save going back and forth between uploads and comments, both
for MOTU/reviewers and for the uploader.

There's also the fact a first review will spot 3/4 of those problems,
then a few month afterwards somebody else will spot one or two issues,
which delay the final reviews even more.

> When people upload to REVU, they have read all the guides and  
> tutorials (at least some have) and what they really want is a human  
> being to look at it, and to get advice on what to do. Many see the  
> warnings by the various tools, but simply don't know what to do about  
> them. Or, they feel unsure on where to go and don't want to spend a  
> lot of time going in the wrong direction.

Even when having read all the tutorials and packaging step-by-step
according to the rules, it's often hard to figure it out perfectly.
There's times you don't understand what the tutorial says in the right
way, and there's times you just make a mistake or overlook something.
Being pointed to your own errors is invaluable in this learning process
(and it's also a good way to ensure you won't forget the issue if you
want to do some reviewing later), and when it's close in time from the
making of the mistake, it means you still have everything fresh in
memory and won't just "apply the fix", but question what you thought you
had understood.

It doesn't replace human interaction, but it's a good start, especially
for uploaders that don't just want to see a package in the archives, but
want to be involved in REVU as well. Having an automated pre-review
process may be a good thing to try, as it won't in any way prevent human
interaction :
- people will go to IRC if they care about their upload, except they
will go there sooner and not few month later when the first comments appear;
- reviewer will have tools that save them some time while ensuring the
uploader is active, since he/she will have corrected some mistakes at
least once already.

Loïc