Bug#881692: command-not-found: I re-wrote command-not-found

[Julian Andres Klode]

On Tue, Nov 14, 2017 at 01:00:54PM +0100, Zygmunt Krynicki wrote:
> Hey everyone.
> 
> Thank you for your interest in command-not-found.
> 
> On Tue, Nov 14, 2017 at 8:50 AM, Julian Andres Klode <jak at debian.org> wrote:
> > (forwarding this to ubuntu-devel-discuss and Zygmunt)
> >
> > On Mon, Nov 13, 2017 at 10:33:39PM -0800, Shawn Landden wrote:
> >> Package: command-not-found
> >> Severity: wishlist
> >>
> >> I re-wrote command-not-found to get rid of the python dependancy, and
> >> to reduce the database size, as to reduce memory usage.
> >>
> >> https://github.com/shawnl/command-not-found
> >>
> >> I was preparing to upload it to mentors as command-not-found-ng
> >
> > I also rewrote it years ago, but using the same database format,
> > just in C. It was a lot faster. I don't understand the memory usage
> > bit - it should not matter how large the database is, it's memory
> > mapped, and not read into memory, as such memory usage should be
> > roughly constant.
> >
> > Questions/Comments for your approach:
> >
> > * Did you test your format on a slow HDD with caches dropped? It
> >   must not be slower than the Python one (that one is way too slow
> >   already) - I did, it seems to be faster (0.4 vs 0.68 seconds)
> >   - I believe the database-based C rewrite was even much faster,
> >   though.
> 
> > * update-command-not-found should use apt-get indextargets
> 
> > * You don't store components, hence you cannot tell people to enable
> >   component. That's a very important use case for Ubuntu, where
> >   not all components are enabled by default, but the database is
> >   shipped in the package.
> >
> >   You could just append /<component> to each package name I think,
> >   and strip it away when displaying.
> 
> I would love if we have a compact representation of mapping from name
> to list of bits of information where each bit can be a small structure
> with some data. Apart from components for ubuntu archive it could be
> used to store facts about snap packages, flatpaks, etc. I would try to
> avoid a simplistic command -> package mapping as that will force us to
> encode things into strings in an ad-hoc way.

That makes sense to me. But then we're back on a db, I guess. I sort
like this minimal approach. An approach of course would be to store
a key/value map after the package, something like:

	file<SEP><LEN>package name
	followed by multiple:
		<LEN>key
		<LEN>value

	where lengths are 32-bit (16 bit?) integers.

Should not be too hard. Alternatively, this also works

	file<SEP><LEN>packagename
		 <LEN>		for each field
		 value		for each field

	and then you can index stuff

		offset(attr_i) = offset(attr_i) + attrs[i]

Lots of options to extend.

> 
> > * You should use getopt_long() to parse command-line options, and
> >   support -h, --help :)
> 
> > * pts_lbsearch belongs into usr/lib/..., not usr/share/...
> >
> > * You don't implement a closest matches function:
> >
> >         $ command-not-found thunderbrd
> >         No command 'thunderbrd' found, did you mean:
> >          Command 'thunderbird' from package 'thunderbird' (main)
> >         thunderbrd: command not found
> >         $ ./command-not-found thunderbrd
> >         thunderbrd: command not found
> >
> >    This one is really important. People do make typos or misremember
> >    command names, so the tool needs to be able to deal with that
> 
> +1 on this, the function should be not too hard to implement in C.
> 
> >    Should be easy to implement though, although you might hav
> 
> > * You need to Conflict with command-not-found and not Break AFAIUI
> 
> Ideally, to ease the transition, you should do something about the
> python APIs. If yo can keep them (either as pure-python bindings or
> just as a compatible implementation) that would be a plus. If you want
> to drop them then please announce that and see if anything rdepends on
> it.

Oh, hmm.

> 
> > * You do have to Depend on apt-file, as that configures apt to download
> >   the Contents files
> 
> I didn't look at the details but I (hope) this is a build dependency
> and this will be processed somewhere on the archive side.

That's a Debian-only dependency forced upon us by ftpmaster, on Ubuntu 
we can ship the data in the package (or preferably a separate 
command-not-found-data source package).


> 
> > I think it's a worthwhile approach, and I can see it replacing
> > command-not-found if those tiny issues have been fixed. Then you
> > could also avoid the -ng moniker, and just take over the main
> > package (if Zygmunt does not mind), which also avoids a month
> > long NEW process :)
> 
> Yes, though I'd like to participate as we're working on
> command-not-found improvements in snapd and would like to have
> something that fits Debian, Ubuntu as well as (eventually but not
> conflicting at least) Fedora and openSUSE (at least the snapd part).

I'd like that.

-- 
Debian Developer - deb.li/jak | jak-linux.org - free software dev
Ubuntu Core Developer                              de, en speaker