[ubuntu-hardened] Making security notices searchable

[Matthew Paul Thomas]

Russenberger Dominik wrote on 14/10/2019 3:00 pm:
>…
> On Sun, Oct 13, 2019 at 10:55:45AM +0100, Matthew Paul Thomas wrote:
>>
>> As part of this, I’m considering adding a search function, so that
>> you can search security notices by Details text, with filters for
>> Release and Package.
>
> Filtering by arches (e.g. amd64+all) might be useful, for the handful
> of bugs that only affect a single architecture.

As far as I can tell, the notices don’t include architecture info in any
structured format. We can’t search data that doesn’t exist.

>> So, I’m interested in knowing:
>>
>> *   Is there any current method of searching USNs? (Other than using
>>     “site:usn.ubuntu.com” with a global search engine, or grepping>>     the usn.ubuntu.com Git repo.)
>
> Well... my current solution is to get
> https://usn.ubuntu.com/releases/ubuntu-18.04-lts/
> and then parse the HTML.

I’m not proposing to port the release pages, since with the introduction
of search, they would be redundant with searching for all notices from a
particular release. (Of course you could continue parsing the search
results page.)

> Atom/RSS also just contain HTML for the content,
> but no method to filter by release. And of course parsing HTML breaks 
> every once in a while.

I wonder if we could provide Atom/RSS feeds for arbitrary search
results. (I realise it might be more expensive for us than generating a
single feed for all notices.)

>> *   If any search showed results sorted newest first, would there be
>>     any use case for searching notices by date? (For example, show me
>>     only notices posted in 2017.)
>
> My usecase requires to show all USNs after a particular date, or even 
> better all USNs after another USN.

I guess your current scraping stops when it gets to a USN that it’s seen
before.

>> *   Anything else you think I should know?
>
> Machine-readable USNs! Alex Murray posted the link to a big json, but
> downloading&parsing 130mb every hour doesn't sound like such a good
> idea.
>…

Cheers
-- 
mpt