[Bug 1281767] [NEW] FFE: simplestreams by-hash storage

[Scott Moser]

Public bug reported:

[Description]
Currently, simplestreams is is able to mirror a source to a local filesystem by paths listsed.
For example:
 sstream-mirror --keep --max=1 http://cloud-images.ubuntu.com/releases/ out.d

That will mirror content from the download stream into out.d/ with paths like server/releases/saucy/release-20140129/ubuntu-13.10-server-cloudimg-amd64.tar.gz .
http://cloud-images.ubuntu.com/releases/streams/v1/com.ubuntu.cloud:released:download.json

That is safe (per definition of simplestreams format) and external paths
are checked for safety.

The issue is that if multiple paths exist to the same sha256 sum, then
we'll download those files mutiple times.  The files could be listed on
completely separate sites or inside of different content on the same
site.

This is wasteful if downloads are hundreds of megabytes.

[Changes]

The goal is to just change simplestreams to be able to store "by-hash",
and create hardlink or symlink trees from the paths listed in the
source.

The format used for the by-hash storage would be just like git's .
 <storage>[A-Z][A-Z]/....

** Affects: simplestreams (Ubuntu)
     Importance: Medium
         Status: New

** Changed in: simplestreams (Ubuntu)
    Milestone: None => ubuntu-14.04

** Changed in: simplestreams (Ubuntu)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to simplestreams in Ubuntu.
https://bugs.launchpad.net/bugs/1281767

Title:
  FFE: simplestreams by-hash storage

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/simplestreams/+bug/1281767/+subscriptions