[Bug 798293] Bug#630756: [uscan] support parsing S3 bucket listings
Scott Moser
smoser at canonical.com
Fri Jun 17 00:46:06 UTC 2011
Package: devscripts
Version: 2.10.69ubuntu2
Severity: wishlist
File: /usr/bin/uscan
Tags: patch
A few (admittedly very few) upstreams host their files on S3.
By default, S3 does not do apache like file listings, but instead gives
file listing in XML format.
uscan can be modified to allow watch files that reference these
listings.
Attached is a suggested patch.
See also ubuntu bug 798293 (http://bugs.launchpad.net/bugs/798293).
There is very little chance for false positives, and the content that is
found in s3 bucket listings is well defined so it is not likely to stop
working. Before considering the content to be an S3 bucket listing, it
checks:
a.) that the file begins with "<?xml"
b.) that it contains the string
'xmlns=http://s3.amazonaws.com/doc/2006-03-01/'
-- Package-specific info:
--- /etc/devscripts.conf ---
--- ~/.devscripts ---
DEBSIGN_KEYID=024BC6F0
DEBUILD_DPKG_BUILDPACKAGE_OPTS="--source-option=--abort-on-upstream-changes"
-- System Information:
Debian Release: squeeze/sid
APT prefers natty-updates
APT policy: (500, 'natty-updates'), (500, 'natty-security'), (500, 'natty')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.38-8-server (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages devscripts depends on:
ii dpkg-dev 1.16.0~ubuntu7 Debian package development tools
ii libc6 2.13-0ubuntu13 Embedded GNU C Library: Shared lib
ii perl 5.10.1-17ubuntu4.1 Larry Wall's Practical Extraction
Versions of packages devscripts recommends:
ii 3.1.12-1ubuntu2 Delayed job execution and batch pr
ii 8.1.2-0.20100314cvs-1 simple mail user agent
ii 2.3.1-1ubuntu1 easy to use distributed version co
ii 7.21.3-1ubuntu1 Get a file from an HTTP, HTTPS or
ii 2.14.5 Command-line tools to process Debi
ii 0.9.6.1ubuntu1 Debian package upload tool
ii 1.14.4-1ubuntu1 Gives a fake root environment
ii 4.0.1+build1+nobinonly-0ubuntu0.11.04.3 Safe and easy web browser from Moz
ii 1:1.7.4.1-3 fast, scalable, distributed revisi
ii 1.4.11-3ubuntu1 GNU privacy guard - a free PGP rep
ii 2.1500-1 Authen::SASL - SASL Authentication
ii 2.27-1 Perl module to parse and convert t
ii 2.005-2 Easy OO parsing of Debian control-
ii 0.2-4build3 Perl extension for retrieving term
ii 1.2000-1 collection of modules to manipulat
ii 1.56-1 module to manipulate and access UR
ii 5.837-1 simple and consistent interface to
ii 2.5.0~rc2ubuntu3 Debian package checker
ii 4.0-0ubuntu11 Linux Standard Base version report
ii 4.43-14ubuntu2 Compression method of 7z format in
ii 2.5.9-4 on-line manual pager
ii 1:5.8p1-1ubuntu3 secure shell (SSH) client, for sec
ii 2.6-3 Apply a diff file to an original
ii 0.3.1-2build1 Utilities to work with patches
ii 0.0.6ubuntu2 Utilities for sensible alternative
ii 4.5.20-2ubuntu2 A system call tracer
ii 1.6.12dfsg-4ubuntu2.1 Advanced version control system
ii 6.0-4ubuntu1 De-archiver for .zip files
ii 0.5.3-1 WWW browsable pager with excellent
ii 0.6.3-1 Compares two files word by word
ii 1.12-2.1ubuntu2 retrieves files from the web
ii 5.0.0-2 XZ-format compression utilities
Versions of packages devscripts suggests:
ii build-essential 11.5ubuntu1 Informational list of build-essent
pn cvs-buildpackage <none> (no description available)
ii debian-keyring [debian-maint 2010.12.29 GnuPG keys of Debian Developers
pn devscripts-el <none> (no description available)
pn equivs <none> (no description available)
pn gnuplot <none> (no description available)
pn libcrypt-ssleay-perl <none> (no description available)
ii libfile-desktopentry-perl 0.04-2 Perl module to handle freedesktop
pn libnet-smtp-ssl-perl <none> (no description available)
pn libsoap-lite-perl <none> (no description available)
pn libyaml-syck-perl <none> (no description available)
pn mutt <none> (no description available)
pn svn-buildpackage <none> (no description available)
ii ubuntu-dev-tools 0.122 useful tools for Ubuntu developers
ii w3m 0.5.3-1 WWW browsable pager with excellent
-- no debconf information
** Attachment added: "uscan-s3-support.diff"
https://bugs.launchpad.net/bugs/798293/+attachment/2172362/+files/uscan-s3-support.diff
** Bug watch added: Debian Bug tracker #630756
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630756
** Also affects: devscripts (Debian) via
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=630756
Importance: Unknown
Status: Unknown
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to devscripts in Ubuntu.
https://bugs.launchpad.net/bugs/798293
Title:
uscan should support parsing s3 bucket listings
Status in “devscripts” package in Ubuntu:
New
Status in “devscripts” package in Debian:
Unknown
Bug description:
Binary package hint: devscripts
A few (admittedly very few) upstreams host their files on S3.
By default, S3 does not do apache like file listings, but instead gives file listing in XML format.
ec2-api-tools:
Homepage: http://aws.amazon.com/developertools/351
S3 listing: http://s3.amazonaws.com/ec2-downloads/
ec2-ami-tools: http://aws.amazon.com/developertools/368
Homepage: http://aws.amazon.com/developertools/368
S3 listing: http://s3.amazonaws.com/ec2-downloads/
rdscli (bug 797387):
Homepage: http://aws.amazon.com/developertools/2928
S3 listing: http://s3.amazonaws.com/rds-downloads/
I suggest that uscan could be made to understand the well defined
output of an S3 bucket listing. Below is an example. I've added
carriage returns, the content usually is only 2 lines (xml header and
content).
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>ec2-downloads</Name><Prefix></Prefix><Marker></Marker><MaxKeys>1000</MaxKeys><IsTruncated>false</IsTruncated>
<Contents><Key>2006-06-26.ec2.wsdl</Key><LastModified>2006-10-23T12:22:30.000Z</LastModified><ETag>"d4fa76ef26b78d3905e009de9db8bf7d"</ETag><Size>28344</Size><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>2006-10-01.ec2.wsdl</Key><LastModified>2006-12-11T15:16:07.000Z</LastModified><ETag>"c10f3f9a199906e09eb3140b1f694e78"</ETag><Size>38244</Size><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>2007-01-03.ec2.wsdl</Key><LastModified>2007-02-10T14:57:28.000Z</LastModified><ETag>"3315c7afe5a27444fddf8e0f21d20d1c"</ETag><Size>39314</Size><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>2007-01-19.ec2.wsdl</Key><LastModified>2007-03-22T16:29:10.000Z</LastModified><ETag>"01d64aeb8a01b41f80e8bc6627daa313"</ETag><Size>40501</Size><StorageClass>STANDARD</StorageClass></Contents><Contents><Key>2007-03-01.ec2.wsdl</Key><LastModified>2007-07-06T05:25:50.000Z</LastModified><ETag>"a716de16be05c734b687f0612134d655"</ETag><Size>43408</Size><StorageClass>STANDARD</StorageClass></Contents>
</ListBucketResult>
ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: devscripts 2.11.0ubuntu1
ProcVersionSignature: Ubuntu 3.0-0.1-generic 3.0.0-rc2
Uname: Linux 3.0-0-generic x86_64
Architecture: amd64
Date: Thu Jun 16 11:09:58 2011
EcryptfsInUse: Yes
InstallationMedia: Ubuntu 10.04 "Lucid Lynx" - Beta amd64 (20100318)
ProcEnviron:
LANGUAGE=en_US:en
PATH=(custom, no user)
LANG=en_US.utf8
LC_MESSAGES=en_US.utf8
SHELL=/bin/bash
SourcePackage: devscripts
UpgradeStatus: Upgraded to oneiric on 2010-11-15 (212 days ago)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/devscripts/+bug/798293/+subscriptions
More information about the foundations-bugs
mailing list