[Bug 1493303] Re: [OSSA 2016-004] Swift proxy memory leak on unfinished read (CVE-2016-0738)
OpenStack Infra
1493303 at bugs.launchpad.net
Tue Jan 26 14:14:55 UTC 2016
Reviewed: https://review.openstack.org/272201
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=f9b7fd3074b5b0e5d6ea879d4144f7bfeec5d46b
Submitter: Jenkins
Branch: feature/crypto
commit e13a03c379273ee10e678818078b9c40a96a7dc9
Author: Tim Burke <tim.burke at gmail.com>
Date: Wed Jan 20 16:06:26 2016 -0800
Stop overriding builtin range
Change-Id: I315f8b554bb9e96659b455f4158f074961bd6498
commit 0a404def7d54d1ef1c85c11a378052260c4fda4c
Author: John Dickinson <me at not.mn>
Date: Wed Jan 20 15:19:35 2016 -0800
remove unneeded duplicate dict keys
Change-Id: I926d7aaa9df093418aaae54fe26e8f7bc8210645
commit 221f94fdd39fd2dcd9a2e5565adceab615d55913
Author: John Dickinson <me at not.mn>
Date: Tue Jan 19 14:50:24 2016 -0800
authors and changelog updates for 2.6.0
Change-Id: Idd0ff9e70abc0773be183c37cd6125fe852da7c0
commit 58359269b0e971e52f0eb7f97221566ca2148014
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Dec 8 16:36:05 2015 -0800
Fix memory/socket leak in proxy on truncated SLO/DLO GET
When a client disconnected while consuming an SLO or DLO GET response,
the proxy would leak a socket. This could be observed via strace as a
socket that had shutdown() called on it, but was never closed. It
could also be observed by counting entries in /proc/<pid>/fd, where
<pid> is the pid of a proxy server worker process.
This is due to a memory leak in SegmentedIterable. A SegmentedIterable
has an 'app_iter' attribute, which is a generator. That generator
references 'self' (the SegmentedIterable object). This creates a
cyclic reference: the generator refers to the SegmentedIterable, and
the SegmentedIterable refers to the generator.
Python can normally handle cyclic garbage; reference counting won't
reclaim it, but the garbage collector will. However, objects with
finalizers will stop the garbage collector from collecting them* and
the cycle of which they are part.
For most objects, "has finalizer" is synonymous with "has a __del__
method". However, a generator has a finalizer once it's started
running and before it finishes: basically, while it has stack frames
associated with it**.
When a client disconnects mid-stream, we get a memory leak. We have
our SegmentedIterable object (call it "si"), and its associated
generator. si.app_iter is the generator, and the generator closes over
si, so we have a cycle; and the generator has started but not yet
finished, so the generator needs finalization; hence, the garbage
collector won't ever clean it up.
The socket leak comes in because the generator *also* refers to the
request's WSGI environment, which contains wsgi.input, which
ultimately refers to a _socket object from the standard
library. Python's _socket objects only close their underlying file
descriptor when their reference counts fall to 0***.
This commit makes SegmentedIterable.close() call
self.app_iter.close(), thereby unwinding its generator's stack and
making it eligible for garbage collection.
* in Python < 3.4, at least. See PEP 442.
** see PyGen_NeedsFinalizing() in Objects/genobject.c and also
has_finalizer() in Modules/gcmodule.c in Python.
*** see sock_dealloc() in Modules/socketmodule.c in Python. See
sock_close() in the same file for the other half of the sad story.
This closes CVE-2016-0738.
Closes-Bug: 1493303
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Change-Id: Ib86c4c45641485ce1034212bf6f53bb84f02f612
commit bc4b298b6e208d3188641712c9d66ae82d172c14
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Jan 19 15:33:13 2016 -0800
Fix a comment's indentation
Change-Id: I34514525b606cf82767ddce7769bc42fa5457717
commit 3a0486e532f22af0d3c8a5c5d78613c22e786ff6
Author: Sivasathurappan Radhakrishnan <siva.radhakrishnan at intel.com>
Date: Fri Dec 4 17:43:00 2015 +0000
Deleted comment about part power in FakeRing
Deleted comment about parameter part power in Class FakeRing as its
behavior got dropped in I8bfc388a04eff6491038991cdfd7686c9d961545.
Change-Id: Iec7d2565a77e48493b0056021066d8d8eab65d0b
Closes-Bug: #1488704
commit 999479f9b17b42ccc5da54ce01651960cf7cf970
Author: John Dickinson <me at not.mn>
Date: Tue Jan 19 10:30:30 2016 -0800
Bump eventlet min version to 0.17.4
IPv6 support in Swift is dependent on IPv6 support in eventlet.
eventlet itself only claims support for IPv6 post v0.17
(https://github.com/eventlet/eventlet/issues/8). This update matches
the OpenStack global requirements version.
Change-Id: I9d8433cdd3bf7d7a93b8f50b991cc21721a80d22
commit 133a3ea601a3fea84af36a42845f27b8182fd901
Author: Christopher Bartz <bartz at dkrz.de>
Date: Mon Dec 21 14:17:00 2015 +0100
Use the correct split_path in handle_request
Change-Id: I86d423309f0b2091ee2e82b2245caf925b6a75ef
Closes-Bug: #1528189
commit bf10974cdefffdaaebc58d21e8a9912638a0405a
Author: Tim Burke <tim.burke at gmail.com>
Date: Wed Dec 16 15:46:13 2015 -0800
Expose token expiration time in tempauth auth response
Previously, we gave no indication of when a token would expire. Users
would have to just use it until it stopped working, then re-auth.
Now, a successful auth response will include a new header,
X-Auth-Token-Expires, with the number of seconds remaining until the
token is invalid. This allows the client to attempt to re-auth before
sending a request that will definitely fail.
For comparison, swauth already uses the X-Auth-Token-Expires header with
identical semantics. Additionally, Keystone (v2 and v3) already exposes
expiration times in its JSON responses. The security impact should be
minimal.
Change-Id: I5a4a74276bc0df6dda94e4bc150065c0d77de0eb
commit 47e226418bad35ccad2a1525f392ba69f6165027
Author: OpenStack Proposal Bot <openstack-infra at lists.openstack.org>
Date: Mon Jan 18 06:20:14 2016 +0000
Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure
Change-Id: Ic416c9afc8a1c76f552803a7c70fc905eda7b3cb
commit 5d449471b12c67b31ebb5a383d9bb35bace36213
Author: Samuel Merritt <sam at swiftstack.com>
Date: Thu Jan 14 17:26:01 2016 -0800
Remove some Python 2.6 leftovers
Change-Id: I798d08722c90327c66759aa0bb4526851ba38d41
commit 3c0cf549f1e822cce8f905b069b317e676cf306b
Author: Samuel Merritt <sam at swiftstack.com>
Date: Wed Jan 13 18:08:45 2016 -0800
Speed up get_more_nodes() when there is an empty zone
The ring has some optimizations in get_more_nodes() so that it can
find handoffs that span all the regions/zones/et cetera and then stop
looking. The stopping is the important part.
Previously, it would quickly find a handoff in each unused region,
then spend way too long looking for more unused regions; the same was
true for zones, IPs, and so on. Thus, in commit 9cd7c6c, we started
counting regions and zones, then stopping when we found them all.
This count included all regions and zones in the ring, regardless of
whether or not there were actually any parts assigned or not. In rings
with an empty region, i.e. a region for which there are only
zero-weight devices, get_more_nodes() would be very slow.
This commit ignores devices with no assigned partitions when counting
regions, zones, and so forth, thus greatly speeding things up.
The output of get_more_nodes() is unchanged. This is purely an
optimization.
Closes-Bug: 1534303
Change-Id: I4a5c57205e87e1205d40fd5d9458d4114e524332
commit 70047709fc9885df7019f791e17a3240682cc6cb
Author: keliang <ke.liang at easystack.cn>
Date: Fri Jan 15 00:31:51 2016 +0800
Drop python 2.6 support
Change-Id: Id6329c863dacb189fccfc304453ed7b6f9607c14
commit a4c2fe95ab2fbe59379a69914ed0fac49c28efbb
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Tue Jan 12 21:26:33 2016 +0100
Allow to change auditor sleep interval in config
Change-Id: Ic451c5e0b686509f8982ed1bf65a223a2d77b9a0
commit edc823e8030640184071fee4920d34f9a1cc6b3e
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Sun Nov 29 18:46:47 2015 +0100
Show UTC time in swift-recon.
It's not consistent now for example local time in replication part and
UTC time at begging of line. Use _ptime in swift-recon for all time
printing and this function returns UTC now.
Change-Id: I732d9851db157130a08e825e8093b7e244b63e9c
commit fa5b32d27964478dfcccf71155d2aaa946c561f0
Author: Tim Burke <tim.burke at gmail.com>
Date: Tue Jan 12 14:18:30 2016 -0800
Make object-auditor storage-policy-aware
Previously, the object-auditor would always use a (replication)
DiskFileManager when walking through AuditLocations, which would cause
it to skip EC fragment archives with a warning like:
Unexpected file <hash_path>/1452557187.03610#3.data:
Invalid Timestamp value in filename '1452557187.03610#3.data'
Now, the AuditLocation's policy will be used to find an appropriate
manager to get the diskfile. Additionally, several .commit()s were added
to the auditor tests so the .durable will be written, which is required
when auditing EC fragment archives.
Change-Id: I960e7d696fd9ad704ca1872b4ac821f9078838c7
Closes-Bug: 1533002
commit 33476460239c9cdb08dd8065d22d84a4717da7be
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Fri Jan 8 16:15:54 2016 -0800
fixups for ipv6 memcache_servers docs
Change-Id: I20d91c1e276014eaf210fa9eb43788bc17f4e8df
commit d5ff5447be30b44bf4acc8b912b6241a44f710be
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Jan 12 16:42:06 2016 -0800
Install liberasurecode packages in SAIO.
Change-Id: If673afa2b61a3e388612debf4860d561960963a3
commit 4ffc4ba411f67c8407ba38d082a3a51a96ad7e04
Author: Jonathan Hinson <jlhinson at us.ibm.com>
Date: Tue Jan 12 11:46:21 2016 -0600
Functional tests for if-match with multiple etags
Multiple etags can be provided on an if-match or if-none-match
request. This is currently being tested in the unit tests, but not
in the functional tests. Since these etags can be modified by
middleware, we need functional tests to assert multiple-etag
requests are handled correctly.
Change-Id: Idc409c85e8aa82b59dc2bc28af6ca2617de82699
commit e6194113a3c81563590eabf8f761ccb988bb917c
Author: Tim Burke <tim.burke at gmail.com>
Date: Fri Jan 8 16:38:31 2016 -0800
Validate X-Timestamps
Previously, attempting to PUT a new object with an X-Timestamp header
less than or equal to zero (ie, for a timestamp on or before 1970-01-01
00:00:00) would cause the object-server to 500.
While we're at it, cap X-Timestamp at 9999999999 (2286-11-20 17:46:40)
so we don't get an eleventh digit before the decimal point.
Closes-Bug: 1532471
Change-Id: I23666ec8a067d829eaf9bfe54bd086c320b3429e
commit 1f3304c5153e01988b8f4493875b6489e93f76d0
Author: Ben Martin <blmartin at us.ibm.com>
Date: Mon Dec 14 15:28:17 2015 -0600
Print min_part_hours lockout time remaining
swift-ring-builder currently only displays min_part_hours and
not the amount of time remaining before a rebalance can occur.
This information is readily available and has been displayed
as a quality of life improvement.
Additionally, a bug where the time since the last rebalance
was always updated when rebalance was called regardless of
if any partitions were reassigned. This can lead to partitions
being unable to be reassigned as they never age according to
the time since last rebalance.
Change-Id: Ie0e2b5e25140cbac7465f31a26a4998beb3892e9
Closes-Bug: #1526017
commit 167bb5eeb82886d67c1b382417fb22b8ea85f0d3
Author: Timur Alperovich <timuralp at swiftstack.com>
Date: Wed Dec 16 12:07:27 2015 -0800
Fix IPv6 handling in MemcacheConnPool.
The patch removes the assumption of IPv4-only addresses in the
MemcacheConnPool. The changes are around address handling.
Namely, if a server is specified with an address
[<address>]:port (port is optional), it is assumed to be an IPv6
address [1]. If an IPv6 address is specified without "[]", an exception
is raised as it is impossible to parse such addresses correctly.
For testing, memcache can be configured to listen on the link-local,
unique-local, or ::1 (equivalent to 127.0.0.1) addresses. Link-local
addresses are assigned by default to each interface and are of the form
"fe80::dead:beef". These addresses require a scope ID, which would look
like "fe80::dead:beef%eth0" (replacing eth0 with the correct interface).
Unique-local addresses are any addresses in the fc00::/7 subnet. To add
a ULA to an interface use the "ip" utility. For example:
"ip -6 address add fc01::dead:beef dev eth0". Lastly, and probably
simplest, memcache can be configured to listen on "::1". The same
address would be used in the swift configuration, e.g. "[::1]:11211".
Note: only memcached version 1.4.25 or greater supports binding to an
IPv6 address.
Fixes #1526570
[1] IPv6 host literals:
https://tools.ietf.org/html/rfc3986#section-3.2.2
Change-Id: I8408143c1d47d24e70df56a08167c529825276a2
commit fb6751d8ba133c57e1ebb76be71a96f2f120b8ca
Author: Paul Dardeau <paul.dardeau at intel.com>
Date: Fri Jan 8 22:49:05 2016 +0000
Look for device holes that can be reused when adding new device.
Change-Id: I1980ebdd9dc89848173d8ca2fe2afb74029dcfa2
Closes-Bug: 1532276
commit b35f3c90bde8a7ccb50440bda5800cbb8274a5a1
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Fri Jan 8 01:29:11 2016 -0800
Add note COPY with conditional headers
Swift now supports Range header for COPY (or PUT with X-Copy-From)
to make a partial copied object. This patch adds the note to show
it obviously supported in Swift community.
Change-Id: I6bf28f0932c90e7b305cd61aabce4ed028ae691e
Partial-Bug: #1532126
commit 23c7a58f8f1412c28b3a16b79be09c224c9f7d55
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Fri Dec 11 18:26:34 2015 +0900
Fix ClientException handling in Container Sync
swift/container/sync.py uses swift.common.internal_client.delete_object
and put_object and expected these methods raise ClientException.
But delete_object and put_object never raise the exception so this patch
raises ClientException when urllib2 library raises HTTPError.
Co-Authored-By: Eran Rom <eranr at il.ibm.com>
Closes-Bug: #1419901
Change-Id: I58cbf77988979a07998a46d9d81be84d29b0d9bf
commit 6786cdf036b4faabe3928c3d0dd9615d94834801
Author: Harshada Mangesh Kakad <harshadak at metsi.co.uk>
Date: Thu Dec 31 01:44:00 2015 -0800
Fixing the deprecated library function.
os.popen() is deprecated since version 2.6. Resolved with use of
subprocess module.
Change-Id: I4409cdd9edbc1a26d6f99c125c9100fadda5d758
Partial-Bug: #1529836
commit 85a0a6a28e166bc076cf8786de2b46248d8786a2
Author: Eran Rom <eranr at il.ibm.com>
Date: Sun Jul 26 13:31:17 2015 +0300
Container-Sync to iterate only over synced containers
This change introduces a sync_store which holds only containers that
are enabled for sync. The store is implemented using a directory
structure that resembles that of the containers directory, but has
entries only for containers enabled for sync.
The store is maintained in two ways:
1. Preemptively by the container server when processing
PUT/POST/DELETE operations targeted at containers with
x-container-sync-key / x-container-sync-to
2. In the background using the containers replicator
whenever it processes a container set up for sync
The change updates [1]
[1] http://docs.openstack.org/developer/swift/overview_container_sync.html
Change-Id: I9ae4d4c7ff6336611df4122b7c753cc4fa46c0ff
Closes-Bug: #1476623
commit e75888b281d59df0889f28d0b32241dac3a34aa2
Author: HugoKuo <tonytkdk at gmail.com>
Date: Wed Jan 6 14:33:23 2016 +0800
Add more description for write_affinity_node_count parameter in the doc.
Change-Id: Iad410a2be4f9a2cd5c53e860b9f91993aa7f2369
Closes-Bug: #1531173
commit f53cf1043d078451c4b9957027bf3af378aa0166
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Tue Jan 5 20:20:15 2016 +0100
Fixed few misspellings in comments
Change-Id: I8479c85cb8821c48b5da197cac37c80e5c1c7f05
commit 3b1591f235f4b85796917507be5e7fd80365ff9e
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Wed Sep 30 19:08:09 2015 +0200
swift-init: New option kill-after-timeout
This option send SIGKILL to daemon after kill_wait period.
When daemon hangs and doesn't respond to SIGTERM/SIGHUP
there is no way to stop it using swift-init now. Classic
init scripts in Linux kills hanged process after grace
period and this patch add same behaviour. This is most
usefull when using "restart" on hanged daemon.
Change-Id: I8c932b673a0f51e52132df87ea2f4396f4bba9d8
commit 79222e327f9df6335b58e17a6c8dd0dc44b86c17
Author: ChangBo Guo(gcb) <eric.guo at easystack.cn>
Date: Sat Dec 26 13:13:37 2015 +0800
Fix AttributeError for LogAdapter
LogAdapter object has no attribute 'warn' but has attribute
'warning'.
Closes-Bug: #1529321
Change-Id: I0e0bd0a3dbc4bb5c1f0b343a8809e53491a1da5f
commit 684c4c04592278a280032002b5313b171ee7a4c0
Author: janonymous <janonymous.codevulture at gmail.com>
Date: Sun Aug 2 22:47:42 2015 +0530
Python 3 deprecated the logger.warn method in favor of warning
DeprecationWarning: The 'warn' method is deprecated, use 'warning'
instead
Change-Id: I35df44374c4521b1f06be7a96c0b873e8c3674d8
commit d0a026fcb8e8a9f5475699cc56e1998bdc4cd5ca
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Wed Dec 16 18:50:37 2015 +0900
Fix duplication for headers in Access-Control-Expose-Headers
There are following problems with Access-Control-Expose-Headers.
* If headers in X-Container-Meta-Access-Control-Expose-Headers are
configured, the headers are kept with case-sensitive string.
Then a CORS request comes, the headers are merged into
Access-Control-Expose-Headers as case-sensitive string even if
there is a same header which is not case-sensitive string.
* Access-Control-Expose-Headers is handled by a list.
If X-Container/Object-Meta-XXX is configured in container/object
and X-Container-Meta-Access-Control-Expose-Headers, same header
is listed in Access-Control-Expose-Headers.
This patch provides a fix for the problems.
Change-Id: Ifc1c14eb3833ec6a851631cfc23008648463bd81
commit 0bcd7fd50ec0763dcb366dbf43a9696ca3806f15
Author: Bill Huber <wbhuber at us.ibm.com>
Date: Fri Nov 20 12:09:26 2015 -0600
Update Erasure Coding Overview doc to remove Beta version
The major functionality of EC has been released for Liberty and
the beta version of the code has been removed since it is now
in production.
Change-Id: If60712045fb1af803093d6753fcd60434e637772
commit 84ba24a75640be4212e0f984c284faf4c894e7c6
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Fri Dec 18 11:24:34 2015 +0000
Fix rst errors so that html docs are complete
rst table format errors don't break the gate job
but do cause sections of the documents to go missing
from the html output.
Change-Id: Ic8c9953c93d03dcdafd8f47b271d276c7b356dc3
commit 9fe0e25604dff35db7eab1bca312821a81db6c1d
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Tue Dec 8 22:27:44 2015 -0800
Sleep enough for trampoline
When running unite test suite in local poor resource environment,
sometimes test/unit/proxy/test_server.py fails due to a lack of
waiting time to trampoline of eventlet thread.
This patch enables to sleep 1 more second when it doesn't seem
to have enough time to tranpoline.
Change-Id: I0bbc8fc245919d3c0a071ff87ff6e20b8d58f9b8
commit 87f7e907ee412f5847f1f9ffca7a566fb148c6b1
Author: Matthew Oliver <matt at oliver.net.au>
Date: Wed Dec 16 17:19:24 2015 +1100
Pass HTTP_REFERER down to subrequests
Currently a HTTP_REFERER (Referer) header isn't passed down to
subrequests. This means *LO subrequests to segment containers
return a 403 on a *LO GET when accessed by requests using referer
ACLs.
Currently the only way around referer access to *LO's is to make the
segments container world readable.
This change makes sure the referer header is passed into subrequests
allowing a segments container to only need to be locked down with
the same referer as the *LO container.
This is a 1 line change to code, but also adds a unit and 2 functional
functional tests (one for DLO and one for SLO).
Change-Id: I1fa5328979302d9c8133aa739787c8dae6084f54
Closes-Bug: #1526575
commit e15960a5d86e00a7d420edc4af034b27da0af8fd
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Thu Dec 17 12:08:45 2015 +0000
Fix incorrect kwarg in auth middleware example
When calling memcache_client.set(), timeout was deprecated
and is now removed as a keyword arg, use time instead.
Change-Id: Iedbd5b064853ef2b386963246f639fbcd3931cd3
commit 169a7c7f9e12ebc9933bd9ca4592e13b0de8b47b
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Wed Dec 16 15:28:25 2015 +0000
Fix func test --until-failure and --no-discover options
This patch changes functional test classes to subclass
unittest2.TestCase rather than unittest.TestCase.
This fixes errors when attempting to use
tox -e func -- -n <test_path_including_test_method>
and
tox -e func -- --until-failure
Also migrate from using nose.SkipTest to unittest2.SkipTest
Change-Id: I903033f5e01833550b2f2b945894edca4233c4a2
Closes-Bug: 1526725
Co-Authored-By: Ganesh Maharaj Mahalingam <ganesh.mahalingam at intel.com>
commit b68311db95860ac1cab585a5ab66bd3b3abb765e
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Tue Dec 15 18:55:41 2015 -0800
Fix reconciler test to calc lastmodified as UTC
Swift reconciler calculates the last-modified date as UTC but
current test calculates it as local time zone. It triggers unit
test failure in non-UTC environment.
This patch fixes the test to calculate the last-modified as UTC
as well.
Change-Id: Ia0053f350daf2cb8c61ac01a933924b6e4b0cb37
Closes-Bug: #1526588
commit 1bb665331af92422290fb585de7cb6a2497236e6
Author: Venkateswarlu Pallamala <p.venkatesh551 at gmail.com>
Date: Mon Nov 9 19:22:38 2015 -0800
remove unused parameters in the method
make the helper methods as private by using convention
Change-Id: I73b9604f8d5a0e85d012aac42b7963b618f5ad97
commit 9d7f71d5754c8b45f8e7c6ab80202de09933afb8
Author: Richard Hawkins <hurricanerix at gmail.com>
Date: Fri Aug 7 18:14:13 2015 -0500
Modify functional tests to use ostestr/testr
Defcore uses Tempest, which uses Test Repository.
This change makes it easier for Defcore to pull functional
tests from Swift and run them. Additionally, using testr
allows tests to be run in parallel.
Concurrency set to 1 for now, >1 causes failures for
reasons that are still TBD.
With switch to ostestr all the server logs are being sent to stdout
which makes it completely unreadable. Suppressing the logs by default
now with a flag to enable it if desired.
Co-Authored-By: John Dickinson <me at not.mn>
Co-Authored-By: Robert Collins <rbtcollins at hpe.com>
Co-Authored-By: Matthew Oliver <matt at oliver.net.au>
Co-Authored-By: Ganesh Maharaj Mahalingam <ganesh.mahalingam at intel.com>
Change-Id: I53ef4a116996a772cf1f3abc2eb0ad60047322d5
Related-Bug: 1177924
commit 2f4b79233e30d42140bbc07059417443bf7a0757
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Tue Dec 15 15:49:42 2015 +0000
Minor cleanup of repeated identical test assertions
assertDictContainsSubset is being called multiple times with
same arguments in a loop. Since assertDictContainsSubset is
deprecated form python 3.2, replace it with checks on
individual key, value pairs.
Change-Id: I7089487710147021f26bd77c36accf5751855d68
commit 60b2e02905d57f55169e506f4874b2334a1a68a5
Author: Alistair Coles <alistair.coles at hp.com>
Date: Mon Oct 5 16:15:29 2015 +0100
Make ECDiskFile report all fragments found on disk
Refactor the disk file get_ondisk_files logic to enable
ECDiskfile to gather *all* fragments found on disk (not just those
with a matching .durable file) and make the fragments available
via the DiskFile interface as a dict mapping:
Timestamp --> list of fragment indexes
Also, if a durable fragment has been found then the timestamp
of the durable file is exposed via the diskfile interface.
Co-Authored-By: Clay Gerrard <clay.gerrard at gmail.com>
Change-Id: I55e20a999685b94023d47b231d51007045ac920e
commit 450737f886050e486f518cdce0c97596ccad848d
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Tue Dec 15 11:33:56 2015 +0900
Fix a typo in development_auth.rst
This patch uses correct name for "CORS".
Change-Id: I5fee5c581a2b3adb7596a273baf05708bfa97f79
commit 40476ea0797690d3a90a9ed91906d26103dfa058
Author: John Dickinson <me at not.mn>
Date: Mon Dec 14 10:52:22 2015 -0800
Document pretend_min_part_hours_passed
Added a docstring for the swift-ring-builder CLI command
"pretend_min_part_hours_passed". This is a dangerous operation, and
that's why it hasn't been documented, but it can be useful at times.
It should be made known to those who need it.
Change-Id: I45bdbaacbbdda64c7510453e6d93e6d8563e3ecd
commit 6ade2908cca696ce1b48a7a19f4d460081fa5b0a
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Sun Dec 13 21:13:42 2015 +0100
Deprecated param timeout removed from memcached
Change-Id: Idf042a79f0db148bf9f28a9e360cb2a3c18d385a
commit 88c9aed7c846402355a3c7831f34f3e833bbdf11
Author: Victor Stinner <vstinner at redhat.com>
Date: Mon Oct 19 16:19:28 2015 +0200
Port swift.common.utils.StatsdClient to Python 3
* StatsdClient._send(): on Python 3, encode parts to UTF-8 and
replace '|' with b'|' to join parts.
* timing_stats(): replace func.func_name with func.__name__. The
func_name attribute of functions was removed on Python 3, whereas
the __name__ attribute is available on Python 2 and Python 3.
* Fix unit tests to use bytes
Change-Id: Ic279c9b54e91aabcc52587eed7758e268ffb155e
commit ca2dcc371921aa1aded6161287cc03c0940bf198
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Fri Dec 11 18:21:28 2015 +0100
Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0
More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching
Change-Id: I9451a0f0dee5c5a3c0ca0a52f58bd353602661a2
commit 211758f8cb02298fe16e59bf2954a146c6b24b83
Author: Catherine Northcott <catherine at northcott.nz>
Date: Thu Nov 5 23:04:14 2015 +1300
Add support for storage policies to have more than one name
This patch alters storage_policy.py to allow storage policies
to have multiple names. Now users are able to add a number of
human-readable aliases for storage policies. Policies now have
a .name (the default name), .aliases (a string of comma
seperated aliases), and .aliases_list (a list of all human
readable names). Policies will always have an .aliases value
if no aliases are set it will contain the default name.
The policy docs and tests have been updated to reflect changes
and policy.get_policy_info has been altered to display the
name and aliases
Change-Id: I02967ca8d7c790595e5ee551581196aa64552eea
commit 7f636a557296ecc6ae4727700cfcf9f82573bd16
Author: Samuel Merritt <sam at swiftstack.com>
Date: Mon Nov 30 18:06:09 2015 -0800
Allow smaller segments in static large objects
The addition of range support for SLO segments (commit 25d5e68)
required the range size to be at least the SLO minimum segment size
(default 1 MiB). However, if you're doing something like assembling a
video of short clips out of a larger one, then you might not need a
full 1 MiB.
The reason for the 1 MiB restriction was to protect Swift from
resource overconsumption. It takes CPU, RAM, and internal bandwidth to
connect to an object server, so it's much cheaper to serve a 10 GiB
SLO if it has 10 MiB segments than if it has 10 B segments.
Instead of a strict limit, now we apply ratelimiting to small
segments. The threshold for "small" is configurable and defaults to 1
MiB. SLO segments may now be as small as 1 byte.
If a client makes SLOs as before, it'll still be able to download the
objects as fast as Swift can serve them. However, a SLO with a lot of
small ranges or segments will be slowed down to avoid resource
overconsumption. This is similar to how DLOs work, except that DLOs
ratelimit *every* segment, not just small ones.
UpgradeImpact
For operators: if your cluster has enabled ratelimiting for SLO, you
will want to set rate_limit_under_size to a large number prior to
upgrade. This will preserve your existing behavior of ratelimiting all
SLO segments. 5368709123 is a good value, as that's 1 greater than the
default max object size. Alternately, hold down the 9 key until you
get bored.
If your cluster has not enabled ratelimiting for SLO (the default), no
action is needed.
Change-Id: Id1ff7742308ed816038a5c44ec548afa26612b95
commit 73d0f1620a269f990dbd3d2796abf27e9a05e227
Author: Béla Vancsics <vancsics at inf.u-szeged.hu>
Date: Tue Dec 8 10:17:08 2015 +0100
Not used parameter
The account variable was not used in the method.
Change-Id: I8e91d7616529f33b615bc52af76bfda01141d364
** Tags added: in-feature-crypto
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to swift in Ubuntu.
https://bugs.launchpad.net/bugs/1493303
Title:
[OSSA 2016-004] Swift proxy memory leak on unfinished read
(CVE-2016-0738)
Status in Ubuntu Cloud Archive:
New
Status in OpenStack Security Advisory:
Fix Released
Status in OpenStack Object Storage (swift):
Fix Released
Status in swift package in Ubuntu:
Confirmed
Bug description:
It looks like the Swift proxy will leak memory if the connection is
closed and the full response is not read. This opens for a potential
DoS attacks.
Reproduce:
$ swift -A http://localhost:8888/auth/v1.0 -U .. -K .. upload --use-slo --segment-size 1048576 <container> <big-file>
$ curl -H'X-Auth-Token: AUTH_...' "http://localhost:8888/v1/AUTH_../<container>/<big-file>" -m 0.001 > /dev/null
Repeat the curl command a couple of times and you will have more
information in netstat and sockstat. The important part is the -m
which sets the max time curl spends at downloading. After that point,
it'll close the connection.
$ sudo netstat -ant -p | grep :6000
$ cat /proc/net/sockstat
tcp 0 0 127.0.0.1:6000 0.0.0.0:* LISTEN 1358/python
tcp 0 43221 127.0.0.1:6000 127.0.0.1:48350 FIN_WAIT1 -
tcp 0 43221 127.0.0.1:6000 127.0.0.1:48882 FIN_WAIT1 -
tcp 939820 0 127.0.0.1:48350 127.0.0.1:6000 ESTABLISHED 17897/python
tcp 939820 0 127.0.0.1:48882 127.0.0.1:6000 ESTABLISHED 17890/python
tcp 983041 0 127.0.0.1:48191 127.0.0.1:6000 CLOSE_WAIT 17897/python
tcp 983041 0 127.0.0.1:48948 127.0.0.1:6000 CLOSE_WAIT 17892/python
Restarting the proxy frees up the lingering memory.
This problem did not exist in 2.2.0.
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: swift 2.2.2-0ubuntu1~cloud0 [origin: Canonical]
ProcVersionSignature: Ubuntu 3.16.0-48.64~14.04.1-generic 3.16.7-ckt15
Uname: Linux 3.16.0-48-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.12
Architecture: amd64
CrashDB:
{
"impl": "launchpad",
"project": "cloud-archive",
"bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
}
Date: Tue Sep 8 09:55:05 2015
InstallationDate: Installed on 2015-06-22 (77 days ago)
InstallationMedia: Ubuntu-Server 14.04.2 LTS "Trusty Tahr" - Release amd64 (20150218.1)
PackageArchitecture: all
SourcePackage: swift
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1493303/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list