[Bug 1493303] Re: [OSSA 2016-004] Swift proxy memory leak on unfinished read (CVE-2016-0738)
OpenStack Infra
1493303 at bugs.launchpad.net
Tue Mar 8 23:31:36 UTC 2016
Reviewed: https://review.openstack.org/290148
Committed: https://git.openstack.org/cgit/openstack/swift/commit/?id=0f7f1de233919a0b046349a3e31ae7fc8675a1c5
Submitter: Jenkins
Branch: feature/hummingbird
commit d6b4587a554b51ba733b151e0d924735b63d07e0
Author: Olga Saprycheva <osapryc at us.ibm.com>
Date: Tue Mar 8 10:57:56 2016 -0600
Removed redundant file for flake8 check
Change-Id: I4322978aa20ee731391f7709bbd79dee140fc703
commit 643dbce134140530eef2ae62c42fef1107f905ed
Author: OpenStack Proposal Bot <openstack-infra at lists.openstack.org>
Date: Tue Mar 8 06:35:49 2016 +0000
Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure
Change-Id: I96b8ff1287bf219c5f8d56a3a4868c1063a953f9
commit 83713d37f0331c5ce9d377f4b4e8724551ae30ca
Author: Daisuke Morita <morita.daisuke at ntti3.com>
Date: Mon Mar 7 18:30:47 2016 -0800
Missing comments for storage policy parameter
There are missing comments about storege_policy_index so appropriate
comments are added.
Change-Id: I3de3f0e6864e65918ca1a13cce70f19c23d295f5
commit 2cff2dec3d1c4588f5103e39679c43b3dded6dcb
Author: Olga Saprycheva <osapryc at us.ibm.com>
Date: Fri Mar 4 15:19:39 2016 -0600
Fixed pep8 and flake8 errors in doc/source/conf.py and updated flake8 commands in tox.ini to test it.
Change-Id: I2add370e4cfb55d1388e3a8b41f688a7f3f2c621
commit 043fbca6d08648baa314ea2236f1ccdca8785f16
Author: Christian Schwede <cschwede at redhat.com>
Date: Fri Mar 4 09:33:17 2016 +0000
Remove Erasure Coding beta status from docs
This removes notes stating support for Erasure coding as beta. Questions
regarding the stability of EC are coming up regularly, and are often referring
to the docs that state EC as still in beta.
Besides this, a note marking statsd support as beta has been removed as well.
Change-Id: If4fb6a5c4cb741d42953db3cee8cb17a1d774e15
commit 09c73b86e9255f28fbd4cf571a52c17d549a8f9a
Author: Pete Zaitcev <zaitcev at kotori.zaitcev.us>
Date: Thu Mar 3 10:24:28 2016 -0700
Fix a crash in exception printout
Says the number of arguments does not match the number of '%'.
Change-Id: I8b5e395a07328fb9d4ac7a19f8ed2ae1637bee3b
commit fad5fabe0a22e8a86635a66523dd3d3d3b1fa705
Author: Tim Burke <tim.burke at gmail.com>
Date: Thu Mar 3 15:07:08 2016 +0000
During functional tests, 404 response to a DELETE is successful
Previously, we would only consider 204 responses successful, which would
cause some spurious gate failures, such as
http://logs.openstack.org/66/287666/3/check/gate-swift-dsvm-functional/c6d2673/console.html#_2016-03-03_13_41_07_846
Change-Id: Ic8c300647924352a297a2781b50064f7657038b4
commit e91de49d6864b3794f8dc5acd9c1bf0c2f7409d1
Author: Alistair Coles <alistair.coles at hp.com>
Date: Mon Aug 10 10:30:10 2015 -0500
Update container on fast-POST
This patch makes a number of changes to enable content-type
metadata to be updated when using the fast-POST mode of
operation, as proposed in the associated spec [1].
* the object server and diskfile are modified to allow
content-type to be updated by a POST and the updated value
to be stored in .meta files.
* the object server accepts PUTs and DELETEs with older
timestamps than existing .meta files. This is to be
consistent with replication that will leave a later .meta
file in place when replicating a .data file.
* the diskfile interface is modified to provide accessor
methods for the content-type and its timestamp.
* the naming of .meta files is modified to encode two
timestamps when the .meta file contains a content-type value
that was set prior to the latest metadata update; this
enables consistency to be achieved when rsync is used for
replication.
* ssync is modified to sync meta files when content-type
differs between local and remote copies of objects.
* the object server issues container updates when handling
POST requests, notifying the container server of the current
immutable metadata (etag, size, hash, swift_bytes),
content-type with their respective timestamps, and the
mutable metadata timestamp.
* the container server maintains the most recently reported
values for immutable metadata, content-type and mutable
metadata, each with their respective timestamps, in a single
db row.
* new probe tests verify that replication achieves eventual
consistency of containers and objects after discrete updates
to content-type and mutable metadata, and that container-sync
sync's objects after fast-post updates.
[1] spec change-id: I60688efc3df692d3a39557114dca8c5490f7837e
Change-Id: Ia597cd460bb5fd40aa92e886e3e18a7542603d01
commit 3c61ab4678a7aa9ff256ace4bc97ab449607fd49
Author: asettle <alexandra.settle at rackspace.com>
Date: Wed Feb 10 17:58:05 2016 +1000
Operational procedures guide
This is the operational procedures guide that HPE used
to operate and monitor their public Swift systems.
It has been made publicly available.
Change-Id: Iefb484893056d28beb69265d99ba30c3c84add2b
commit 8e4347afd59aa30708ef03a78202652aaadda2e9
Author: Takashi Kajinami <kajinamit at nttdata.co.jp>
Date: Tue Nov 17 16:15:59 2015 +0900
Fix proxy-server's support for chunked transferring in GET object
Proxy-server now requires Content-Length in the response header
when getting object and does not support chunked transferring with
"Transfer-Encoding: chunked"
This doesn't matter in normal swift, but prohibits us from putting
any middelwares to execute something like streaming processing of
objects, which can't calculate the length of their response body
before they start to send their response.
Change-Id: I60fc6c86338d734e39b7e5f1e48a2647995045ef
commit 9dda1c9526c5b3e0ac905985921605d833bef3cc
Author: Takashi Kajinami <kajinamit at nttdata.co.jp>
Date: Fri Dec 25 10:30:01 2015 +0900
Update parameters about authtoken middleware in proxy-server.conf.sample
This patch replaces some configration parameters about authtoken
middleware in proxy-server.conf, which will be deprecated.
The following parameters are to be deprecated.
auth_admin_prefix, auth_host, auth_port, auth_protocol,
identity_uri, admin_token, admin_user, admin_password,
admin_tenant_name
Change-Id: I373734933189c87c4094203b0752dd3762689034
commit 087fa4fa01fc36343f2ce0974d3b36ec0b98afc8
Author: Andreas Jaeger <aj at suse.com>
Date: Mon Feb 29 20:00:38 2016 +0100
Remove unused pngmath Sphinx extension
There's no RST file that uses ".. math" and thus
the pngmath Sphinx extension is not used and can
get removed.
Change-Id: I3ea4b529025c8ac3c4092f3720124647861e6668
commit 9c5b01e7c7699fbfd221e98ccde1afa9361c8119
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Wed Dec 9 08:08:22 2015 +0000
Add container quota functional test
This patch provides one container quota functional test.
Change-Id: I16354c15fc434d70f9bb6b9b1e18b09357195c83
commit efdc6ee8dbc434e9189a6b85b924c322c3ff6ae3
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Thu Dec 10 01:15:30 2015 +0000
Add X-Delete-After and X-Delete-At functional test
This patch provides functional tests for X-Delete-After and X-Delete-At.
Change-Id: I55b0d0584428413aa7a31f66defec55f9b1bbcd9
commit bd93d44bb46350f716f1dfb66fb43c2ab4ad8c50
Author: Takashi Kajinami <kajinamit at nttdata.co.jp>
Date: Sun Feb 28 13:41:19 2016 +0900
Make sure all temp files get deleted in test_ringbuilder
This patch makes test_ringbuilder create a temporal directory,
run ring builder commands under it, and delete it after testing
for each test cases, to fix temp file leaking.
Change-Id: I6f59fe095ea6485af0e60b5a8e8fc3892e0a0f90
commit 10b5765c60790f8a3ae2a94d6a51736ec2c99ce0
Author: benjkeller <bjkeller at us.ibm.com>
Date: Thu Feb 25 18:54:00 2016 -0600
Rename test_object_delete_at_aysnc
This renames test_object_delete_at_aysnc to
test_object_delete_at_async to match with the function
async_update which it tests.
Change-Id: I726afcbf08a3449c7af2834e573b97be378a86cd
Closes-Bug: 1550067
commit 9e53bb47efa28983109da8ceca5d316925bc6804
Author: John Dickinson <me at not.mn>
Date: Wed Feb 24 09:36:40 2016 -0800
updated some weird whitespace character to a normal one
Change-Id: I187fb8fc8d10f950bbebc586999eced0bc117432
commit 54e53ae87b6e87229dc77cb7c814ab9733ea7013
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Fri Feb 19 09:30:10 2016 +0000
Silence SkipTest noise in in-process test teardown
setup_package and teardown_package get called twice when
running functional tests using nose. With in-process mode
the first call to teardown_package clears global config
which causes the test swiftclient to raise a SkipTest error
during the second call to teardown_package.
To reproduce:
SWIFT_TEST_IN_PROCESS=1 nosetests ./test/functional/test_account.py
This patch simply tests for config existence before attempting
to create a test swiftclient in the teardown_package function.
Also fix a related issue whereby in_process flag would be reset to
False during second call to setup_package, thus causing some of the
in process cleanup in teardown_package to never be executed.
Change-Id: I074dcd3d39aa46b262632024b047556ca471e8b8
commit 076134784b4eb1c2e55b28218dc0d64a57795d63
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Tue Feb 23 17:28:08 2016 -0800
Add warn method back to LogAdapter
The change in 684c4c04 to improve py3 support breaks older middleware
that consumes the SwiftLogger.
Good for Swift code to not to use warn because it's deprecated.
Bad for LogAdapter to not faithfully provide the current python logging
module API (deprecated or otherwise).
Change-Id: Ib0baf66b7f33f91562867076ff0526a287d8e5a6
commit eb7ca115e6501edd9225dce8784b8b3d3e094bb1
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Feb 23 14:00:34 2016 -0800
Fix StatsD tests to not use real DNS
In common/test_utils.py, TestStatsdLogging had the majority of its
test cases calling the real socket.getaddrinfo(), which uses real
DNS. This is very slightly slower than using a mock getaddrinfo() when
the machine running the tests has functioning DNS, but on a machine
with no network connection at all, the tests are excruciatingly slow
due to timeouts.
This commit mocks things out as appropriate. There's still one user of
the real getaddrinfo(), but it's for ::1, so that's just local
resolution based on /etc/hosts.
Timing numbers for "./.unittests test.unit.common.test_utils:TestStatsdLogging":
* network, without this patch: 1.8s
* no network, without this patch: 221.2s (ouch)
* network, with this patch: 1.1s
* no network, with this patch: 1.1s
Change-Id: I1a2d6f24fc9bb928894fb1fd8383516250e29e0c
commit b97adf70ade2982fea5354c55f1eea9f98442fa7
Author: Thiago da Silva <thiago at redhat.com>
Date: Tue Feb 23 16:23:51 2016 -0500
fix formatting of bulk operations doc
Change-Id: I247d2867c55d6bbf2288dfb74271f57c6f88b243
commit cfd50212358372694ba0713ff8c3bc313c6f1a92
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Thu Feb 18 21:06:40 2016 -0800
dry out container unittests
Change Ifcc4392ffa4df485ce3c62a35d0fcc8c93de74aa introduced
TestContainerController4Replicas as a subclass of
TestContainerController. And there is duplicated code
block to override setUp method.
This patch removed the duplicated code block.
Change-Id: Iad0b64cba52554b89dbd959475a2ce7c0d9a20eb
commit 406e8996fb288e9851e02c3f3faa536a9919b28f
Author: Takashi Kajinami <kajinamit at nttdata.co.jp>
Date: Mon Feb 22 16:10:09 2016 +0900
Use 201 insted of 200 in PUT test case
There are some unit test cases which assumes 200 as the response
of PUT request, but 200 is never returned in PUT.
This patch replaces 200 in PUT by 201, and adapt unit test cases
to real case.
Change-Id: I35fc7a4fe7fcc5558a9e6c6371105793c2710f3e
commit 5b18455cef57d052df3cda3e9179c923d3aadcad
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Fri Jan 22 07:06:45 2016 +0000
Add unit tests for checking responses in ContainerController
This patch provides unit tests to check responses for various situation
in three/four replicas env.
Change-Id: Ifcc4392ffa4df485ce3c62a35d0fcc8c93de74aa
commit 002513d21786df0d6c71e9772ec4688e2c14d8de
Author: Tim Burke <tim.burke at gmail.com>
Date: Thu Jan 7 11:56:33 2016 -0800
Make write-only container access consistent
Previously, if a user could write to (but not read from) a container,
the behavior for object POST would vary depending on whether
object_post_as_copy was enabled (403 response) or disabled (202
response).
Now, POSTs will consistently be allowed, regardless of whether fast-POST
is enabled.
Change-Id: I1d6dcbc4f5034a322a1073850fc3b059ebb1c0fa
commit 9a6c6b55713f896609b7127560ae99fabbf759e7
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Thu Feb 18 17:38:42 2016 +0100
Fixed log format in object auditor
Change-Id: I218e08446c5ffda2db7c774ec8bb9d28ce9f63e7
commit 2d55960a221c9934680053873bf1355c4690bb19
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Thu Jan 14 18:31:21 2016 +0000
Fix inconsistent suffix hashes after ssync of tombstone
Consider two replicas of the same object whose ondisk files
have diverged due to failures:
A has t2.ts
B has t1.data, t4.meta
(The DELETE at t2 did not make it to B. The POST at t4 was
rejected by A.)
After ssync replication the two ondisk file sets will not be
consistent:
A has t2.ts (ssync cannot POST t4.meta to this node)
B has t2.ts, t4.meta (ssync should not delete t4.meta,
there may be a t3.data somewhere)
Consequenty the two nodes will report different hashes for the
object's suffix, and replication will repeat, always with the
inconsistent outcome. This scenario is reproduced by the probe
test added in this patch.
(Note that rsync replication does result in (t2.ts, t4.meta)
on both nodes.)
The solution is to change the way that suffix hashes are
calculated. Currently the names of *all* files found in each
object dir are added to the hash. With this patch the
timestamps of only those files that could be used to
construct a valid diskfile are added to the hash. File
extensions are appended to the timestamp so that in most
'normal' situations the result of the hashing is the same
as before this patch. That avoids a storm of hash mismatches
when this patch is deployed in an existing cluster.
In the problem case described above, t4.meta is no longer
added to the hash, since it is not useful for constructing
a diskfile. (Note that t4.meta is not deleted because it
may become useful should a t3.data be replicated in future).
Closes-Bug: 1534276
Change-Id: I99e88b8d5f5d9bc22b42112a99634ba942415e05
commit ddeb0cde9fc828fa79dce90df4975f66737ba054
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Wed Feb 17 21:17:34 2016 -0800
Fix object-server to handle newer ts file
Swift object-server will return 409 Conflict when the incomming
request X-Timestamp is older than the local disk file to prevent
wasted data transfer and disk space consumption. However, IFF,
the local disk file is a tombstone, current object-server will
write the data in the device and cleanup it according to timestamp
comparison result in hash_cleanup_list_dir, and then return 201
Created. That's wasted and far from the semantics in the normal case.
This patch fixes object-server to catch the DiskFileDeleted exception
at PUT request, and then, set the timestamp from tombstone as
original_timestamp to be able to compare with request timestamp.
Co-Authored-By: Kazuhiro Miyahara <miyahara.kazuhiro at lab.ntt.co.jp>
Change-Id: I078c9cb90707a3c320708e76ea42cbfa73e1ea4b
Closes-Bug: #1546865
commit 878688cae00576ec0bdb0a66eca7ada95e661c0a
Author: Béla Vancsics <vancsics at inf.u-szeged.hu>
Date: Mon Dec 7 17:02:40 2015 +0100
Reduce code duplication
Reduced source code by extracting duplicated code
(swift/common/direct_client.py)
http://openqa.sed.hu/dashboard/index/5?did=1
Change-Id: I47fa6ae12c646e8e42fb0a46a017971f05d883fc
Closes-Bug: 1536067
commit c3201f256cf415988a3cb1c42e9c6b7835195fa3
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro at lab.ntt.co.jp>
Date: Fri Jan 8 18:03:33 2016 +0900
Remove execute permissions from doc files and swift.conf-sample
Some doc files and swift.conf-sample were added execute permissions in past changes.
This patch remove execute permissions from them.
Change-Id: Id8844989a8321578e9207566ebd6660f5b9523f0
commit e46d6b17251ab538595b1deb5581360a286b8fd1
Author: Christian Schwede <cschwede at redhat.com>
Date: Wed Feb 17 20:47:26 2016 +0100
Add note on using printable chars for swift_hash_path_suffix/prefix
Using unprintable characters for swift_hash_path_prefix/suffix might lead to
hard problems when parsing of these values changes, for example due to newer
Python versions or changes in the parsers itself. Let's avoid this and add a
note that deployers should use printable strings for these values.
Change-Id: I976982b753b6af831ab91d7190f50f8f15bf73bf
commit b8fa48080812ab0f6bac7ae19c0290da08f6a6e4
Author: Donagh McCabe <donagh.mccabe at hp.com>
Date: Thu Feb 4 16:19:13 2016 +0000
Added links to API reference and usage guides
Added a link to the API reference (where headers and query
strings are documented; click the "detail" button to see).
Also added a reference to Swift section of the OpenStack end
user guide. This contains some additional details about the API.
No attempt was made to reconcile duplicate information. Instead
this patch links documents that might otherwise be overlooked.
However, I fixed text, originally in a table, that had become
garbled in a prior patch.
Change-Id: I0910cbeb0c8bffc00e510f35585603e7b7a67790
commit aa7204d106ae33eba9219514f31f008510c9db53
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro at lab.ntt.co.jp>
Date: Fri Feb 12 20:48:34 2016 +0900
Remove '#! /usr/bin/env python' from unexecutable files
'cli/recon.py' and 'cli/ring_builder_analyzer.py' have '#! /usr/bin/env
python' in spite of they don't have execute permissions. This patch
removes '#! /usr/bin/env python' from them.
Change-Id: I1917ccc84b1673af3d862be1796f54595f94c5ca
commit db87a5a9889d436f2e2aacdc8923d55ca0d3d1ca
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Tue Feb 16 10:52:01 2016 -0800
Mock sleep in direct_client retry tests
... and fixup some assertions, and other mocks
Change-Id: Iadf839d1d3d735109fed24049fc3d829776b7b65
commit 8eb30afd7c39f6e6ba031f72c6a61f45b28d37f6
Author: Tim Burke <tim.burke at gmail.com>
Date: Tue Feb 16 10:25:39 2016 -0800
Static methods should be @staticmethods
Change-Id: Ifee5d68e00bbb3571aaac885cdd7490c79732985
commit 4a44e27e00795c5f12176ef346cd2249fca54fd8
Author: Chaozhe.Chen <chaozhe.chen at easystack.cn>
Date: Tue Feb 16 23:44:27 2016 +0800
Replace assertEqual(None, *) with assertIsNone in tests
As swift no longer supports Python 2.6, replace assertEqual(None, *)
with assertIsNone in tests to have more clear messages in case of
failure.
Change-Id: I94af3e8156ef40465d4f7a2cb79fb99fc7bbda56
Closes-Bug: #1280522
commit 42f4b3fc1e69a046c1e70e73afe847e4423fa2da
Author: Christian Schwede <cschwede at redhat.com>
Date: Tue Feb 16 10:08:55 2016 +0000
Add SwiftHLM to associated projects
Change-Id: I5ea3de34e61f22abec803b35fec8adde18a793e9
commit 025ebf2901d5a972796b73538bb306847d7d06b6
Author: Ankur Gupta <ankur.gupta at intel.com>
Date: Wed Feb 10 14:36:16 2016 -0600
Docstring omission in class BaseDiskFileManager.
Added documentation for missing Docstring variables.
Change-Id: I29a53b8141c5607815f234a6123e2289200bca34
commit 4906b4c431edc436f165b163a228b0a221950c79
Author: Janonymous <janonymous.codevulture at gmail.com>
Date: Tue Jan 12 12:50:43 2016 +0530
Fix missing txn_id logs in GreenAsyncPile's spawned functions
This commit ensures that the logger thread_locals
value is passed to and set in _get_conn_response methods
executed in a green thread.
Added partial bug tag because in bug description a more
relevant fix is suggested which would fix the bug completely
but for now this makes sense to add this commit for logging.
Co-Authored-By: Clay Gerrard <clay.gerrard at gmail.com>
Change-Id: I13bbf174fdca89318d69bb0674ed23dc9ec25b9a
Partial-Bug: #1409302
commit 511a1b8698889c430bf673ed916ae41a96b99c5c
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro at lab.ntt.co.jp>
Date: Fri Feb 12 19:01:59 2016 +0900
Remove execute permission from swift/cli/ringbuilder.py
In Change-Id: Ia63f59a8faca1fad990784f27532ca07a2125454,
'bin/swift-ring-builder' is renamed to 'swift/cli/ringbuilder.py'.
However, execute permission and '#! /usr/bin/env python' are left on
the file.
This patch removes execute permission and '#! /usr/bin/env python'
from the file.
Change-Id: I66370f825ce9ed9a40ec21f2745aa6aff22c8709
commit 973d9358d2e6e029409d1902831b4680e02952c9
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Thu Feb 11 10:20:10 2016 +0100
Bump PyECLib requirement to >= 1.2.0
This was discussed at last Swift meeting [1] and conclusion was
to require newest PyECLib version.
[1] http://eavesdrop.openstack.org/meetings/swift/2016/swift.2016-02-10-21.00.log.html#l-268
Depends-On: I0fd58f26bc0a3e7715356ff74344981f22786152
Change-Id: I54e89d99b48e00e614d2d457814c00354763c83b
commit d3dd137cad896c7f741592e88f076acab2636a3c
Author: Tomas Matlocha <tomas.matlocha at firma.seznam.cz>
Date: Tue Jan 5 12:32:50 2016 +0100
Additional info log message for drive-audit
An additional info log message was added for case of
running drive-audit without failed device unmounting.
Change-Id: I11abee40a712b6c6de65e63626b6f7f0a9c9f4c7
commit c4158bd674f34c8a0648c98974ba474fcd1c2b7a
Author: Sarafraj Singh <Sarafraj.Singh at intel.com>
Date: Thu Feb 11 20:52:47 2016 +0000
Fixed params on autocreate_account docstring
... in proxy.controllers.base
Change-Id: Ic5c27cbf0f19b65fcaa3e6587583f03e56d646a0
commit f56d18e143d0a38584e4c383ea9afe46e67a0af6
Author: Samuel Merritt <sam at swiftstack.com>
Date: Thu Feb 11 10:59:43 2016 -0800
Fix typos in comments
Change-Id: I4f98d447bd2ddd8eeb2f4da66d069bd7d6f00dc6
commit cc8dc88b55b44ca2a1fd65c049f3cfec22eb5525
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Thu Feb 4 16:02:42 2016 +0100
Script for checking sanity of manpages
Change-Id: Ia27b0ecf6a3267eb51f8a6562721dca72f128915
commit bcada66b9065c94cd39498a5330c727cc575a6fe
Author: root <lr699s at att.com>
Date: Wed Feb 10 09:21:53 2016 -0600
Removed unused parameter in server.py
The variable max_large_object_get_time is no longer used and was
removed to reflect the change.
Change-Id: I43051181dcb38245de6d13fab63876e83f46fc39
Closes-Bug: #1538834
commit b5311f63dbf5787bda3495a58c8ef7a6a557daa4
Author: gh159m <gh159m at att.com>
Date: Wed Feb 10 10:36:59 2016 -0600
Removed default value for log_statsd_host
Multiple files and documents showed that log_statsd_host had
a default value, usually localhost. This was incorrect, instead
setting a value for log_statsd_host enables statsd logging.
Removed any reference of log_statsd_host having a default value.
Also changed descriptions to show setting a value enables logging.
Change-Id: I3ca5c0e8b8e4981de3aa6db0c476072b5a59723d
Closes-Bug: #1542227
commit b98c1c60f9c581c76677c19baae60ae720e3c7f3
Author: Christian Berendt <berendt at b1-systems.de>
Date: Tue Feb 9 07:32:48 2016 +0100
Remove executable flag from sample config file
swift.conf-sample should not be executbale
Change-Id: I960058e2af10b43efc6324a7534d9a23c810e39a
commit 81a4355c2d0907035805c300a9fa5aaa6d0e70c4
Author: Christopher Bartz <bartz at dkrz.de>
Date: Fri Feb 5 09:51:11 2016 +0100
Adds storage policy option to recon
With this patch, recon gets policy-aware in regard to the hosts to retrieve.
If no policy is passed and the server_type is object,
all hosts of all policies are retrieved.
Previously, recon did only retrieve the hosts of the storage-policy 0.
Change-Id: If5735cd6721eac504aed8aaf3884cb91b6a0fcac
Closes-Bug: 1541491
commit 87fc21c7cfcce2a3e23a84fddcfd1309cd884716
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Mon Feb 8 15:31:42 2016 +0000
Speed up functional testing
test/functional/tests.py:TestObjectVersioningUTF8 does not clean
up the versions files it creates because the class's multiple
inheritance is such that it does not call the tearDown method in
TestObjectVersioning.
As a result, any attempt to clean up account containers wastes
time retrying container delete requests. This occurs either in
the setUp for TestSloEnv, if the TestSlo class is included in a
test run, or in the tests.py package tearDown method.
On the author's dev machine this patch reduces the execution
time of functional tests in tests.py by approx 30% or 1 minute.
Change-Id: I8194672bf2ca82435df5868720b6a55a79b94413
commit 3b94bd45406f9d457d37caa10c856a19c4cec22a
Author: Mingyu Li <li.mingyu at 99cloud.net>
Date: Sun Feb 7 21:41:08 2016 +0800
Keep the Usage of exit()/sys.exit() Consistent
It's better to keep the usage of exit()/sys.exit() consistent
in one file. Furthermore, sys.exit() is considered good to be
used in production code, while exit is for interactive shell.
Change-Id: Ia3092853a648922588e2bc11db37d6decdec1b48
commit f05da4c8fa37a285bb2ca9b45a355dbe57f7add4
Author: janonymous <janonymous.codevulture at gmail.com>
Date: Sat Feb 6 16:59:57 2016 +0530
Changing the base version to python2.7 for pep8 checks
tox was taking python3 as it's base version which lead to
tox pep8 check failing on local env:
http://paste.openstack.org/show/486183/
Change-Id: Ia822ec5b5e12a1fc30b676f1e6d95e287b9b3368
commit 13edc9a865cd203ddf28a3f782426a1b3b94f7a1
Author: Sarafraj Singh <Sarafraj.Singh at intel.com>
Date: Fri Feb 5 17:28:26 2016 +0000
Improved test coverage of form_signature
Added four new test cases to execute path of invalid arguments. This
improved test coverage of file specified from 80% to 91%.
Change-Id: I63c2e7bab3f01121301d78b687687208a58401c0
commit b173995666b026def3f6558d9a8c972b75449323
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Thu Feb 4 05:18:42 2016 -0800
Fix missing Accept-Ranges
Since commit 4f2ed8bcd0468f3b69d5fded274d8d6b02ac3d10, the response
header for GET EC object doesn't include the Accept-Ranges header.
This patch fixes it and also adds a few unittests to prevent regression.
Closes-Bug: #1542168
Change-Id: Ibafe56ac87b14bc0028953e620a653cd68dd3f84
commit d78e861d850db96989f4fb77ad31e3170034f15d
Author: Richard Hawkins <richard.hawkins at rackspace.com>
Date: Wed Jan 13 13:15:07 2016 -0600
Add option for label to static web listings
By default listings are rendered with a label of:
"Listing of /v1/account/container/path"
This change adds a new custom metadata key/value pair
"X-Container-Meta-Web-Listings-Label: <label>" that
when set, will cause the following:
"Listing of <label>/path" to be rendered instead.
Change-Id: I47c5d21834f47e566b380920196926882ff83abf
commit ae632abbd802228ef98a23c98ee49988e7e5b942
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Tue Feb 2 21:58:52 2016 +0100
Fixed manpages errors.
account-server.conf.5
105: warning: numeric expression expected (got `)')
container-server.conf.5
111: warning: numeric expression expected (got `)')
object-expirer.conf.5
79: warning: numeric expression expected (got `)')
object-server.conf.5
114: warning: numeric expression expected (got `)')
proxy-server.conf.5
121: warning: numeric expression expected (got `)')
331: warning: numeric expression expected (got `[')
1005: warning: macro `*' not defined
Change-Id: I203dcfde83035e3b1dcb91109b72b5d08bb7840e
commit f2fca9aafa17e45b668c6f76b256927681cc463a
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Tue Feb 2 11:43:44 2016 +0000
Fix time-shifting of objects PUT with container-sync
When container-sync PUTs an object to a destination container
it uses the timestamp from the container row rather than the
actual timestamp of the object being copied. The actual timestamp
of the object can be newer, so the sync'd object may end up with
the right content but at the wrong, older, timestamp.
This patch changes the timestamp sent with the sync'd object
to be that of the actual source object being sent.
Drive-by fix to make code more readable by removing a variable
rename mid-function, fix a typo and remove a redundant function
call.
Change-Id: I800e6de4cdeea289864414980a96f5929281da04
Closes-Bug: #1540884
commit 050e3b35dd88f7b31b6810d9ac88548ef371e3ce
Author: venkatamahesh <venkatamaheshkotha at gmail.com>
Date: Thu Feb 4 16:47:15 2016 +0530
Use uppercase 'S' in word "OpenStack"
Change-Id: I4692aa58b5173b1b662d5eee19fac770ae5aaed0
commit e47aaaacf15e8f0c7226400541a97238447a7f56
Author: Tim Burke <tim.burke at gmail.com>
Date: Wed Feb 3 12:52:29 2016 -0800
Stop nesting functions unnecessarily
Change-Id: Iff120d0bac8a075c37bbddcd2bb0fe85145f1749
commit 26327e1e8b1d37faa764ec586f5bee0e1560eea2
Author: Darrell Bishop <darrell at swiftstack.com>
Date: Thu Jan 21 11:18:18 2016 -0800
Allow IPv6 addresses/hostnames in StatsD target
The log_statsd_host value can now be an IPv6 address or a hostname
which only resolves to an IPv6 address. In both cases, the new
behavior is to use an AF_INET6 socket on which .sendto() is called
with the originally-configured hostname (or IP). This means the
Swift process is not caching a DNS resolution for the lifetime of
the process (a good thing).
If a hostname resolves to both an IPv6 or IPv4 address, an AF_INET
socket is used (i.e. only the IPv4 address will receive the UDP
packet).
The old behavior is preserved: any invalid IP address literals and
failures in DNS resolution or actual StatsD packet sending do not
halt the process or bubble up; they are caught, logged, and
otherwise ignored.
Change-Id: Ibddddcf140e2e69b08edf3feed3e9a5fa17307cf
commit 2f8e6040470948cf17c5fc5fddd0991a415d2a4c
Author: Eran Rom <eranr at il.ibm.com>
Date: Wed Feb 3 08:59:53 2016 +0200
Fixing typo in unit test
The typo does not influence the correctness of the test
yet its ugly. bhhha.
Change-Id: I62e511bae23523892b87309dd3c50aea61742f2d
commit f27ad34e1d4e5dc306eed3ff620a294a4c301f31
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Tue Feb 2 11:44:39 2016 -0800
Document use-case for slow option
Change-Id: Iec4087a896a2277179e3720d802cca101fa7ad54
commit 16976a0f14ee2b5f7abc76086f9abfb9f337a655
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Sun Jan 31 10:30:06 2016 +0100
Changed EC backend from jerasure to liberasurecode in examples and docs
liberasurecode_rs_vand is build-in liberasurecode, so you don't need
another depedency libjerasure2.
liberasurecode_rs_vand is supported by pyeclib from 1.0.8
version, so bumping version up.
Closes-Bug: #1534325
Change-Id: If2d96875694df8fd48c5278395859aaa165cb566
commit 30d74af6534b754e6ad9bfdcbff4ec494277ca83
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Wed Jan 27 13:50:57 2016 +0000
Insert versioned_writes in correct pipeline position
If not explicitly configured the versioned_writes middleware
should be auto-inserted in the pipeline after slo and dlo, which
is where the versioned_writes filter section's comments say it
should be in proxy-server.conf-sample. At the moment it can end up
being placed ahead of slo and dlo if they have been explicitly
configured, which results in the linked bug manifesting.
Closes-Bug: #1537042
Change-Id: I6ac95a331f4ef0d4887311940acc6f8bc00fb4eb
commit ccdf4a9f309af5fe1401a6e4b776b08c3f6a655c
Author: Christian Schwede <cschwede at redhat.com>
Date: Tue Feb 2 09:38:55 2016 +0100
Document slow option in etc/object-server.conf
Change-Id: Ic9940b0b830a468887878f7b0d7ca42c2cbbebd5
commit b19dc1ddecdc68760ca2f5a2b7b4bbdce2199b35
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Mon Feb 1 15:30:59 2016 -0800
Always fix devices with multiple part-replica assignments
I've found that given a sufficiently bad replica2part2dev table we can
accidently not entirely fix palcement when more than two replicas of a
part are assigned to the duplicate devices.
It shows up most on > 3 replica rings when you have two *different*
devices both holding two replicas. But you can see it on a three
replica ring when all three replicas are assigned to the same device.
Change-Id: Ieb213c1a259815a2ed657291242919cda568c7b5
commit c722fa1ab318b233fc11f8e1bd8758dbe33e87bb
Author: dangming <dangming at unitedstack.com>
Date: Mon Feb 1 16:13:54 2016 +0800
Fix spelling error
Change-Id: I4a1400c4d383acd59727ada3661a8dffc67825fb
commit c7e5e125003de1a0f288f79783e9a41650e6f686
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Fri Jan 29 14:22:19 2016 +0000
Enable in-process func tests to optionally use fast-post
Running functional tests in the in-process mode uses
the default value for proxy-server object_post_as_copy,
which is True. This patch adds support for an environment
variable to change this value to False so that the fast-post
mode is tested when running in-process functional tests.
The patch also adds a new tox environment func-in-process-fast-post
which forces in-process functional testing with
object_post_as_copy=False.
The motivation for this change, apart from enabling configurable
local testing, is to put support in place for an upstream CI job
that will actually functionally test the fast-post mode, which is
otherwise only covered by unit tests.
There are currently two gate jobs that run the functional tests:
- gate-swift-dsvm-functional runs the tests *twice* against a
devstack swift service, once using tempauth and once using
keystoneauth. The devstack swift service uses the default
object_post_as_copy=True.
- gate-swift-tox-func runs the func tests in in-process mode
which also uses tempauth and object_post_as_copy=True. This
duplicates one of the config scenarios above.
With this change either the gate-swift-tox-func job or a new job
could run the functional tests using object_post_as_copy=False.
Change-Id: Ia37f6df1dc38e44ef7404fbf0a52f6fc22fae0c2
commit 4ba290bac19b1b6e8f709848fd35cda33b8bd1c4
Author: Qiaowei Ren <qiaowei.ren at intel.com>
Date: Mon Feb 1 16:44:52 2016 +0800
Replace exit() by sys.exit()
sys.exit() is considered good to use here.
Change-Id: I365b0dbb50c8f1953b41665ba49145ac6db1b24f
commit 9aff76bd18e609eb657db14f2d60e605ee28d472
Author: Qiaowei Ren <qiaowei.ren at intel.com>
Date: Mon Feb 1 17:15:32 2016 +0800
fix typo
informations -> information
Change-Id: I6bcb9bb9fd24b873b49de4a686b223550dd703ac
commit a8e0dc550bf35010595ce791433cd68c4b8d5287
Author: OpenStack Proposal Bot <openstack-infra at lists.openstack.org>
Date: Sun Jan 31 06:15:17 2016 +0000
Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure
Change-Id: If1cd20e187720f5ee02c6a1574f612c8a8e26e30
commit 02c06585e6e0403d1a983fccaba5093e3d3fa7c5
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Fri Jan 29 22:46:50 2016 +0100
Renamed variable for better code readability
Change-Id: I22d8db0dd9edc39672fc9997895a24f669975e15
commit 595ec464e3fdc5a766eee9baa06edd074576c73e
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Fri Jan 29 15:28:56 2016 +0000
Trivial fix for sphinx warnings
Including a * in the param name causes a sphinx warning,
messes up the html formatting, and strictly the * isn't part of
the param name.
Change-Id: I6be6a7aa2a371eaae9f8e501435838c1e15f4a1c
commit cc90834a8ed7bd84440bf132a94978a47ee2f3da
Author: Mingyu Li <li.mingyu at 99cloud.net>
Date: Fri Jan 29 03:48:47 2016 +0900
A minor modification of the Deployment Guide
In the section of General Service Tuning, there is a sentence
"Our Storage servers all run together on the same servers".
The word "servers" is used twice with different meanings, which
would be a little confusing. This patch makes it more understandable.
Change-Id: I6d806456aeb7d0ecf297c04dacbd271b94a0425c
Closes-Bug: #1526697
commit a1776b9c1f23fbd604114b377434e4269d2f1a2e
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Thu Jan 28 16:47:37 2016 +0000
Let equal Timestamps not be unequal
Make the result of Timestamp(x) != Timestamp(x) be False.
In python 2.7 this requires the __ne__ method to be defined [1].
"The truth of x==y does not imply that x!=y is false." The
functools.total_ordering decorator does not autocreate a __ne__
method.
In python 3 the __ne__ method is not required [2]. "By default,
__ne__() delegates to __eq__() and inverts the result".
This patch puts back the __ne__ method removed in [3]. Whilst no tests
fail on master with python2.7, they do on this patch [4] and it seems
dangerous to have this absurd behaviour lurking.
[1] https://docs.python.org/2/reference/datamodel.html#object.__ne__
[2] https://docs.python.org/3.4/reference/datamodel.html#object.__ne__
[3] Change-Id: Id26777ac2c780316ff10ef7d954c48cc1fd480b5
[4] Change-Id: Ia597cd460bb5fd40aa92e886e3e18a7542603d01
Change-Id: I01fbfa310df3c74390f8e8c2e9ffff81bbf05e47
commit c3886eea15f11104764fb201e85202dd871c3027
Author: Thomas Goirand <thomas at goirand.fr>
Date: Thu Jan 28 09:15:28 2016 +0000
Fix a few English mistakes in man
These errors are producing lintian warnings, so fixing them
helps having less errors when checking for Debian packages.
Change-Id: Iff99a8d5f2276515f42d758d110a43cae757db28
commit 400cc707273bab0b70a3cdf5dec1c55c2edc4e22
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Wed Jan 27 15:35:40 2016 +0000
Remove a print statement from tests
Change-Id: I213b7e4bc4f9e6de76ebab572c377af61072d9c5
commit c871e423ef0bed6c774693af3d54750235d6ce1f
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Fri Jan 22 05:19:06 2016 +0000
Add unit tests for checking responses in AccountController
This patch provides unit tests to check responses for various situation
in three/four replicas env.
Change-Id: I1af48728b28da9780a2a78f3110e1c7487047ff8
commit e45d30d6b7b596b80c63e75018ad07582d705ee1
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Wed Jan 27 10:18:37 2016 +0000
Remove unnecessary code from test_account.py
Change-Id: Ia197ba08a606421244d5b94a900bdc39bc5da647
commit f39cffbb5f7f1853f7f67261c4652b9875d302be
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Tue Jan 26 20:16:08 2016 -0800
Follow up mem_server diskfile fixes
This is a follow-up patch for change
https://review.openstack.org/#/c/270422
That patch fixes POST behavior on object mem server and adds
some tests for last-modified. However, that one doesn't check
the POST updates the last-modified on object server.
This patch adds assertions to make sure the actual POST semantics.
*bonus* Add docs and cleanup variable name to fit actual instance
at mem diskfile.
Change-Id: I78185cdfb0ea604b2122821870ec34322236f48d
commit 9ef15453fadb847db0f0fb92e935785b544da3fc
Author: Kazuhiro MIYAHARA <miyahara.kazuhiro at lab.ntt.co.jp>
Date: Thu Jan 21 14:18:31 2016 +0900
Fix handling of "Permission Denied" error from NamedTemporaryFile function
If "Permission Denied" has happen in NamedTemporaryFile function in
dump_recon_cache method, swift will log a message of reference to a variable
without assignment and not log a message of "Permission Denied".
This patch fixes the handling and add an unit test.
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Change-Id: Iafdd94905e9e9c81f5966a923324b50c18fcf592
commit 01410129dac6903ce7f486997a48e36072fa0401
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Fri Aug 21 18:15:25 2015 -0700
Make handoffs_first a more useful "mode"
Based on experience using handoffs_first and feedback from other
operators it has become clear that handoffs_first is only used during
periods of problematic cluster behavior (e.g. full disks) when
replication attempts are failing to quickly drain off the partitions
from the nodes which they have been rebalanced from.
In order to focus on the most important work (getting handoff partitions
off the node) handoffs_first mode will abort the current replication
sweep before attempting any primary suffix syncing if any of the handoff
partitions were not removed for any reason - and start over with
replication of handoffs jobs as the highest priority.
Note that handoffs_first being enabled will emit a warning on start up,
even if no handoff jobs fail, because of the negative impact it can have
during normal operations by dog piling on a node that was temporarily
unavailable.
Change-Id: Ia324728d42c606e2f9e7d29b4ab5fcbff6e47aea
commit 78cfb137d5d3d6f4770923a1fb086adaa9c0676f
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Wed Nov 11 19:05:09 2015 -0800
Ring Overview updates
Change-Id: Ic5a8ec5be9fec7fa812fdb7b91890d5fc2fbf967
commit 24227e8bb0d4d369eeab61ab5c20730b4f84b36a
Author: Richard Hawkins <richard.hawkins at rackspace.com>
Date: Wed Jan 13 10:16:29 2016 -0600
_get_info_cache should encode unicode metadata
Closes-Bug: 1533768
Change-Id: I67bbd0e8802d2a34cb3a0b0fafe26bbc62221032
commit fc9a3852e44b2bf1258fcb24a6c12ecfb2795f44
Author: Tim Burke <tim.burke at gmail.com>
Date: Mon Jan 25 09:25:20 2016 -0800
Don't .lower() a literal
It feels silly, and we don't do it for any of the other headers in
headers_to_container_info.
While we're at it, clean up a stray '
Change-Id: I0745038cc3832a77d064e515c37cacbdcb97c4d9
Related-Change: Iea3d06de80210e9e504e296d4572583d7ffabeac
commit 5c6576d1eb3c4c042bb46dd48f23d2fd474eba07
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Mon Jan 25 18:20:45 2016 +0100
Fixed docs: Title underline too short
Change-Id: If96b0e799a4671e7b7f3f5bea2628db75afa9578
commit 9c403d5e1bb1f404445f1731e3ff1b5c034336a6
Author: Victor Stinner <vstinner at redhat.com>
Date: Mon Oct 19 16:31:12 2015 +0200
Port swift.common.utils.Timestamp to Python 3
* Fix Timestamp.isoformat(): on Python 3, round manually to use the
same rounding method than Python 2.
* Use @functoos.total_ordering on Timestamp instead of the __cmp__()
special method. On Python 3, the __cmp__() method is no more used,
and other special methods must be defined (__eq__, __lt__, etc.).
Only define __eq__() and __lt__() methods but use
@functools.total_ordering to generate other methods.
* Fix also a Python 3 issues in swift.common.swob._datetime_property:
replace (float, int, long) with (float,) + six.integer_types. The
long type was renamed to int in Python 3.
Change-Id: Id26777ac2c780316ff10ef7d954c48cc1fd480b5
commit c790db89369221473bcbfc23f301424ab8d30d60
Author: Andrew Welleck <awellec at us.ibm.com>
Date: Mon Jan 25 10:03:18 2016 -0600
Additional step for Debugging Issues in SAIO
Adding an additional debugging step for the SAIO "Debugging Issues"
section. Adding step to check if memcache is running and to start
it if its not.
Change-Id: I0bfa563f595dbefcf907494451fad45e6310db88
commit d47155af262d52c284f262bf6c303d0587000bba
Author: Victor Stinner <vstinner at redhat.com>
Date: Mon Oct 19 17:22:24 2015 +0200
Add __next__() methods to utils iterators for py3
On Python 3, next(obj) calls obj.__next__(), not obj.next(). Add an
alias from __next__() to next() to be compatible with Python 2 and
Python 3.
Change-Id: Ida104d3bd7cdba557e523f18df43d56847060054
commit 0f8319dceb2d21a9d1ec74f78203b033f25c0969
Author: Tim Burke <tim.burke at gmail.com>
Date: Fri Jan 22 13:31:34 2016 -0800
Use calendar.timegm not time.mktime
time.mktime produces a POSIX timestamp that takes into account the local
timezone, but we're parsing a Last-Modified header with a known (GMT)
timezone. As a result, if one proxy-server is configured with a timezone
other than GMT/UTC, we may have previous versions in the wrong order.
Change-Id: I320e2368b243f4245725e73bfabc7ad19bc5bacb
commit 72d972166ff66d3c5313356c1e1e3984fee9f012
Author: paul luse <paul.e.luse at intel.com>
Date: Thu Jan 21 13:30:15 2016 -0700
Update AUTHORS
+Luse core emeritus
Change-Id: Ie175f410ddc24a2d5e64bc191364418a3c450f3f
commit 2f8105e5fcf4ecee2202797ba1ab48fa8fb1a1f6
Author: Alistair Coles <alistair.coles at hpe.com>
Date: Wed Jan 20 18:14:16 2016 +0000
Fix bug in mem_diskfile write_metadata method
The mem_diskfile DiskFile.write_metadata method was
apparently never called in any existing test, as if it
were it would blow up as reported in the bug.
This patch fixes the method and adds a test that
exercises it. The test addition itself should be useful
since it verifies the behaviour of Last-Modified after
POSTs to an object.
Drive-by fixes for bad docstring and undefined references
in the _quarantine method.
Change-Id: I17fd62e5f02be5b48bfd9ba7fa25315e30a0a4bf
Closes-Bug: #1536037
commit e13a03c379273ee10e678818078b9c40a96a7dc9
Author: Tim Burke <tim.burke at gmail.com>
Date: Wed Jan 20 16:06:26 2016 -0800
Stop overriding builtin range
Change-Id: I315f8b554bb9e96659b455f4158f074961bd6498
commit 0a404def7d54d1ef1c85c11a378052260c4fda4c
Author: John Dickinson <me at not.mn>
Date: Wed Jan 20 15:19:35 2016 -0800
remove unneeded duplicate dict keys
Change-Id: I926d7aaa9df093418aaae54fe26e8f7bc8210645
commit 221f94fdd39fd2dcd9a2e5565adceab615d55913
Author: John Dickinson <me at not.mn>
Date: Tue Jan 19 14:50:24 2016 -0800
authors and changelog updates for 2.6.0
Change-Id: Idd0ff9e70abc0773be183c37cd6125fe852da7c0
commit 58359269b0e971e52f0eb7f97221566ca2148014
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Dec 8 16:36:05 2015 -0800
Fix memory/socket leak in proxy on truncated SLO/DLO GET
When a client disconnected while consuming an SLO or DLO GET response,
the proxy would leak a socket. This could be observed via strace as a
socket that had shutdown() called on it, but was never closed. It
could also be observed by counting entries in /proc/<pid>/fd, where
<pid> is the pid of a proxy server worker process.
This is due to a memory leak in SegmentedIterable. A SegmentedIterable
has an 'app_iter' attribute, which is a generator. That generator
references 'self' (the SegmentedIterable object). This creates a
cyclic reference: the generator refers to the SegmentedIterable, and
the SegmentedIterable refers to the generator.
Python can normally handle cyclic garbage; reference counting won't
reclaim it, but the garbage collector will. However, objects with
finalizers will stop the garbage collector from collecting them* and
the cycle of which they are part.
For most objects, "has finalizer" is synonymous with "has a __del__
method". However, a generator has a finalizer once it's started
running and before it finishes: basically, while it has stack frames
associated with it**.
When a client disconnects mid-stream, we get a memory leak. We have
our SegmentedIterable object (call it "si"), and its associated
generator. si.app_iter is the generator, and the generator closes over
si, so we have a cycle; and the generator has started but not yet
finished, so the generator needs finalization; hence, the garbage
collector won't ever clean it up.
The socket leak comes in because the generator *also* refers to the
request's WSGI environment, which contains wsgi.input, which
ultimately refers to a _socket object from the standard
library. Python's _socket objects only close their underlying file
descriptor when their reference counts fall to 0***.
This commit makes SegmentedIterable.close() call
self.app_iter.close(), thereby unwinding its generator's stack and
making it eligible for garbage collection.
* in Python < 3.4, at least. See PEP 442.
** see PyGen_NeedsFinalizing() in Objects/genobject.c and also
has_finalizer() in Modules/gcmodule.c in Python.
*** see sock_dealloc() in Modules/socketmodule.c in Python. See
sock_close() in the same file for the other half of the sad story.
This closes CVE-2016-0738.
Closes-Bug: 1493303
Co-Authored-By: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Change-Id: Ib86c4c45641485ce1034212bf6f53bb84f02f612
commit d9b22ac51cca579e97ec7bd5a6df3ad7bf56c3bf
Author: Victor Stinner <vstinner at redhat.com>
Date: Mon Oct 19 16:45:33 2015 +0200
Port parse_mime_headers() to Python 3
Port swift.common.utils.parse_mime_headers() to Python 3:
* On Python 3, tries to decode headers from UTF-8. If an header was
was not encoded to UTF-8, decode the header from Latin1.
* Update the parse_mime_headers() tests: on Python 3, HTTP header
values are Unicode strings.
This change is a follow-up of the change
Ia5ee2ead67e36e8c6416183667f64ae255887736.
Change-Id: I042dd13e9eb0e9844ccd832d538cdac84359ed42
commit cec2aa0b4dbf0e1bf2de62945f5abdf494925a8e
Author: Victor Stinner <vstinner at redhat.com>
Date: Fri Oct 9 17:56:39 2015 +0200
py3: Use the six module in the xprofile middleware
* Replace bytes_() with a isinstance(content, six.text_type)
and content.encode('utf-8')
* Get parse_qs() from six.moves.urllib.parse
Change-Id: I4a506975814bff098ade25c950c154e50dc9626e
commit bc4b298b6e208d3188641712c9d66ae82d172c14
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Jan 19 15:33:13 2016 -0800
Fix a comment's indentation
Change-Id: I34514525b606cf82767ddce7769bc42fa5457717
commit 6c32da14f48b435aa4e5efea48904bd288a532a2
Author: Victor Stinner <vstinner at redhat.com>
Date: Mon Oct 19 16:38:24 2015 +0200
Port FileLikeIter to Python 3
Port FileLikeIter and _MultipartMimeFileLikeObject and
swift.common.utils to Python 3:
* Add a __next__() alias to the next() method. On Python 3, the
next() method is no more used, __next__() is required.
* Use literal byte strings: FileLikeIter _MultipartMimeFileLikeObject
are written to handle binary files.
* test_close(): replace .FileLikeIter('abcdef') with
FileLikeIter([b'a', b'b', b'c']). On Python 3, list(b'abc') returns
[97, 98, 99], whereas ['a', 'b', 'c'] is returned on Python 2.
* Update unit FileLikeIter tests to use byte strings.
Change-Id: Ibacddb70b22f624ecd83e374749578feddf8bca8
commit caa127d59a63a25cc6d08e985e536f90300a6156
Author: Samuel Merritt <sam at swiftstack.com>
Date: Thu Nov 12 11:34:44 2015 -0800
Monkeypatch everything the same
There's three spots in the code where eventlet monkeypatches
things. In one place, it was monkeypatching socket and thread; in the
other two, only socket. This commit monkeypatches socket and thread
everywhere.
Hopefully, this lets us avoid code that, for example, works in the
object server but fails in the object updater.
Change-Id: Ia30d069c5ac89f4bfbe52233d03d96738144b614
commit 3a0486e532f22af0d3c8a5c5d78613c22e786ff6
Author: Sivasathurappan Radhakrishnan <siva.radhakrishnan at intel.com>
Date: Fri Dec 4 17:43:00 2015 +0000
Deleted comment about part power in FakeRing
Deleted comment about parameter part power in Class FakeRing as its
behavior got dropped in I8bfc388a04eff6491038991cdfd7686c9d961545.
Change-Id: Iec7d2565a77e48493b0056021066d8d8eab65d0b
Closes-Bug: #1488704
commit 999479f9b17b42ccc5da54ce01651960cf7cf970
Author: John Dickinson <me at not.mn>
Date: Tue Jan 19 10:30:30 2016 -0800
Bump eventlet min version to 0.17.4
IPv6 support in Swift is dependent on IPv6 support in eventlet.
eventlet itself only claims support for IPv6 post v0.17
(https://github.com/eventlet/eventlet/issues/8). This update matches
the OpenStack global requirements version.
Change-Id: I9d8433cdd3bf7d7a93b8f50b991cc21721a80d22
commit 133a3ea601a3fea84af36a42845f27b8182fd901
Author: Christopher Bartz <bartz at dkrz.de>
Date: Mon Dec 21 14:17:00 2015 +0100
Use the correct split_path in handle_request
Change-Id: I86d423309f0b2091ee2e82b2245caf925b6a75ef
Closes-Bug: #1528189
commit 09b188f03c4a40838ea6b3e54e0714d93077b066
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Thu Jul 16 09:35:37 2015 +0900
Add functional test for access control (RBAC) with Keystone
This patch adds test cases for PUT, DELETE, GET, HEAD, POST and OPTIONS
requests to accounts, containers and objects using various combinations
of users/projects, roles and/or service tokens.
Change-Id: Iea8141ac74ad949a3ae7fa47fda3135d0f2612f6
commit bf10974cdefffdaaebc58d21e8a9912638a0405a
Author: Tim Burke <tim.burke at gmail.com>
Date: Wed Dec 16 15:46:13 2015 -0800
Expose token expiration time in tempauth auth response
Previously, we gave no indication of when a token would expire. Users
would have to just use it until it stopped working, then re-auth.
Now, a successful auth response will include a new header,
X-Auth-Token-Expires, with the number of seconds remaining until the
token is invalid. This allows the client to attempt to re-auth before
sending a request that will definitely fail.
For comparison, swauth already uses the X-Auth-Token-Expires header with
identical semantics. Additionally, Keystone (v2 and v3) already exposes
expiration times in its JSON responses. The security impact should be
minimal.
Change-Id: I5a4a74276bc0df6dda94e4bc150065c0d77de0eb
commit f56f29ef7a692b6c3204e25d41eace3d68dceece
Author: Peter Lisák <peter.lisak at firma.seznam.cz>
Date: Thu Dec 3 16:48:18 2015 +0100
Add info about state of ring file to default command.
Try to find ring file, load and compare it with builder file, then show result state.
Examples:
Ring file object.ring.gz not found, probably it hasn't been written yet
Ring file object.ring.gz is up-to-date
Ring file object.ring.gz is obsolete
Ring file object.ring.gz is invalid: ValueError('string length not a multiple of item size',)
Change-Id: I4d769aa5fe1c2b1167ec088aa372874f7d13ae48
commit 47e226418bad35ccad2a1525f392ba69f6165027
Author: OpenStack Proposal Bot <openstack-infra at lists.openstack.org>
Date: Mon Jan 18 06:20:14 2016 +0000
Imported Translations from Zanata
For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure
Change-Id: Ic416c9afc8a1c76f552803a7c70fc905eda7b3cb
commit 165fa1fd40b5e4b13cf5188712593e2275bef821
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Fri Jan 15 11:44:21 2016 -0800
add test for zero weight region get_more_nodes
Change-Id: If537981e8deadd9c3528dcb30a15011c7781e334
commit 5d449471b12c67b31ebb5a383d9bb35bace36213
Author: Samuel Merritt <sam at swiftstack.com>
Date: Thu Jan 14 17:26:01 2016 -0800
Remove some Python 2.6 leftovers
Change-Id: I798d08722c90327c66759aa0bb4526851ba38d41
commit 3c0cf549f1e822cce8f905b069b317e676cf306b
Author: Samuel Merritt <sam at swiftstack.com>
Date: Wed Jan 13 18:08:45 2016 -0800
Speed up get_more_nodes() when there is an empty zone
The ring has some optimizations in get_more_nodes() so that it can
find handoffs that span all the regions/zones/et cetera and then stop
looking. The stopping is the important part.
Previously, it would quickly find a handoff in each unused region,
then spend way too long looking for more unused regions; the same was
true for zones, IPs, and so on. Thus, in commit 9cd7c6c, we started
counting regions and zones, then stopping when we found them all.
This count included all regions and zones in the ring, regardless of
whether or not there were actually any parts assigned or not. In rings
with an empty region, i.e. a region for which there are only
zero-weight devices, get_more_nodes() would be very slow.
This commit ignores devices with no assigned partitions when counting
regions, zones, and so forth, thus greatly speeding things up.
The output of get_more_nodes() is unchanged. This is purely an
optimization.
Closes-Bug: 1534303
Change-Id: I4a5c57205e87e1205d40fd5d9458d4114e524332
commit 70047709fc9885df7019f791e17a3240682cc6cb
Author: keliang <ke.liang at easystack.cn>
Date: Fri Jan 15 00:31:51 2016 +0800
Drop python 2.6 support
Change-Id: Id6329c863dacb189fccfc304453ed7b6f9607c14
commit a4c2fe95ab2fbe59379a69914ed0fac49c28efbb
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Tue Jan 12 21:26:33 2016 +0100
Allow to change auditor sleep interval in config
Change-Id: Ic451c5e0b686509f8982ed1bf65a223a2d77b9a0
commit edc823e8030640184071fee4920d34f9a1cc6b3e
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Sun Nov 29 18:46:47 2015 +0100
Show UTC time in swift-recon.
It's not consistent now for example local time in replication part and
UTC time at begging of line. Use _ptime in swift-recon for all time
printing and this function returns UTC now.
Change-Id: I732d9851db157130a08e825e8093b7e244b63e9c
commit fa5b32d27964478dfcccf71155d2aaa946c561f0
Author: Tim Burke <tim.burke at gmail.com>
Date: Tue Jan 12 14:18:30 2016 -0800
Make object-auditor storage-policy-aware
Previously, the object-auditor would always use a (replication)
DiskFileManager when walking through AuditLocations, which would cause
it to skip EC fragment archives with a warning like:
Unexpected file <hash_path>/1452557187.03610#3.data:
Invalid Timestamp value in filename '1452557187.03610#3.data'
Now, the AuditLocation's policy will be used to find an appropriate
manager to get the diskfile. Additionally, several .commit()s were added
to the auditor tests so the .durable will be written, which is required
when auditing EC fragment archives.
Change-Id: I960e7d696fd9ad704ca1872b4ac821f9078838c7
Closes-Bug: 1533002
commit 725a166ebd3fde2c34f2c9578ff6f9911cf206d7
Author: Timur Alperovich <timuralp at swiftstack.com>
Date: Fri Jan 8 18:42:33 2016 -0800
Make _get_addr() method a function in utils.
The patch moves the MemcacheConnPool._get_addr() method a function in
swift.common.utils. The function is renamed to parse_socket_string()
and the documentation is updated accordingly. The test for it has also
been moved.
Change-Id: Ida65b2fded28d0a059e668646f5b89714298f348
commit 0647aea9c5fc95f0860758cc54bc23f965da460d
Author: Timur Alperovich <timuralp at swiftstack.com>
Date: Fri Jan 8 14:54:56 2016 -0800
Fail early if the memcache address is invalid.
In cases when the memcache address is invalid, we should fail early.
This patch addresses the cases when the IPv6 addresses are not
enclosed in "[]". It does not, however, fix the case of an invalid
hostname. These improvements could also be added to the _get_addr()
method.
Change-Id: I4743dcda45a1fc1640989325c4a2e1fea591fc69
commit 33476460239c9cdb08dd8065d22d84a4717da7be
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Fri Jan 8 16:15:54 2016 -0800
fixups for ipv6 memcache_servers docs
Change-Id: I20d91c1e276014eaf210fa9eb43788bc17f4e8df
commit d5ff5447be30b44bf4acc8b912b6241a44f710be
Author: Samuel Merritt <sam at swiftstack.com>
Date: Tue Jan 12 16:42:06 2016 -0800
Install liberasurecode packages in SAIO.
Change-Id: If673afa2b61a3e388612debf4860d561960963a3
commit 4ffc4ba411f67c8407ba38d082a3a51a96ad7e04
Author: Jonathan Hinson <jlhinson at us.ibm.com>
Date: Tue Jan 12 11:46:21 2016 -0600
Functional tests for if-match with multiple etags
Multiple etags can be provided on an if-match or if-none-match
request. This is currently being tested in the unit tests, but not
in the functional tests. Since these etags can be modified by
middleware, we need functional tests to assert multiple-etag
requests are handled correctly.
Change-Id: Idc409c85e8aa82b59dc2bc28af6ca2617de82699
commit 1292361176aec385137dd5873c5cc9714de1fa17
Author: Clay Gerrard <clay.gerrard at gmail.com>
Date: Mon Jan 11 16:25:50 2016 -0800
functest for x-timestamp validation
Change-Id: I05af7d837af00cd1059a8912df05c131a52ffb90
commit e6194113a3c81563590eabf8f761ccb988bb917c
Author: Tim Burke <tim.burke at gmail.com>
Date: Fri Jan 8 16:38:31 2016 -0800
Validate X-Timestamps
Previously, attempting to PUT a new object with an X-Timestamp header
less than or equal to zero (ie, for a timestamp on or before 1970-01-01
00:00:00) would cause the object-server to 500.
While we're at it, cap X-Timestamp at 9999999999 (2286-11-20 17:46:40)
so we don't get an eleventh digit before the decimal point.
Closes-Bug: 1532471
Change-Id: I23666ec8a067d829eaf9bfe54bd086c320b3429e
commit 1f3304c5153e01988b8f4493875b6489e93f76d0
Author: Ben Martin <blmartin at us.ibm.com>
Date: Mon Dec 14 15:28:17 2015 -0600
Print min_part_hours lockout time remaining
swift-ring-builder currently only displays min_part_hours and
not the amount of time remaining before a rebalance can occur.
This information is readily available and has been displayed
as a quality of life improvement.
Additionally, a bug where the time since the last rebalance
was always updated when rebalance was called regardless of
if any partitions were reassigned. This can lead to partitions
being unable to be reassigned as they never age according to
the time since last rebalance.
Change-Id: Ie0e2b5e25140cbac7465f31a26a4998beb3892e9
Closes-Bug: #1526017
commit 167bb5eeb82886d67c1b382417fb22b8ea85f0d3
Author: Timur Alperovich <timuralp at swiftstack.com>
Date: Wed Dec 16 12:07:27 2015 -0800
Fix IPv6 handling in MemcacheConnPool.
The patch removes the assumption of IPv4-only addresses in the
MemcacheConnPool. The changes are around address handling.
Namely, if a server is specified with an address
[<address>]:port (port is optional), it is assumed to be an IPv6
address [1]. If an IPv6 address is specified without "[]", an exception
is raised as it is impossible to parse such addresses correctly.
For testing, memcache can be configured to listen on the link-local,
unique-local, or ::1 (equivalent to 127.0.0.1) addresses. Link-local
addresses are assigned by default to each interface and are of the form
"fe80::dead:beef". These addresses require a scope ID, which would look
like "fe80::dead:beef%eth0" (replacing eth0 with the correct interface).
Unique-local addresses are any addresses in the fc00::/7 subnet. To add
a ULA to an interface use the "ip" utility. For example:
"ip -6 address add fc01::dead:beef dev eth0". Lastly, and probably
simplest, memcache can be configured to listen on "::1". The same
address would be used in the swift configuration, e.g. "[::1]:11211".
Note: only memcached version 1.4.25 or greater supports binding to an
IPv6 address.
Fixes #1526570
[1] IPv6 host literals:
https://tools.ietf.org/html/rfc3986#section-3.2.2
Change-Id: I8408143c1d47d24e70df56a08167c529825276a2
commit fb6751d8ba133c57e1ebb76be71a96f2f120b8ca
Author: Paul Dardeau <paul.dardeau at intel.com>
Date: Fri Jan 8 22:49:05 2016 +0000
Look for device holes that can be reused when adding new device.
Change-Id: I1980ebdd9dc89848173d8ca2fe2afb74029dcfa2
Closes-Bug: 1532276
commit b35f3c90bde8a7ccb50440bda5800cbb8274a5a1
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Fri Jan 8 01:29:11 2016 -0800
Add note COPY with conditional headers
Swift now supports Range header for COPY (or PUT with X-Copy-From)
to make a partial copied object. This patch adds the note to show
it obviously supported in Swift community.
Change-Id: I6bf28f0932c90e7b305cd61aabce4ed028ae691e
Partial-Bug: #1532126
commit 23c7a58f8f1412c28b3a16b79be09c224c9f7d55
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Fri Dec 11 18:26:34 2015 +0900
Fix ClientException handling in Container Sync
swift/container/sync.py uses swift.common.internal_client.delete_object
and put_object and expected these methods raise ClientException.
But delete_object and put_object never raise the exception so this patch
raises ClientException when urllib2 library raises HTTPError.
Co-Authored-By: Eran Rom <eranr at il.ibm.com>
Closes-Bug: #1419901
Change-Id: I58cbf77988979a07998a46d9d81be84d29b0d9bf
commit 6786cdf036b4faabe3928c3d0dd9615d94834801
Author: Harshada Mangesh Kakad <harshadak at metsi.co.uk>
Date: Thu Dec 31 01:44:00 2015 -0800
Fixing the deprecated library function.
os.popen() is deprecated since version 2.6. Resolved with use of
subprocess module.
Change-Id: I4409cdd9edbc1a26d6f99c125c9100fadda5d758
Partial-Bug: #1529836
commit 85a0a6a28e166bc076cf8786de2b46248d8786a2
Author: Eran Rom <eranr at il.ibm.com>
Date: Sun Jul 26 13:31:17 2015 +0300
Container-Sync to iterate only over synced containers
This change introduces a sync_store which holds only containers that
are enabled for sync. The store is implemented using a directory
structure that resembles that of the containers directory, but has
entries only for containers enabled for sync.
The store is maintained in two ways:
1. Preemptively by the container server when processing
PUT/POST/DELETE operations targeted at containers with
x-container-sync-key / x-container-sync-to
2. In the background using the containers replicator
whenever it processes a container set up for sync
The change updates [1]
[1] http://docs.openstack.org/developer/swift/overview_container_sync.html
Change-Id: I9ae4d4c7ff6336611df4122b7c753cc4fa46c0ff
Closes-Bug: #1476623
commit e75888b281d59df0889f28d0b32241dac3a34aa2
Author: HugoKuo <tonytkdk at gmail.com>
Date: Wed Jan 6 14:33:23 2016 +0800
Add more description for write_affinity_node_count parameter in the doc.
Change-Id: Iad410a2be4f9a2cd5c53e860b9f91993aa7f2369
Closes-Bug: #1531173
commit 3b1591f235f4b85796917507be5e7fd80365ff9e
Author: Ondřej Nový <ondrej.novy at firma.seznam.cz>
Date: Wed Sep 30 19:08:09 2015 +0200
swift-init: New option kill-after-timeout
This option send SIGKILL to daemon after kill_wait period.
When daemon hangs and doesn't respond to SIGTERM/SIGHUP
there is no way to stop it using swift-init now. Classic
init scripts in Linux kills hanged process after grace
period and this patch add same behaviour. This is most
usefull when using "restart" on hanged daemon.
Change-Id: I8c932b673a0f51e52132df87ea2f4396f4bba9d8
commit 4691d0592ff3deeea2713195312bf1effd5a7a7b
Author: Hisashi Osanai <osanai.hisashi at jp.fujitsu.com>
Date: Fri Dec 25 14:00:16 2015 +0900
Add object replicator's succeeded and failed counts in log
account-replicator and container-replicator outputted the
following info in log.
info account-replicator: 17 successes, 4 failures
info container-replicator: 178 successes, 29 failures
But there is no output info for object-replicator. This info
is important to check replicator's status from log.
So this patch adds object-replicator's succeeded and failed
counts in log like account-replicator and container-replicator.
Change-Id: I7fbaf161540bdb06c0a65a1032aae83764bd970c
commit 9fe0e25604dff35db7eab1bca312821a81db6c1d
Author: Kota Tsuyuzaki <tsuyuzaki.kota at lab.ntt.co.jp>
Date: Tue Dec 8 22:27:44 2015 -0800
Sleep enough for trampoline
When running unite test suite in local poor resource environment,
sometimes test/unit/proxy/test_server.py fails due to a lack of
waiting time to trampoline of eventlet thread.
This patch enables to sleep 1 more second when it doesn't seem
to have enough time to tranpoline.
Change-Id: I0bbc8fc245919d3c0a071ff87ff6e20b8d58f9b8
commit 7f636a557296ecc6ae4727700cfcf9f82573bd16
Author: Samuel Merritt <sam at swiftstack.com>
Date: Mon Nov 30 18:06:09 2015 -0800
Allow smaller segments in static large objects
The addition of range support for SLO segments (commit 25d5e68)
required the range size to be at least the SLO minimum segment size
(default 1 MiB). However, if you're doing something like assembling a
video of short clips out of a larger one, then you might not need a
full 1 MiB.
The reason for the 1 MiB restriction was to protect Swift from
resource overconsumption. It takes CPU, RAM, and internal bandwidth to
connect to an object server, so it's much cheaper to serve a 10 GiB
SLO if it has 10 MiB segments than if it has 10 B segments.
Instead of a strict limit, now we apply ratelimiting to small
segments. The threshold for "small" is configurable and defaults to 1
MiB. SLO segments may now be as small as 1 byte.
If a client makes SLOs as before, it'll still be able to download the
objects as fast as Swift can serve them. However, a SLO with a lot of
small ranges or segments will be slowed down to avoid resource
overconsumption. This is similar to how DLOs work, except that DLOs
ratelimit *every* segment, not just small ones.
UpgradeImpact
For operators: if your cluster has enabled ratelimiting for SLO, you
will want to set rate_limit_under_size to a large number prior to
upgrade. This will preserve your existing behavior of ratelimiting all
SLO segments. 5368709123 is a good value, as that's 1 greater than the
default max object size. Alternately, hold down the 9 key until you
get bored.
If your cluster has not enabled ratelimiting for SLO (the default), no
action is needed.
Change-Id: Id1ff7742308ed816038a5c44ec548afa26612b95
** Tags added: in-feature-hummingbird
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to swift in Ubuntu.
https://bugs.launchpad.net/bugs/1493303
Title:
[OSSA 2016-004] Swift proxy memory leak on unfinished read
(CVE-2016-0738)
Status in Ubuntu Cloud Archive:
New
Status in OpenStack Security Advisory:
Fix Released
Status in OpenStack Object Storage (swift):
Fix Released
Status in swift package in Ubuntu:
Confirmed
Bug description:
It looks like the Swift proxy will leak memory if the connection is
closed and the full response is not read. This opens for a potential
DoS attacks.
Reproduce:
$ swift -A http://localhost:8888/auth/v1.0 -U .. -K .. upload --use-slo --segment-size 1048576 <container> <big-file>
$ curl -H'X-Auth-Token: AUTH_...' "http://localhost:8888/v1/AUTH_../<container>/<big-file>" -m 0.001 > /dev/null
Repeat the curl command a couple of times and you will have more
information in netstat and sockstat. The important part is the -m
which sets the max time curl spends at downloading. After that point,
it'll close the connection.
$ sudo netstat -ant -p | grep :6000
$ cat /proc/net/sockstat
tcp 0 0 127.0.0.1:6000 0.0.0.0:* LISTEN 1358/python
tcp 0 43221 127.0.0.1:6000 127.0.0.1:48350 FIN_WAIT1 -
tcp 0 43221 127.0.0.1:6000 127.0.0.1:48882 FIN_WAIT1 -
tcp 939820 0 127.0.0.1:48350 127.0.0.1:6000 ESTABLISHED 17897/python
tcp 939820 0 127.0.0.1:48882 127.0.0.1:6000 ESTABLISHED 17890/python
tcp 983041 0 127.0.0.1:48191 127.0.0.1:6000 CLOSE_WAIT 17897/python
tcp 983041 0 127.0.0.1:48948 127.0.0.1:6000 CLOSE_WAIT 17892/python
Restarting the proxy frees up the lingering memory.
This problem did not exist in 2.2.0.
ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: swift 2.2.2-0ubuntu1~cloud0 [origin: Canonical]
ProcVersionSignature: Ubuntu 3.16.0-48.64~14.04.1-generic 3.16.7-ckt15
Uname: Linux 3.16.0-48-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.12
Architecture: amd64
CrashDB:
{
"impl": "launchpad",
"project": "cloud-archive",
"bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
}
Date: Tue Sep 8 09:55:05 2015
InstallationDate: Installed on 2015-06-22 (77 days ago)
InstallationMedia: Ubuntu-Server 14.04.2 LTS "Trusty Tahr" - Release amd64 (20150218.1)
PackageArchitecture: all
SourcePackage: swift
UpgradeStatus: No upgrade log present (probably fresh install)
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1493303/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list