Error -3 while decompressing data
Liam Routt
liam.routt at mediasaints.com
Tue Jan 11 03:36:00 UTC 2011
I'm a relatively new Bazaar user.
I've been really enjoying bazaar, in the main, but I've run into a
severe problem.
We have a number of Windows XP machines here, and we're talking to a
small linux (OpenSUSE) server which houses a set of master repositories.
Initially the problem we're having manifested when trying to update a
branch of a repository with large binary (source art) files. The windows
machines have branches which are bound back to the linux server. At some
point we were unable to commit on the windows machines, and then updates
were impossible as well. Each attempt resulted in a reported 'Error -3
while decompressing data'.
We spent some time trying to work out what the problem was. Searches
online have indicated that filesystem errors are most commonly blamed
for this error, but we can't find any indication that is what is
happening here.
Trial and error has shown us that the problem is entirely limited to our
linux server. We can run the same tests on it locally (rather than over
the network at the other end of a bound branch) to get the same error.
We can use a repository for a while, but eventually it will get to a
point where the error happens, and then it is useless.
As far as we can tell the problem is the result of repository or branch
size. Other repository/branches contain smaller (text - code source)
files and either haven't encountered this problem, or in one case only
encountered it after quite a while. My test case (below) creates a new
repository/branch and adds a bunch of binary files (some large); the
commit always causes the error (at what seems to be the same point) on
the linux machine. I can do the same set of actions locally on another
(Windows) machine without having the problem.
My test case:
=====
bzr init-repo bzr://onix/NewArt
bzr init bzr://onix/NewArt/trunk
bzr checkout bzr://onix/NewArt/trunk NewArt
cd NewArt
cp -r /data/BigArtTest ./
bzr add
bzr commit -m test
=====
Running du -h on /data/BigArtTest tells me it totals to 779 MB; the
largest file is about 75 MB (we did test just adding the largest files,
and that wasn't the problem on its own - it took more than just the
largest file).
The error reported is:
====
bzr: ERROR: bzrlib.errors.ErrorFromSmartServer: Error received from
smart server: ('error', 'Error -3 while decompressing data: incorrect
data check')
Traceback (most recent call last):
File "/usr/local/lib64/python2.6/site-packages/bzrlib/commands.py",
line 917, in exception_to_return_code
return the_callable(*args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/commands.py",
line 1117, in run_bzr
ret = run(*run_argv)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/commands.py",
line 691, in run_argv_aliases
return self.run(**all_cmd_args)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/commands.py",
line 710, in run
return self._operation.run_simple(*args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/cleanup.py",
line 135, in run_simple
self.cleanups, self.func, *args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/cleanup.py",
line 165, in _do_with_cleanups
result = func(*args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/builtins.py",
line 3241, in run
exclude=tree.safe_relpath_files(exclude))
File "/usr/local/lib64/python2.6/site-packages/bzrlib/decorators.py",
line 194, in write_locked
result = unbound(self, *args, **kwargs)
File
"/usr/local/lib64/python2.6/site-packages/bzrlib/workingtree_4.py", line
197, in commit
result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/decorators.py",
line 194, in write_locked
result = unbound(self, *args, **kwargs)
File
"/usr/local/lib64/python2.6/site-packages/bzrlib/mutabletree.py", line
201, in commit
*args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/commit.py",
line 286, in commit
possible_master_transports=possible_master_transports)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/cleanup.py",
line 131, in run
self.cleanups, self.func, self, *args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/cleanup.py",
line 165, in _do_with_cleanups
result = func(*args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/commit.py",
line 448, in _commit
self.branch.repository, new_revno, self.rev_id)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/branch.py",
line 1031, in import_last_revision_info
self.repository.fetch(source_repo, revision_id=revid)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/remote.py",
line 1382, in fetch
find_ghosts=find_ghosts, fetch_spec=fetch_spec)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/decorators.py",
line 194, in write_locked
result = unbound(self, *args, **kwargs)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/repository.py",
line 3446, in fetch
find_ghosts=find_ghosts)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/fetch.py", line
73, in __init__
self.__fetch()
File "/usr/local/lib64/python2.6/site-packages/bzrlib/fetch.py", line
99, in __fetch
self._fetch_everything_for_search(search)
File "/usr/local/lib64/python2.6/site-packages/bzrlib/fetch.py", line
127, in _fetch_everything_for_search
stream, from_format, [])
File "/usr/local/lib64/python2.6/site-packages/bzrlib/remote.py",
line 1843, in insert_stream
(verb, path, resume_tokens) + lock_args, byte_stream)
File
"/usr/local/lib64/python2.6/site-packages/bzrlib/smart/client.py", line
181, in call_with_body_stream
expect_response_body=False)
File
"/usr/local/lib64/python2.6/site-packages/bzrlib/smart/client.py", line
81, in _call_and_read_response
expect_body=expect_response_body),
File
"/usr/local/lib64/python2.6/site-packages/bzrlib/smart/message.py", line
306, in read_response_tuple
_translate_error(self.args)
File
"/usr/local/lib64/python2.6/site-packages/bzrlib/smart/message.py", line
361, in _translate_error
raise errors.ErrorFromSmartServer(error_tuple)
ErrorFromSmartServer: Error received from smart server: ('error', 'Error
-3 while decompressing data: incorrect data check')
bzr 2.3b3 on python 2.6.5
(Linux-2.6.34.7-0.7-desktop-x86_64-with-SuSE-11.3-x86_64)
arguments: ['/usr/local/bin/bzr', 'commit', '-m', 'test']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'en_US.UTF-8'
plugins:
bash_completion
/usr/local/lib64/python2.6/site-packages/bzrlib/plugins/bash_completion
[2.3b3]
launchpad
/usr/local/lib64/python2.6/site-packages/bzrlib/plugins/launchpad [2.3b3]
netrc_credential_store
/usr/local/lib64/python2.6/site-packages/bzrlib/plugins/netrc_credential_store
[2.3b3]
news_merge
/usr/local/lib64/python2.6/site-packages/bzrlib/plugins/news_merge [2.3b3]
*** Bazaar has encountered an internal error. This probably indicates a
bug in Bazaar. You can help us fix it by filing a bug report at
https://bugs.launchpad.net/bzr/+filebug
including this traceback and a description of the problem.
====
bzr version returns:
====
Bazaar (bzr) 2.3b3
Python interpreter: /usr/bin/python 2.6.5
Python standard library: /usr/lib64/python2.6
Platform: Linux-2.6.34.7-0.7-desktop-x86_64-with-SuSE-11.3-x86_64
bzrlib: /usr/local/lib64/python2.6/site-packages/bzrlib
====
We have had the same error with earlier versions as well (I think we
started on 2.1 or 2.0.x - I can find out if necessary, I think).
The CPU is an Intel Atom D510 running at 1.66GHz.
You may note that in the test case I'm still talking to a bzr:
repository. I haven't gone back over my earlier tests, but my
recollection is that I thought we'd fixed the problem when I was testing
a purely local repository, but found that when the branch is bound to
one being handled by the server the problem was still present. Which
would seem to indicate the problem is in the server code somewhere.
The server is running with the following command line:
====
/usr/bin/python /usr/local/bin/bzr serve --allow-writes
--directory=/data/bzr/repo
====
OK. Looking at the log, I'm seeing some output I don't recall looking at
before:
====
171.473 Using fetch logic to copy between
CHKInventoryRepository('file:///data/home/caligari/NewArt/.bzr/repository/')(RepositoryFormat2a())
and
RemoteRepository(bzr://onix/NewArt/.bzr/)(RemoteRepositoryFormat(_network_name='Bazaar
repository format 2a (needs bzr 1.16 or later)\n'))
171.473 fetch up to rev {liam-20110111023040-bsycdvcc27urj9n5}
172.019 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 523187956, 20552048) to an LRUSizeCache failed. value
50372345 is too big to fit in a the cache with size 41943040 52428800
172.854 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 85935550, 52486586) to an LRUSizeCache failed. value
120836304 is too big to fit in a the cache with size 41943040 52428800
173.884 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 258141629, 34162369) to an LRUSizeCache failed. value
77474460 is too big to fit in a the cache with size 41943040 52428800
174.381 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 494444632, 19105066) to an LRUSizeCache failed. value
46922120 is too big to fit in a the cache with size 41943040 52428800
174.931 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 180081344, 44961676) to an LRUSizeCache failed. value
123643953 is too big to fit in a the cache with size 41943040 52428800
176.465 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 151499003, 13772795) to an LRUSizeCache failed. value
42618185 is too big to fit in a the cache with size 41943040 52428800
177.137 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 360241805, 51613596) to an LRUSizeCache failed. value
127447891 is too big to fit in a the cache with size 41943040 52428800
178.515 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 445929982, 28796858) to an LRUSizeCache failed. value
79505093 is too big to fit in a the cache with size 41943040 52428800
179.533 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 42, 60781546) to an LRUSizeCache failed. value 142575608 is
too big to fit in a the cache with size 41943040 52428800
181.454 Adding the key (<bzrlib.btree_index.BTreeGraphIndex object at
0x13c86d0>, 476227199, 17871087) to an LRUSizeCache failed. value
53829166 is too big to fit in a the cache with size 41943040 52428800
====
It goes on to report a traceback which appears to match the one shown above.
Hopefully that, at least, could help someone tell us what the nature of
the problem is. I haven't reported the problem on launchpad at this
point because I don't understand the conditions well enough myself at
this point. Any advice at all would be excellent. I'm almost to the
point where I'm going to have to look at another VCS, and I'm not eager
to do that at all.
Take care,
Liam Routt
Media Saints
More information about the bazaar
mailing list