[MERGE] [Bug 215426] Re: MemoryError - recv for HTTP through Proxy
Eric Holmberg
eholmberg at arrow.com
Fri Apr 11 22:34:37 BST 2008
Work-around patch for cause of MemoryError on HTTP download under Windows is attached.
Regards,
Eric Holmberg
> -----Original Message-----
> From: bounces at canonical.com [mailto:bounces at canonical.com] On
> Behalf Of Alexander Belchenko
> Sent: Friday, April 11, 2008 2:48 PM
> To: Eric Holmberg
> Subject: Re: [Bug 215426] Re: MemoryError - recv for HTTP
> through Proxy
>
> Please, send your patch to Bazaar ML with prefix [MERGE] in
> the subject line.
>
> Eric Holmberg пишет:
> > There is another bug that looks related -
> > http://bugs.launchpad.net/bzr/+bug/198727.
> >
> > In addition, Python seems to have a few bugs that look like
> they have
> > been resolved, but not yet released.
> >
> > * http://bugs.python.org/issue1092502
> > * http://bugs.python.org/issue1389051
> >
> > Version tried:
> > * Fails: Windows Python 2.5 (r25:51908, Sep 19 2006, 09:52:17)
> > * Fails: Windows Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45)
> > * OK: Linux Python 2.4.4c1 (#2, Oct 11 2006, 21:51:02)
> > * OK: Linux Python 2.5.2 (r252:60911, Mar 12 2008, 13:36:25)
> >
> > Here is a temporary-work-around to the that I'm using (I have also
> > attached the patch to this posting). Please let me know if
> you would
> > like me do to anything.
> >
> > Regards,
> >
> > Eric
> >
> > --- bzrlib\transport\http\response.original.py Fri Apr
> 11 14:05:55 2008
> > +++ bzrlib\transport\http\response.py Fri Apr 11 14:07:20 2008
> > @@ -206,7 +206,18 @@
> > limited = self._start + self._size - self._pos
> > if size >= 0:
> > limited = min(limited, size)
> > - data = self._file.read(limited)
> > +
> > + lst = []
> > + while limited > 0:
> > + # limit reads to 4 MB for Windows problem
> > + # See bug:
> http://bugs.launchpad.net/bzr/+bug/215426
> > + # nBytesToRead = min(limited,1024*1024*4 +
> 1024*100) OK
> > + # nBytesToRead = min(limited,1024*1024*4 +
> 1024*128) Fails
> > + nBytesToRead = min(limited,1024*1024*4)
> > + lst.append(self._file.read(nBytesToRead))
> > + limited -= nBytesToRead
> > + data = ''.join(lst)
> > +
> > else:
> > # Size of file unknown, the user may have
> specified a size or not,
> > # we delegate that to the filesocket object (-1 means
> > read until
> >
> >
> > ** Attachment added: "BZR-1.4rc1 Bug 215426 Patch"
> >
> >
> http://launchpadlibrarian.net/13364915/bzr-1.4rc1-bug215426.patch.txt
> >
>
> --
> MemoryError - recv for HTTP through Proxy
> https://bugs.launchpad.net/bugs/215426
> You received this bug notification because you are a direct
> subscriber of the bug.
>
> Status in Bazaar Version Control System: New
>
> Bug description:
> When using a repository with a 4MB pack where the traffic has
> to go through a proxy server, the proxy server fragments the
> HTTP packets into 1460-byte blocks (as sniffed with
> Ethereal/Wireshark). This results in a very large number of
> HTTP Continuation packets when bzr is downloading the 4MB
> pack. Eventually, bzr runs out of memory. Actual memory by
> bzr is only 24MB with more than 500MB available (total of 2GB
> in system), so the memory error seems to be more of a
> fixed-size buffer issue.
>
> OS: Windows XP
> BZR versions: All (have tried >= 1.2 and most recently 1.3)
> Python version: 2.5
>
> bzr: ERROR: exceptions.MemoryError:
>
> Traceback (most recent call last):
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\commands.
> py", line 834, in run_bzr_catch_errors
> return run_bzr(argv)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\commands.
> py", line 790, in run_bzr
> ret = run(*run_argv)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\commands.
> py", line 492, in run_argv_aliases
> return self.run(**all_cmd_args)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\builtins.
> py", line 927, in run
> hardlink=hardlink)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\bzrdir.py
> ", line 941, in sprout
> revision_id=revision_id)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\decorator
> s.py", line 127, in read_locked
> return unbound(self, *args, **kwargs)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repositor
> y.py", line 1036, in sprout
> dest_repo.fetch(self, revision_id=revision_id)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repositor
> y.py", line 949, in fetch
> return inter.fetch(revision_id=revision_id, pb=pb,
> find_ghosts=find_ghosts)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\decorator
> s.py", line 165, in write_locked
> return unbound(self, *args, **kwargs)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repositor
> y.py", line 2759, in fetch
> revision_ids).pack()
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repofmt\p
> ack_repo.py", line 589, in pack
> return self._create_pack_from_packs()
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repofmt\p
> ack_repo.py", line 722, in _create_pack_from_packs
> self._copy_text_texts()
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repofmt\p
> ack_repo.py", line 686, in _copy_text_texts
> self.new_pack.text_index, readv_group_iter, total_items))
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repofmt\p
> ack_repo.py", line 807, in _copy_nodes_graph
> write_index, output_lines, pb, readv_group_iter, total_items):
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\repofmt\p
> ack_repo.py", line 830, in _do_copy_nodes_graph
> izip(reader.iter_records(), node_vector):
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\pack.py",
> line 272, in _iter_records
> for record in self._iter_record_objects():
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\pack.py",
> line 277, in _iter_record_objects
> record_kind = self.reader_func(1)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\pack.py",
> line 218, in reader_func
> return self._source.read(length)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\pack.py",
> line 177, in read
> self._next()
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\pack.py",
> line 172, in _next
> length, data = self.readv_result.next()
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\transport
> \http\__init__.py", line 250, in _readv
> data = rfile.read(size)
> File
> "c:\python25\lib\site-packages\bzr-1.3.1-py2.5-win32.egg\bzrli
> b\transport
> \http\response.py", line 209, in read
> data = self._file.read(limited)
> File "C:\Python25\lib\socket.py", line 308, in read
> data = self._sock.recv(recv_size)
> File "C:\Python25\lib\httplib.py", line 529, in read
> s = self.fp.read(amt)
> File "C:\Python25\lib\socket.py", line 308, in read
> data = self._sock.recv(recv_size)
> MemoryError
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bug215426-fix.bundle
Type: application/octet-stream
Size: 3057 bytes
Desc: bug215426-fix.bundle
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20080411/22734995/attachment.obj
More information about the bazaar
mailing list