committing a large (600 MB) file, p4-fast-export, fast-import

[Matt McClure]

I have a Perforce repository containing a 658 MB file, that I am
trying to mirror in Bazaar.  I was able to export it with
p4-fast-export, but I found that bzr fast-import raises MemoryError
trying to import it.  Before I go too far trying to reduce bzr
fast-import's memory usage, I did a quick check to see if I could
commit a 600 MB file to a Bazaar branch.  No luck.  It also raises
MemoryError.

I have 2 GB of memory.  How much memory would Bazaar need to commit
this file?  Is there hope of reducing Bazaar's and bzr fast-import's
memory usage?

C:\u\mlm\tmp\bzr>dir
 Volume in drive C has no label.
 Volume Serial Number is 5CEB-4368

 Directory of C:\u\mlm\tmp\bzr

02/22/2009  04:21 PM    <DIR>          .
02/22/2009  04:21 PM    <DIR>          ..
02/22/2009  04:21 PM       629,145,600 file
               1 File(s)    629,145,600 bytes
               2 Dir(s)  19,440,357,376 bytes free

C:\u\mlm\tmp\bzr>bzr add file
added file

C:\u\mlm\tmp\bzr>bzr ci -m 'test'
Committing to: C:/u/mlm/tmp/bzr/
aborting commit write group: MemoryError()
bzr: ERROR: exceptions.MemoryError:

Traceback (most recent call last):
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commands.py", line
893, in run_bzr_catch_errors
    return run_bzr(argv)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commands.py", line
839, in run_bzr
    ret = run(*run_argv)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commands.py", line
539, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\builtins.py", line
2434, in run
    exclude=safe_relpath_files(tree, exclude))
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\decorators.py", line
192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\workingtree_4.py",
line 235, in commit
    result = WorkingTree3.commit(self, message, revprops, *args, **kwargs)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\decorators.py", line
192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\mutabletree.py",
line 201, incommit
    *args, **kwargs)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commit.py", line
369, in commit
    self._update_builder_with_changes()
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commit.py", line
674, in _update_builder_with_changes
    self._populate_from_inventory()
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commit.py", line
823, in _populate_from_inventory
    content_summary)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\commit.py", line
867, in _record_entry
    ie, self.parent_invs, path, self.work_tree, content_summary)
  File "C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\repository.py", line
388, in record_entry_contents
    lines = file_obj.readlines()
MemoryError

bzr 1.10dev on python 2.5.2 (win32)
arguments: ['C:\\u\\mlm\\w\\bzr\\repo\\bzr.mlm.stable\\bzr', 'ci',
'-m', "'test'"]
encoding: 'cp1252', fsenc: 'mbcs', lang: None
plugins:
  bzrp4
C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\plugins\bzrp4 [unknown]
  bzrtools
C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\plugins\bzrtools [1.10]
  fastimport
C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\plugins\fastimport [unknown]
  launchpad
C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\plugins\launchpad [unknown]
  rebase               C:\Documents and Settings\mlm\Application
Data\bazaar\2.0\plugins\rebase [0.4.3dev]
  xmloutput
C:\u\mlm\w\bzr\repo\bzr.mlm.stable\bzrlib\plugins\xmloutput [0.8.1]
*** Bazaar has encountered an internal error.
    Please report a bug at https://bugs.launchpad.net/bzr/+filebug
    including this traceback, and a description of what you
    were doing when the error occurred.

C:\u\mlm\tmp\bzr>

Matt