Rev 28: Merge John's updates, allowing global blooms for the whole index, requires new pybloom from https://code.edge.launchpad.net/~jameinel/+junk/pybloom in http://people.ubuntu.com/~robertc/baz2.0/plugins/index2/trunk
Robert Collins
robertc at robertcollins.net
Sat Jul 12 06:14:35 BST 2008
At http://people.ubuntu.com/~robertc/baz2.0/plugins/index2/trunk
------------------------------------------------------------
revno: 28
revision-id: robertc at robertcollins.net-20080712051426-pbkgal01tj82brpw
parent: robertc at robertcollins.net-20080704061030-xled0wh4s6yeq4f1
parent: john at arbash-meinel.com-20080708024112-p9k0l8a8vkttnpr6
committer: Robert Collins <robertc at robertcollins.net>
branch nick: trunk
timestamp: Sat 2008-07-12 15:14:26 +1000
message:
Merge John's updates, allowing global blooms for the whole index, requires new pybloom from https://code.edge.launchpad.net/~jameinel/+junk/pybloom
modified:
.bzrignore bzrignore-20080703034434-q63sohljnxg5loze-1
_parse_btree_c.pyx _parse_btree_c.pyx-20080703034413-3q25bklkenti3p8p-2
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
indexbench.py indexbench.py-20080702083855-5tju02y79rw7kkzh-1
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.36
revision-id: john at arbash-meinel.com-20080708024112-p9k0l8a8vkttnpr6
parent: john at arbash-meinel.com-20080708023113-ovl2cdcf8jifh3d1
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 21:41:12 -0500
message:
Make sure you have loaded the root node in _iter_sorted_entries
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
------------------------------------------------------------
revno: 8.1.35
revision-id: john at arbash-meinel.com-20080708023113-ovl2cdcf8jifh3d1
parent: john at arbash-meinel.com-20080707224256-0uz4gera4thhi6vr
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 21:31:13 -0500
message:
node.bloom never is smarter than the global bloom, so don't use it.
Actually *use* the filtered list we just spent the time to create.
Don't actually pre-load the key => offset map unless we have a
bloom to look in.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
indexbench.py indexbench.py-20080702083855-5tju02y79rw7kkzh-1
------------------------------------------------------------
revno: 8.1.34
revision-id: john at arbash-meinel.com-20080707224256-0uz4gera4thhi6vr
parent: john at arbash-meinel.com-20080707193706-nsl64vpa3z2ntj12
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 17:42:56 -0500
message:
A prototype of a different function than iter_entries
The idea was that we could preserve the known sorting, and thus not
need to sort multiple times per pass.
This doesn't help the miss_torture test, because it is not going
through the normal channels, it actually makes miss_torture
much worse right now, still investigating.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
indexbench.py indexbench.py-20080702083855-5tju02y79rw7kkzh-1
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.33
revision-id: john at arbash-meinel.com-20080707193706-nsl64vpa3z2ntj12
parent: john at arbash-meinel.com-20080707183457-cwddxt5rvuan7nra
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 14:37:06 -0500
message:
Write down bloom pages to disk.
Now all indexes that have >1 page will also have a
bloom page written for them.
Next step is to hook this up to the reader, so that
it can be loaded on request, and get the benchmarks
to see if it helps. Then consider writing a 'paged'
bloom handler that doesn't have to read everything
at once.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.32
revision-id: john at arbash-meinel.com-20080707183457-cwddxt5rvuan7nra
parent: john at arbash-meinel.com-20080707180217-21trnahth6phuvpa
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 13:34:57 -0500
message:
Write out the number of pages taken up for a global bloom.
Tests show the parser round trips it correctly.
Add a requireFeature() test, to show if the extensions are available or not.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.31
revision-id: john at arbash-meinel.com-20080707180217-21trnahth6phuvpa
parent: john at arbash-meinel.com-20080707175247-mdsvs2tvf8nukbcm
parent: robertc at robertcollins.net-20080704061030-xled0wh4s6yeq4f1
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 13:02:17 -0500
message:
Merge Robert's changes, and fix a failing test that assumed you didn't have to pass size
added:
tests/test_repofmt.py test_repofmt.py-20080704030345-bza6rrd6nf4sdmyy-1
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
indexbench.py indexbench.py-20080702083855-5tju02y79rw7kkzh-1
repofmt.py repofmt.py-20080701113732-m1iu3n94ikbxdelb-1
tests/__init__.py __init__.py-20080624222253-p0x5f92uyh5hw734-10
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.30
revision-id: john at arbash-meinel.com-20080707175247-mdsvs2tvf8nukbcm
parent: john at arbash-meinel.com-20080707175118-v92oedb6m1vyeu8j
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 12:52:47 -0500
message:
The writer now decides which row is the bloom row,
rather than writing a useless bloom to all layers after the first.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
------------------------------------------------------------
revno: 8.1.29
revision-id: john at arbash-meinel.com-20080707175118-v92oedb6m1vyeu8j
parent: john at arbash-meinel.com-20080707165306-it2a1ucnn1h1rn0q
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 12:51:18 -0500
message:
Only create blooms in a specific layer.
Also, reserve the right number of *bytes* for the blooms on disk.
Sizes in constants are in bits, reserved is in bytes.
Small tweaks to help test_iter_all_entries_reads to be more independent
of number of pages on disk.
It still will fail, but the necessary change is now localized better.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.28
revision-id: john at arbash-meinel.com-20080707165306-it2a1ucnn1h1rn0q
parent: john at arbash-meinel.com-20080703233011-l2sxt8dt8pyvs0bx
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Mon 2008-07-07 11:53:06 -0500
message:
Some minor reworking, separate out the size of the
global bloom from the sizes for individual blooms.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
indexbench.py indexbench.py-20080702083855-5tju02y79rw7kkzh-1
------------------------------------------------------------
revno: 8.1.27
revision-id: john at arbash-meinel.com-20080703233011-l2sxt8dt8pyvs0bx
parent: john at arbash-meinel.com-20080703211121-hkin5osvlz0soy6w
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Thu 2008-07-03 18:30:11 -0500
message:
Allow dynamically growing the blooms when they start to fill up.
There is a bit of tuning to do here. How many b/e before we jump, how far do we jump, etc.
at 16 b/e and jump 2x the blooms were getting unexpectedly full.
32 b/e and 4x jump creates bigger blooms, but keeps them fairly empty.
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
------------------------------------------------------------
revno: 8.1.26
revision-id: john at arbash-meinel.com-20080703211121-hkin5osvlz0soy6w
parent: john at arbash-meinel.com-20080703171239-p0yr682smdn1jb3v
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Thu 2008-07-03 16:11:21 -0500
message:
Pre-compute the row offsets, paving the way for sneaking in extra pages
modified:
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
------------------------------------------------------------
revno: 8.1.25
revision-id: john at arbash-meinel.com-20080703171239-p0yr682smdn1jb3v
parent: john at arbash-meinel.com-20080703171207-a315ypnhybt1r6vk
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Thu 2008-07-03 12:12:39 -0500
message:
Copy _my_memrchr from the _dirstate_helpers so that this will compile and run on win32
modified:
_parse_btree_c.pyx _parse_btree_c.pyx-20080703034413-3q25bklkenti3p8p-2
------------------------------------------------------------
revno: 8.1.24
revision-id: john at arbash-meinel.com-20080703171207-a315ypnhybt1r6vk
parent: john at arbash-meinel.com-20080702210419-4xfq1jb9k4cuksk1
parent: robertc at robertcollins.net-20080703115714-czomk4m21tfu2ebe
committer: John Arbash Meinel <john at arbash-meinel.com>
branch nick: index2
timestamp: Thu 2008-07-03 12:12:07 -0500
message:
Merge in Robert's code.
added:
.bzrignore bzrignore-20080703034434-q63sohljnxg5loze-1
_parse_btree_c.pyx _parse_btree_c.pyx-20080703034413-3q25bklkenti3p8p-2
_parse_btree_py.py _parse_btree_py.py-20080703034413-3q25bklkenti3p8p-3
modified:
__init__.py __init__.py-20080624222253-p0x5f92uyh5hw734-5
btree_index.py index.py-20080624222253-p0x5f92uyh5hw734-7
indexbench.py indexbench.py-20080702083855-5tju02y79rw7kkzh-1
repofmt.py repofmt.py-20080701113732-m1iu3n94ikbxdelb-1
setup.py setup.py-20080624222253-p0x5f92uyh5hw734-8
tests/test_btree_index.py test_index.py-20080624222253-p0x5f92uyh5hw734-13
Diff too large for email (1088 lines, the limit is 1000).
More information about the bazaar-commits
mailing list