Database Filesystem? (Was Tracker in Edgy?)

[Ulrik Mikaelsson]

Perhaps not really on topic of Ubuntu Development, but a question I've
been pondering for a while on this subject.

One of the things that _really_ suprises me about files and databases
is why we are still using a traditional hierarchial filesystem for
most of our chores? Yes, I understand it's a paradigm that in theory
are often simple for the user to understand, but is it REALLY the way
the _system_ should deal with files?

A bunch of years ago, the creators of BeOS did some brilliant things,
one of them were their filesystems. This information is second-hand, I
have not investigated the BeOS filesystem much myself, but from what
I've heard, their filesystem were split in half, where as one half
were treated as a regular file-tree, and the other as an indexed,
unsorted database of object-like entities. I think this concept is
extremely interesting and I wonder why noone have tried following it
up? As I see, this could be achieved in two ways.

A) (Reiser4) split the filesystem up in layers, and let the
space-allocation happen in a layer different from actual content or
meta-data. Provide a module (or plugin) that hooks up on the provided
allocator and build a database on the raw allocated blocks from the
allocator. In this way, the regular files and the database can be
efficiently shared from the same block device without hard
partitioning in between them.

B) Partition the block device, and use a partition as a pure block
file for the database, be it MySQL, BDB, PostgreSQL, or some
completely new database engine, better suited for the job of keeping a
database directly on disk without a "intelligent" (in terms of caching
etc.) filesystem in between.

I certainly think it would be a useful addition to the modern desktop,
since the only place I see most users preferring hierarchy are
possibly within mail-program or similar. I see many users having
problems with grasping the concept of the filesystem, and how it maps
into their applications objects. For instance, only few popular Media
Applications today considers the music-collection a hierarchy. Almost
all of them (Rythmbox, Amarok, ITunes, WMP...) see them purely as a
database, heavily using the index-and-search methodology of a
database. Unifying the back-end of this seems like a good thing(TM) to
me.

My question: could we use Tracker as a database-frontend for
supporting applications to interface with the database-store, and
configure it's MySQL-backend to store on a separate partition, to
avoid the issues John R. Moser are pointing out?

Would this be feasible?

Could MySQL cope with this?

Is it possible to implement relatively quick? Maybe in the timeframe
for Edgy, or Edgy + 1?

Regards
/ Ulrik

On 6/30/06, Jamie McCracken <jamiemcc at blueyonder.co.uk> wrote:
> John Richard Moser wrote:
> > Data journaling does not magically solve anything, it just slows things
> > down.  As for meta-data journaling, different implementations have
> > different problems; some methods can truncate if you're shrinking a
> > file, some methods can zero-fill a file for whatever reason, and some
> > will just remove the file.
>
>
> With FS that support atomic file operations , it gives you protection
> against half written records ( and I presume resizes too else it
> wouldn;t be atomic then!). Updates to a file either happen completely or
> not at all.
>
> To quote Reiser4 : Reiser4 is an atomic filesystem, which means that
> your filesystem operations either entirely occur, or they entirely
> don't, and they don't corrupt due to half occuring. We do this without
> significant performance losses, because we invented algorithms to do it
> without copying the data twice.
>
> This is what Im referring to wrt to making mysql super robust (I presume
> *data* journaling gives you atomic updates). You shouldn't have to worry
> about DBs corrupting with an atomic journalised FS (providing its not
> buggy of course!).
>
>
> --
> Mr Jamie McCracken
> http://jamiemcc.livejournal.com/
>
>
> --
> ubuntu-devel mailing list
> ubuntu-devel at lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
>