Database Filesystem? (Was Tracker in Edgy?)

[Ulrik Mikaelsson]

Well, I can agree to it's not beeing a big issue, but yes, there are
records of Ext3 causing data loss. Many of them are due to misbehaving
harddrives ignoring the ATA fsync, and one or two due to bugs in the
filesystem implementation.

The ones regarding the misbehaving harddrives are of course hard to
deal with. However, I suspect the DBMS would be a bit better of trying
to recreate most of it's own structure, saving a bunch of objects, in
a block device where a few blocks have not been written properly, than
in a filesystem where half the file might be slashed right off, or
lost completely.

In the case of buggy filesystems, well, we can never completely ignore
the risk, and quite a lot of users prefer XFS or Reiser due to it's
performance-improvements. Some people like JFS just because of it's
strengths in partition resizing. While these users might have to
accept the risks of loosing important files, why should they be
enforced to risk a complete _other_ filesystem (the database) nested
inside their chosen first-level one?

Basically, as I see it, there is no need to have the database running
on top of a different filesystem. Doing so will just increase risks,
and to me seems a bit fishy from a architectural view. Supporting
running the database in the filesystem in order to avoid hard-drive
fragmentation in partitioning, is of course a good thing (tm), but I
think both modes should be kept an option. (Much like using a
swap-partition or a swap file in the filesystem is actually an option
today.)

Anyways, it's good to see tracker making progress. From what I've read
it seems to be decent solution both from a transitional, and a
long-term POV.

Regards
/ Ulrik

On 7/2/06, Jamie McCracken <jamiemcc at blueyonder.co.uk> wrote:
> Ulrik Mikaelsson wrote:
>
> > By the way, how well does tracker deal with large files such as
> > DVD-images, and seeking?
>
> Tracker is designed more for objects with properties like emails,
> bookmarks etc rather than storing blobs of files (there really is no
> benefit in storing them in the DB over the filesystem)
>
>
> > Regarding DB-corruption I think I must agree with John a bit. I too
> > find the risk of having a sudden loss of power during a
> > file-relocation FS truncating my database in half, probably completely
> > erasing all my Music Collection, Photos and everything I consider
> > valuable in my computer, gone in the blink of an eye.
>
> Agian this is a total non-issue with Ext3.
>
> EXT3 offer full data journalling such that after a power failure, the
> filesystem structure *and* the file data is guaranteed to be intact.
> This is the safest available and you should never get any corruption
> with this.
>
> EXT3 however by default uses the "ordered" mode. It is basically
> metadata journalling but adds a constraint that the writes to disk will
> be ordered in such a way that it the data is also guaranteed to be
> intact as well as the filesystem structure. It may not be the *latest*
> data that the app was told was written to disk but it is guaranteed not
> to be garbage. IE it may be the data that was there before the last write.
>
> So for the final time - talk of databases like Tracker corrupting on
> Ext3 due to power loss is pretty much FUD (unless anyone can show
> otherwise of course).
>
> --
> Mr Jamie McCracken
> http://jamiemcc.livejournal.com/
>
>