Database Filesystem? (Was Tracker in Edgy?)

[Ulrik Mikaelsson]

Doable, yes, but is it reasonable to include using FUSE?

I'm no FUSE-wiz, let's all agree to that first, but the little things
I think I've got right is that FUSE still adheres to VFS, basically
implying the good ol' file-structure after all. I'm sure there are
ways to hack around it by using special virtual directories and such,
and sure, it is a really quick solution that gives even legacy
applications this power.

However, is it really the best option to reuse an old API, explicity
designed to work with files as hierachial entities, for this flat and
indexed way of storing/retrieving files? Perhaps, maybe a better
option would be to simply hook the legacy applications up using FUSE,
but primarily focusing on a simple to use API to be used by the
applications that actually benefits from this type of storage. (People
tend to have less use of storing a temporary created ISO-file in a
database, compared to having the entire mail and music collection
indexed.) So sure, I think FUSE is a great idea for this, but let's
primarily use it for backwards compatibility. I think we should
consider tracker more a common solution for the database and storage
in Amarok, Evolution, Rythmbox and Kmail, than an all-purpose
filesystem. Then there should of course be some good querier/browser,
and a desktop icon to allow DnD for the action "Store this file in my
database".

By the way, how well does tracker deal with large files such as
DVD-images, and seeking?

Regarding DB-corruption I think I must agree with John a bit. I too
find the risk of having a sudden loss of power during a
file-relocation FS truncating my database in half, probably completely
erasing all my Music Collection, Photos and everything I consider
valuable in my computer, gone in the blink of an eye. If it is
possible, using a complete raw-partition as direct datastore sounds
like a much safer option, since the DBMS, if adapted for it, can use
information from the block devices to guarantee the safety of my data.
If a power-failure occurs, the databases own transactional support is
what will hopefully rescue my precious pictures and files. I don't
care of the file-structure my database uses, so using a filesystem
seems to be a mere extra risk (albeit tiny risk) to me?

Does _anyone_ in this list know of a database designed to work off raw
block-devices rather than a filesystem, suited to scale to a
database-size of a couple of terabytes?

Regards
/ Ulrik

On 7/2/06, Jamie McCracken <jamiemcc at blueyonder.co.uk> wrote:
> Ulrik Mikaelsson wrote:
> >
> > B) Partition the block device, and use a partition as a pure block
> > file for the database, be it MySQL, BDB, PostgreSQL, or some
> > completely new database engine, better suited for the job of keeping a
> > database directly on disk without a "intelligent" (in terms of caching
> > etc.) filesystem in between.
>
> Certainly doable using FUSE to create a userspace FS with Tracker.
> Virtual folder support using FUSE is on my agenda (as it will be a lot
> more powerful and system wide compared to Nautilus's limited smart
> folder capability)
>
>
> I dont buy most of the concerns people have wrt corruption simply
> because we have a very reliable FS in ext3 and automatic self repairing
> mysql database both of which should in theory fix most things that could
> go wrong if the worst happens. (and of course in the long term Reiser4
> will put this issue to rest anyhow) SO yeah it would be cool to have
> something like this.
>
> --
> Mr Jamie McCracken
> http://jamiemcc.livejournal.com/
>
>