[storm] Moving Storm Forward

[James Henstridge]

On Fri, Jun 12, 2009 at 12:09 AM, Vernon Cole <vernondcole at gmail.com> wrote:
> True, each increment is a reasonable step, and I for one would like to
> pursue the project. ZeOmega has step 1 pretty much done, as far as MS-SQL is
> concerned.
>
> My worry is what happens when I hook to something OTHER than MS-SQL using my
> ADO connection? There are a lot of [insert shudder here] Access (a.k.a, Jet)
> databases out there, can we include them? I already test adodbapi against
> Access, MS-SQL, and MySQL and am planning on adding Postgress tests. Can we
> add complexity to the internals of Storm to allow using ADO to access those?
> How to I clue Storm to use ADO to access different engines? Will I use
> 'ado-MySQL://dbname' vs 'ado-msSQL://db' vs 'ado-Access://filePath')? Will I
> have to change my create_database argument to switch between different
> drivers for the same engine?

So, unless ADO provides the features Storm needs in a database neutral
way, it will probably be necessary to have some per-database code at
the Python level.

The main non-standard behaviour Storm needs from the backend is a way
to identify a row that has just been inserted into the database where
the primary key has been set to an auto increment or sequence value.
There are also a few minor bits to do with reserved words and if the
database has any weird precedence rules in its expression parser.  If
ADO doesn't provide database independent ways of doing this, then it
will still be necessary to use per-database code.

It would be nice to share as much code as possible in this case.  A
good way to do this would be to copy either the mysql or postgresql
backends and tests to a different name, then port them to adodbapi and
get all the test to pass (this should be relatively easy, since the
backends are already know how to speak the respective databases' SQL
dialects).  The diffs between the original and adodbapi backends
should then give an idea of what needs to be shared.


> All of these concerns point to making create_database more complex than it
> now is: so that it can do logical mapping of "scheme:" to different database
> engines. - - - While we are in that neighborhood, I want to able to
> virtualize the entire
> "scheme://username:password@hostname:port/database_name" construct so that I
> can deliver programs to customers who use different engines, usernames,
> passwords, hosts, ports, and database names, without modifying the source
> code. [ I am stuck on that idea because I have been using a database system
> which does that really well (RDM) for the past 28 years. Perhaps there is a
> better way?]

Well, one option here is to put the connection strings into a
configuration file.  For example, the storm.zope.zstorm code lets you
configure the connections through a ZCML file.  The Python code then
just accesses named stores through the IZStorm utility.  This sounds
similar to what you are proposing.

We've been using the zstorm code in a number of non-zope projects
(Django apps and twisted servers), so it might also cover your needs.


> I mention that here, rather than filing a Bug, because there needs to be
> open discussion about how to go about it. This is a big design question. The
> existing design of "we support three schemes, if you want another scheme,
> write a module with  the correct name" is simple and effective. ADO fouls up
> the works. If that simplicity is to be lost, the change needs to be done
> right, if it is to be worth the cost.

Adodbapi certainly makes things more complicated.  However, if it is
the adapter that makes sense for Windows users then it is probably
worth taking on that complexity.  Hopefully it doesn't need to be
passed on to users.

James.