[storm] psycopg2-2.2, Storm, and ENUMs

[James Henstridge]

On Tue, May 25, 2010 at 12:15 AM, Dan Halbert <halbert at halwitz.org> wrote:
>
>
> On Monday, May 24, 2010 11:35am, "James Henstridge" <james at jamesh.id.au>
> said:
>
>> On Sat, May 22, 2010 at 3:06 AM, Dan Halbert <halbert at halwitz.org> wrote:
>> > Psycopg2-2.2 now adds explicit casts to strings in places where it did
>> > not before.
>> > ...
>> > This change causes several storm tests to break for type conversion
>> > reasons.
>
> *** That the above change caused the storm test breakage perhaps got lost in
> my following verbiage
> and is probably the more important point. It will be necessary to fix the
> storm tests to work with psycopg2-2.2.

Thanks for the heads up.  I'll try to take a look at it soon.


>> The RawStr property type in Storm is really intended for use with
>> binary data. Given that the 2.x str type has essentially become the
>> bytes type in Python 3, I wonder if the psycopg2 changes are treating
>> it in a similar fashion?
>
> I believe the psycopg2 change was not motivated by Python 3, but was to fix
> some different
> problems. But you're right that the effect is to treat 2.x strings as bytes.
> Here's the start of the thread on their list:
> http://lists.initd.org/pipermail/psycopg/2010-April/006985.html
>
> Storm treats any object for which isinstance(obj, str) is true as a
> psycopg2.Binary,
> which psycopg2 now decorateswith a "::bytea".
> (See to_database() in databases/postgres.py.)

Right.  Storm expects text data to be represented as unicode at that level.


>> That said, If you are dealing with textual data (which the enumeration
>> values essentially are), perhaps the Unicode type would have worked
>> better. Another option would be the Enum type, which lets you provide
>> an arbitrary map between Python and database values.
>
> In my case, I used a "CREATE TYPE" to create some string-based enums.
> (The schema design predates my use of storm.)
> When I passed these psycopg2 as RawStr's, psycopg2 used to leave these
> strings alone,
> and PostgreSQL would coerce the string into the appropriate type.
> But now psycopg2 decorates them with an explicit type, which doesn't match
> the column type.
> So my RawStr() hack broke. Not a big deal, but I thought other people might
> run into the same
> problem.

My point here is that the enumeration value names are effectively text
rather than encoded binary data.  So our Unicode type is probably a
better fit than the RawStr type, and would probably work with current
psycopg2.

James.