On Smalltalk

In a previous post I created a simple blog containing two business objects, BlogPost and BlogComment. I implemented persistence for that blog in the simplest manner possible, I put an OrderedCollection of BlogPosts in an accessor called repository on the BlogPost class…

BlogPost class>>repository
    ^repository ifNil: [repository := OrderedCollection new]

Allowing the class itself to serve as the database for all its instances. This is great for development speed, and gives you the ability to play and try out many different things until your object model settles down and you figure out what your model is really going to look like. Using the image as an object database allows you to remain nimble as you learn more about the problem and change your mind as you gain new insight into the domain model. Put off transactional persistence as long as you can, it’s likely you can build almost your entire program without actually leaving the image or thinking about a database.

Let’s assume the we’re at that point now, and we’re going to map BlogPost and BlogComment into a real relational database as if this were a production app. I’m going to use PostgreSQL and Glorp, my first time using either, and see what it takes. I’m not claiming anything I do here is best practice, only sharing my experiences as I learn.

For the record, Alan Knight, the author of Glorp, has a working implementation of ActiveRecord, somewhat like Ruby on Rails, that would allow me to avoid all this meta-data, but as far as I know, it hasn’t been ported to Squeak yet, and is still in Alpha; I hope he’s done soon.

Until ActiveRecord is available, I’ll have to write the meta-data the old fashioned way. After reading the Glorp Tutorial, I see I need to subclass DescriptorSystem and create a class containing all the meta data for classes, tables, and the mappings between them. I call it SBGlorpDescriptions.

It seems I need to override two methods to tell Glorp about the tables and classes…

allTableNames
    ^#(POST COMMENT)

constructAllClasses
    ^(super constructAllClasses)
        add: BlogPost;
        add: BlogComment;
        yourself

Then I need to write methods for the table meta-data. Glorp will create the actual schema in PostgreSQL for me from this metadata, as well as use it to map between tables and classes.

tableForPOST: aTable
    (aTable createFieldNamed: #postId type: platform sequence) bePrimaryKey.
    aTable
        createFieldNamed: #title type: (platform varchar: 100);
        createFieldNamed: #body type: (platform text)

tableForCOMMENT: aTable
    | postId |
    (aTable createFieldNamed: #commentId type: platform sequence) bePrimaryKey.
    postId := aTable createFieldNamed: #postId type: platform int4.
    aTable
        createFieldNamed: #name type: (platform varchar: 100);
        createFieldNamed: #comment type: platform text;
        addForeignKeyFrom: postId
            to: ((self tableNamed: #POST) fieldNamed: 'postId')

I also need to write methods for the class model which gives me a chance to control how Glorp loads the classes. For example, I may want to force Glorp to use accessors rather than direct instance variable access. Here I simply tell Glorp that a Post has a collection of Comments.

classModelForPost: aModel
    aModel
        newAttributeNamed: #persistentId;
        newAttributeNamed: #title;
        newAttributeNamed: #body;
        newAttributeNamed: #comments collectionOf: BlogComment

classModelForComment: aModel
    aModel
        newAttributeNamed: #persistentId;
        newAttributeNamed: #name;
        newAttributeNamed: #comment

Now Glorp has a meta-data model for both classes and tables. Now, given the table meta-data and the class meta-data, I create a map between them for each class on each class…

BlogPost class>>glorpSetupDescriptor: aDescriptor forSystem: aDescriptorSystem
    | table |
    table := aDescriptorSystem tableNamed: #POST.
    aDescriptor table: table.
    (aDescriptor newMapping: DirectMapping) from: #persistentId
        to: (table fieldNamed: #postId).
    (aDescriptor newMapping: DirectMapping) from: #title
        to: (table fieldNamed: #title).
    (aDescriptor newMapping: DirectMapping) from: #body
        to: (table fieldNamed: #body).
    (aDescriptor newMapping: OneToManyMapping)
        attributeName: #comments;
        referenceClass: BlogComment;
        collectionType: OrderedCollection

BlogComment class>>glorpSetupDescriptor: aDescriptor forSystem: aDescriptorSystem
    | table |
    table := aDescriptorSystem tableNamed: #COMMENT.
    aDescriptor
        table: table;
        addMapping: (DirectMapping from: #persistentId to: (table fieldNamed: #commentId));
        addMapping: (DirectMapping from: #name to: (table fieldNamed: #name));
        addMapping: (DirectMapping from: #comment to: (table fieldNamed: #comment))

I’m not using the real power of Glorp here, because my models mostly match, however, were I programming against an existing legacy schema, I might start to appreciate all this meta-data a bit more, knowing I could map any shape class to any shape table. This is where Glorp truly shines over Rail’s ActiveRecord, which is an extremely simple and not so flexible object mapping system. Everything I’m doing here, Rails could do easily, but Glorp is much more powerful and can be used against legacy schemas that look nothing like the class model.

Rails does one thing, one way, very well, with inferred meta-data and virtually no configuration. Glorp can do anything, any way you like it, but requires explicit meta-data, with a bit of configuration. Alan’s about to give us the best of both worlds, by inferring the basic meta-data like Rails does, but allowing you to mix and match it with custom meta-data for more complex mappings, I can’t wait, and I’m glad Rails has pushed Alan in this direction. Glorp will be much easier to use for greenfield applications if it infers the meta-data with reasonable defaults, and Rails has proven how popular this approach is with developers.

OK, now that all the meta-data and mappings are created, I need to have Glorp create the schema for me. I fire up a Workspace and create a Glorp session using my new SBGlorpDescriptions class…

login := (Login new)
    database: PostgreSQLPlatform new;
    username: 'xxxxxx';
    password: 'xxxxxx';
    connectString: '127.0.0.1_seasideBlog'.

accessor := DatabaseAccessor forLogin: login.
accessor login.
session := GlorpSession new.
session system: (SBGlorpDescriptions forPlatform: login database).
session accessor: accessor.

Then I tell Glorp to create my schema…

session inTransactionDo:
    [session system allTables
        do: [:each | accessor createTable: each ifError: [:error | error inspect]]].

So far so good, everything works, time to change the blog and query through Glorp. On my Seaside session, I created a few delegation methods to pass through to the Glorp session which it contains…

database
	^database ifNil: [database := self buildDbSession]

buildDbSession
	| login accessor |
	login := (Login new)
				database: PostgreSQLPlatform new;
				username: ‘xxx’;
				password: ‘xxx’;
				connectString: ‘localhost_seasideBlog’.
	accessor := DatabaseAccessor forLogin: login.
	accessor login.
	^(GlorpSession new)
		system: (SBGlorpDescriptions forPlatform: login database);
		accessor: accessor;
		yourself

unregistered
    super unregistered.
    self database accessor logout

commit: aBlock
    ^self database inUnitOfWorkDo: aBlock

execute: aQuery
    ^self database execute: aQuery 

register: anObject
    ^self database register: anObject

And then make a few changes to the main blog component…

blogPosts
    ^BlogPost repository reversed

Becomes…

blogPosts
    ^self session execute:
        ((SimpleQuery returningManyOf: BlogPost limit: 10)
            orderBy: [:each | each persistentId descending];
            yourself)

And I modify #newPost and #addCommentTo: to look like so…

newPost
    | post |
    post := self call: ((BlogPost new asComponent)
                        addValidatedForm;
                        yourself).
    post ifNotNil: [self session commit: [self session register: post]]

addCommentTo: aPost
    | comment |
    comment := self call: ((BlogComment new asComponent)
                        addValidatedForm;
                        yourself).
    comment ifNotNil:
            [self session commit:
                    [self session register: aPost.
                    aPost addComment: comment]]

And that’s it, the sample blog now works against PostgreSQL. In all honesty, it was a bit of work, more than I expected, but given Glorp’s capabilities, I understand the need for so much meta-data. I’ll sure be glad when Alan finishes his ActiveRecord implementation because most of the time, that’s all I need.

In the mean time, I’ll have to write up a quick code generator to generate most of this meta-data for me directly from the classes using reflection. I’ve got a bigger project coming up where I plan to use Glorp, and there’s no way I’m writing all this by hand again.

Tags: Programming, Ruby, Seaside, Smalltalk, Sql

I’ve been working in Smallalk and Seaside for quite a while now, but something I haven’t quite gotten around to yet is trying to hook Squeak up to a database in a manner that I think could actually scale for a professional project. Now, I mean directly hook it up, so far, professionally, I’ve been using it against web services written in .Net against Microsoft SQL Server, which scales just fine, but leaves me still working in .Net, and I’d much rather work in pure Smalltalk.

Object Databases

I’ve tried several object databases, GOODS, Magma, and OmniBase, and while interesting experiences, I find them not quite acceptable for various reasons. OmniBase is file based, and has odd semantics that make hooking it up to multiple images and programming web apps against it difficult.

GOODS is very low level and bare bones, it works great, but you have to pre-index all your data, it has no query capabilities beyond what you provide in your object model, which can make performance quite horrible unless you know exactly what you’re doing and make very strict choices about how your data is stored. GOODS is also a one man show, so I’m not thrilled by the support I’d have available, were I trying to do something serious with it, though it works great for hobby and prototype programming.

Magma has queries, and is very similar to GOODS as far as ease of use goes but like GOODS, it’s a one man show, and I just wouldn’t feel comfortable doing anything truly serious on a one man show kind of database. This, for Squeak at least, seems to rule out object databases, for me anyway, though I hear Gemstone is going to support Seaside. I’m hopeful, for Gemstone is truly an enterprise ready object database, it’s just vaporware at this point, nothing production ready.

All of them seem rather slow when it comes to bulk inserts, and there are various solutions and workarounds depending on which version of Squeak you’re running, but a guy can only jump through so many hoops before he says “fuck it”. I’ve hit that point more than a few times when working with larger datasets and trying to do bulk inserts or queries.

Now, don’t get me wrong, Goods, Magma, and OmniBase are great products, and I’m sure have their uses, they’re just not something I’d throw up to my boss and say “hey, let’s use this for this big upcoming project”, because it’s hard enough throwing Squeak at them and having to support that decision over something like .Net which everyone already knows how to use, let alone taking away their familiar relational databases. I’d actually prefer something like Gemstone, time will tell if that preference pans out.

Relational Databases

So, on to relational databases. Squeak has ODBC support, but it’s single threaded and blocks the VM when querying, so while it works for demos and low traffic apps against pretty much any database, I wouldn’t try anything too big with it; it just can’t scale. Blocking the whole VM, every time you run a query, just leaves me feeling a bit dirty and not too proud of whatever I just wrote.

Squeak basically supports two popular relational databases that I know of, natively, MySQL and PostgreSQL. Now, I use MySql for this blog, so I have some experience managing it, and I’m just not a big fan, compared to Microsoft SQL Server, which I work with professionally daily, MySQL just sucks, but PostgreSQL is a different story, I’m quite impressed with the latest release, which looks and runs very nice on Windows servers and has a nice admin tool.

MySQL is IMHO not much better than Microsoft Access, it’s not an enterprise database. PostgreSQL, I think is much more comparable to SqlServer and Oracle and could be used for any size app. I have much more faith in its abilities and it has some cool features like table inheritance, which to an object bigot like myself, just makes me think relational databases aren’t totally void of innovation.

Now if I can just talk my DBA into giving up Sql Server, ummm… yea, not gonna happen, but I’ve got a side project coming up that’ll be totally green field development, no legacy database to worry about, hence my renewed interest in PostgreSQL, my new database of choice with Squeak.

Getting Started

So I installed the latest PostgreSQL, installed the PostgreSQL Client for Squeak from SqueakMap, then the GLORP port from SqueakMap, and gave it a shot.

I was immediately confronted with a nasty error that reminded me why I gave up last time I tried PostgreSQL. Something about the PostgreSQL driver’s state machine not being valid, luckily enough time had passed that a few minutes Googling turned up a simple answer this time, unlike the last.

PostgreSQL installs with MD5 connection encryption turned on, which Squeak doesn’t support out of the box. There are two fixes, either install the Cryptography package from SqueakMap, or turn encryption in PostgreSQL to “password” with a simple configuration change to the pg_hba.conf. I chose the latter, as its default install only accepts local connections anyway I’m not too concerned with encryption.

PostgreSQL fires up and runs without a problem, GLORP tests all run fine, so now I just have to learn GLORP and how to map my objects into PostgreSQL, but that, is going to have to be another story. I’m going to map the simple blog in Seaside into a PostgreSQL database to learn GLORP, and I’ll post that code once I figure it out. I’ve done a bit already, and so far, I’m quite happy with GLORP.

Tags: Databases, Programming, Seaside, Smalltalk, Sql

Here’s an interesting post from a Java guy trying to decide between Ruby on Rails and Seaside. He has quite a few interesting things to say concerning the shortcomings of Ruby on Rails, and how well Seaside handles that complexity with ease.

He also has a few complaints about Seaside, most of them valid. Seaside still isn’t the full stack solution that Ruby on Rails is. We still have to handle object relational mapping, something Ruby on Rails gives you for free. Nor does Seaside deal with object validation and errors, I use Magritte for this. Magritte rocks, but I’m not sure the average guy trying out Seaside will find it, or learn how to use it. From an outside point of view, Ruby on Rails looks like a much more complete solution.

We have Glorp, which can do this, but only against Postgres Sql in Squeak. Nothing against Postgres, but seriously, in the real business world, it’s either Microsoft Sql Server or Oracle; and it’s also usually a legacy database, so we really need something like Glorp for those databases, because something like ActiveRecord is just too brain dead to work.

Most of the schemas I have to work with suck, and can’t be changed because people used the database as an integration point for multiple apps (God I’m tired of seeing people make this mistake). Doing a simple class = table, object = row mapping just doesn’t cut the mustard for legacy development against existing databases.

Seaside is far more advanced than Ruby on Rails, and is a much better web framework for doing anything complex, but we’re still missing the market on CRUD apps. CRUD against a popular business database is still far too difficult using Squeak. I’m sure Visualworks has far better database support, but I want something free… I want Squeak, I want Squeak to work with Sql Server, I want a Pony… :(

Featured Resources…

There are different programming languages for different purposes. For advertising the one that has the best features becomes bit difficult as they usually have their own purposes to fulfill. Similar competition applies to the hosting companies, like ipowerweb, or another best one would be dot5hosting to choose from. With the cheap internet phones comes the cheap call rates. Taking a 350-001 test got easier with preparation guides available online.

Tags: Databases, Magritte, Programming, Ruby, Seaside, Sql

Sadly, I wasn’t able to make the trip to OOPSLA this year, but I found a few podcasts that covered the Objects and Databases: State of the Union in 2006, as well as a few other interesting ones. It’s great to hear these issues being discussed, dealing with databases and objects in still one of the fundamental problems in our industry.

There aren’t any easy answers, there are benefits to both object databases, and relational databases. The butchering of an object model so it can be shoehorned into a relational database via some mapping layer isn’t the answer. Industry wide, the relational folk seem to think this is a solved problem and object databases are modern dinosaurs. Mathematically, they may be right, pragmatically, they’re absolutely wrong, and the academics are wrong as well. I don’t care how pretty the relational model is, programming to it is a nightmare and doesn’t fit the way programmers actually work.

Databases will learn to deal with objects as objects, without crippling them, or they will be replaced with ones that can, period.

Whatever the future of databases are, they will need to learn to deal with random queries (a weakness of object databases) and fast access to objects with a nearly transparent API (a weakness of relational databases). As Esther Dyson says, “Using tables to store objects is like driving your car home and then disassembling it to put it in the garage. It can be assembled again in the morning, but one eventually asks whether this is the most efficient way to park a car.”

There has to be a better solution!

Tags: Databases, Programming