The Design of Software (CLOSED)

A public forum for discussing the design of software, from the user interface to the code architecture. Now closed.

The "Design of Software" discussion group has been merged with the main Joel on Software discussion group.

The archives will remain online indefinitely.

google datastore transactions and scalability

I've been studying google's datastore that comes with their apps engine, and there's no discussion of it here yet. Their design has a lot of potential getting away from the scalability issues of regular databases where you end up having to deal with distribution, clustering and replication.

The key to transactions in their datastore lies in "entity groups." An entity is like a flexible row or record. Upon adding an entity to the database you have to associate it with a group if you ever plan to combine an action or query on that entity with another in the group in one transaction. This is not to be confused with having them in the same table. The "kind" of entity works like a table in that in GQL you select from entities of a particular kind.

Anyway, this means that if I am tracking modifications of Item entities using Revision entities, I will need to put the Revisions for an Item in an entity group with that Item. It also suggests that I cannot have a Revision entity that involves 2 Items unless I can do it without combining the Items modification into one transaction with the creation of the Revision entity.

On the face of it I think designing an application with this in mind has great benefits. I wonder if there are any significant disadvantages.

BTW I posted 1.5 years ago about not wanting column widths (an antiquated feature of database development) without realizing SQLite was already there. Great to see too that there is no sign of column widths in the google datastore.
Ben Bryant Send private email
Wednesday, April 09, 2008
I must have missed that thread the first time. It was interesting. I completely agree with your not liking fixed column widths. We use a (somewhat off-the-beaten-track) database product that has variable length everything, and for the most part, I can't understand what the point is of doing otherwise.
Greg Send private email
Wednesday, April 09, 2008
Google datastore is fine for simple read/write webs (like blogs, wikis or online shops) but not for websites required to do massive calculations. I appreciate this product for its simplicity to get things started but eventually you will feel constrained by their framework. I still think the best longterm bet is to configure your own stack having virtual hardware supplied on-demand by vendors like Amazon. I wouldn't start anything big on Google and not only because it's not portable.
Monday, April 14, 2008
Thanks lubos. The reading I've been doing is giving me the same impression.
Ben Bryant Send private email
Wednesday, April 16, 2008

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
Powered by FogBugz