How Simple Can Amazon SimpleDB Be?

I am a developer using Amazon Simple DB currently in a project, so I'll add a few pro's and con's to help folks out:

Pro's:
===========
- It's dirt cheap for low budget apps
- It's easily available from anywhere on the internet, major plus for some distributed applications
- It's self managed, no administrative overhead, maintenance, etc.

Con's
===========
- It's an utter bear to use for the developer, lots of limitations requiring multiple queries to ensure an entire set is returned, or to ensure various exception conditions are covered.
- The API is dirt simple, meaning it doesn't support even the most common basic functions that we're used to with databases, you'll find yourself creating more code to work around the simplified database than you would vs. a standard relational DB. There's a lot of "Query-Read-Post" operations necessary where SQL would just require a Post where some basic clauses or simple arithmetic operations might suffice.
- I would hate to think of how to implement concurrency with this thing, while entirely possible, it's not going to be a walk in the park.
- It only supports strings, so be prepared for lots of conversions, and the potential runtime bugs that come with those conversions.
- It doesn't natively support keys on multiple attributes, you'll have to concatonate those key attributes into another single field that constitutes the key (a duplication of data in most use cases, and another potential outlet for bugs)


With all that said, I'm using it. I'm not liking every moment of it, but I can't beat the price, distributed availability, and self management aspects of it anywhere, so I've accepted a little extra code, and the bugs that come with it for my project.

You'll have to decide yourself. In my view, it's great for the little guys or straight forward use cases, but don't expect to take it too far beyond that. It sure as heck won't be an end to relational databases in my lifetime.

mfield, DaveK, you can cut Kalen some slack. First, you can't install SimpleDB, it is an online offering. Second, the difference between the relational model and the cloud(ish) model is as big as the difference between sequential files and the relational model. It requires a fundamental shift in the way you think about data - plus attending 10 hours worth of sessions by one of the pioneers on the subject.

Kalen, you are too kind when you describe my attempt to cram a week's worth of Teched in a couple of sentences. I agree with you that SimpleDB (or the Azure Table Service for that matter) may appear simple but building applications that use it correctly will be anything but:
- Versioning the data is easy, we just create a new entity with the attributes we need. Versioning the code is much harder, as it now has to cope with all data versions ever created.
- The cloud DBs take care of replicating our code for us and that makes availability really simple. But, updates take time to propagate to all replicas and we may well receive stale data in a subsequent read. Imagine a client in Australia requesting data that has just changed. The cloud DB has to decide which replica to return, the one in the US or the one in Sweden? There is no guarantee that the replicas are up to date.The application will have to cope with stale data too.
- Limited or no transaction support means we don't need to concern ourselves with locking and deadlocks anymore. Instead, we need to handle partially failed updates and compensating transactions.

Getting all this right will not be easy and we are all just now discovering what it takes to build systems for the cloud. There is no unifying theory or model for the cloud yet, just a general concensous among pioneers on the field that we need to relax some of the guarantees we took for granted if we want to scale out. A frightening but interesting prospect.

I too was surprised at the lack of research that was done here and the missing level of quality expectation I admire from SQL Server magazine contributors.

This isn't about some new type of service, its a database principle which I'd expect is known amongst DB professionals.

SimpleDB, MS SQL Service Data Services, Google BigTable all sacrifice traditional DBMS at the advantage of massive scalability. They are for the most part columnar databases, meaning you get one giant table with keys and each row has many columns. Each row doesn't necessarily have to have the same columns or data types. So there is no fixed schema, no JOINs (although references are sometimes possible but not efficient like a JOIN), etc.

These services and traditional databases are apples and oranges, each having their purposes and particular uses. You need a thick business layer to process your data, where much of this would be done in the database with a DBMS.

I'd suggest a proper review so DB professionals can truly evaluate these and make the right decisions around using these services. Unlike the article post states, there is ALOT of information available on how these services work and when they are appropriate architecture wise. Don't follow definitions the service provider gives - its all marketing speak - go out and see what people using it say.

You are forgetting that your audience will treat it as a review. Your negativity about the product are all they will remember. That's human nature. Certainly you could have found more positives to describe to make it a more rounded. I certainly understand the limitation of time. Perhaps you should stick to the SQL Server topics that we all respect you for, and you can speak from the hip about if you don't have time to do the deep dive into a different product?

Wow..thanks for all the comments!

I don't have hours and hours of spare time, and can only spend a few hours in researching and writing my commentary. If this was a full SQL Server Magazine article describing SimpleDB, then of course I would have installed it. But this is a commentary, not a research article.

My point in the commentary was that some of the questions I was asking should have been answerable WITHOUT having to install the product. Yes, if I was seriously considering using SimpleDB, I would intall it. But there should be more than hype available up front to let me know whether I should even spend any of my limited spare time on hands-on research.

So, mfield, you obviously missed my point. I was not rating the product. I was rating the announcement of the product and the availability of information available to help people decide whether to try it out or not.

Thanks to Brent, for your wonderful followup! And pkanavos, I appreciate your additional data points!

~Kalen