APP For Scalability

One of the common first steps for scaling up an application is to move the database off to a dedicated server – often followed by having multiple application server instances to handle requests. With a standard SQL database that’s pretty straight forward, with data stored in Amazon S3 that’s not always as simple.

S3 obviously provides a network API, but it doesn’t necessarily provide all the functionality you need from your data layer. For instance if you need to update search indexes you need a central server to track the changes and update the indexes. You may also need synchronization above what S3 provides etc. Whatever the reason, you need to provide a server to handle those data layer tasks and then pass the storage off to S3.

So where does APP (Atom Publishing Protocol) fit in? It occurs to me that for a large number of applications it’s probably a very good interface to use. Firstly, there’s some libraries being created that make it easy to get up and running. More importantly, because it’s a standard, other applications can also connect to your data store.

Of course it all depends on exactly what data your application deals with, and I still haven’t worked out the best way to deal with versions, but it shows some interesting potential.