Integration Dogfight in the CloudsIntegration Dogfight in the Clouds

A startup called SnapLogic is poised to challenge incumbent Informatica with an approach that seems to combine data integration -- Informatica's forté -- with online app stores (think Apple). Things are poised to get interesting.

The idea behind the SnapStore, now thrown open for a 100-day beta test by SnapLogic, is similar to the Apple application store: developers can load their applications on a storefront hosted by SnapLogic; when a sale is made, they receive 70% of the price while SnapLogic retains 30%. The difference is that these applications - called Snapps - are in fact data integration (i.e. ETL) components such as connectors, transformations and pipelines (i.e. programs). These components will all need to use the DataFlow API ("SnAPI"); in turn, this ensures that components can be build on top of, and together with, each other. But does this imply vendor lock-in? Nope. SnapLogic Chairman/CEO Gaurav Dhillon tells me (via email) developers are free to write the components in a popular Web language like Java or Python and then isolate the SnAPI dependencies. In many cases, existing libraries can easily be wrapped with a SnAPI wrapper.(Want more on the SnapStore? Here's an excerpt from their PR release: "The SnapStore provides listing, rating, reviewing, provisioning, billing, and updating for Snaps and related services. SnapLogic provides quality assurance and certification of Snaps available in the SnapStore.")

The idea is novel (in data integration, that is) -- the SnapStore concept takes cloud computing a significant step further -- and the concept is a winner. Yet, it will require rigorous implementation, and the challenges are numerous: organizing the components in the online store; helping prospective customers understand exactly the capabilities and limitations of a particular component; money-back situations where a component doesn't work or has a slightly different behavior than expected, etc. etc. Businesses looking to use such pre-built components could have their work cut out too: integrating multiple third-party components -- in the cloud, no less -- is no piece of cake.

Success seems far from guaranteed, but if it works, it's a win-win proposition. Entrepreneurial ETL developers can capitalize on their creativity and expertise by putting out components for sharing… for a price (imagine the ETL developer-small-businessman!). Customers and development teams benefit by using these pre-built pieces of ETL to build componentized solutions faster.

With the SnapStore, SnapLogic poses a direct challenge to Informatica (originally co-founded, in an interesting twist, by Dhillon himself) as well as to the likes of Talend and RightScale, which have joined hands to provide a flavor of SaaS data integration. In particular, I think, Informatica will need to do more and better at shedding what seems like a growingly staid image. (Note, IBM/Ascential and Oracle face the challenge too, but they have moved on into a totally different space, with their data integration solutions hitched on to other bandwagons. It's harder when you call yourself "the data integration company" -- your only self-professed identity is based on how well you do and how fast you move in data integration.)

As somebody that works closely with clients, architects and developers in areas of data integration, I love the concept of a marketplace for pre-built components in ETL -- who wouldn't? Now to see if this bold chick learns to fly in the clouds. Check this space for an update... but don't hold your breath!

To subscribe to the weekly Intelligent Enterprise newsletters, click here.A startup called SnapLogic is poised to challenge incumbent Informatica with an approach that seems to combine data integration -- Informatica's forté -- with online app stores (think Apple). Things are poised to get interesting.