SnapLogic: Innovation, data integration networks and open source

I recently had a chance to talk to Chris Marino, CEO and co-founder, of SnapLogic about his company's approach to Internet-scale, data integration. The SnapLogic team as a whole has significant experience and credentials within the two areas they aim to bring together, namely Internet-infrastructure and data integration. This is important, as throwing open source at a problem is different from coupling the benefits of an open source model with practical industry experience. Accordingly, during the talk and presentation they were able to portray a good understanding of the dynamics [shortcomings, opportunities for innovation/disruption] of the integration space.

Data integration + open source
A background for SnapLogic includes an overview of the current state of integration. Presently, commercial integration solutions are primarily designed for client-server applications while minimally addressing the bulk of problems which don't warrant a $10,000+ price tag (more on that later). As a result, there has been a preponderance of hand coded solutions put into place to close this gap. However, neither are well suited to scale in meeting what has been an explosion in data, service endpoints and the like, outside the firewall.

Since Matt Asay already posted a good synopsis of how SnapLogic is positioned to meet this demand using open source, I'll shy away from re-wording the information provided there. Nevertheless, it suffices to say that SnapLogic intends to offer a solution which reduces the inherent complexity of integration efforts by introducing standards-based consistency, using the concept of a data integration network. As illustrated below:

The SnapLogic bubble, portrayed in diagram's center, is the aforementioned integration network. It replaces the tangled [and expensive] mass that connector-based offerings from vendors like Tibco, Informatica, WebMethods, and IBM create when they are stretched beyond the limits of client-server architectures. SnapLogic offers an open source, full-featured option that outflanks the labors of hand coding and out-scales inflexible, costly commercial solutions. Simply put, bigger vendors just aren't positioned to meet this part of the demand curve when their entire business models and product suites have molded themselves almost exclusively to its higher end.

This has effectively created a vacuum with substantial room for SnapLogic to step into what could be close to a $19 billion data integration space by 2008 (according to IDC estimates), and deliver some well-heeled, innovation-driven disruption. As time passes, a squeeze on proprietary incumbents will most likely occur at the hands of a gathering momentum along a "long-tail." Thereby, resulting in an increasingly tighter segment in which the aforementioned market leaders will have to compete with each other...as room for growth and expansion down the tail will be progressively non-existent.

A new wave of open source innovation
From a larger, long-term perspective SnapLogic represents the next step in the evolution of the open source. In its earliest stages and up until today, open source has been pigeonholed solely as a source of less-expensive alternatives. Even today, there is a tendency to view it as more of a reactionary response to proprietary price and inflexibility pressures within a given market [see early stage Linux, MySQL]. Yet, as is evident with SnapLogic, open source is also a burgeoning source of and context for innovation.

During my talk with SnapLogic it became clear that it could have just as easily taken the proprietary road seeing how they:

• Are introducing an entirely new approach.
• Compete in an established market with ongoing growth potential.
• Are targeting an underserved market segment with a standards based approach.
• Have a working product with an experienced team behind it.

Consequently, it's nowhere near a stretch to visualize them as a closed source start-up capable of drumming up interest in a potential series A round of investment from the VC community. In more ways than one, an open source model is SnapLogic's by the choosing as opposed to out of raw necessity. The company can exist/compete without it, but has realized that the proprietary model is fundamentally broken and has adjusted accordingly. Even more telling is the SnapLogic team's focus on establishing community infrastructure around the project from the jump, an indication of a comprehension of the concept that community drives dollars.

Additionally, a fully open source development model co-joined with a commitment to open processes encourages the community to create, share and modify adapters. The net effective of which, will end up benefiting SnapLogic as a platform for innovation, as much as any core product feature set. Fellow open source operation, Jitterbit has experienced continuous success by encouraging a similar process of give-back/sharing of not just source code but also domain-specific components. So the model definitely has its merits as a productive fuel for the growth of an open source community.

Where things stand
It will be interesting to observe how similarly positioned open source products from the likes of Jitterbit, MuleSource and Talend are going to respond to SnapLogic's entrance, especially considering the company's potential for strong product differentiation (Internet-scale, data integration networks) and structurally sound approach to growth (community-centric approach). Either way, open source is well equipped to begin a collective push across the data integration landscape with solutions for EAI, ETL and more. If nothing else, it is an area definitely worth keeping an eye on.

Technorati Tags: SnapLogic |  open source software | data integration networks |  ETL | EAI

Comments

[Chris Marino]

Thanks for the kind words. There seems to be ample room in this space for different approaches because the problem is so large. Messaging routing, data services, SOA all will benefit from open source.

[James Governor]

you might want to go back and compare with Grand Central- which had similar ambitions and goals but didn't pull it off.

[Chris Marino]

James, you're not the first person to make this comparison. Grand Central tried to do way too much, way too early. Their vision of a Web Services Network sounds similar to a Data Integration Network, but they are actually quite different. A Data Integration Network provides data services not transaction services. Grand Central's vision was to become a central hub for Web Services with complexity of SOAP/WSDL/etc. They even had their own UDDI registry. That's not even close to what we are suggesting with SnapLogic.

[Chris Marino]

One more thing. It's not 100% clear from Alex's post here, so I'll say it explicitly. SnapLogic is not a hosted service. It's software that you download and run on your own premises and configure anyway you like. The diagram might indicate a shared infrastructure (like Grand Central's), but that is not the case at all. With SnapLogic, the software can run anywhere. On a dedicated servers or on the endpoints, or any combination of the two. The transformation logic can be partitioned across any of the instances. Collectively they become a Data Integration Network.