Open source and the numbers game

As a follow up (a day later than previously advertised...thanks to a bad strain of casserole) to my previous post, it is appropriate to investigate how the unique nature of open source software has created a void in the development of metrics for related companies, products and overall industry growth. Even more important is what can be done to fill the void, considering some of the specific requirements and constraints involved with doing so.

Before writing this post, I was reading a well done article on RoughlyDrafted which details some of the shortcomings of relying too heavily on market share statistics to gauge the impact of products. From the article:

Market share simply refers to the portion of a vendor’s unit sales compared to the overall market. However, most large markets include specialized niches that each act as a market. For example, within the overall market for vehicles are passenger cars, and buried within that major segment is the small but profitable luxury car market.

BMW doesn't compete against ship and plane builders, nor even the entire line of cars built by GM. It would therefore be absurd to talk about BMW's small share of the "vehicle market," or even to compare its market share among other car makers. It's simply pointless and irrelevant.

This is a valid point that is often overlooked by some industry analysts in an attempt to identify market leaders and strong performers. Without digging beneath the surface levels, it is easy to miss a great deal of the variances and dynamics that compose a given market. In the same breath, this approach fails to account for niche segments which can be quite substantial if the overall market is large enough. Meaning the conversation center less around big vs. small than it does generic vs. detailed, where the birds eye view approach is a bit too broad to compel competent and objective analysis.

Traditional numbers and open source
The emergence of open source is transforming the consensus understanding of the dynamics of both software development models and the business of software in general. Intertwined with this change is a need to formulate new methods for measurement and analysis. As a result, relying on traditional tools and approaches will continue to produce incomplete and vague coverage of the industry as a whole. This quandary affects all parties involved, everyone from open source companies to the industry analyst community to the user community depend on being able to quantify what all are experiencing first hand: namely, the significant paradigm shift that is open source.

A root cause is the fact that numbers such as product sales per quarter/per year or even customer wins can only begin to serve as diagnostics for the health and future prospects of open source technology. So while open source products can more or less be evaluated on a pure functional basis (using scorecards, feature matrices, etc.) in an identical manner as proprietary software. The nature of being in business as an open source vendor is that your product(s) can often be in use [legally] without resulting in a dime of revenue and without you being remotely aware of it. An essentially good possibility indeed, but one that has made it necessary to accurately track things like downloads and user registrations. All in the attempt to gain a better depiction of adoption rates, which demonstrate acceptance and serve as growth metrics. Consequently, the community component of an open source ecosystem must be included as an indicator equal in importance to more traditional ones.

However, monitoring the number of downloads and registrations is only a start. Just as the elements of a balance sheet attempt to provide a concise view of financial wellbeing and outlook, certain aspects of an open source community absolutely must be monitored. Granted, this requires a context switch which can only be made once the importance of community is fully grasped. Nonetheless, as time passes and some open source outfits prosper while others fade, the empirical evidence for the relationship between strong community and commercial success will be as evident as that of revenue and profitability.

Where it leads
In more ways than one, the added transparency and openness of open source models call for more evolved techniques which can gauge the condition of the resulting realities. In the same way that downloads might not mean much in the proprietary realm, a different set of activity areas are important within the open source universe. Despite the fact that good technology which meets a need will always win over the long haul and good software springs forth from both closed and open source beginnings. There will always be a need to distill strengths and weakness into a concise format. Graphs, charts, reports and the like, all exist to meet this need. Part of the reason for this: numbers don't lie. Even if they can be assuaged deceptively, they are and will continue to be inherently objective.

What begs to be investigated is what the other numbers are which matter as it relates to illustrating growth, progress and other key indicators for open source products and the companies behind them. OpenBRR already provides a method for evaluating the maturity of an open source product, in terms of adopting it. But what if I want to view product growth over the last three quarters I need to dig and find out as much as possible about that set of reliable determinants. Stats regarding version releases, bug reporting/resolution, code contribution and even the number of prominent issues voiced by users must be considered. The good news is that with open source all of these are publicly visible, the bad news is that they aren't actively being collected and integrated into the equation.

That being said, Entiva has its own process + toolset for doing the aforementioned and I would venture to guess that other analyst firms who are active in the open source landscape have their own. Still, it remains my perspective that more exchange across the entire spectrum needs to take place in order to sharpen and improve some of these current methods. Open dialogue will benefit everyone over the short and the long term. So if anyone wants to get a conversation started, just drop me a line.

Technorati Tags: market share |  open source software | analysis and measurement |  industry analyst community | commercial open source

Comments

[www.asiawebmedia.com]

Asiawebmedia Technologies Provides .Net Development, ASP/PHP

Programmers hiring,SQL/MySQL development,software outsourcing,

Offshore software development,Web Development and Web Design etc.

http://www.asiawebmedia.com

[Vintage Typewriters]

pretty cool stuff for a post. thanks.