After posting my original thoughts on industry analysts and open source my initial inclination was to leave the subject at that. However, Joe Niski from Burton Group left a thoughtful comment which motivated me to add further commentary. Even after responding to Joe's comment on the post I considered a particular statement of his worth exploring in a separate post. Joe mentioned that:
"...unless you're talking about a really well-established best-of-breed product, it's a lot harder to get useful information on OSS projects *unless you're in a position to download, install, and evaluate them first-hand*. i suspect that makes them inaccessible to some analysts."
I'm with Joe on this one but I also think that the availability of products, source code and transparent development communities as a result of the open source model isn't a side effect so much as it is a call for analysts to adjust their approach accordingly. Think of it this way, if [insert name of any given Fortune 500 proprietary software vendor] decided to provide industry analysts with open access to at least 85% of their development activities including visibility into the code base as well as an openly available version of [insert name of a product from the same Fortune 500 software vendor], wouldn't research consumers expect a little bit more from analyst coverage of that aforesaid technology? To me, the option to evaluate software first hand should work to make it MORE accessible to ALL analysts. Not only that but it works at obliterating barriers to producing useful analysis by more than a "chosen few."
Obviously, industry analysts shouldn't be charged with becoming power users nor with attaining ingrained familiarity with the technologies they cover, but there is still room to expand capabilities in light of being granted added transparency...it doesn't have to be an either-or-proposition. The RedMonk team stays well tuned to the developer grapevine and reaps the benefits of doing so. Maintaining close contact with those who sport first hand experience can be counted as nothing but a positive. Along those same lines, how much would analysts benefit from getting an actual feel for technology? Considering how much is gained from assuming the perspective of a user, I would say quite a bit. From my perspective, as companies get smarter and more enterprise open source goes mainstream this is set to become part of the future role of analysts. As more sources of quality analysis arise across the long tail, the more earned insight that can be provided the better.
Does downloading, installing and spending time using a piece of software guarantee understanding of product strengths/weaknesses, vendor capabilities or feature completeness? Absolutely not. Yet how many products which don't require domain-specific knowledge, i.e. developer-centric tools, is it useful to neglect an evaluation in the first person...especially as it relates to open source? I can't help but think that it is essentially hypocritical for the analyst community to give lip service to open source disruption without doing our part to adjust to it.