In the late ’90s and early 2000s, the Agile Software Development movement completely changed how individuals and corporations performed software development. This change from a monolithic, waterfall style of development to a more rapid, iterative approach helped developers focus more on problem solving, collaboration, and innovation. Simply doing a Google search for “Agile Development” will reveal thousands of resources on the benefits of this methodology, including what is considered one of the most important events of the movement — the Agile Software Development Manifesto.
I would argue that the Agile Software Development movement helped spur not only a healthier development culture, lowering the bar for many to become software engineers, but also contributed to the rise of open source development, as well as the current startup renaissance.
Analytics, if you look at it in a broad sense, is not dissimilar to software development. From design, architecture, coding, and testing, analytic lifecycles mirror many of the same processes seen in software development. But if you look at how most companies perform analytics, an agile approach is uncommon. Most enterprises are still mired in monolithic (data warehouses), non-collaborative (siloed data sets, teams, and domain expertise), opaque (who did what when?) processes.
But in today’s data driven, big data world, analytics velocity — and I don’t mean computing speed — is paramount. Monolithic, non-collaborative, opaque processes are enemies of velocity. How can we change this? Thankfully, there have been many new technologies introduced in the past few years that are helping companies analyze their data faster. Technologies such as Hadoop, Spark, Apache Zeppelin, and many others are helping enterprises analyze more of their ever increasing data sets faster. But that’s not enough.
Current progress in analytics is just pockets of technology innovation leveraged by few. Much like the Agile Software Development movement, I propose that we also need an Agile Analytics Manifesto. I’m not egotistical enough to think I can come up with one alone, but maybe we can start by borrowing from the Agile Software Development Manifesto:
- Individuals and Interactions over processes and tools
- Working Software over comprehensive documentation (replace with ‘Explorative Analytics over comprehensive dashboards’?)
- Customer Collaboration over contract negotiation
- Responding to Change over following a plan
What would you add or change? All movements need a beginning.
Would love to hear your thoughts here, through twitter (@zeplco), or directly via sejun@zepl.com.
