‘One Size Fits All’ Platform for Data Analytics?

A previous post by Frank Osborne discussed the value of data within organisations and how treating data as A Valued Core Product can lead to significant efficiencies and increased productivity, rather than simply a by-product of doing business got me thinking. How can organisations begin to productise their data when they are overwhelmed by decades of accumulation in multiple systems?

The EDW was, for many years, seen as the place where order would be achieved. Industry experts told us we’d see a “single version of the truth”, a place for a “golden record” or at the very least a “single reporting/analytics platform”. Unfortunately, this hasn’t been my experience of reality:

• We’ve seen processes for ingesting and integrating data take days, weeks or even months.

• We’ve seen standards applied to data that are meant to contribute to order, but usually do the opposite (sometimes driven by technology restrictions that no longer exist).

• We’ve seen access to data stymied by “Need to Know” governance rules.

• We’ve seen multiple versions of the same data. Not only in user-managed workspaces, but within the core of the EDW. Particularly as the cost of full integration means it often gets relegated to becoming a ‘nice to have’.

Tools to accelerate ingestion and smart techniques for matching and integrating data can certainly help with these issues but the reality again is that they rarely assist in reversing years of dysfunctional data management practices.

Data Lakes teased a solution to this problem by wrestling control away from centralised IT teams and putting the power into the hands of the user where they could load their own data, perform analysis with sophisticated tools and achieve outcomes quickly. Unfortunately, the open nature of the technology meant that governance of the data became harder to achieve; the complexity of the tooling made it often out of reach for many traditionally skilled analysts; and the skills market proved to be both scarce and expensive to resource assistance.

None of this lived up to the messages of “You don’t need expensive EDW platforms”; they are “Old-fashioned” or their proponents are “Dinosaurs”; new open-source technology is “Extremely Cheap” and will cost you a fraction of your EDW.

So, what’s the next silver bullet? The technology to rule them all? The Machine Learning/Artificial Intelligence based scientific method that will solve the data integration challenge for you? The simple answer is, there isn’t one. In fact, there never was. However, the cloud offers us a chance to choose the right technology without having to make huge ‘One Size Fits All’ platform investment commitment years in advance.

So, NOW is the time to let your business requirements drive your technology, process and governance choices. There is almost certainly a place in your organisation for a cloud-based relational database for simple access to structured core business data; an analytics platform that allows fail-fast discovery analytics with access to an open, collaborative user base; and governance processed across these platforms that is appropriate to the level of risk of making (or not making) the data available.

Once we stop the cycle of technology being viewed as the cause and solution to all data problems, then we can start thinking about real data challenges – How do we make data products that are high-quality, easily understood and consumable? How do we safeguard data without compromising its use? How can we ensure that the value of data is commensurate to the effort required for managing and maintaining it? How can we make sure that data is available to people that need it?

Comment on the article on LinkedIn - ‘One Size Fits All’ Platform for Data Analytics?