It was the best of times, it was the worst of times . . .it was the season of Light, it was the season of Darkness, it was the spring of hope
– Charles Dickens, A Tale of Two Cities
I remember it well. In early 2003, Michael Lewis released his book Moneyball. A fun story about how the underdog Oakland A’s used data to create a winning baseball team. I had always been interested in baseball, but I was also really interested in data and how it could be used to improve the company. I was starting my first data warehousing project and Moneyball was inspiring. Billy Bean used data to win at baseball, we were going to use our data warehouse to conquer the world . . .
A lot of game-changing advancements were happening in the data management space in the early/mid 2000’s – new vendors were popping up with MPP (massively parallel processing) platforms that could be built with commodity equipment. Netezza, Greenplum, and Vertica (and a handful of others) gave us the ability to process more data than ever. The phrase “map-reduce” was often heard in tech circles and the vendors told us we were entering the Era of Big Data. Our data warehouse would conquer the world . . .
In 2007, Michael Davenport released his book Competing on Analytics, and it added fuel to the fire. Data could be used to win at baseball. And now he showed us how Capital One, Harrah’s , and The Home Depot were competing – and winning – in their spaces using data and analytics. And Davenport gave us a roadmap to help convince our executives that getting more data in the hands of more people would magically turn us into data-driven antilytic competitors!
Well, it kind of worked out that way. We had a couple of mis-steps and a couple of reboots, but we definitely built a great data warehouse. But we didn’t have internal customers banging down our door to use it. Some groups groused that converting from their existing databases and reports would take too much time/money, but in almost every case we were hearing:
“I’m not sure where your data is coming from.”
“Why doesn’t your data match what we already have?”
“These values are wrong.”
And thus began a long process of getting users to adopt our data and tools. It wasn’t that we had bad data. But we had different data. Sometimes we were better than the other reporting systems. Sometimes they had figured things out better than we had. We had to uncover a lot of special rules and uses of the data. A field that meant one thing to Finance meant another to the Risk group.
In most cases, the “If you build it they will come” mentality doesn’t work well. At its root, user adoption comes down to trust. Do users trust the data you are giving them? Do they trust that you will manage it well? Do they trust that you’ll fix issues before they get out of control? Are they willing to stake their professional credibility on your data?
Formal (but not necessarily complex) data governance processes and programs go a long way to building and establishing trust around data. Systematically aligning an organization to a common data vocabulary is a great first step. Providing clear access to data definitions and mappings goes a long way to getting users to adopt your system. Finding and correcting data quality problems before they do is just as (or maybe more) important than giving them the latest and greatest technologies. And there’s no better time to start than today.