Whether we’re paraphrasing Albert Einstein or restating Occam’s razor, the idea of simplicity resonates. We’ve all been in those environments where everything was more complicated than it needed to be. And those systems that morph into Rube Goldberg machines drive us all crazy. A few bonus features here, add a few political considerations there, and sprinkle in some bells and whistles just because we can; and we end up with something bigger than we need and more costly than we can afford.
Metadata and data mapping tools are no exception to this routine. We’ve been building data warehouse and reporting systems for 20 years and have seen both ends of the spectrum. At some clients, it’s too simple. “Just throw it into Excel and figure out how to share it with the team.” And when that doesn’t work it becomes “Can someone knock out an Access database this week and get everyone to put their mappings in it?” Trying to short-cut things causes rework and more headaches for everyone.
Or . . .
Everyone agrees that spreadsheets aren’t the way to go and we launch into a months-long requirements process to document every conceivable need the organization has. Business glossary, business and technical names, mapping and lineage are no-brainers but then someone wants to make sure it is fully integrated with the data warehouse (never mind the endless meetings just to figure out what “fully integrated” means to everyone). The project managers want it to have configurable workflows and the development team wants it to generate ETL code.
It takes 3 months just to run the RFP and the vendor bake-off and then you still have to install the hardware, configure and customize the software, test all of the integrations, and put the entire organization through a week of onsite training.
Before you know it, you’re a year into the effort and have spent a hefty war chest before you’ve mapped a single field.
We aim to be the Anti-Rube-Goldberg: Provide a simple tool that does the basics of metadata management and data mapping really, really well. Simple, but not too simple. The alternative to Rube Goldberg is this:
- Provide a central repository for curating and cataloging definitions, reference values, and mappings
- Be accessible and usable to the widest audience possible – for technical and business users
- Provide a historical view of definitions/mappings and easily allow for future changes (releases, new development, business changes, etc.)
- Simplify setup, configuration, training, and maintenance
- Avoid developing features and complexities that limit any of the above
If you have the time and the money, there’s nothing wrong with having robust integration with every tool in your stack. Generating code is a great feature for your data mapping tool – but it only benefits part of your organization. Too many nice-to-have features limit what you set out to do in the first place.