What we Learned in Chicago

We passed another milestone in our MapWorks journey last week. We attended our first industry event with MapWorks, the TDWI conference in Chicago. I think I speak for the whole crew when I say we had a great experience. From the keynotes and session speakers to the other attendees and exhibitors and on to the historic setting at the Hilton Chicago and the great food we ate (probably a little too much), everything was terrific. We offer a sincere thank you and big pat on the back to the TDWI staff and others who pulled off such an excellent event.

There are a lot of exciting things being done in the data world. As an industry, we are making great strides to make the promises of the data-driven organization a reality (not to mention the data-driven world). In just this conference we heard loads about advances in automation, machine learning and virtualization that not only allow bleeding-edge organizations to get real value and real insight from their data, but also allows those organizations who are striving to get up to speed to do so quickly. And these automation and virtualization technologies aren’t just for the bleeding-edge – they are now the mainstream approach and we see everyone headed that way.  We picked up on two main themes:

Yesterday’s bleeding-edge is now mainstream.

  • Virtualization is not the “data federation” approach from a few years ago that only worked in special cases.  Microsoft, AWS, and Rackspace pushing large-scale frameworks that make virtualizing in the cloud that much more scalable.  Specialty products like Denodo give companies the ability to virtualize their data environments in their own data centers.
  • We’ll be seeing more and more EDW and data integration shops building virtual CDI and analytics platforms directly on top of their source applications or on Hadoop data lakes.  Traditional ETL will be replaced with data virtualization as transformations, cleansing, and business rules are applied in-flight.  Better, faster, cheaper – that’s the driving force behind it all.  Traditional EDW architectures are going to more closely resemble the real-time data streaming systems that power our communication and transportation networks.

Information is still worthless unless it can cause change

  • Data quality, data governance, clear/clean documentation, metadata management, lineage analysis are more important than ever.  A development team’s agility is directly correlated to it’s ability to quickly understand what has to be built or what will be impacted by their changes.
  • Pushing more data at a faster rate to decision makers means more analysis is needed.  Correlation will never mean causation (you’ve heard it a thousand times, I know), and the need for rigorous, statistical analysis is only going to increase.  To apply advances in AI and machine learning, we’re going to have to have better data and understand how it is related.

The great thing about conferences like this TDWI conference is that you meet people from organizations at every level of maturity, and there truly is something for everyone.

Speaking of the people, we were blown away by the reception we received from the TDWI attendees. It was a true pleasure and a great honor for us to meet so many great people and hear the challenges they are facing. Many of those challenges are ones we have faced and are the reason we started MapWorks. We believe that we have a great solution and are excited to hopefully be a part of helping them conquer their challenges. Our conversations affirmed for us the need so many people have for a simple to use, yet powerful tool like MapWorks and gave us great insight as to what we need to do next to make MapWorks even better.

We’re already looking forward to the next TDWI event. See you in Anaheim in August!