Beginning on the journey to a Data Mesh can be an overwhelming task. Where do you start? During Datanova for Data Leaders, we featured a panel on how Data Mesh is affecting organizations across industries. This panel featured Richard Jarvis, CTO at EMIS Health, Mangesh Patil, Director of Data Science and Engineering at Disney, and Zhamak Dehghani, Director of Next Tech Incubation at ThoughtWorks and founder of the term Data Mesh and was moderated by Paul Gillin, Enterprise Editor at SiliconAngle. They discussed the Data Mesh journey, cultural changes, technological changes, and the best advice to begin implementing a Data Mesh.
What is a Data Mesh?
Previously, the data warehouse tried to put all an organization’s data in one place in order to make decisions. However, this concept is increasingly out of step with the reality of data management. Now, decentralized data which advocates for greater access for more people is preferred, which is what Data Mesh aims to achieve.
Zhamak puts it briefly that Data Mesh is, “a sociotechnical approach in managing, accessing, and sharing data for analytical purposes at scale.” This paradigm is one that impacts the organizational structure, ownership of data, as well as the overarching technology, and attempts to solve the desire to use data in new and novel ways.
How is Data Mesh disrupting traditional data architectures?
Data Mesh is disrupting the way people consume data. Instead of having to wait days to create models and analytics, data users have access to real-time data to query more accurately and really be innovative with their data. For Richard and EMIS Health, having access to the most up-to-date data is vital. Without the most accurate data, doctors and physicians can’t provide the best care to their patients. By having all of their data decentralized, they can, “react much more quickly and change how [they] can deliver health care benefits.”
Where do you start your Data Mesh journey?
Zhamak explains that, “The problems that Data Mesh pinpoints and articulates are real.” Organizations want to stop moving and copying their data and they want more data visibility across different teams. Early adopters of the Data Mesh principle have started changing their mindset around data use and access. The next step is to determine which technologies will work best in their organization while incorporating a Data Mesh.
EMIS Health actually began with technology rather than organization in their Data Mesh journey. They already had silos of data that they needed to join together in order to get a cost effective analytical output and thereby provide better healthcare at cheaper prices. Once they began implementing new technologies to get rid of these data silos, they quickly realized that they were actually building a Data Mesh, and then they could begin incorporating the characteristics of a Data Mesh.
Mangesh and Disney are early on in their journey to achieving a Data Mesh. One of the pillars at the heart of a Data Mesh is treating data as a product, and this is a goal Mangesh is trying to achieve. Having decentralized data access and data products allows for more users to have access to data across their entire organization through domain owners and teams. Although Disney has not made any technological changes, they are starting to change their data culture. By using Starburst, Disney allows its data to stay where it lives, and can demonstrate the value of a Data Mesh by pulling data quickly. They can rapidly do feature engineering, build models, and put those models into production.
Zhamak describes a test for validating that your organization is on the right path to a Data Mesh. This can be measured by how quickly a data user can go from a hypothesis to finding data, to then consuming, understanding, and creating models of the data. She recommends getting started with actual use cases. By beginning with building a platform that enables peer-to-peer communication, you can have domains engaged early on which gives autonomy to domain teams. With data driven insights, organizations can discover where in the business they can generate value for customers, partners, and employees. Organizations will be able to see the value in a reduced lead time to consumption of data, increased volume of data users, and the overall satisfaction of data users.
Implications of a Data Mesh
A Data Mesh provides business value across organizations. For Richard and EMIS Health, externally their clients are able to treat people early and with less expensive treatments. Internally, they can better be able to understand their customer’s behavior and optimize their experience. Mangesh describes one of the key benefits is that they can achieve organic value input from all places in an organization simultaneously. Because a Data Mesh allows for the creation of data products, different domain teams across the organization can achieve the most value out of their data. By putting data users at the center, organizations can optimize their data understanding.
Data Mesh helps increase data understanding across the business. Technology aside, it allows for people to start to have the conversation around the sharing of data, which is one of the last walls of data organization that needs to come down.
The Journey of a Thousand Miles Begins with a Single Step
Zhamak aptly concludes that you should start the Date Mesh journey with business value but embrace it as an evolutionary approach. Transitioning to a Data Mesh paradigm can seem daunting at first; however, it really all begins with changing the data culture within your organization. Richard suggests, “Start small and prove value early on,” and Mangesh advises to give it time to learn this new skill set.
Get Started with Starburst Galaxy today
Sign up for Starburst Galaxy and get up to 500 credits free.