×
×

Clarifying Data Mesh Roles And Responsibilities

By: Colleen Tartow, Ph.D., Andy Mott, MBA
November 2, 2022
Share: Linked In

Recently we co-authored the book Data Mesh For Dummies, along with our colleague Adrian Estala. We greatly enjoyed writing the book and collaborating to succinctly deliver the ideas around Data Mesh in a straightforward format. However, like many authors we were asked to trim our book’s word count to satisfy a strict page limit.

Fortunately, there’s a very popular medium known as blogs and we decided to salvage the text on Data Mesh roles and responsibilities and repurpose it here. Below, we are thrilled to present one of our lost chapters on Data Mesh roles and responsibilities.

Data Mesh for Dummies - Starburst

Roles and Responsibilities in a Data Mesh

Embarking on a Data Mesh journey will result in organizational changes and adjustments to employees’ roles. Existing workers will be critical to the success of adopting a Data Mesh, as they have invaluable tacit knowledge to contribute to the Data Mesh journey. Therefore, the transition of data ownership from a central data team to decentralized domains should be approached as a reallocation of existing data-focused employees.

With that in mind, here are some of the roles and related responsibilities that will likely be involved:

  • Executive Sponsor: Responsible for overall corporate data strategy and ensuring alignment with the business strategy. This is likely to be a member of the C-suite due to the significance of the organizational change required.
  • Domain Owner: Represents the needs of the domain when defining federated computational governance standards.
  • Data Product Manager: Responsible for the scope,delivery, and lifecycle of one or more data products within a domain.
  • Domain Data Architect: Designs how data flows from source to products, defines data product data models, and ensures adherence to architecture standards within a domain.
  • Data Product Developer (Data Engineer): Creates and maintains data products.
  • Infrastructure Engineer: Responsible for the operation and maintenance of the self-serve data platform.
  • Domain SME / Analyst: Experts in the business area that relates to the domain, and understands the consumers’ needs, to ensure that data products meet their business objectives. Typically the first consumer of a data product.
  • Legal / InfoSec / Audit: A set of expert resources available to the domains to provide guidance on areas such as corporate risk and compliance standards.

It’s important to note that these are roles, and not necessarily individual people. In some organizations, especially in the early adoption phase of Data Mesh, some employees will take on multiple roles. In addition, the choice of technologies within a self-serve data platform can dramatically affect the workload and efficiency of the employees in these roles.

Designing the decentralized organization for success

The design for your Data Mesh organization should focus on efficiency through empowerment. Data engineering skills will be embedded within business-focused domain teams. This organizational change will take time and will evolve as you move along your Data Mesh journey.

Here are some key design principles to consider for your organization:

Distribute data engineering responsibilities to the domains

Data engineers are the experts in shaping data, and over time they will become subject matter experts in the data related to the domain in which they operate. Giving them autonomy and control will accelerate results.

Design the domain to manage its data with autonomy

A federated computational governance model enables the domains to make salient and relevant decisions within their domain, while ensuring that they follow the cross-domain data governance rules.

Use external advisory teams

Domains should be autonomous, but not all domains will require full-time support for every specialist role. For example, each domain may need access to a legal advisor to ensure that they stay within GDPR rules when dealing with personal data.

As organizations move along their Data Mesh journey it’s likely that new domains will be developed and the number of data products will increase. Data product developers and data product architects will manage the data lifecycle within the domain, and will play a key role in creating new data products.

Data Mesh For Dummies

Want to discuss or learn more? Contact us

Download the Book
Colleen Tartow, Ph.D.

Director of Engineering, Starburst

Colleen Tartow, Ph.D. has over 20 years of experience in data, advanced analytics, engineering, consulting, and she has been obsessed with data her entire life. Adept at assisting organizations in deriving value from a data-driven culture, she has successfully led large data, engineering, and analytics teams through the development of complex global data management solutions, and architecting front- and back-end SaaS and enterprise data systems. Colleen is also experienced in building and leading diverse teams through business reorganization and transforming existing data ecosystems, maturing them into modern and robust technology stacks. She is determined to make engineering organizations better for both humans and business through mentoring, leadership, and streamlining processes. Her demonstrated excellence in data and engineering leadership makes her a trusted senior advisor among executives, and her work has led to her speaking at a variety of events in the technology leadership space, and mentoring aspiring leaders in data, analytics, and technology.

Andy Mott, MBA

EMEA Head of Partner Solutions Architecture and Data Mesh Lead, Starburst

With more than 20 years of experience in data analytics, Andy Mott is skilled at optimizing the utility of analytics within organizations. When determining how to generate value or fortifying existing revenue through technologies, Andy considers the alignment of an organization’s culture, structure and business processes. He ensures that the strategic direction of the organization will ultimately enable organizations to out compete their respective markets. In his spare time, Andy enjoys tinkering with his home Hadoop cluster and Linux is his preferred operating system. Andy Mott is currently EMEA head of partner solutions architecture and a Data Mesh lead at Starburst, and lives in the United Kingdom.

Start Free with
Starburst Galaxy

Up to $500 in usage credits included

  • Query your data lake fast with Starburst's best-in-class MPP SQL query engine
  • Get up and running in less than 5 minutes
  • Easily deploy clusters in AWS, Azure and Google Cloud
For more deployment options:
Download Starburst Enterprise

Please fill in all required fields and ensure you are using a valid email address.