×

Employee Perspective: Continue Your Innovation with Data Mesh

By: Jungbin Oh
September 7, 2022
Share: Linked In

Data Mesh: Need of the Hour

The technology sector is having a hard time with the stock price drop, inflation, fear of recession, and rising interest rates. However, one thing that companies cannot and should not give up is the momentum of innovation. The key to continuous innovation lies not on vertical scaling, but on horizontal scaling. Data Mesh is the best approach to accomplishing disruptive innovation with its pursuit of decentralized, self-serving, and domain-oriented architecture.

How I Learned English

Language is composed of four pillars, which are speaking, listening, reading, and writing. When I was a riot police officer in Korea, my teammates often asked which of the four pillars they should master first to learn English quickly. One of them told me that he plans to master speaking first so he can apply while traveling abroad immediately. However, my advice was always the same: pursue balanced growth. Language has an intricately interconnected cultural context; thus, mastering just one pillar alone might sound easier than balanced learning, but it is highly inefficient. Four pillars must collaborate and lean on each other to grow as a whole.

I believe growing a data-driven company resembles the way we grow new language skills. Is vertically scaling one type of database the best way to grow? I feel this is the same as studying reading skills only while ignoring learning how to speak, write, or listen. In data-driven companies, all departments have to collaborate with each other, where each department possesses the most optimized data structure. Today, there are numerous solutions for Big Data management, each developed to provide specialized services. The web application team may need a no-SQL database like MongoDB for their specialty in API, and DevOps may need object storage like Parquet on S3 to handle terabytes of ETL pipeline. To ensure different superpowers collaborate on the growth and innovation of the company’s full potential, the Data Mesh approach is the best solution in the market today.

Data Mesh with Starburst

With a strong belief in the value and potential of Data Mesh, Starburst provides more than 30 different connectors to existing data technologies, from databases to data visualization tools.

Database

When you Google “SQL and no-SQL,” the headline is “Which one is better: SQL or No-SQL?” The Data Mesh version of the answer is, both! Each has its own strengths and weaknesses, and the best practice is to curate the effective system and federate them. Starburst provides connectors of both types of databases, where the end user can speak through the unified language of ANSI-SQL.

Starburst Multi-Database Federation

Figure 1: Starburst Multi-Database Federation

Cloud Vendors

Also for companies that pursue multi-cloud solutions or hybrid infrastructure, Starburst is available for all the options including AWS, Azure, GCP, Alibaba, and on-prem infrastructure vendors. Many companies worry about the increased dependency on a specific database or cloud vendor risking future flexibility, but Starburst allows them to alleviate such risk by liberating them from relying on a single technology vendor.

Data Consumption Layer

Figure 2: Starburst Data Consumption Layer

Data Visualization

One of the biggest hurdles to make changes to data infrastructure comes from the integration of data visualization tools. Different teams or departments may prefer different software from Tableau to Power BI, and if any of them have to give up using its original tool for any reason, two issues occur. First, the existing workbooks cumulated and curated for the specific domain use case become invalid, which slows down the job performances. Second, the organization may get locked into a single data visualization tool which has its own limitations that cannot be replaced unless other tools are available. Starburst’s Data Mesh framework allows end users to keep using existing tools while allowing renovation of backend data infrastructure.

Achieve Horizontal Scalability with Data Mesh

In my view, horizontal scalability is a key to innovation that data-driven companies need to seriously consider. Centralized data solutions may sell sweet dreams of easy control and cost efficiency that may look achievable, but doing so costs loss of diversity where creativity and innovation suffer. Great products need both SQL and No-SQL. Useful insights come from analysis visualized by cumulated history of workbooks on the data visualization tool best curated for each domain (marketing, business intelligence, data analysis, etc). Starburst’s solution around Data Mesh ideology focuses on collaboration of different power tools ranging from databases to data visualization tools, helping continuous innovation and growth of the company.

Jungbin Oh

Customer Success Manager, Starburst

Start Free with
Starburst Galaxy

Up to $500 in usage credits included

  • Query your data lake fast with Starburst's best-in-class MPP SQL query engine
  • Get up and running in less than 5 minutes
  • Easily deploy clusters in AWS, Azure and Google Cloud
For more deployment options:
Download Starburst Enterprise

Please fill in all required fields and ensure you are using a valid email address.