Optimizing query performance to power data lake analytics

About

Since its inception in 2014, advertising technology company El Toro has become a leader in IP targeting. Its patented technology allows advertisers to target individual homes, based on their IP addresses and the demographic information of the people living there, with extreme precision. This allows customers to “get the right message, to the right person, at the right time,” shares El Toro’s CTO, Richard Teachout. El Toro was struggling to facilitate ad-hoc analytics due to its hundreds of databases, and queries involving over 100 million records that reside across the various data stores. 

Challenge

El Toro’s technology leverages machine learning algorithms to constantly monitor, digest, and interpret data from hundreds of 1st, 2nd, and 3rd party data sources. Its data architecture includes:

• Amazon S3
• Apache Hive 
• Apache Cassandra 
• PostgreSQL 
• MongoDB 

Processing large amounts of complex data across these platforms was slow and time-consuming — El Toro relied on complex ETL pipelines to facilitate queries for ad-hoc analytics. After visualizing the data in PowerBI, the company had to reverse engineer queries to generate results for clients. 

The data engineering team evaluated other options for data access, such as Snowflake and Databricks. However, the alternatives did not support their technical requirements at the scale El Toro demanded, and would have had a price tag in the millions.   

Solution

The team deployed Starburst Enterprise to accelerate data access with features such as enterprise-grade performance, connectivity, security, cluster management, and support. It also offered these additional benefits:

• Deploying and managing Starburst Enterprise using Kubernetes removed the burden of more manual Trino deployments and cluster management. 
• Auto-scaling with graceful shutdown features enabled better cluster management. 
• 40+ supported connectors and high-performance parallel connectors unlocked El Toro’s entire data environment.
• Self-service BI allowed data consumers to join and access data in PowerBI. 

Results

Starburst Enterprise’s analytics engine helped El Toro run fast, live, interactive queries on its data lake. The company also found that it eliminates the need for the additional resources and ongoing technical support to deploy at scale. 

In addition, Starburst Enterprise delivers the following impacts to El Toro’s business:

• 300% increase in query performance. Starburst Enterprise offers enhancements on top of Trino, such as exclusive connectors, pushdown query processing, and dynamic filtering, that exponentially speeds up query run times.
• $5 million cost savings. Starburst Enterprise circumvented the need for El Toro to spend $5 million moving its data stack to a data warehouse.
• Decreased time-to-market. Quicker, easier access to data results in improved time-to-market for launching advertising campaigns. El Toro is now able to launch thousands of campaigns per day.
• Data products. Starburst Enterprise enables data producers to create and maintain an array of data products, including a client-facing data marketplace for downstream users to query live data. 

Starburst Enterprise also provides easy configuration in any environment, automated deployments anywhere, advanced monitoring, and 24×7 support by Trino experts, decreasing the workload for both platform administration and data engineering staff.  

Ultimately, Starburst Enterprise improves the analytics that enhance El Toro’s ability to visualize and extrapolate audiences based on human intent, ensuring decisions and dollars are allocated to the right audiences at the right time. Coming next, El Toro plans to create a recommendation engine based on all the customer data they collect — powered by Starburst.