I wanted to ask why Starburst Enterprise Platform currently does not include a built-in backup and restore process as part of its feature set. From an enterprise perspective, having this capability integrated would greatly ease operational management and disaster recovery.
At the moment, we are managing point-in-time recovery (PITR) backups for our Hive metastore and catalogs. While this works, it adds complexity and operational overhead. An integrated backup solution within SEP would simplify these processes and improve reliability.
Is there any roadmap or recommended best practice around this? I’m also interested to know how it is being done for starburst galaxy because as it is SAS product in case of failure how this is being handled .Would love to hear others’ thoughts or if there are any third-party tools that are commonly used alongside SEP for streamlined backups.
I think for the same reasons lots of open-source big-data frameworks (and the vendors that support them) do not do this for the deploy-anywhere software model. I worked at Hortonworks/Cloudera for almost a decade and for the non-SaaS deployment models this was always up to a platform architecture team to take care of.
Why?
Because the system could be deployed almost anywhere, for any scale, and for any purpose. Surely don’t need this for a little single-node POC and maybe don’t even want for your integration testing env.
Because there are plenty of options for the underlying database system in use. Don’t want to tackle that for all possibilities.
There are already best of breed tools and approaches to ensuring HA/DR solutions for the systems housing the control databases. Good separation of concerns to NOT include stuff like this in the software stack.