Infrastructure Prerequisites
Essential infrastructure setup for a successful Starburst Enterprise Kubernetes (SEP) installation
This document details the essential infrastructure requirements for a typical Starburst and Kubernetes deployment. It's crucial to ensure that all these prerequisites are in place before beginning the integration process.
General Requirements
The following technologies are mandatory for SEP to operate effectively:
| Technology | Notes |
|---|---|
| Python | Python 2.7+ required (3.6+ recommended; future updates will mandate 3.X) |
| Java Development Kit (JDK) | JDK version 17.0.3 or higher is required. Versions 18 and 19 are not officially supported and may not be fully compatible. |
| Helm | Version 3.2.4 or higher |
| kubectl | Essential for managing Kubernetes clusters |
| Internet Access | Necessary for accessing Helm charts |
Kubernetes Requirements
For optimal performance, ensure that all Kubernetes nodes are uniform in size and dedicated exclusively to SEP, not shared with other applications.
| Component | Notes |
|---|---|
| Kubernetes versions | Supported versions range from 1.24 to 1.27 |
| RAM | A minimum of 64 GB, up to 256 GB |
| Cores | Ranging from 16 to 64 cores |
| Processor Architecture | Compatible with both x86_64 and ARM (Graviton) architectures |
Regularly tested and supported platforms include:
- Amazon Elastic Kubernetes Service (EKS)
- Google Kubernetes Engine (GKE)
- Microsoft Azure Kubernetes Service (AKS)
- Red Hat OpenShift
- Rancher RKE2
Databases
Starburst may require multiple databases depending on the chosen architecture. The backend service is vital for SEP functionality.
Note: The backend and caching services can utilize the same database type but must be separate, either as different schemas in PostgreSQL or Oracle, or as distinct databases in MySQL.
| Component | Compatible Databases |
|---|---|
| Backend Services | MySQL 8.0.12+, PostgreSQL 12.0+, OracleDB 12.2.0.1+ |
| Internal Hive Meta Store | PostgreSQL 12.0+ |
| External Hive Meta Store | PostgreSQL, MySQL, Oracle, Microsoft SQL Server |
| Caching Service | MySQL 8.0.12+, PostgreSQL 9.6+, OracleDB 12.2.0.1+ |
Backend Services
The SEP backend service is crucial for managing and storing critical information such as query completion details, cluster metrics, and data products. It facilitates role-based access control, managed statistics, and an optional caching service.
Supported authentication methods for each RDBMS type include:
| RDBMS | Authentication method |
|---|---|
| MySQL | Basic, AWS IAM |
| OracleDB | Basic, Kerberos |
| PostgreSQL | Basic |
Recommended hardware specifications for the RDBMS server:
- 64GB of RAM
- 8 CPU cores
- Minimum of 5GB storage
Helm Chart Repository
Required Helm charts and Docker images are accessible through the Starburst Harbor repository at https://harbor.starburstdata.net. Ensure you have access to this repository; if not, contact Starburst customer support for assistance.
Load Balancers
Load balancers are typically utilized for setting up TLS/HTTPS in SEP integrations. For more information, refer to Establishing TLS/HTTPS.