Running MySQL on Kubernetes
A complete architecture guide
MySQL is one of the most widely deployed databases in the world. Running it on Kubernetes is no longer a question of whether, but of how, starting with which replication model to use.
1. Introduction: MySQL meets Kubernetes
MySQL has been in development since 1995 and runs a large part of the web. Running it on Kubernetes is no longer experimental. It is a production-proven approach for organizations that want portability, automation, and freedom from managed-service lock-in.
Is MySQL on Kubernetes production-ready?
Yes. The Percona Operator for MySQL based on Percona XtraDB Cluster has been generally available since 2019. Oracle ships an official operator for InnoDB Cluster, and Percona's Group Replication operator reached general availability in 2025. MySQL's own high-availability features, Group Replication and semi-synchronous replication, are mature and well understood.
Compared with VM-based deployments or a managed service, running MySQL on Kubernetes has three main advantages:
- It avoids vendor lock-in. The same cluster runs on AWS, GCP, Azure, or bare metal, so your data and configuration stay portable.
- Operations are automated. Operators handle failover, scaling, backups, and upgrades through Kubernetes resources instead of runbooks.
- It costs less. You can consolidate workloads, right-size resources, and avoid the premium of managed services like RDS or Cloud SQL.
The real decisions are which replication model, which storage, and which operator to use. Unlike PostgreSQL, where every operator builds on the same streaming replication, MySQL operators differ in the replication model itself, so that choice comes first.
2. MySQL on Kubernetes architectures
A MySQL deployment on Kubernetes has two layers: a routing layer that sends each connection to the right instance, and a database layer that replicates data between instances.
The routing layer (HAProxy, ProxySQL, MySQL Router)
Applications need to reach the current primary for writes, and ideally the replicas for reads, without knowing which pod is which. Most operators put a proxy in front of MySQL for this:
| Router | What it does | Used by |
|---|---|---|
| HAProxy | Fast TCP load balancing with health checks. Separate ports for writes and reads. | Both Percona operators |
| ProxySQL | MySQL-aware: query routing rules, read/write splitting, and connection multiplexing | Percona XtraDB Cluster operator |
| MySQL Router | Follows Group Replication membership and routes to the primary or replicas | Oracle operator, Percona Server operator |
| Kubernetes Services | One Service that always points to the primary, one for the replicas | MOCO |
Run at least two proxy pods, on different nodes from each other, so the routing layer has no single point of failure.
Asynchronous and semi-synchronous replication
This is MySQL's classic replication model. The primary writes changes to its binary log, and replicas copy and apply them. With GTIDs, every transaction has a global ID, which makes promoting a replica and repointing the others safe and automatic.
Asynchronous replication can lose the last transactions if the primary fails before replicas receive them. Semi-synchronous replication closes that gap: in its loss-less mode, the primary waits until at least one replica has received each transaction before it commits. The cost is one network round trip per commit. Semi-sync is fully supported in current MySQL versions. Only the old master and slave names of its plugins and settings were deprecated, in favour of source and replica.
Group Replication (InnoDB Cluster)
Group Replication is MySQL's built-in consensus replication. Before a transaction commits, a majority of the group must agree on it, so a committed transaction survives the loss of any minority of members. If the primary fails, the group elects a new one, and MySQL Router or a proxy follows it. InnoDB Cluster is Oracle's name for Group Replication managed with MySQL Shell and MySQL Router.
Single-primary mode is the standard. Group Replication supports up to nine members, and it has limits worth knowing: every table needs a primary key, and very large transactions are rejected above group_replication_transaction_size_limit, about 143 MB by default.
Galera (Percona XtraDB Cluster)
Galera, used by Percona XtraDB Cluster and MariaDB, replicates each transaction to every node at commit and certifies it there. Every node can accept writes, though in practice sending writes to one node avoids certification conflicts, and that is what the Percona operator's HAProxy does by default. There is no primary to elect: if a node fails, the others keep serving as long as they have quorum.
The trade-offs are flow control, which slows the whole cluster down to its slowest node, and limits on large transactions. Like Group Replication, Galera requires primary keys and only works with InnoDB.
| Semi-sync replication | Group Replication | Galera | |
|---|---|---|---|
| Writes | One primary | One primary (single-primary mode) | Any node, usually one in practice |
| Commit waits for | One replica to receive it | A majority of members | Certification on all nodes |
| Failover | Operator promotes a replica | Group elects a primary | No election needed |
| Restrictions | Fewest | Primary keys, transaction size limit | Primary keys, transaction size limit, flow control |
| Operators | MOCO | Oracle, Percona Server | Percona XtraDB Cluster |
Pod anti-affinity
A three-instance cluster needs at least three Kubernetes nodes, with each MySQL pod on a different one, so a node failure takes out at most one instance. Most operators set this for you. Check that it is required, not preferred, in production:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- topologyKey: kubernetes.io/hostname
labelSelector:
matchLabels:
app: mysqlMulti-AZ deployments
Spread the instances across three availability zones with topologySpreadConstraints, so the cluster keeps its quorum if a whole zone fails. With only two zones, the zone holding the majority takes the cluster down with it.
topologySpreadConstraints:
- maxSkew: 1
topologyKey: topology.kubernetes.io/zone
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: mysql3. Compute and storage strategy
Getting resources right is the difference between a solid database and one that the Linux OOM killer stops at 3 AM. For MySQL, the InnoDB buffer pool decides most of the memory picture.
Guaranteed QoS (requests equal limits)
For stateless apps, low requests and high limits are a standard way to pack more pods onto each node. For databases, this causes problems. If the memory limit is higher than the request, the Linux OOM killer targets the database pod first when the node runs low on memory.
For MySQL on Kubernetes, set requests equal to limits for both CPU and memory, in every container of the pod. This puts the pod in the Guaranteed QoS class, the last to be killed or evicted when a node is under pressure.
resources:
requests:
memory: 2Gi
cpu: 500m
limits:
memory: 8Gi
cpu: 4000mBurstable: killed before Guaranteed pods under memory pressure.
resources:
requests:
memory: 8Gi
cpu: 4000m
limits:
memory: 8Gi
cpu: 4000mGuaranteed: the last to be killed.
| QoS class | When a pod gets it |
|---|---|
| Guaranteed | Requests equal limits |
| Burstable | Requests lower than limits |
| BestEffort | No requests or limits set |
InnoDB buffer pool sizing
innodb_buffer_pool_size sets how much memory InnoDB uses to cache data and indexes. On a dedicated server it is often 70–80% of RAM. In a container, leave more headroom: about 50–75% of the memory limit, because per-connection buffers, temporary tables, and sidecars share the same limit, and going over it gets the pod OOM-killed.
Some operators derive the buffer pool from the container's memory limit if you don't set it. Check what yours does, and check it again whenever you change the limit.
Network-attached storage (EBS, Ceph)
The standard approach is to separate storage from compute with network-attached volumes such as AWS EBS, GCP Persistent Disk, or Ceph. When a node fails, the pod is rescheduled on another node and the same PersistentVolume is re-attached with all its data, so the instance only has to catch up on the changes it missed.
The trade-off is I/O latency, which is higher than local disk because every read and write goes over the network. For most MySQL workloads, network storage with provisioned IOPS is the right default.
Local NVMe SSDs
For latency-sensitive workloads, local NVMe SSDs give the best raw performance, with much lower I/O latency and higher throughput than network-attached storage. They also avoid paying for provisioned IOPS.
The trade-off is durability. If the node fails, the local volume is lost, and the replacement instance has to copy the full dataset from another member, with the clone plugin, XtraBackup, or a Galera state transfer. That takes time and loads the donor. Use local NVMe only with at least three instances and current backups.
| Network-attached (EBS) | Local NVMe SSD | |
|---|---|---|
| Latency | About 1 ms | About 0.1 ms |
| Throughput | Medium | High |
| Node failure | Data survives, and the volume re-attaches to the new pod right away | Data on the node is lost, and the instance needs a full copy |
| Cost | Provisioned IOPS add cost | Better cost per IOPS |
| Best for | General workloads and simplicity | Write-heavy, latency-sensitive workloads |
4. Monitoring MySQL on Kubernetes
Kubernetes changes how monitoring works. Pods come and go and their IPs change, so the monitoring stack has to discover targets dynamically. Prometheus with mysqld_exporter, or Percona Monitoring and Management (PMM), is the usual choice.
Prometheus, Grafana, and PMM
mysqld_exporter reads MySQL's status variables and Performance Schema and exposes them as Prometheus metrics. Operators usually run it as a sidecar next to each MySQL pod. Prometheus Operator then finds it through a ServiceMonitor or PodMonitor, so you never hardcode scrape targets.
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
spec:
selector:
matchLabels:
app: mysql
endpoints:
- port: metrics
path: /metricsWith the Percona operators, enabling PMM adds a client sidecar that ships metrics and query data to a PMM server, with ready-made MySQL dashboards and Query Analytics.
MySQL-specific metrics
Generic Kubernetes metrics such as CPU, memory, and network tell you the pod is alive. MySQL metrics tell you whether the database is healthy. Instrument these five first:
| Metric | Source | What to watch |
|---|---|---|
| Replication lag | Seconds_Behind_Source in SHOW REPLICA STATUS; the applier queue in replication_group_member_stats for Group Replication; wsrep_local_recv_queue and wsrep_flow_control_paused for Galera | Alert if lag or queues keep growing. For Galera, time spent in flow control means the cluster is waiting for its slowest node. |
| Buffer pool hit ratio | Innodb_buffer_pool_reads and Innodb_buffer_pool_read_requests | Should stay above 99%. Lower means the working set doesn't fit in memory. |
| Connections | Threads_connected vs max_connections, and Aborted_connects | Alert well before you reach the limit. Running out causes hard application errors. |
| Threads running | Threads_running | Queries executing right now. Spikes point to lock contention or saturation. |
| Slow queries | Slow_queries, with long_query_time | The trend matters more than the absolute number. Query Analytics shows which queries. |
Adapting monitoring for Kubernetes
- Use
ServiceMonitororPodMonitorand never hardcode Prometheus scrape targets. - Alert on replication health, not just pod restarts. A Running pod can have stopped replicating.
- For Group Replication, alert when a member leaves the ONLINE state or the group loses its majority.
- Alert on unexpected primary changes, which point to evictions, node pressure, or network problems.
- Track PVC capacity. Binary logs can fill a volume faster than you expect if they aren't purged.
- Monitor the proxy layer too: HAProxy backend health and ProxySQL connection pools.
5. Backup strategies
Backups are the last line of defence. In cloud-native environments, the standard approach is S3-compatible object storage combined with continuous binary log archiving, which lets you recover to any second in your retention window.
Physical and logical backups
| Tool | Type | Notes |
|---|---|---|
| Percona XtraBackup | Physical, hot | Open source. Copies InnoDB files while the database runs. Fast to back up and restore large databases. Used by both Percona operators. |
| MySQL Shell dump utilities | Logical | Parallel and much faster than mysqldump, but restores still rebuild every table and index. Used by the Oracle operator and MOCO. |
| MySQL Enterprise Backup | Physical, hot | Oracle's commercial tool, with incremental backups and point-in-time recovery in the Oracle operator. |
For databases larger than a few hundred gigabytes, physical backups make the difference between restoring in minutes and restoring in hours.
S3-compatible backup storage
The cloud-native standard for backup storage is any S3-compatible endpoint: AWS S3, Google Cloud Storage, Azure Blob, or self-hosted options such as MinIO. These give you durability, versioning, and lifecycle management.
Keep backups outside the Kubernetes cluster: in a separate account or region, or at the very least a separate namespace. If the cluster is compromised or destroyed, your backups survive.
Point-in-time recovery (binary logs)
A full backup is a snapshot of one moment. Real incidents often need a restore to an exact second, such as just before a DELETE without a WHERE clause. This is point-in-time recovery (PITR). It works by uploading binary logs to object storage continuously and replaying them on top of the last full backup, up to the chosen moment or GTID.
Keep binary logs on the server at least until they are uploaded, with binlog_expire_logs_seconds, and check that the upload process keeps up. A gap in the binary logs breaks recovery for everything after it.
DELETE.With the Percona XtraDB Cluster operator, binary logs are uploaded every 60 seconds by default, and a point-in-time restore is a Kubernetes resource:
apiVersion: pxc.percona.com/v1
kind: PerconaXtraDBClusterRestore
metadata:
name: before-bad-delete
spec:
pxcCluster: production
backupName: daily-backup-mon
pitr:
type: date
date: "2026-10-21 14:31:00"
backupSource:
storageName: s3-binlogsBackup architecture decisions
| Approach | Setup | Why |
|---|---|---|
| Recommended | Physical backups with continuous binary log archiving, stored in S3 in a separate cloud account or region | Fast restores, recovery to any second, and it survives the loss of the cluster. Test restores regularly, because an untested backup is not a backup. |
| Acceptable for smaller databases | Logical backups with MySQL Shell, plus binary logs | Simple and portable, but restores get slow as data grows. |
| Acceptable for dev and staging | S3-compatible storage such as MinIO, RustFS, or Ceph running in the same Kubernetes cluster | Convenient, but it shares the cluster's fate. If you lose the cluster, you lose the backups. |
| Not recommended | A mysqldump cron job writing to a local PVC | No point-in-time recovery, slow restores, and the same blast radius as the database. |
6. MySQL operators compared
The Kubernetes ecosystem has several MySQL operators, and they differ most in the replication model they use. For scores, a feature-by-feature deep dive, and a picker, see our MySQL operators comparison.
| Operator | Maintainer | License | Replication | Backups |
|---|---|---|---|---|
| Percona Operator for MySQL (XtraDB Cluster) | Percona | Apache 2.0 | Galera | XtraBackup, with PITR |
| Percona Operator for MySQL (Percona Server) | Percona | Apache 2.0 | Group Replication; async in tech preview | XtraBackup. PITR in tech preview |
| Oracle MySQL Operator | Oracle | UPL 1.0 | Group Replication (InnoDB Cluster) | MySQL Shell dumps. PITR needs Enterprise Backup |
| MOCO | Cybozu | Apache 2.0 | Semi-synchronous | MySQL Shell dumps, with PITR |
| Vitess Operator | PlanetScale | Apache 2.0 | Sharding on top of MySQL replication | Built into Vitess |
| KubeDB | AppsCode | Proprietary | Group Replication or semi-sync | KubeStash |
| OpenEverest | CNCF Sandbox project | Apache 2.0 | Galera or Group Replication, through the Percona operators | XtraBackup, with PITR. A control plane with a UI and API |
Percona Operator for MySQL based on Percona XtraDB Cluster
The longest-running production MySQL operator, generally available since 2019. It runs Galera-based Percona XtraDB Cluster with HAProxy or ProxySQL in front, backs up with XtraBackup, and has supported point-in-time recovery since 2021. PMM integration and cross-site replication are included.
Percona Operator for MySQL based on Percona Server
Percona's newer operator, built on upstream Group Replication with HAProxy or MySQL Router. It became generally available for Group Replication in November 2025. Asynchronous replication with Orchestrator, incremental backups, and point-in-time recovery are still tech previews.
Oracle MySQL Operator
The official operator for InnoDB Cluster, using Group Replication, MySQL Router, and MySQL Shell. The free version backs up with MySQL Shell dumps. Physical backups, incremental backups, and point-in-time recovery need MySQL Enterprise Backup, which is part of Oracle's commercial offering.
MOCO
Built by Cybozu around GTID-based, loss-less semi-synchronous replication, with no proxy layer and no Group Replication restrictions. It backs up with MySQL Shell dumps plus binary logs, including point-in-time recovery. A good fit if you want standard MySQL behaviour and a small operator.
Vitess
A sharding layer for MySQL that started at YouTube and graduated from the CNCF. Use it when one MySQL cluster can no longer hold your data or write load. It is not a replacement for the operators above when you just need highly available MySQL.
KubeDB
Part of a multi-database operator suite (PostgreSQL, MongoDB, Redis, and others) with one management plane. It is not open source, and enterprise features need a license.
The Bitpoke (formerly Presslabs) MySQL Operator, once popular for WordPress hosting, hasn't had a release since 2023. Avoid it for new deployments.
OpenEverest
OpenEverest is a CNCF Sandbox project for running databases on Kubernetes through one control plane. For MySQL it offers two providers, one for each Percona operator: Percona XtraDB Cluster for Galera, and Percona Server for Group Replication. It also has a MariaDB provider based on mariadb-operator. Either way you get high availability, backups to S3-compatible storage, and monitoring from the same UI and API.
OpenEverest is modular, so the operator underneath is pluggable. The same approach covers PostgreSQL, MongoDB, and a growing list of other databases. You learn one API and one way of operating, whichever operator or database runs underneath.
Related whitepapers
- The Cloud-Native Data Manual: storage, compute, cost, security, and availability for production databases on Kubernetes.
- Running PostgreSQL on Kubernetes: streaming replication, Patroni HA, PgBouncer, pgBackRest backups, and operator comparison.
- Running MongoDB on Kubernetes: replica sets, sharded clusters, WiredTiger and oplog sizing, PBM backups, and operator comparison.
- Running SQL Server on Kubernetes: availability groups without a Windows cluster, memory and licensing-aware sizing, native backups to S3, and operator comparison.
- Running Valkey on Kubernetes: replication and Valkey Cluster, the persistence trap, maxmemory and container limits, RDB and AOF, and operator comparison.
Run MySQL on Kubernetes with Solanica
Solanica builds on OpenEverest to give you high availability, point-in-time recovery, and monitoring for MySQL through one open source control plane, on infrastructure you own. Talk to us about your setup, or try OpenEverest yourself.