Skip to main content

Run Three WeSQL-Server Nodes in EKS

This document describes how to deploy a WeSQL-Server cluster (including one data node and two logger nodes) on Amazon EKS.

Prerequisites​

Before we get started, let’s get a few prerequisites out of the way:

  1. Prepare an AWS S3 bucket for WeSQL backups and logging. You can apply a S3 Bucket for testing purpose.
  2. Install kubectl and ensure it is in your PATH.
  3. Install the MySQL client locally.
  4. Prepare a Amazon EKS Kubernetes cluster.

Deploying the Cluster​

1. Create the Configuration Files​

Use kubectl to create a ConfigMap with your WeSQL-Server configuration.

You need to replace the objectstore_provider, objectstore_region, and objectstore_bucket with your AWS S3 bucket info.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: wesql-server-config
data:
  MYSQL_CUSTOM_CONFIG: |
    [mysqld]
    objectstore_provider=aws
    objectstore_region=us-west-1
    objectstore_bucket=wesql-storage
    repo_objectstore_id=sysbench
    branch_objectstore_id=main
    log-bin=binlog
    gtid_mode=ON
    enforce_gtid_consistency=ON
    log_slave_updates=ON
    binlog_format=ROW
    skip_name_resolve=ON
EOF
note

Since we need to expose MySQL's service address through AWS Load Balancer (NLB), and the NLB will periodically perform health checks on the port, we need to enable skip_name_resolve to avoid errors like this:

ERROR 1129 (HY000): Host '192.168.47.47' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'

2. Create the Secret​

Use kubectl to create a secret with your AWS S3 credentials. Replace YOUR_S3_ACCESS_KEY, YOUR_S3_SECRET_KEY, and YOUR_MYSQL_ROOT_PASSWORD with your actual AWS S3 credentials and desired MySQL root password.

kubectl create secret generic wesql-server-secret \
  --namespace default \
  --type Opaque \
  --from-literal=WESQL_OBJECTSTORE_ACCESS_KEY=${YOUR_S3_ACCESS_KEY} \
  --from-literal=WESQL_OBJECTSTORE_SECRET_KEY=${YOUR_S3_SECRET_KEY} \
  --from-literal=MYSQL_ROOT_PASSWORD=${YOUR_MYSQL_ROOT_PASSWORD}

3. Prepare a Kubernetes Cluster​

Make sure you have a Kubernetes cluster running. You can check the status of the cluster by running the following command:

kubectl cluster-info

Example Output (Amazon EKS):

Kubernetes control plane is running at https://<eks-endpoint>.eks.amazonaws.com
CoreDNS is running at https://<eks-endpoint>.eks.amazonaws.com/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

4. Deploy the WeSQL-Server Cluster​

Deploy the WeSQL-Server cluster by running:

kubectl apply -f https://raw.githubusercontent.com/wesql/deploy/refs/heads/main/artifact/wesql-server.yaml

Check the status of the pods to ensure the WeSQL-Server pods are running:

kubectl get pods

Example Output:

NAME                           READY   STATUS    RESTARTS   AGE
mycluster-wesql-0-0            1/1     Running   0          17s
mycluster-wesql-1-0            1/1     Running   0          17s
mycluster-wesql-2-0            1/1     Running   0          17s

5. Set Up Port Forwarding (Optional)​

If you prefer to access the cluster locally, forward the MySQL port to your machine:

kubectl port-forward pod/mycluster-wesql-0-0 3306:3306
mysql -h127.0.0.1 -uroot -p${YOUR_MYSQL_ROOT_PASSWORD}

Accessing the Cluster via the Public Network​

To access your WeSQL-Server cluster from the internet, you need to expose the service externally by creating a public LoadBalancer. This allows you to connect to your database from outside the Kubernetes cluster.

1. Deploy the Public Service​

Apply the public service configuration using kubectl:

kubectl apply -f https://raw.githubusercontent.com/wesql/deploy/refs/heads/main/public-service/wesql-server-public-service.yaml
Content of wesql-server-public-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: mycluster-wesql-server-public-service
  namespace: default
  labels:
    app.kubernetes.io/component: wesql
    app.kubernetes.io/instance: mycluster
spec:
  type: LoadBalancer
  ports:
    - name: wesql
      port: 3306
      protocol: TCP
      targetPort: wesql
  selector:
    app.kubernetes.io/component: wesql
    app.kubernetes.io/instance: mycluster

2. Verify the LoadBalancer Creation​

After applying the public service, Kubernetes will request your cloud provider (AWS) to provision a LoadBalancer. This process may take a few minutes.

Check the status of the service to get the external IP address or hostname:

kubectl get svc mycluster-wesql-server-public-service

Example Output:

NAME                            TYPE           CLUSTER-IP       EXTERNAL-IP                                                              PORT(S)          AGE
mycluster-wesql-public-service  LoadBalancer   10.100.248.146   a1b2c3d4e5f6.us-west-1.elb.amazonaws.com   3306:30900/TCP   2m

Note: The EXTERNAL-IP field shows the DNS name of the LoadBalancer. It may take a few minutes for the EXTERNAL-IP to be assigned.

3. Configure AWS Resources (If Necessary)​

While Kubernetes automates the creation of the LoadBalancer, ensure the following AWS configurations are in place:

  • IAM Permissions: The AWS IAM role associated with your EKS worker nodes must have permissions to create and manage Elastic Load Balancers (ELBs). The necessary permissions include:

    • elasticloadbalancing:*
    • ec2:*
    • iam:ListServerCertificates

  • VPC and Subnets:

    • The EKS cluster should be deployed in a VPC with public subnets to allow internet access.

    • Public subnets should be tagged appropriately:

      kubernetes.io/role/elb: 1

  • Security Groups:

    • Ensure the security group associated with the LoadBalancer allows inbound traffic on port 3306 from your client IP address or the desired IP range.

4. Wait for DNS Propagation​

The DNS name provided in the EXTERNAL-IP may require some time to propagate. You can verify when it's ready by checking DNS resolution:

nslookup a1b2c3d4e5f6.us-west-1.elb.amazonaws.com

Once the DNS name resolves to an IP address, you can proceed to connect.

5. Connect to the WeSQL-Server Cluster​

Use the MySQL client to connect to your database using the LoadBalancer's DNS name:

mysql -h a1b2c3d4e5f6.us-west-1.elb.amazonaws.com -P 3306 -uroot -p${YOUR_MYSQL_ROOT_PASSWORD}

Important: Replace a1b2c3d4e5f6.us-west-1.elb.amazonaws.com with the actual EXTERNAL-IP DNS name from your service.

Troubleshooting Tips
- **Connection Errors:**

  • If you receive an error like Host 'X.X.X.X' is not allowed to connect to this MySQL server, ensure your MySQL user is configured to allow connections from your client IP address.

  • Firewall and Security Groups:

    • Double-check that your AWS security groups allow inbound traffic on port 3306 from your IP address.

  • DNS Resolution Issues:

    • If the DNS name isn't resolving, try flushing your DNS cache or wait a few more minutes for propagation.

  • LoadBalancer Status:

    • Verify the LoadBalancer is in the Active state in the AWS console under EC2 > Load Balancers.
Security Considerations
Exposing your database to the internet introduces security risks. Take the following precautions:

  • Restrict Access:

    • Limit inbound traffic to specific IP addresses using security group rules.

  • Use Secure Passwords:

    • Ensure your MySQL root password is strong and not easily guessable.

  • Enable SSL/TLS:

    • Configure MySQL to use SSL/TLS encryption for connections.

  • Monitor Access Logs:

    • Regularly check logs for unauthorized access attempts.

  • Consider VPN or Bastion Hosts:

    • For enhanced security, access the database through a VPN connection or via a bastion host instead of exposing it directly to the internet.

Delete the Cluster​

Delete the WeSQL-Server cluster by running:

kubectl delete -f https://raw.githubusercontent.com/wesql/deploy/refs/heads/main/artifact/wesql-server.yaml

Delete the Load Balancer service (if created):

kubectl delete -f https://raw.githubusercontent.com/wesql/deploy/refs/heads/main/public-service/wesql-server-public-service.yaml

Delete the ConfigMap and Secret:

kubectl delete configmap wesql-server-config
kubectl delete secret wesql-server-secret

Additionally, you can wipe out the data by deleting PersistentVolumeClaims (PVCs) and S3 bucket objects:

kubectl get pvc
kubectl delete pvc data-mycluster-wesql-0-0 data-mycluster-wesql-1-0 data-mycluster-wesql-2-0

Next Steps​

If you want to try out the WeScale feature, you can follow the tutorial: Deploying WeScale in Kubernetes.

By following these steps, you can deploy a WeSQL-Server cluster in Kubernetes and make it accessible over the internet. Always ensure you follow best practices for security and resource management when exposing services publicly.