Skip to main content

WeSQL Architecture Overview

alt text

WeSQL adopts a compute-storage separation architecture.

Compute Layer​

The compute layer consists of two components: WeScale and WeSQL-Server.

  • WeSQL-Server runs the unmodified MySQL Server code, along with a Raft replication plugin and our cloud-native storage engine, SmartEngine. It handles transaction execution, persists data to S3, loads data from S3 when needed, and synchronizes binlogs across nodes using the Raft algorithm. Multiple WeSQL-Server nodes form a Raft group, which can be further divided into two roles: Data Node and Logger Nodes. A typical WeSQL-Server Raft group consists of one Data Node and two Logger Nodes, though higher availability can be achieved with configurations like one Data Node and four Logger Nodes. The transaction is committed only after the binlog has been successfully written to the EBS of a majority of the nodes.

    • The Data Node has both logs and data. They handle transactions and serve SQL queries. Under normal conditions, the Data Node uploads both updated data and logs to S3.
    • Logger Nodes only store logs. In the event of the Data Node failure, Logger Nodes ensure the last few committed logs are written to S3, maintaining data integrity. Because Logger Nodes only handle logs, they require significantly fewer resources than Data Nodes. For instance, you can allocate just 1 vCPU or less to a Logger Node.

  • WeScale is a stateless proxy node that provides access via the MySQL protocol. It handles request routing and offers extended SQL capabilities like Connection Pooling, Online DDL, Branching, Filters, and Transaction Chopping. WeScale is stateless, and you can deploy one or more WeScale nodes. For example, you can deploy a WeScale node in each availability zone (AZ). This way, each AZ connects to its nearest WeScale node, which then routes the requests to the Data Node (which may reside across AZs).

note

WeScale is an optional component. In a deployment without WeScale, applications can connect directly to the WeSQL Data Node to access the MySQL database.

In which situation you need WeScale?​

WeScale is designed to save application developers from having to write additional logic to handle complex database scenarios, making the business code simpler and more streamlined. You would typically use WeScale in the following scenarios:

  1. Serverless Architecture

    WeScale is central to WeSQL's serverless architecture. Applications connect to WeScale and maintain connections. If there is no traffic for an extended period, WeScale can shut down all backend WeSQL-Server nodes, conserving resources and enabling Auto-Suspend. When new requests arrive, WeScale instantly starts a group of WeSQL-Server nodes, connects to the Data Nodes, and forwards the requests accordingly. Additionally, WeScale automatically adjusts the CPU and memory resources of WeSQL-Server nodes based on traffic load, providing Auto-Scale functionality.

  2. Transparent High Availability Failover

    WeScale includes a connection pooling feature that keeps long lived TCP connections with your application, seamlessly handling backend failures. For instance, if the Data Node fails and is replaced by a new node (potentially with a different IP), WeScale can automatically switch to the new node. Any SQL queries in progress will be retried without requiring the application to handle exceptions or reconnect. This makes failover nearly transparent to the application, minimizing downtime and complexity.

  3. Built-in Connection Pooling

    WeScale can manage a connection pool for your application, reducing the number of connections to backend WeSQL-Server nodes. This is particularly useful for applications that frequently open and close connections, as it reduces the overhead of establishing new connections and improves overall performance.

  4. Developer-Friendly SQL Features

    WeScale offers advanced SQL capabilities that go beyond vanilla MySQL, such as Online DDL, Branching, Filters, and Transaction Chopping. This allows developers to offload complex database operations from the application layer to the database itself, simplifying application logic and improving coding efficiency. WeScale is particularly beneficial for applications that aim to write code with elegance and agility while avoiding the complexity of handling sophisticated database operations.

Storage Layer​

WeSQL uses S3 as its storage layer. Why S3 explains why we chose S3 over EBS or a dedicated storage system.