Agile MySQL with S3-Based Storage
Improving the popular MySQL to be more developer-friendly and easier to manage, streamlining your app lifecycle.
Introducing WeSQL
MySQL with innovative compute-storage separation architecture
All Data on S3: Storage for the Cloud-Native Era.
- Unlimited data capacity with ultra-high I/O bandwidth.
- 11 nines durability: No risk of data loss
- Region-level disaster recovery: Resilient to AZ failures
80x Reduction in Storage Costs for Data Archiving
- Columnar format is 10x more space-efficient than InnoDB.
- S3 offers an additional 7 to 10 times cost savings compared to EBS.
Accelerate Application Development
- Streamline DevOps with Branching：Create isolated environments for production, staging, and development.
- Simplified Code: Built-in Connection Pool, Online DDL and Large Transaction Handling.
- Extensible via Wasm: Intercept and manipulate SQL executions like API gateway Filters.
Serverless and Pay Only for Resources Used
- S3 is billed based on actual usage, with no need for pre-provisioning storage capacity.
- Support auto-suspend and auto-scale with K8s, aligning compute resources with workload demands.
Launch Replicas Anywhere
- Launch a local read-only replica by checking out data from S3 at any location with S3 access.
- No network communication is required between replicas; data is synced via S3.
Ease Operational Burden
- Just run enough WeSQL containers connected to S3 — no need for mysqldump or xtrabackup.
- Even if all containers or underlying EC2 instances (including EBS) are lost, your data will remain safe in S3.
Key Technologies
Afraid of a performance drop? Don’t worry, WeSQL on S3 has matched the performance of InnoDB on EBS, and we are continuing to optimize further.
Low Latency Commit: Avoid the high write latency of direct log writes to S3 by using Raft replication.
Write Optimized Engine: Leverage the append-only LSM storage engine, naturally optimized for S3, where both data and metadata are stored as S3 objects.
Local Cache: Three-tier (Row, Block, Disk) caching system optimizes read performance.
Acknowledgments
WeSQL is built on the foundations of great open-source projects and ideas.This approach helps us avoid reinventing the wheel and reach our goals faster.
- MySQL server codebase is used without modification, ensuring full MySQL compatibility.
- The LSM storage engine is based on the MyRocks codebase and its variant, X-Engine.
- Inspired by Neon, WeSQL stores OLTP database data directly on S3.
- The metadata management on S3 are similar to Data Lake.
- WeScale is built on Vitess, developed in Golang, and offers agile features like Branching.
WeSQL is Continuously Evolving
RoadMap includes
Instant Branch
- A new branch is cloned instantly from shared S3 data, with subsequent writes diverging via Copy-On-Write.
Edge Mode
- Data is written locally when disconnected from S3 and synced once the connection is restored.
Real-time Replicas
- Replicas sync incremental data via S3 without the need for direct network communication, such as within a VPC.
Shared-Data Replicas
- Support multiple data nodes sharing the same data on S3, with one writer and multiple readers, similar to Aurora.
