Skip to main content

Parameters and Settings

Enable/Disable the serverless mode​

serverless​

WeSQL, by default, uses object storage as its persistent storage medium. This includes persisting the binlog to object storage, creating snapshots and saving them to object storage, and persisting SmartEngine extent data to object storage. However, to support environments where object storage is unavailable, or to compare the performance between object storage and local disk, WeSQL also supports using local disk as the persistent storage medium. In such cases, serverless mode must be disabled.

  • Command-Line Format: --serverless=[={true|false}]
  • System Variable: serverless
  • Scope: Global
  • Dynamic: NO
  • Type: Boolean
  • Default Value: true

The serverless mode can be disabled either via the server command option:

mysqld --serverless=false

or by modifying the option file (my.cnf):

serverless=false

Object Storage Settings​

WeSQL persists data to object storage. This section covers the configuration parameters related to object storage.

objectstore_provider​

The object storage provider setting. This parameter is Required.

  • Command-Line Format: --objectstore-provider=name
  • System Variable: objectstore_provider
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: local
  • Valid Values: local | aws | aliyun | minio
note

The local option simulates object storage using the local file system and is intended for testing and development purposes only.

When objectstore_provider is set to local, the serverless mode must be enabled. This behavior is different from disabling serverless mode, even though in both configurations the local disk is used.

Examples:

objectstore_provider=local # for Local File System

objectstore_provider=aws # for AWS S3

objectstore_provider=aliyun # for Aliyun OSS

objectstore_provider=minio # for Minio

objectstore_region​

The region where the object storage is located. This parameter is Required.

  • Command-Line Format: --objectstore-region=name
  • System Variable: objectstore_region
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: .local_objectstore_region

Examples:

objectstore_region=us-west-1 # for AWS S3

objectstore_region=cn-hangzhou # for Aliyun OSS

objectstore_region= # for Minio, region is not used by default.

note

When objectstore_provider is set to local, you can just use the default value and don't need to set it.

objectstore_endpoint​

The endpoint of the object storage. This allows access to the object storage via the specified endpoint. This parameter is Required if objectstore_provider is set to aliyun or minio.

  • Command-Line Format: --objectstore-endpoint=name
  • System Variable: objectstore_endpoint
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: empty string

Examples:

objectstore_endpoint=oss-cn-hangzhou-internal.aliyuncs.com # for Aliyun OSS

objectstore_endpoint=http://127.0.0.1:9000 # for Minio

note

When objectstore_provider is set to local or aws, you can just use the default value and don't need to set it.

objectstore_use_https​

Specifies whether to use the HTTPS protocol for accessing object storage. The default value is false.

  • Command-Line Format: --objectstore-use-https=[={true|false}]
  • System Variable: objectstore_use_https
  • Scope: Global
  • Dynamic: NO
  • Type: Boolean
  • Default Value: FALSE

Example:

objectstore_use_https=true

objectstore_bucket​

The bucket in the object storage where all persistent data of the WeSQL instance will be stored. Each bucket can only be used by one instance, and this parameter is Required.

  • Command-Line Format: --objectstore-bucket=name
  • System Variable: objectstore_bucket
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: objectstore_bucket

Example:

objectstore_bucket=wesql-storage

repo_objectstore_id​

The repository identifier under the object storage bucket objectstore_bucket that stores all persistent data of the WeSQL database.You can use meaningful, application-related names as repo_objectstore_id, but it must be unique under the objectstore_bucket.

  • Command-Line Format: --repo-objectstore-id=name
  • System Variable: repo_objectstore_id
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: wesql_serverless_repo

Example:

repo_objectstore_id=sysbench

branch_objectstore_id​

The Branch identifier under the object storage repo_objectstore_id that stores persistent data of the WeSQL Branch cluster. WeSQL allows you to create a copy of the current WeSQL cluster, forming a Branch cluster similar to GitHub branches. The branch_objectstore_id serves as the data path for the Branch in object storage, with the default set to main. All Branch data is located under the repo_objectstore_id.

  • Command-Line Format: --branch-objectstore-id=name
  • System Variable: branch_objectstore_id
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: main

Example:

branch_objectstore_id=main

initialize_from_objectstore​

Initialzie instance data from source object storage. Only valid when initialize a WeSQL with the --initializeor --initialize-insecure option.

  • Command-Line Format: --initialize-from-objectstore=[={true|false}]
  • System Variable: initialize_from_objectstore
  • Scope: Global
  • Dynamic: NO
  • Type: Boolean
  • Default Value: FALSE

Example:

initialize_from_objectstore=true

initialize_objectstore_provider​

The provider of source object storage during initialzie instance, when initialize_from_objectstore is 'true'.

note

Configure access permissions for source object storage using the environment variables SOURCE_ACCESS_KEY_ID and SOURCE_SECRET_ACCESS_KEY. In Docker mode, set source object storage access permissions using WESQL_SOURCE_OBJECTSTORE_ACCESS_KEY and WESQL_SOURCE_OBJECTSTORE_SECRET_KEY.

  • Command-Line Format: --initialize-objectstore-provider=name
  • System Variable: initialize_objectstore_provider
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: local
  • Valid Values: local | aws | aliyun | minio

More details description about valid values, refer to objectstore_provider.

Example:

initialize_objectstore_provider=aws

initialize_objectstore_region​

The region where the source object storage is located, when initialize_from_objectstore is 'true'.

  • Command-Line Format: --initialize-objectstore-region=name
  • System Variable: initialize_objectstore_region
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: .local_objectstore_region_1

Example:

initialize_objectstore_region=true

initialize_objectstore_endpoint​

The endpoint of the source object storage, when initialize_from_objectstore is 'true'. This allows access to the source object storage via the specified endpoint. This parameter is Required if initialize_objectstore_provider is set to aliyun or minio.

  • Command-Line Format: --initialize-objectstore-endpoint=name
  • System Variable: initialize_objectstore_endpoint
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: empty string

Example:

initialize_objectstore_endpoint=oss-cn-hangzhou-internal.aliyuncs.com

initialize_objectstore_use_https​

Specifies whether to use the HTTPS protocol for accessing source object storage, when initialize_from_objectstore is 'true'.

  • Command-Line Format: --initialize-objectstore-use-https=[={true|false}]
  • System Variable: initialize_objectstore_use_https
  • Scope: Global
  • Dynamic: NO
  • Type: Boolean
  • Default Value: FALSE

Example:

initialize_objectstore_use_https=true

initialize_objectstore_bucket​

The bucket identifier in the source object storage, when initialize_from_objectstore is 'true'.

  • Command-Line Format: --initialize-objectstore-bucket=name
  • System Variable: initialize_objectstore_bucket
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: objectstore_bucket_1

Example:

initialize_objectstore_bucket=wesql-clone

initialize_repo_objectstore_id​

The repository identifier under the source object storage bucket initialize_objectstore_bucket, when initialize_from_objectstore is 'true'.

  • Command-Line Format: --initialize-repo-objectstore-id=name
  • System Variable: initialize_repo_objectstore_id
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: wesql_serverless_repo

Example:

initialize_repo_objectstore_id=sysbench

initialize_branch_objectstore_id​

The branch identifier under the source object storage initialize_repo_objectstore_id, when initialize_from_objectstore is 'true'.

  • Command-Line Format: --initialize-branch-objectstore-id=name
  • System Variable: initialize_branch_objectstore_id
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: main

Example:

initialize_branch_objectstore_id=main

initialize_smartengine_objectstore_data​

Specifies whether to also initialize smartengine extent data from the source object storage.

  • Command-Line Format: --initialize-smartengine-objectstore-data=[={true|false}]
  • System Variable: initialize_smartengine_objectstore_data
  • Scope: Global
  • Dynamic: NO
  • Type: Boolean
  • Default Value: FALSE

Example:

initialize_branch_objectstore_id=true

Snapshot Settings​

snapshot_archive​

Indicates whether to enable snapshots archive persistents, default is true. Snapshots will be periodically generated and persisted to object storage. Snapshots archive should be disabled for Logger nodes.

  • Command-Line Format: --snapshot-archive=[={true|false}]
  • System Variable: snapshot_archive
  • Scope: Global
  • Dynamic: Yes
  • Type: Boolean
  • Default Value: TRUE

Example:

snapshot_archive=true

Allow globally dynamic modification, not persistent. Used to temporarily disable snapshot generation.

Example:

set global snapshot_archive = false;

snapshot_archive_dir​

The local temporary archive path for snapshot objects. This directory must be created before starting the instance, and it must not be a subdirectory of the datadir. The default value is null, which will automatically use MySQL's tmp directory (--tmpdir) as the archive_dir.

  • Command-Line Format: --snapshot-archive-dir=dir_name
  • System Variable: snapshot_archive_dir
  • Scope: Global
  • Type: Directory name
  • Default Value: empty string

Example:

snapshot_archive_dir=/u01/wesql_archive_dir

snapshot_archive_period​

The frequency, in seconds, at which a snapshot is generated. The default value is 300 seconds (5 minutes). A snapshot is created at every cycle. Additionally, a snapshot is generated after the system starts up and before shuts down gracefully.

  • Command-Line Format: --snapshot-archive-period=#
  • System Variable: snapshot_archive_period
  • Scope: Global
  • Dynamic: YES
  • Persist: YES
  • Type: Integer
  • Default Value: 300
  • Minimum Value: 2
  • Maximum Value: 31536000
  • Unit: Seconds

Example:

snapshot_archive_period=1800

Example:

SET GLOBAL snapshot_archive_period = 600;
SET PERSIST snapshot_archive_period = 600;

snapshot_archive_expire_auto_purge​

Indicates whether to enable the automatic cleanup of expired snapshots stored in object storage. Snapshots will be cleaned up when the creation time plus 'snapshot_archive_expire_seconds' exceeds the current time.

  • Command-Line Format: --snapshot-archive-expire-auto-purge=[={true|false}]
  • System Variable: snapshot_archive_expire_auto_purge
  • Scope: Global
  • Dynamic: Yes
  • Persist: YES
  • Type: Boolean
  • Default Value: TRUE

Example:

snapshot_archive_expire_auto_purge=true

snapshot_archive_expire_seconds​

The expiration period, in seconds, for snapshots stored in object storage. The default value is 0, meaning only the latest snapshot is retained, and older snapshots are automatically purged. If you want to retain historical snapshots, you can set a specific period in seconds to keep them for a defined time. This setting can be used for Point-in-Time Recovery (PITR).

  • Command-Line Format: --snapshot-archive-expire-seconds=#
  • System Variable: snapshot_archive_expire_seconds
  • Scope: Global
  • Dynamic: YES
  • Persist: Yes
  • Type: Integer
  • Default Value: 0
  • Minimum Value: 0
  • Maximum Value: 4294967295
  • Unit: Seconds

Example:

snapshot_archive_expire_seconds=2592000

snapshot_archive_innodb_tar_mode​

In what format is the InnoDB data of a snapshot persisted to object storage.

'TAR' indicates that InnoDB data is first packaged using tar locally before being persisted. 'TAR_AND_COMPRESS' indicates that InnoDB data is first compressed and then packaged using tar before being persisted. 'OFF' indicates that InnoDB data is directly persisted by directory. The defaut value is 'OFF'.

  • Command-Line Format: --snapshot-archive-innodb-tar-mode=name
  • System Variable: snapshot_archive_innodb_tar_mode
  • Scope: Global
  • Dynamic: YES
  • Persist: Yes
  • Type: String
  • Default Value: OFF
  • Valid Values: TAR | TAR_AND_COMPRESS | OFF

Example:

snapshot_archive_innodb_tar_mode=TAR

Allow globally dynamic modification, not persistent.

snapshot_archive_smartengine_tar_mode​

In what format is the SmartEngine data of a snapshot persisted to object storage.

'TAR' indicates that SmartEngine data is first packaged using tar locally before being persisted. 'TAR_AND_COMPRESS' indicates that SmartEngine data is first compressed and then packaged using tar before being persisted. 'OFF' indicates that SmartEngine data is directly persisted by directory. The defaut value is 'OFF'.

  • Command-Line Format: --snapshot-archive-smartengine-tar-mode=name
  • System Variable: snapshot_archive_smartengine_tar_mode
  • Scope: Global
  • Dynamic: YES
  • Persist: Yes
  • Type: String
  • Default Value: OFF
  • Valid Values: TAR | TAR_AND_COMPRESS | OFF

Example:

snapshot_archive_smartengine_tar_mode=TAR

Allow globally dynamic modification, not persistent.

Binlog Archive Settings​

binlog_archive​

Indicates whether to enable binlog persistent, default is true. Binlog will be periodically persisted to object storage.

note

Configure access permissions for object storage using the environment variables ACCESS_KEY_ID and SECRET_ACCESS_KEY. In Docker mode, set object storage access permissions using WESQL_OBJECTSTORE_ACCESS_KEY and WESQL_OBJECTSTORE_SECRET_KEY.

  • Command-Line Format: --binlog-archive=[={true|false}]
  • System Variable: binlog_archive
  • Scope: Global
  • Type: Boolean
  • Default Value: TRUE

Example:

binlog_archive=false

binlog_archive_dir​

The local temporary archive path for binlog objects. This directory must be created before starting the instance, and it must not be a subdirectory of the datadir. The default value is null, which will automatically use MySQL's tmp directory (--tmpdir) as the archive_dir.

  • Command-Line Format: --binlog-archive-dir=dir_name
  • System Variable: binlog_archive_dir
  • Scope: Global
  • Type: Directory name
  • Default Value: empty string

Example:

binlog_archive_dir=/u01/wesql_archive_dir

binlog_archive_slice_max_size​

The binlog object size when uploaded to object storage, with a default value of 4MB. During the binlog persistence process, events are batched and written to the local cache. Once the size limit defined by this parameter is reached, these events are packaged and uploaded to object storage as a binlog slice object.

  • Command-Line Format: --binlog-archive-slice-max-size=#
  • System Variable: binlog_archive_slice_max_size
  • Scope: Global
  • Dynamic: YES
  • Persist: YES
  • Type: Integer
  • Default Value: 4194304
  • Minimum Value: 4096
  • Maximum Value: 1073741824
  • Unit: Bytes

Example:

binlog_archive_slice_max_size=1048576

For example, the binlog file binlog.000001 would have the following slice objects:

binlog.000001.00000000000000000002.0004048843
binlog.000001.00000000000000000002.0009345154
binlog.000001.00000000000000000002.0010137258

binlog_archive_period​

The minimum time period for binlog persistence, during which binlog events must be persisted even if the locally cached binlog events have not reached 'binlog_archive_slice_max_size'.The goal is to achieve near real-time persistence, even if the binlog is persisted to object storage in batches.

  • Command-Line Format: --binlog-archive-period=#
  • System Variable: binlog_archive_period
  • Scope: Global
  • Dynamic: YES
  • Persist: YES
  • Type: Integer
  • Default Value: 1000
  • Minimum Value: 10
  • Maximum Value: ULONG_MAX
  • Unit: Milliseconds

Example:

binlog_archive_period=3000

binlog_archive_expire_auto_purge​

Indicates whether to enable the automatic cleanup of expired binlog objects stored in object storage. The default value is true. Binlog objects will be cleaned up when the creation time plus 'binlog_archive_expire_seconds' exceeds the current time.

  • Command-Line Format: --binlog-archive-expire-auto-purge=[={true|false}]
  • System Variable: binlog_archive_expire_auto_purge
  • Scope: Global
  • Dynamic: Yes
  • Persist: YES
  • Type: Boolean
  • Default Value: TRUE

Example:

binlog_archive_expire_auto_purge=true

binlog_archive_expire_seconds​

The expiration period, in seconds, for binlog objects stored in object storage. The default value is 2592000 seconds (30 days). The value of binlog_archive_expire_seconds should be greater than snapshot_archive_expire_seconds. Otherwise, even if a binlog is marked as expired, it cannot be purged as long as it is still referenced by a snapshot.

  • Command-Line Format: --binlog-archive-expire-seconds=#
  • System Variable: binlog_archive_expire_seconds
  • Scope: Global
  • Dynamic: YES
  • Persist: Yes
  • Type: Integer
  • Default Value: 2592000
  • Minimum Value: 0
  • Maximum Value: 4294967295
  • Unit: Seconds

Example:

binlog_archive_expire_seconds=2592000

binlog_archive_parallel_workers​

The number of parallel workers used to persist binlog slice objects to object storage.

  • Command-Line Format: --binlog-archive-parallel-workers=#
  • System Variable: binlog_archive_parallel_workers
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 4
  • Minimum Value: 1
  • Maximum Value: 1024

Example:

binlog_archive_parallel_workers=8

Raft Group Settings(TODO: the description)​

raft_replication_cluster_info​

Defines the Raft group members and their respective addresses (this is logger node 2 in the cluster).

The parameter value format is as follows: '$IP:$Port;$IP:$Port;$IP:$Port@$Num', where $IP:$Port represents a node, and each node is separated by a semicolon. Here, $IP is the node's IP or DNS; $Port is the current node's protocol communication port for the raft algorithm. @$Num specifies the position of the current node in the raft-replication-cluster-info parameter. When adding a node to an existing cluster, the raft-replication-cluster-info parameter should be in the format '$IP:$Port', without specifying @$Num. If raft-replication-cluster-info is set to '$IP:$Port@1', it indicates the cluster is a single node cluster.

  • Command-Line Format: --raft-replication-cluster-info=name
  • System Variable: raft_replication_cluster_info
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: empty string

Example:

--raft-replication-cluster-info='192.168.0.2:13006;192.168.0.3:13007;192.168.0.4:13008@2'

raft_replication_auto_leader_transfer​

Allows automatic leader transfer in the Raft group.

  • Command-Line Format: --raft-replication-auto-leader-transfer=[={true|false}]
  • System Variable: raft_replication_auto_leader_transfer
  • Scope: Global
  • Dynamic: Yes
  • Persist: YES
  • Type: Boolean
  • Default Value: TRUE

Example:

--raft_replication_auto_leader_transfer=true

raft_replication_log_type_node​

Specifies that this node is a logger node in the Raft group.

  • Command-Line Format: --raft-replication-log-type-node=[={true|false}]
  • System Variable: raft_replication_log_type_node
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Boolean
  • Default Value: FALSE

Example:

--raft_replication_log_type_node=true

raft_replication_election_timeout​

raft election timeout.

  • Command-Line Format: --raft-replication-election-timeout=#
  • System Variable: raft_replication_election_timeout
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 5000
  • Minimum Value: 20
  • Maximum Value: 7200000
  • Unit: Milliseconds

Example:

raft_replication_election_timeout=3000

raft_replication_io_thread_cnt​

Number of raft replication IO thread.

  • Command-Line Format: --raft-replication-io-thread-cnt=#
  • System Variable: raft_replication_io_thread_cnt
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 4
  • Minimum Value: 2
  • Maximum Value: 128

Example:

raft_replication_io_thread_cnt=2

raft_replication_worker_thread_cnt​

Number of raft replication worker thread.

  • Command-Line Format: --raft-replication-worker-thread-cnt=#
  • System Variable: raft_replication_worker_thread_cnt
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 4
  • Minimum Value: 2
  • Maximum Value: 128

Example:

raft_replication_worker_thread_cnt=2

raft_replication_max_log_size​

Max one log size.

  • Command-Line Format: --raft-replication-max-log-size=#
  • System Variable: raft_replication_max_log_size
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 20971520
  • Minimum Value: 1
  • Maximum Value: 1073741824
  • Unit: Bytes

Example:

raft_replication_max_log_size=20971520

raft_replication_log_cache_size​

Max cached logs size.

  • Command-Line Format: --raft-replication-log-cache-size=#
  • System Variable: raft_replication_log_cache_size
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 67108864
  • Minimum Value: 1
  • Maximum Value: ULONG_MAX
  • Unit: Bytes

Example:

raft_replication_log_cache_size=20971520

raft_replication_log_level​

raft log level.

  • Command-Line Format: --raft-replication-log-level=name
  • System Variable: raft_replication_log_level
  • Scope: Global
  • Type: Enumeration
  • Default Value: LOG_ERROR
  • Valid Values: LOG_ERROR | LOG_WARN | LOG_INFO | LOG_DEBUG | LOG_TRACE

Example:

raft_replication_log_level=LOG_INFO

raft_replication_pipelining_timeout​

The timeout the raft server cache the log (milliseconds).

  • Command-Line Format: --raft-replication-pipelining-timeout=#
  • System Variable: raft_replication_pipelining_timeout
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 1
  • Minimum Value: 0
  • Maximum Value: 1073741824
  • Unit: Milliseconds

Example:

raft_replication_pipelining_timeout=1

raft_replication_send_timeout​

raft send packet timeout.

  • Command-Line Format: --raft-replication-send-timeout=#
  • System Variable: raft_replication_send_timeout
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 0
  • Minimum Value: 0
  • Maximum Value: 200000
  • Unit: Milliseconds

Example:

raft_replication_send_timeout=1

raft_replication_configure_change_timeout​

raft configure change timeout (ms). Default 1 min.

  • Command-Line Format: --raft-replication-configure-change-timeout=#
  • System Variable: raft_replication_configure_change_timeout
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Integer
  • Default Value: 60000
  • Minimum Value: 0
  • Maximum Value: 200000
  • Unit: Seconds

Example:

raft_replication_configure_change_timeout=60000

raft_replication_force_change_meta​

raft cluster force to change meta.

  • Command-Line Format: --raft-replication-force-change-meta=[={true|false}]
  • System Variable: raft_replication_force_change_meta
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Boolean
  • Default Value: FALSE

Example:

--raft_replication_force_change_meta=true

raft_replication_force_single_mode​

raft cluster force to use single mode.

  • Command-Line Format: --raft-replication-force-single-mode=[={true|false}]
  • System Variable: raft_replication_force_single_mode
  • Scope: Global
  • Dynamic: No
  • Persist: No
  • Type: Boolean
  • Default Value: FALSE

Example:

--raft_replication_force_single_mode=true

SmartEngine Settings​

smartengine_data_dir​

The path to the SmartEngine data directory. It is recommended to configure this as an absolute path. If a relative path is used, it will be resolved with respect to the MySQL datadir.

  • Command-Line Format: --smartengine-data-dir=dir_name
  • System Variable: smartengine_data_dir
  • Scope: Global
  • Type: Directory name
  • Default Value: smartengine

smartengine_wal_dir​

The path to the SmartEngine WAL directory. It is recommended to configure this as an absolute path. If a relative path is used, it will be resolved with respect to the MySQL datadir.

  • Command-Line Format: --smartengine-wal-dir=dir_name
  • System Variable: smartengine_wal_dir
  • Scope: Global
  • Type: Directory name
  • Default Value: smartengine

smartengine_persistent_cache_dir​

The path to the SmartEngine persistent cache directory. It is recommended to configure this as an absolute path. If a relative path is used, it will be resolved with respect to the MySQL datadir.

  • Command-Line Format: --smartengine-persistent-cache-dir=dir_name
  • System Variable: smartengine_persistent_cache_dir
  • Scope: Global
  • Type: Directory name
  • Default Value: smartengine
note

The persistent cache data is stored in a separate file named SMARTENGINE_PERSISTENT_CACHE. By default, this file is located in the smartengine_data_dir directory.

smartengine_persistent_cache_mode​

In read-write-through mode, data is cached during both read and write operations. In read-through mode, data is only cached during read operations.

  • Command-Line Format: --smartengine-persistent-cache-mode=#
  • System Variable: smartengine_persistent_cache_mode
  • Scope: Global
  • Dynamic: NO
  • Type: String
  • Default Value: read-write-through
  • Valid Value: read-write-through, read-through

smartengine_persistent_cache_size​

The maximum disk size used for persistent cache.

  • Command-Line Format: --smartengine-persistent-cache-size=#
  • System Variable: smartengine_persistent_cache_size
  • Scope: Global
  • Dynamic: NO
  • Type: Integer
  • Default Value: 0
  • Minimum Value: 0
  • Maximum Value: INT64_MAX
  • Unit: bytes
note

Persistent cache is used to accelerate access to S3. By default, it is disabled. The value for smartengine_persistent_cache_size should be set based on the performance requirements of the use case. If higher performance is needed, it is recommended to set the size to match the volume of data being accessed. The exact setting should be evaluated based on the specific scenario.

smartengine_memtable_size​

The maximum memory size used for a single active memtable before it is converted to an immutable memtable, which will then be flushed.

  • Command-Line Format: --smartengine-memtable-size=#
  • System Variable: smartengine_memtable_size
  • Scope: Global
  • Dynamic: YES
  • Type: Integer
  • Default Value: 134217728
  • Minimum Value: 4096
  • Maximum Value: INT64_MAX
  • Unit: bytes

smartengine_total_memtable_size​

The maximum memory size used for all memtables, including all active and immutable memtables.

  • Command-Line Format: --smartengine-total-memtable-size=#
  • System Variable: smartengine_total_memtable_size
  • Scope: Global
  • Dynamic: YES
  • Type: Integer
  • Default Value: 134217728
  • Minimum Value: 4096
  • Maximum Value: INT64_MAX
  • Unit: bytes

smartengine_row_cache_size​

The maximum memory size used for row cache, which buffers data flushed to persistent storage at the row level.

  • Command-Line Format: --smartengine-row-cache-size=#
  • System Variable: smartengine_row_cache_size
  • Scope: Global
  • Dynamic: YES
  • Type: Integer
  • Default Value: 0
  • Minimum Value: 0
  • Maximum Value: INT64_MAX
  • Unit: bytes

smartengine_block_cache_size​

The maximum memory size used for block cache, which buffers data flushed to persistent storage at the block level.

  • Command-Line Format: --smartengine-block-cache-size=#
  • System Variable: smartengine_block_cache_size
  • Scope: Global
  • Dynamic: YES
  • Type: Integer
  • Default Value: 134217728
  • Minimum Value: 0
  • Maximum Value: INT64_MAX
  • Unit: bytes
note

The memory control parameters in SmartEngine primarily include smartengine_total_memtable_size, smartengine_block_cache_size, and smartengine_row_cache_size. It is generally recommended to allocate 60% to 70% of the total system memory to these parameters, reserving the remaining portion for temporary memory usage, jemalloc overhead, and other scattered memory allocations. A typical configuration would be to set smartengine_total_memtable_size to 30% of the total memory, smartengine_block_cache_size to another 30%, and leave smartengine_row_cache_size disabled by default.

smartengine_total_wal_size​

The maximum size limit for the write-ahead log (WAL). Once this limit is reached, it will force a flush of the memtable.

  • Command-Line Format: --smartengine-total-wal-size=#
  • System Variable: smartengine_total_wal_size
  • Scope: Global
  • Dynamic: NO
  • Type: Integer
  • Default Value: 134217728
  • Minimum Value: 8388608
  • Maximum Value: INT64_MAX
  • Unit: bytes
note

The setting of smartengine_total_wal_size is directly related to the Recovery Time Objective (RTO). A large value for smartengine_total_wal_size means that in the event of failure, a larger amount of WAL may need to be replayed, which can increase the RTO. On the other hand, setting it too small can lead to frequent flush operations, negatively impacting performance. It is generally recommended to set smartengine_total_wal_size to be the same size as smartengine_total_memtable_size.

smartengine_flush_threads​

The number of threads used to execute flush tasks for flushing memtables to persistent storage.

  • Command-Line Format: --smartengine-flush-threads=#
  • System Variable: smartengine_flush_threads
  • Scope: Global
  • Dynamic: NO
  • Type: Integer
  • Default Value: 1
  • Minimum Value: 1
  • Maximum Value: 1024

smartengine_compaction_threads​

The number of threads used to execute compaction tasks for organizing the persistent data.

  • Command-Line Format: --smartengine-compaction-threads=#
  • System Variable: smartengine_compaction_threads
  • Scope: Global
  • Dynamic: NO
  • Type: Integer
  • Default Value: 1
  • Minimum Value: 1
  • Maximum Value: 1024
note

It is generally recommended to set smartengine_flush_threads and smartengine_compaction_threads to the same value. If the data volume is large, you may consider increasing the value of smartengine_compaction_threads. For machines with fewer than 16 cores, it is recommended to set both smartengine_flush_threads and smartengine_compaction_threads to the number of CPU cores. For machines with more than 16 cores, set these values to half the number of CPU cores.