Configuration Reference

This page describes the configuration layout of the current main branch. It is derived from the actual config structs in crates/common/curvine-config/src/*.rs and the sample file etc/curvine-cluster.toml.

Config SourcesCopy Anchor Link

Curvine commonly uses these inputs together:

• conf/curvine-cluster.toml: main cluster config
• conf/curvine-env.sh: shell-side environment setup for scripts under bin/
• conf/curvine-site.xml: Hadoop / Java client config

Important environment overrides:

How the config is loadedCopy Anchor Link

The load pipeline is:

file (TOML) -> environment allowlist -> deserialize -> normalize whitespace
            -> validate -> discovery.init -> resolve client.master_addrs

• Unknown keys are reported as warnings by validation::warn_unknown_keys; they are not rejected. A misspelled key silently keeps its default value, so always check the startup warnings.
• client.master_addrs may be omitted. If it is empty, Curvine derives it from journal.journal_addrs using master.rpc_port; if journal_addrs is also empty, it uses master.hostname:master.rpc_port.
• The standalone Transfer service uses a reduced loader (ClusterConf::from_transfer) that validates only the client and transfer sections.

net_interfaceCopy Anchor Link

When net_interface is non-empty (for example "eth0"), Curvine resolves the first IPv4 address of that interface and applies it to the master, mds, journal, worker, client, and transfer hostnames. When it is set, the CURVINE_*_HOSTNAME environment variables are ignored.

net_interface only fills in this node's own hostname. Cluster membership lists (journal.journal_addrs and client.master_addrs) must still be written explicitly, and each node's entry must use the exact IPv4 that its interface resolves to. Otherwise the node cannot locate itself in the Raft group, and startup fails with a clear error.

Top-Level LayoutCopy Anchor Link

The current ClusterConf contains these sections:

Cluster-Level FlagsCopy Anchor Link

format_master deletes RocksDB metadata and format_worker deletes the Worker data directory. Always set both to false in an existing production deployment; the shipped etc/curvine-cluster.toml does so.

[master]Copy Anchor Link

worker_policy = "local" prefers a local Worker for the first replica and assigns the remaining replicas through the weighted policy. weighted allocates according to worker.weight.

[master.rocksdb] controls metadata RocksDB behavior. Its defaults come from MasterConf::rocksdb_default() (4 KB blocks, WAL disabled, bloom filters enabled, index/filter blocks cached and pinned).

[master.compatibility] is described in the Version Compatibility Policy section below.

[mds]Copy Anchor Link

The MDS is an optional standalone metadata service, started with curvine-server --service mds. It is disabled by default.

kv_backend = "memory" is intended for tests only and loses all metadata when the process exits. fdb_txn_timeout_ms bounds how long an operation waits on an unreachable FoundationDB cluster; leave it at the default unless you have a specific reason to change it.

[journal]Copy Anchor Link

Key points:

• journal_addrs must list the Raft peers, and the effective local hostname must match one of the entries (or net_interface must resolve to a matching IPv4). Otherwise startup fails.
• retain_checkpoint_num also defines the retained log window (retain_checkpoint_num × snapshot_entries entries). Raising it increases journal disk usage.
• leader_create_snapshot defaults to creating snapshots only for a single-node group. A multi-node group with this key unset has no leader-side log compaction and logs an error every few minutes; set leader_create_snapshot = true for production HA clusters.
• ignore_reply_error = true can make a Master permanently diverge from its peers. Use it only for disaster recovery, never as a normal setting.
• recover_from_peers is normally only needed for an operator-directed rebuild of a stale but non-empty member. Wiped members are quarantined automatically on rejoin when the cluster is healthy.

[worker]Copy Anchor Link

data_dir entries support three forms:

data_dir = [
  "[SSD]/data/data1",          # storage type, capacity unbounded
  "[HDD:500GB]/data/data2",    # storage type + capacity
  "/data/data3"                # defaults to DISK, capacity unbounded
]

Storage types are SSD, HDD, DISK, and MEM. The Worker appends cluster_id below each path, so the effective data directory is <path>/<cluster_id>.

dir_reserved is a fixed byte reservation. free_ratio is a fraction of the physical device and also covers space consumed by other applications on the same disk. Both are enforced at write admission for new blocks and FileLayout rewrite staging copies.

[worker.spdk_disk] is only meaningful in server builds that enable SPDK (build.sh --spdk or --spdk-rdma). It contains enabled, app_name, reactor_mask, qpair_acquire_timeout, and a targets array (trtype, adrfam, traddr, trsvcid, subnqn).

[client]Copy Anchor Link

Size values are strings such as "128MB", "4MB", or "10GB". Duration fields ending in _ms accept integer milliseconds. The older unsuffixed aliases (mount_update_ttl, sync_check_interval_min, and so on) are still accepted.

[fuse]Copy Anchor Link

Notes:

• meta_cache_capacity and node_cache_size were removed. Use meta_cache_timeout and node_cache_timeout instead.
• negative_timeout = 0 disables negative-lookup caching; it does not mean "cache forever".
• direct_io and write_back_cache cannot both be enabled; startup rejects the combination.
• fuse_opts supports the VFS pairs ro/rw, nodev/dev, nosuid/suid, noexec/exec, noatime/atime, sync/async, plus dirsync, allow_other, default_permissions, and big_write. The legacy libfuse options auto_unmount, allow_root, and max_write are not supported by the direct mount backend.
• Duration values accept strings ("1s", "500ms", "1h") or bare integers interpreted as milliseconds. The older *_ms keys remain aliases.

[log], [cli], [job]Copy Anchor Link

Global [log]Copy Anchor Link

The top-level [log] section is shared by the Rust client and FUSE. Each service also has its own nested log table (master.log, worker.log, mds.log, transfer.log) and the CLI has [cli].log.

Rotation only applies when log_dir is a directory. With size and time rotation both unset, files rotate by size at 100 MB. If max_file_size_mb = 0 and no valid time_rotation is configured, files rotate daily so they cannot grow without limit.

[cli]Copy Anchor Link

CliConf contains only log. The default CLI log level is warn, with output to stdout.

[job]Copy Anchor Link

[job] configures the legacy Master-side job implementation used by clients with transfer.enabled = false. Transfer jobs use [transfer] for ownership, recovery, and retention.

[transfer]Copy Anchor Link

[transfer] is read by the standalone Transfer service and by clients that submit Load and Export work to it. A Transfer service also needs the normal [client] Master addresses. During a staged rollout, keep this section disabled in the Master and Worker configuration and enable it only in the Transfer and selected client configurations.

Do not use the deprecated inputs store_type, sqlite_path, or mysql_url in new configurations. store_url selects the backend from its scheme. Internal queue, probe, and scheduler values are not part of the supported user configuration surface.

For PostgreSQL with a private CA, append sslrootcert=/path/to/ca.pem to the store_url; TLS validates the system trust store by default.

[discovery]Copy Anchor Link

Service discovery is disabled by default. When enabled, the only supported provider is etcd.

When discovery is enabled, cluster_id must contain only lowercase letters, digits, _, and -. Effective keys are {prefix}/{cluster_id}/services/{kind}/{service_id}.

[fault_injection]Copy Anchor Link

Test-only fault control plane. It is disabled by default and must not be enabled in normal deployments.

The HTTP routes require binaries built with the fault-injection Cargo feature (for example curvine-server/fault-injection). Enabling [fault_injection] without that feature fails server startup instead of silently omitting the route.

Version Compatibility PolicyCopy Anchor Link

[master.compatibility] and [worker.compatibility] share the same fields:

The protocol version range is a code-level contract, not an operator setting. Invalid mode strings and unparseable version bounds degrade to the lenient default instead of failing closed.

enforce mode requires Workers that report component_info on their heartbeat. Workers older than the release that introduced this field resolve to MissingInfo and are rejected, so keep diagnose until the whole Worker fleet has been upgraded.

Example SkeletonCopy Anchor Link

format_master = false
format_worker = false
# net_interface = "eth0"

[master]
meta_dir = "data/meta"
# [master.compatibility]
# mode = "diagnose"

[mds]
enabled = false
rpc_port = 8998
web_port = 9004
kv_backend = "fdb"
fdb_cluster_file = "/etc/foundationdb/fdb.cluster"

[journal]
journal_addrs = [
  { id = 1, hostname = "master1", port = 8996 },
  { id = 2, hostname = "master2", port = 8996 },
  { id = 3, hostname = "master3", port = 8996 },
]
journal_dir = "data/journal"
leader_create_snapshot = true

[worker]
data_dir = ["[SSD]/data/data1"]
weight = 1

[client]
master_addrs = [
  { hostname = "master1", port = 8995 },
  { hostname = "master2", port = 8995 },
  { hostname = "master3", port = 8995 },
]

[transfer]
enabled = false

[fuse]
mnt_path = "/curvine-fuse"
web_port = 9003

[log]
level = "info"
log_dir = "stdout"

[cli]
log = { level = "warn", log_dir = "stdout", file_name = "cli.log" }

[discovery]
enabled = false