Debugging Guide

This guide summarizes the runtime files, logging paths, ports, and common debugging entrypoints in the Curvine reference repository.

Configuration FileCopy Anchor Link

The default cluster configuration file is:

build/dist/conf/curvine-cluster.toml

The sample source file lives at etc/curvine-cluster.toml. Relevant sections in that file include:

• [master]
• [mds]
• [journal]
• [worker]
• [client]
• [transfer]
• [fuse]
• [discovery]
• [fault_injection]
• [log]
• [cli]

Runtime overrides referenced by the scripts include:

• CURVINE_CONF_FILE
• CURVINE_MASTER_HOSTNAME
• CURVINE_WORKER_HOSTNAME
• CURVINE_CLIENT_HOSTNAME

Wrapper Scripts and Runtime FilesCopy Anchor Link

The wrappers under build/dist/bin/ source conf/curvine-env.sh and then delegate to the binaries under build/dist/lib/.

For master, worker, and fuse, build/bin/launch-process.sh also manages:

• pid files at $CURVINE_HOME/<service>.pid
• stdout/stderr capture at $CURVINE_HOME/logs/<service>.out
• default config path CURVINE_CONF_FILE=$CURVINE_HOME/conf/curvine-cluster.toml

This means logs/master.out, logs/worker.out, and logs/fuse.out are the first places to look when a wrapper-started process fails before structured logging is configured.

Logging ConfigurationCopy Anchor Link

The sample config uses these defaults:

MasterCopy Anchor Link

[master]
log = { level = "info", log_dir = "stdout", file_name = "master.log" }

WorkerCopy Anchor Link

[worker]
log = { level = "info", log_dir = "stdout", file_name = "worker.log" }

Client and FUSECopy Anchor Link

[log]
level = "info"
log_dir = "stdout"
file_name = "curvine.log"

CLICopy Anchor Link

[cli]
log = { level = "warn", log_dir = "stdout", file_name = "cli.log" }

When log_dir = "stdout" and you start a component through the shell wrappers, structured logs land in the matching logs/*.out file because stdout is redirected there. If you set log_dir to a directory path instead, inspect the configured log file directly.

For deeper debugging, raise the relevant section to debug or trace.

Startup CommandsCopy Anchor Link

bin/curvine-fuse.sh defaults to mount path /curvine-fuse, but accepts --mnt-path and passes the rest of the arguments through.

Default PortsCopy Anchor Link

These defaults come from crates/common/curvine-config/src/cluster_conf.rs and the sample config:

Common Debugging FlowsCopy Anchor Link

Service will not startCopy Anchor Link

1. Check the wrapper output file:

   Copy Text

   tail -n 100 build/dist/logs/master.out
   tail -n 100 build/dist/logs/worker.out
   tail -n 100 build/dist/logs/fuse.out

2. Confirm the config path:

   Copy Text

   echo "$CURVINE_CONF_FILE"

3. Confirm the pid file is not stale:

   Copy Text

   ls build/dist/*.pid

Master or Worker connectivity issuesCopy Anchor Link

Use nc or telnet from the client or another node:

nc -vz <master-host> 8995
nc -vz <worker-host> 8997

Then verify the cluster config and any hostname overrides:

• CURVINE_MASTER_HOSTNAME
• CURVINE_WORKER_HOSTNAME
• journal and client addresses in conf/curvine-cluster.toml

Cluster looks unhealthyCopy Anchor Link

Use the built CLI and the helper script first:

cd build/dist
bin/local-cluster.sh status
bin/cv report
bin/cv node

Metrics and performance inspectionCopy Anchor Link

curl http://<master>:9001/metrics
curl http://<worker>:9002/metrics

Then inspect the host with tools such as htop, iostat -x 1, sar -n DEV 1, or perf record -g.

FUSE mount problemsCopy Anchor Link

1. Check whether the mount exists:

   Copy Text

   mount | grep curvine

2. If needed, run FUSE manually with an explicit mount path:

   Copy Text

   build/dist/lib/curvine-fuse --conf build/dist/conf/curvine-cluster.toml --mnt-path /mnt/curvine

3. Verify FUSE permissions:

   Copy Text

   ls -l /dev/fuse
   groups "$USER"

Low-Level ToolsCopy Anchor Link

• gdb for native debugging and core inspection
• strace -f -e trace=network ... for connection issues
• perf for CPU profiling
• bin/cv fs ... for path-level checks from the Curvine client side

Core DumpsCopy Anchor Link

ulimit -c unlimited
gdb build/dist/lib/curvine-server /tmp/core.curvine-server.12345

Inside gdb, start with bt and info threads.

Reporting IssuesCopy Anchor Link

When you file a bug, include:

• OS and architecture
• Curvine version or commit
• the exact wrapper command or binary command used
• relevant config with secrets removed
• the matching logs/*.out or structured log file excerpt
• any metrics or reproduction steps