Skip to content
2026-09-12 full-chain

Curvine full-chain daily test report - 2026-09-12

Quality conclusion

Executive summary

Caution

Release decision: NO-GO. Pipeline FAIL; ran 7 profiles, 3 passed, 3 failed.

Quality gates

GateCriterionActualVerdict
Full-chain resultAll required profiles passed3/7 passedFAIL
Failure attributionFailures classified2 root-cause groupsPASS
Resource cleanupAll profile cleanups succeeded7/7PASS

Conclusion

This full-chain run did not pass. Failed profiles require resolution before release.

Test results

Profile summary

ProfilePreflightResultDurationClassCleanup
fastPASSPASS12m 31spassedPASSED
integrationPASSFAIL5m 07sharness_failurePASSED
dailyPASSFAIL9m 07sunknown_failurePASSED
fusePASSFAIL3m 12sunknown_failurePASSED
ltpPASSPASS49m 42spassedPASSED
csiPASSPASS0m 36spassedPASSED
perf-benchmarkNOT_RECORDEDPASSED5m 29spassedPASSED

LTP

  • Status: COMPLETED
  • Suites completed: 7
  • Suites remaining: 0
  • Stats: 1129 passed / 0 real failed / 141 skipped / 0 report-consistency errors
SuiteStatusPassedReal failedSkippedReport errorsReturn code
fs_perms_simplepassed180000
fsxpassed10000
fs_bindpassed10000
smoketestpassed120100
iopassed20000
fs-jfspassed270200
syscalls-jfspassed1068013800

Performance

Note

Gate policy: report only, non-blocking for the full-chain result. Mark yellow/red vs baseline for human follow-up.

  • Status: passed
  • Gate mode: report_only

Metadata performance (this run)

ITEMVALUEAVG COSTP50 msP95 msP99 msMAX msSAMPLESERRORSBaseline VALUEVALUE deltaBaseline P99 msP99 deltaStatus
Create file23112.09 ops/s1.72 ms/op2.054.094.094.60200000021668.74 ops/s+6.7%4.09+0.1%pass
Stat file71711.99 ops/s0.56 ms/op1.021.021.022.63200000063487.48 ops/s+13.0%2.05-50.1%pass
Open file68931.50 ops/s0.58 ms/op1.021.021.023.42200000063113.20 ops/s+9.2%2.05-50.1%pass
Rename file32706.40 ops/s1.22 ms/op2.054.014.014.01200000030314.48 ops/s+7.9%4.09-2.0%pass
Delete file32228.14 ops/s1.23 ms/op2.054.094.094.59200000031231.33 ops/s+3.2%4.09+0.1%pass

FIO read/write (this run)

ITEMSPEED GiB/sIOPSAVG COSTP50 msP95 msP99 msMAX msSAMPLESERRORSBaseline SPEED GiB/sSPEED deltaBaseline P99 msP99 deltaStatus
Sequential write 64KB1.7428459.888.81 ms/op8.8510.1610.9417.2326214401.71+1.6%10.940.0%pass
Sequential read 64KB2.3037740.286.34 ms/op6.1310.2912.3922.4526214402.28+1.0%12.78-3.1%pass
Random write 64KB1.8129735.038.40 ms/op8.229.3710.16290.4326214401.77+2.5%10.29-1.3%pass
Random read 64KB1.0917910.9013.16 ms/op13.0416.5819.2750.3626214401.12-2.4%19.53-1.3%pass
Sequential write 256KB2.8211566.5420.49 ms/op21.6331.8537.4945.556553602.73+3.4%36.44+2.9%pass
Sequential read 256KB2.7711328.6120.88 ms/op22.9426.3527.9245.386553603.11-11.1%26.08+7.0%pass
Random write 256KB2.7111102.1521.74 ms/op21.1026.8730.02453.646553602.51+8.0%30.28-0.9%pass
Random read 256KB2.4610068.5224.52 ms/op24.5129.7533.1649.616553602.37+3.7%34.34-3.4%pass
Sequential write 1MB3.363443.4666.61 ms/op58.98145.75204.47327.901638403.45-2.5%191.89+6.6%pass
Sequential read 1MB2.532590.7792.24 ms/op101.19113.77117.96198.491638402.72-7.0%108.53+8.7%pass
Random write 1MB3.133208.7772.60 ms/op69.73110.62242.22559.231638403.09+1.4%229.64+5.5%pass
Random read 1MB2.342399.88102.10 ms/op103.28114.82125.30180.771638402.15+9.0%135.27-7.4%pass

Failures and attribution

Failure analysis

g-rustc-recursion-compile

  • Class: unknown_failure
  • Failure layer: build
  • Root-cause confidence: medium
  • Root cause: Rust compiler query recursion limit is exceeded while type-checking async test code in fallback_read_test and write_cache_test crates. Whether increased type complexity in curvine-tests or rustc 1.98 toolchain limits is the primary cause remains unproven.
  • Recommendation: Bisect commits between last passing daily SHA and ae330604 to identify async test changes that increased type depth. If product-side, add recursion_limit attributes or refactor test helpers; if toolchain-specific, adjust rustc settings in the build environment.

g-fuse-buffered-io-eio

  • Class: pre_existing_failure
  • Failure layer: product
  • Root-cause confidence: high
  • Root cause: curvine-fuse buffered read/write on /curvine-fuse returns Input/output error during sustained or verified writes. This matches the pre-existing defect tracked in GitHub issue #1646 related to FuseConf mount CLI override generation.
  • Recommendation: Review FuseConf-to-mount flag mapping in curvine-fuse mount CLI code introduced around e01cf918; restore correct buffered I/O behavior per open issue #1646 repair plan.

Failed case summary

CaseSuite / PackageStatusKey errorRoot group
integration profileintegration / integrationFAILEDprofile failedg-rustc-recursion-compile
daily profiledaily / dailyFAILEDprofile failedg-rustc-recursion-compile
FIO Sequential Write Test (256KB blocks)fio / fuseFAILFIO Sequential Write test failedg-fuse-buffered-io-eio
FIO Sequential Read Test (256KB blocks)fio / fuseFAILFIO Sequential Read test failedg-fuse-buffered-io-eio
FIO Random Write Test (256KB blocks)fio / fuseFAILFIO Random Write test failedg-fuse-buffered-io-eio
FIO Random Read Test (256KB blocks)fio / fuseFAILFIO Random Read test failedg-fuse-buffered-io-eio
FIO Mixed Random Read/Write Test (256KB blocks, 70% read, 30% write)fio / fuseFAILFIO Mixed Random Read/Write test failedg-fuse-buffered-io-eio
fusefuse / fuseFAILED“status”: “failed”g-fuse-buffered-io-eio

Failed case reconciliation

ProfileReported failuresSource failuresDelta
integration110
daily101
fuse660

Common root-cause groups

g-rustc-recursion-compile

  • Model class: unknown_failure; confidence: medium
  • Recommendation: Bisect commits between last passing daily SHA and ae330604 to identify async test changes that increased type depth. If product-side, add recursion_limit attributes or refactor test helpers; if toolchain-specific, adjust rustc settings in the build environment.

g-fuse-buffered-io-eio

  • Model class: pre_existing_failure; confidence: high
  • Recommendation: Review FuseConf-to-mount flag mapping in curvine-fuse mount CLI code introduced around e01cf918; restore correct buffered I/O behavior per open issue #1646 repair plan.

Follow-up

Defects and fixes

  • GitHub Issue: 1646 (open, reused)

Risks

  • A subset of green profiles does not override a full-chain NO-GO.
  • Report-only performance regressions still require human follow-up.

Next actions

  • Bisect commits between last passing daily SHA and ae330604 to identify async test changes that increased type depth. If product-side, add recursion_limit attributes or refactor test helpers; if toolchain-specific, adjust rustc settings in the build environment.
  • Review FuseConf-to-mount flag mapping in curvine-fuse mount CLI code introduced around e01cf918; restore correct buffered I/O behavior per open issue #1646 repair plan.
  • Rerun the full chain after fixes and require all release gates to pass.

Publication note

This public report contains only sanitized test outcomes. Private execution evidence remains internal.