Bench run: dcompact / 31251963803

generated (UTC): 2026-08-08T11:48:09.805805+00:00

ToplingDB zipkeyonly db_bench | ToplingDB zipkeyvalue db_bench | RocksDB v8.10 db_bench | RocksDB master (5fe22535) db_bench | DB INFO LOGs (Actions artifact) — ToplingDB zipkeyonly LOG-fillrandom, ToplingDB zipkeyonly LOG-fillseq, ToplingDB zipkeyvalue LOG-fillrandom, ToplingDB zipkeyvalue LOG-fillseq, RocksDB v8.10 LOG-fillrandom, RocksDB v8.10 LOG-fillseq, RocksDB master (5fe22535) LOG-fillrandom, RocksDB master (5fe22535) LOG-fillseq

ToplingDB bench yaml (runtime graft): ToplingDB zipkeyonly yaml | ToplingDB zipkeyvalue yaml

Bench configuration (dcompact)

Write-side db_bench CPU quota: 50% (from runner_env.txt cpu_quota_write). Topling compact: dcompact_worker — dcompact_worker runs outside the write cgroup and uses remaining CPU cores. RocksDB compact: compaction_service_spool — CompactionService spool + out-of-cgroup broker/worker on remaining cores.

Runner hardware/software

OS	Ubuntu 24.04.4 LTS
Kernel	6.17.0-1020-azure
CPU	AMD EPYC 7763 64-Core Processor
Logical CPUs	4
Memory	15.6GiB
/dev/shm	7.8GiB
RocksDB block cache	7.8GiB

On-disk DB size (3.7GiB) ≤ block cache (7.8GiB) — cache can hold the entire dataset. So this benchmark mainly shows CPU and memory cost.

/dev/shm usage (disk space; after db_bench)

Allocated disk usage (IEC blocks). RocksDB uses default Snappy compression. Space ratio = engine / v8.10; <1 = less space than RocksDB, >1 = larger.

workload	ToplingDB zipkeyonly	ToplingDB zipkeyvalue	RocksDB v8.10	RocksDB master (5fe22535)	zipkeyonly / v8.10 (space)	zipkeyvalue / v8.10 (space)
fillrandom	3.0GiB	615.4MiB	3.7GiB	3.7GiB	0.80x	0.16x
fillseq	4.7GiB	1.2GiB	3.2GiB	3.2GiB	1.45x	0.36x

Peak RSS (RAM; during db_bench)

RSS is Resident Set Size. RocksDB block cache = half physical memory (7.8GiB). Ratio = engine / v8.10; <1 = less RSS, >1 = more RSS.

workload	ToplingDB zipkeyonly	ToplingDB zipkeyvalue	RocksDB v8.10	RocksDB master (5fe22535)	zipkeyonly / v8.10 (RSS)	zipkeyvalue / v8.10 (RSS)
fillrandom	1011.1MiB	762.5MiB	4.1GiB	4.2GiB	0.24x	0.18x
fillseq	500.8MiB	1.2GiB	6.4GiB	6.6GiB	0.08x	0.19x

RAM usage over time

Sampled once per second from /proc/statm plus open-file page cache. Colored bands show benchmark segments (start time from db_bench output).

rss: resident set size (pages currently in RAM for the process); rss = shared + anony.
shared: shared resident pages; mostly readonly (cheap; OS prefers reclaiming these, no swap needed).
anony: rss - shared; mostly readwrite anonymous pages (costly, needs swap).
pagecache: kernel file page cache for regular files the process currently has open (cachestat(2) on /proc/pid/fd, deduped by inode; covers buffered readwrite and mmap). Pages brought in only via buffered I/O are not charged to process RSS; mmap'd file pages can appear in both pagecache and RSS/shared, so series may overlap.
anony+pc: anony+pagecache (sum of process anonymous RSS and open-file page cache; not a disjoint partition).

Comparison: db_bench fillrandom suite (perf)

Benchmarks: fillrandom, flush, compact, readseq×3, readrandom. RocksDB uses default Snappy compression. compact row shows operations/seconds. RocksDB time / zipkey*: >1 = that zipkey* engine faster. Values show ops/sec.

benchmark	ToplingDB zipkeyonly ops/sec	ToplingDB zipkeyvalue ops/sec	RocksDB v8.10 ops/sec	RocksDB master (5fe22535) ops/sec	zipkeyvalue time / zipkeyonly	v8.10 time / zipkeyonly	master (5fe22535) time / zipkeyonly	v8.10 time / zipkeyvalue	master (5fe22535) time / zipkeyvalue
compact	1/54.4s	1/80.7s	1/43.6s	1/43.1s	1.48x	0.80x	0.79x	0.54x	0.53x
fillrandom	241835	236926	235428	230443	1.02x	1.03x	1.05x	1.01x	1.03x
readrandom	1994788	825921	161676	157886	2.42x	12.34x	12.63x	5.11x	5.23x
readseq	10421035	8130583	4801306	3113648	1.28x	1.37x	2.12x	1.07x	1.65x

Comparison: db_bench fillseq suite (perf)

Same as fillrandom. Watch compact / readseq / readrandom cost for minDictZip=10 vs space savings above.

benchmark	ToplingDB zipkeyonly ops/sec	ToplingDB zipkeyvalue ops/sec	RocksDB v8.10 ops/sec	RocksDB master (5fe22535) ops/sec	zipkeyvalue time / zipkeyonly	v8.10 time / zipkeyonly	master (5fe22535) time / zipkeyonly	v8.10 time / zipkeyvalue	master (5fe22535) time / zipkeyvalue
compact	1/89.5s	1/133.4s	1/40.3s	1/39.9s	1.49x	0.45x	0.45x	0.30x	0.30x
fillseq	555081	549811	896661	880758	1.01x	0.62x	0.63x	0.61x	0.62x
readrandom	2939023	616793	145178	156694	4.76x	20.24x	18.76x	4.25x	3.94x
readseq	11744643	9890606	4952012	3714615	1.19x	2.37x	3.16x	2.00x	2.66x

Per-engine details

ToplingDB zipkeyonly

db_bench (fillrandom+compact+readseq/readrandom)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillrandom	4.135040	241835	413.504	100000000	13.4 MB/s
compact	54423477.000000	0	54.424	1
readseq	0.094664	10563639	9.466	100000000	584.3 MB/s
readseq	0.095006	10525578	9.501	100000000	582.2 MB/s
readseq	0.095959	10421035	9.596	100000000	576.4 MB/s
readrandom	0.501306	1994788	50.131	100000000	69.7 MB/s (63205900 of 100000000 found)

db_bench (fillseq suite)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillseq	1.800703	555081	180.154	100000000	30.7 MB/s
compact	89513450.000000	0	89.514	1
readseq	0.085061	11756158	8.506	100000000	650.3 MB/s
readseq	0.084770	11796538	8.477	100000000	652.5 MB/s
readseq	0.085145	11744643	8.515	100000000	649.6 MB/s
readrandom	0.340249	2939023	34.025	100000000	162.6 MB/s (100000000 of 100000000 found)

ToplingDB zipkeyvalue

db_bench (fillrandom+compact+readseq/readrandom)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillrandom	4.220717	236926	422.072	100000000	13.1 MB/s
compact	80680133.000000	0	80.680	1
readseq	0.123275	8111904	12.328	100000000	448.7 MB/s
readseq	0.122937	8134205	12.294	100000000	449.9 MB/s
readseq	0.122992	8130583	12.299	100000000	449.7 MB/s
readrandom	1.210769	825921	121.077	100000000	28.9 MB/s (63209842 of 100000000 found)

db_bench (fillseq suite)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillseq	1.818804	549811	181.880	100000000	30.4 MB/s
compact	133393096.000000	0	133.393	1
readseq	0.103101	9699181	10.310	100000000	536.5 MB/s
readseq	0.100872	9913550	10.087	100000000	548.3 MB/s
readseq	0.101106	9890606	10.111	100000000	547.1 MB/s
readrandom	1.621288	616793	162.129	100000000	34.1 MB/s (100000000 of 100000000 found)

RocksDB v8.10

db_bench (fillrandom+compact+readseq/readrandom)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillrandom	4.248	235428	424.757	100000000	13.0 MB/s
compact	43588269.000	0	43.588	1
readseq	0.293	3411124	18.531	63212929	188.7 MB/s
readseq	0.207	4825355	13.100	63212929	266.9 MB/s
readseq	0.208	4801306	13.166	63212929	265.6 MB/s
readrandom	6.185	161676	618.518	100000000	5.7 MB/s (63208248 of 100000000 found)

db_bench (fillseq suite)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillseq	1.115	896661	111.525	100000000	49.6 MB/s
compact	40303497.000	0	40.304	1
readseq	0.348	2875161	34.781	100000000	159.0 MB/s
readseq	0.204	4899223	20.411	100000000	271.0 MB/s
readseq	0.202	4952012	20.194	100000000	273.9 MB/s
readrandom	6.888	145178	688.807	100000000	8.0 MB/s (100000000 of 100000000 found)

RocksDB master (5fe22535)

db_bench (fillrandom+compact+readseq/readrandom)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillrandom	4.339	230443	433.946	100000000	12.7 MB/s
compact	43118125.000	0	43.118	1
readseq	0.427	2340435	27.008	63210519	129.5 MB/s
readseq	0.324	3082586	20.506	63210519	170.5 MB/s
readseq	0.321	3113648	20.301	63210519	172.2 MB/s
readrandom	6.334	157886	633.368	100000000	5.5 MB/s (63210155 of 100000000 found)

db_bench (fillseq suite)

benchmark	micros/op	ops/sec	seconds	operations	extra
fillseq	1.135	880758	113.538	100000000	48.7 MB/s
compact	39916466.000	0	39.917	1
readseq	0.423	2366528	42.256	100000000	130.9 MB/s
readseq	0.271	3690876	27.094	100000000	204.2 MB/s
readseq	0.269	3714615	26.921	100000000	205.5 MB/s
readrandom	6.382	156694	638.185	100000000	8.7 MB/s (100000000 of 100000000 found)