Sign inSign up

Resources / Field guides

From input artifacts to an auditable report.

This path reruns the offline evaluation mechanism. Reproducing the published result exactly also requires its matrix, checkpoint, settings and software revision.

Check the inputs

Run these commands from a project checkout with Enthalpy installed. They do not prepare business data or production credentials for you.

Reward matrix
Use a matrix containing per-task, per-candidate rewards and costs. Inspect task identities, candidate coverage and cost fields. Data without costs cannot support dollar-cost conclusions.
Checkpoint & pool
Match the checkpoint to the evaluation matrix and pool. A checkpoint trained on a public matrix is not automatically a production checkpoint for your provider pool.
Environment & archive location
Prepare dependencies and encoder weights. Record versions, device and settings, and use new experiment names and output paths to preserve existing results.

Inspect before training

The commands below use your prepared mine.jsonl.gz. If you have no matrix, use enthalpy data list to inspect public sources, then follow their download instructions.

Separate acquisition from replay
Public downloads need network access and storage. data harvest makes real provider calls and incurs charges; it is not required to replay an existing matrix.
enthalpy data matrix stats data/matrices/mine.jsonl.gz

enthalpy train sft \
  --matrix data/matrices/mine.jsonl.gz \
  --name gen1

Generate the comparison & cost sweep

Training and evaluation replay the matrix without provider calls. A published source split takes precedence over the hash-split flags below. Keep the split log.

Keep the artifacts, not only the headline
The report writes JSON and Markdown; the cost sweep is stored separately as sweep.json. Keep these with checkpoint identity, commands and logs.
enthalpy eval \
  --matrix data/matrices/mine.jsonl.gz \
  --checkpoint gen1 \
  --test-fraction 0.3 --split-seed 0 \
  --reference best_single --sweep-cost \
  --out reports/gen1.json

Know the output artifacts

Use generated artifacts for review and quotation instead of transcribing results.

gen1.json / gen1.md
JSON preserves evaluation data; Markdown provides tables, comparisons and provenance. Check split wording and retain bound labels and intervals when quoting.
gen1.sweep.json
Stores cost weights, operating points and frontier information. Format it with the repository’s sweep_table.py; the evaluation command does not automatically print the sweep table.
Checkpoint & logs
Save checkpoint_id, pool fingerprint, training settings and actual split origin. Establish experiment identity before investigating numerical differences.
python3 scripts/sweep_table.py reports/gen1.sweep.json

When the result differs

Rule out input and environment differences before attributing changes to the algorithm.

Pool fingerprint mismatch

Check model IDs, providers, model names and descriptors. Do not bypass validation to load an incompatible checkpoint; use the matching pool or retrain.

Different sample counts or results

Check data revision, source split, filters, training settings and checkpoint. Retraining an experiment with the same name does not recreate the published checkpoint identity.

Continue reading

Next: review the report for quotation

4ROUTER / SYSTEMS RESEARCH

Cross-layer reproduction: state transfer must be measurable

A reproducible run records more than model and GPU: cache tiers, transfer paths, reused tokens and failure fallbacks belong in the trace.

NOTE / 05TIERS / TRANSFER

LMCache: placing KV state in a storage hierarchy

Avoiding prefill requires paying for state access.

When reusable context exceeds GPU-resident capacity, the question becomes where state lives, when to move it and whether the transfer is worthwhile. LMCache connects inference engines to KV storage and transport across CPU, disk and remote tiers. It changes state placement and lifetime.

FIG. 01TIERS / TRANSFER
TIER 00GPU · HBM
TIER 01CPU · DRAM
TIER 02NVMe / Remote
Store state by tier; load when neededSchematic, not a measured scale or a specific engine layout.

Separate engine integration from storage

The inference engine executes the model; LMCache connectors and storage components manage KV retrieval and movement. Tiering can retain colder state in larger storage, but a storage hit still needs a path to usable GPU state. Integration versions, chunk granularity and layouts affect that boundary; backend names alone do not establish compatibility. [1]

Reuse has a break-even point

A hit entails lookup, reads, transfers and possibly layout conversion. It benefits a request when that cost is below recomputing prefill. Asynchronous prefetch may hide part of the cost while consuming bandwidth and buffers. Measure effective transfer rates and queueing instead of substituting advertised link speed for serving behavior. [2]

Context reuse and P/D separation differ

Context caching retains existing state across requests. Prefill/decode separation places two phases of one request on different execution instances, requiring KV transfer. They can be combined, but historical-prefix reuse does not validate cross-instance transport or recovery. Observe hits, misses, timeouts and fallback for each transfer path. [2]

Model & notation

T_lookup + T_read + T_transfer + T_reformat < T_prefill_saved

This is a reuse criterion without overlap, not a throughput predictor. Concurrent transfers contend for bandwidth; with pipelining, measure the critical path rather than blindly summing stages.

Experimental protocol

  1. Test cache states separately

    Separate GPU-hot, CPU-hot, disk-hot, remote-hot and fully cold requests. Record model, sequence length, tier, hit bytes and recomputation scope.

  2. Trace data movement

    Record lookup, reads, device transfers and waiting; inspect pinned memory, NUMA, PCIe or network contention alongside TTFT.

  3. Verify fallback and cleanup

    Test missing objects, exhausted capacity, unavailable backends and incompatible versions. Verify recomputation or explicit failure; define tenant isolation, retention and deletion.

References

  1. LMCache · Versioned architecture overview
  2. LMCache technical report · arXiv:2510.09665
  3. LMCache · Current documentation and integration guidance

Original explanatory figures; equations describe mechanisms or capacity models. Results in papers and documentation are not 4Router measurements.