Stateless Validation

Run a stateless validator to independently verify every MegaETH block on commodity hardware using SALT witnesses.

The stateless validator is a Rust client that independently verifies every MegaETH block without maintaining full chain state. Instead of replaying blocks against a locally-stored state trie, it re-executes each block against a compact cryptographic witness supplied by the network, then checks that the resulting post-state matches the commitments in the block header.

This design enables independent verification of sequencer execution on commodity hardware — a laptop-class machine can verify MegaETH Mainnet in real time. For how validators fit into the broader network, see Architecture.

Why run a stateless validator

  • Independent verification — you re-execute the state transition function (STF) of every block yourself, rather than trusting an RPC provider to tell you the truth.

  • Low hardware cost — thanks to SALT (Small Authentication Large Trie)arrow-up-right witnesses, proof data per block is significantly smaller than Merkle Patricia Trie witnesses, so validators do not need sequencer-class hardware.

  • Parallel-friendly — validation workers are embarrassingly parallel; throughput scales linearly with CPU cores.

  • Auditable trusted computing base (TCB) — the validator is built on the upstream Rust EVM interpreter revmarrow-up-right with an in-memory backend, keeping the TCB small and reviewable.

Installation

The validator is distributed as source only — there are no prebuilt binaries today. Install the Rust toolchain and build the release binary:

# Install rustup (if you don't already have it)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Clone and build
git clone https://github.com/megaeth-labs/stateless-validator.git
cd stateless-validator
cargo build --release --bin stateless-validator

The project pins a specific nightly Rust toolchain via rust-toolchain.toml, so cargo build downloads it automatically on first run.

The compiled binary lives at ./target/release/stateless-validator. Copy it onto your PATH if you plan to invoke it directly.

Quick start

First run

On the first launch, the validator needs two pieces of bootstrap information:

  1. --genesis-file — the MegaETH genesis JSON, which encodes the chain ID and hardfork activation schedule. Use test_data/mainnet/genesis.jsonarrow-up-right from the stateless-validator repo. The alloc list is stripped from this file — the validator never reads initial balances, so only the chain config is needed.

  2. --start-block — a trusted block hash that anchors your local chain. The validator fetches this block's header and stores its block_number, block_hash, state_root, and withdrawals_root as the anchor. The anchor itself is not re-executed; its values are taken on faith. Verification starts from the next block, which must satisfy both invariants below or the pipeline halts:

    • parent_hash equals the anchor's block_hash.

    • witness pre_state_root / pre_withdrawals_root match the anchor's state_root / withdrawals_root.

    The simplest way to get a usable anchor is to fetch the latest finalized header from the same RPC you'll point the validator at, then cross-check the returned hash against an independent source (a block explorer, a second RPC provider):

    Pass that hash to --start-block. An older anchor is also valid, but the validator must then re-check every block between the anchor and the tip before going live.

    circle-exclamation

Replace <ANCHOR_HASH_FROM_CURL_ABOVE> below with the hash returned by the curl command above, then launch from the cloned stateless-validator directory (so ./target/release/... and ./test_data/... resolve):

circle-info

--data-max-concurrent-requests and --witness-max-concurrent-requests are independent semaphores guarding the data path (eth_get*) and the witness path (mega_getBlockWitness) respectively — keeping them separate prevents a burst on one path from starving the other. The values shown (4 / 4) are tuned for the public mainnet RPC at mainnet.megaeth.com/rpc: unbounded concurrency may trigger HTTP 429 rate-limiting and stall the validator's forward progress.

On start, the validator:

  1. Persists the genesis config to its database.

  2. Fetches the header for --start-block and installs it as the trusted anchor.

  3. Begins the fetch → process → advance pipeline, verifying every new block.

--log.file-directory writes a rotated stateless-validator.log into the directory you pass it (./validator-data/logs in the example above). Tail it from another terminal to watch the pipeline make progress:

Healthy output looks like this — Replay block, Successfully validated block, and Chain advanced lines marching forward:

File logs are at debug level by default; console output stays at info. See Logging flags to tune levels, formats, and rotation.

Subsequent runs

Once the database is initialized, omit --genesis-file and --start-block — the validator resumes from the last validated block. All other operational flags (logging, concurrency caps) are not persisted to the database, so re-supply them on every run:

Multiple RPC endpoints

Both --rpc-endpoint and --witness-endpoint accept multiple endpoints as repeated flags or a comma-separated list. Both share the same retry primitive: each "round" attempts every provider once in order (no inter-provider sleep), and only when an entire round has failed does the client sleep for round-level exponential backoff (initial → 2× → 4× …, capped at --rpc-max-backoff-ms, with up to 50% jitter) before starting the next round. There is no global retry cap — rounds repeat indefinitely until a request succeeds. The two paths only differ in which provider each round starts at:

  • --rpc-endpoint (data: blocks / headers / code / tx) — round-robin load balancing. The starting provider rotates per call via an atomic counter, so healthy endpoints share traffic evenly; within a round the order is fixed (start → start+1 → …).

  • --witness-endpoint — primary-failover. Every round starts from provider 0, so the first endpoint takes all traffic while healthy and later endpoints only see traffic when the primary is failing. This keeps the primary cache-hot.

The two paths have independent concurrency caps (--data-max-concurrent-requests, --witness-max-concurrent-requests) so a burst on one cannot starve the other.

Command-line options

Every flag has an equivalent environment variable, convenient for service managers and containerized deployments. Command-line flags take precedence over environment variables. Boolean flags (e.g., --metrics-enabled) accept true or false via their env var — set STATELESS_VALIDATOR_METRICS_ENABLED=true to turn the endpoint on without command-line arguments.

Core flags

Flag
Env variable
Required?
Description

--data-dir

STATELESS_VALIDATOR_DATA_DIR

Yes

Directory holding the validator database and any cached data.

--rpc-endpoint

STATELESS_VALIDATOR_RPC_ENDPOINT

Yes

JSON-RPC endpoint(s) for block headers and bodies. Repeat the flag or pass a comma-separated list.

--witness-endpoint

STATELESS_VALIDATOR_WITNESS_ENDPOINT

Yes

MegaETH JSON-RPC endpoint(s) for SALT witnesses (mega_getBlockWitness). Multiple endpoints accepted.

--genesis-file

STATELESS_VALIDATOR_GENESIS_FILE

First run

Path to the genesis JSON. Stored in the database after the first run.

--start-block

STATELESS_VALIDATOR_START_BLOCK

First run

Trusted block hash used as the validation anchor.

--report-validation-endpoint

STATELESS_VALIDATOR_REPORT_VALIDATION_ENDPOINT

No

RPC endpoint that receives mega_setValidatedBlocks callbacks for validated blocks. If not provided, validation reporting is disabled.

--metrics-enabled

STATELESS_VALIDATOR_METRICS_ENABLED

No

Expose a Prometheus /metrics endpoint. Default: false (endpoint not bound).

--metrics-port

STATELESS_VALIDATOR_METRICS_PORT

No

Port for the metrics endpoint. Default: 9090.

Advanced tuning

These flags override pipeline and RPC retry defaults — most operators can leave them unset.

Flag
Env variable
Default
Description

--data-max-concurrent-requests

STATELESS_VALIDATOR_DATA_MAX_CONCURRENT_REQUESTS

unlimited

Cap on concurrent in-flight data requests (blocks, headers, code, tx). Omit for unlimited.

--witness-max-concurrent-requests

STATELESS_VALIDATOR_WITNESS_MAX_CONCURRENT_REQUESTS

unlimited

Cap on concurrent in-flight witness fetches, independent of the data cap. Omit for unlimited.

--poll-interval-ms

STATELESS_VALIDATOR_POLL_INTERVAL_MS

100

Fetcher caught-up poll interval (ms). Also rate-limits eth_blockNumber. Lower values reduce tip-following lag.

--error-restart-delay-ms

STATELESS_VALIDATOR_ERROR_RESTART_DELAY_MS

1000

Pipeline restart delay (ms) after a transient cycle error.

--rpc-initial-backoff-ms

STATELESS_VALIDATOR_RPC_INITIAL_BACKOFF_MS

500

Initial round-level RPC retry backoff (ms). Applied after every provider in a round has failed; doubles each round.

--rpc-max-backoff-ms

STATELESS_VALIDATOR_RPC_MAX_BACKOFF_MS

30000

Cap on round-level RPC retry backoff (ms).

--canonical-chain-max-length

STATELESS_VALIDATOR_CANONICAL_CHAIN_MAX_LENGTH

1000

Soft cap on canonical-chain rows retained locally. Larger values widen the reorg-lookup window; smaller values reduce db growth. Must be ≥ 1.

Logging flags

Logging is configured via --log.* flags, mirrored by STATELESS_LOG_* environment variables.

Flag
Env variable
Default
Description

--log.stdout-filter

STATELESS_LOG_STDOUT

info

Console log level (trace / debug / info / warn / error).

--log.stdout-format

STATELESS_LOG_STDOUT_FORMAT

terminal

Console format: terminal or json.

--log.color

STATELESS_LOG_COLOR

auto

ANSI color: auto, always, or never.

--log.file-directory

STATELESS_LOG_FILE_DIRECTORY

(unset)

Directory for rotated log files. File logging is off when unset.

--log.file-name

STATELESS_LOG_FILE_NAME

stateless-validator.log

Base name of the active log file.

--log.file-filter

STATELESS_LOG_FILE

debug

Log level for file output.

--log.file-format

STATELESS_LOG_FILE_FORMAT

terminal

File format: terminal or json.

--log.file-max-size

STATELESS_LOG_FILE_MAX_SIZE

200

Max log file size (MB) before rotation.

--log.file-max-files

STATELESS_LOG_FILE_MAX_FILES

5

Number of rotated log files to keep.

Monitoring

Checking validation progress

With metrics enabled, the validator binds a Prometheus endpoint on 0.0.0.0:9090 (reachable as http://<host>:9090/metrics, or http://localhost:9090/metrics from the same machine). Three gauges tell you whether the validator is keeping up:

validation_lag is the number of blocks the validator is behind the remote tip (remote_chain_height − local_chain_height). Interpret it in two phases:

  • During initial catch-up (only if you anchored at an older block), validation_lag starts large and shrinks as the validator replays history to reach the tip. A large lag here is expected, not a symptom.

  • Once caught up, the gauge sits around 3–5 and briefly spikes during bursty periods. This floor is intentional: the validator currently buffers 3 blocks below the remote tip, refusing to fetch any block within that window so the upstream witness generator has headroom to finish. Factor in the 100 ms poll cadence and one RPC round-trip, and a steady-state lag of a few blocks is expected — not a symptom. Persistent lag much above that range means the validator can't keep pace with the sequencer — investigate per the Troubleshooting section.

The scripts/validator-status.sharrow-up-right helper in the repo renders these metrics as a formatted dashboard.

Useful metrics

Metric
Type
What it tells you

stateless_validator_local_chain_height

Gauge

Local chain tip.

stateless_validator_remote_chain_height

Gauge

Remote chain tip reported by the RPC endpoint.

stateless_validator_validation_lag

Gauge

Blocks behind the remote tip (steady-state floor ≈ 3–5).

stateless_validator_block_validation_time_seconds

Histogram

End-to-end time to validate a block.

stateless_validator_witness_verification_time_seconds

Histogram

Time spent verifying SALT witnesses.

stateless_validator_block_replay_time_seconds

Histogram

EVM execution time per block.

stateless_validator_salt_update_time_seconds

Histogram

Time to apply post-state deltas to the SALT trie.

stateless_validator_block_state_reads

Histogram

KV reads per block (diagnoses I/O-bound slowdown).

stateless_validator_block_state_writes

Histogram

KV writes per block.

stateless_validator_transactions_total

Counter

Total transactions validated.

stateless_validator_gas_used_total

Counter

Total gas used in validated blocks.

stateless_validator_reorgs_detected_total

Counter

Number of reorgs handled.

stateless_validator_reorg_depth

Histogram

Depth of chain reorganizations.

stateless_validator_rpc_requests_total{method=...}

Counter

RPC requests made (one per logical call), labelled by method.

stateless_validator_rpc_errors_total{method=...}

Counter

RPC final failures (not retried attempts), labelled by method.

stateless_validator_rpc_retry_attempts_total{method=...}

Counter

Transient retry attempts before final outcome, labelled by method.

stateless_validator_block_fetch_time_seconds

Histogram

Per-call eth_getBlockByNumber / eth_getBlockByHash latency.

stateless_validator_code_fetch_time_seconds

Histogram

Per-call eth_getCodeByHash latency.

stateless_validator_witness_fetch_rpc_time_seconds

Histogram

Per-call mega_getBlockWitness latency.

stateless_validator_contract_cache_hits_total

Counter

Bytecode served from the local cache.

stateless_validator_contract_cache_misses_total

Counter

Bytecode fetched from RPC on miss.

stateless_validator_salt_witness_size_bytes

Histogram

Serialized SALT witness size per block.

stateless_validator_salt_witness_keys

Histogram

Key count in each SALT witness.

stateless_validator_salt_witness_kvs_size_bytes

Histogram

KV payload size inside each SALT witness.

stateless_validator_mpt_witness_size_bytes

Histogram

Serialized MPT withdrawals-witness size per block.

stateless_validator_worker_tasks_completed_total

Counter

Tasks completed per worker (label: worker_id).

stateless_validator_worker_tasks_failed_total

Counter

Tasks failed per worker (label: worker_id).

For the complete list, see metrics.rsarrow-up-right in the upstream repo.

Logs

When --log.file-directory is set, the validator writes rotated log files to that directory. Rotation is size-based (--log.file-max-size, default 200 MB), keeping --log.file-max-files rotated files (default 5). Console output honors --log.stdout-filter.

Trust model

The stateless validator is an execution client: it verifies that every block's state transition was applied correctly and that commitments in the block header match the resulting post-state. It does not decide which chain is canonical — it validates whatever sequence of blocks you feed it.

If you trust the data endpoint (--rpc-endpoint) to serve the canonical block sequence the sequencer produced, the validator detects any block whose post-state does not match the header commitments — including any execution mistake by the sequencer. The witness endpoint (--witness-endpoint) does not need to be trusted for correctness: witness contents are cryptographically verified against the previous block's state root, so a faulty or malicious witness endpoint can only stall progress, not produce a false validation.

circle-exclamation

For a fully trust-minimized setup, pair the stateless validator with:

  • op-node to derive the canonical L2 chain from L1 and the data availability layer.

  • A MegaETH replica node that follows the derived chain and serves blocks locally.

In that configuration, you rely only on L1's security and your own software — no external RPC is in the trusted path.

circle-info

The MegaETH replica node is currently permissioned — the binary is not generally available. If you want to run a replica alongside the stateless validator, contact the MegaETH teamarrow-up-right to request access.

Troubleshooting

The validator can't find the start block. You will see warnings like WARN ...: failed to fetch start block ... repeating in the log. Check that --rpc-endpoint is reachable and that the block hash in --start-block exists on that endpoint. The validator retries fetch failures automatically, so the warnings clear once the RPC starts responding.

validation_lag keeps growing. Either the remote RPC is throttling witness fetches (look for mega_getBlockWitness errors in stateless_validator_rpc_errors_total) or the machine is under-provisioned. Compare stateless_validator_block_validation_time_seconds p99 against the chain's block period — if validation time exceeds the period, the validator cannot keep pace at all and you need either faster hardware or fewer worker tasks. Histograms like block_validation_time_seconds, witness_verification_time_seconds, and block_replay_time_seconds break down where time is being spent.

Last updated