MIGRATION_GUIDE.md

miden-base to miden-vm next migration guide

This is a hand‑off summary for someone starting fresh on a newer miden-base branch. It captures the hard‑won lessons from the old UPGRADE_LOG and the recent branch work, plus what is still broken. Treat it as a map, not a history book.

Scope

• Upgrade miden-base to miden-crypto v0.22 and miden-vm next.
• Align the kernel and host code with two breaking changes:
  1. hash function switch to Poseidon2
  2. stack ordering switch from BE to LE
• Get make check, make test, make lint, and scripts/check-features.sh passing.

What we learned (high‑value, low‑drama)

LE word layouts

• Asset word is [amount, 0, faucet_id_suffix, faucet_id_prefix].
• Account id/nonce word is [account_nonce, 0, account_id_suffix, account_id_prefix].
• u32split now yields [lo, hi] in LE. Prefix/version checks must use low 32 bits, and suffix MSB checks must stay correct.
• When miden-vm uses LE, downstream miden-base should follow. If you see a BE helper in a kernel or account path, assume it is wrong.

mem_loadw/loc_loadw and padding

mem_loadw_le and loc_loadw_le overwrite the top word. If the top word matters, insert padw first or move the value elsewhere. Several bugs came from forgetting this.

Hash order

hmerge hashes top || next. Most of the commitment mismatches we fixed were just word order mistakes.

Advice stack behavior (miden-vm next)

• Advice stack is a deque. Index 0 is the top. adv_push.n pops from the front; the last popped ends up on top of the operand stack.
• adv_loadw pops 4 items from the front and forms Word([w0, w1, w2, w3]) where w0 was top.
• adv_pipe pops 8 items (2 words) and preserves each word’s order.
• adv.push_mapval reverses the values before pushing so values[0] is returned first later; push_mapvaln padding is inserted before values.

Regenerating kernel procedure tables

If you hit DynamicNodeNotFound for a kernel procedure, the generated kernel table is stale. Run: BUILD_GENERATED_FILES_IN_SRC=1 cargo build -p miden-protocol This regenerates crates/miden-protocol/src/transaction/kernel/procedures.rs.

What was already changed (not exhaustive)

• asset_vault.masm: LE asset handling in add/remove/peek/balance; LE vault key building.
• asset.masm: LE asset validation.
• account_id.masm: LE prefix parsing with correct u32split usage.
• output_note.masm: LE u32split usage for execution hint low part.
• account_delta.masm: slot id suffix/prefix order, LE metadata ordering, and commitment absorption order aligned with Rust.
• prologue.masm: major note‑processing refactors, advice handling fixes, and local cleanups. See UPGRADE_LOG for details.

Current blockers at the time of hand‑off

1) Output vault build loop misalignment

Tests kernel_tests::tx::test_account_delta::{fungible_asset_delta, non_fungible_asset_delta} are failing. Non‑fungible sometimes passes; fungible still fails.

Symptoms:

• Stack depth grows beyond 16 by the end of execution.
• Fungible asset validation fails or produces wrong vault keys.

Likely cause:

• The asset loop in build_output_vault mixes padw mem_loadw_le with loop pointers kept on the operand stack. The extra word introduced by padw leaks into later iterations and balloons the stack.

Suggested fix direction:

• Keep loop pointers in locals only.
• During add_asset, make the operand stack contain exactly [ASSET, vault_root_ptr] and nothing else.
• Rebuild the loop so every iteration starts from locals and ends with a clean stack.

2) Account delta commitment mismatch

The kernel‑computed account delta commitment still mismatched the host. The likely culprits are:

• ordering of storage/value/map delta absorption
• ordering inside metadata words
• absorption order inside the rate/permutation flow

This needs a fresh comparison against Rust ordering.

3) Block build / note auth

kernel_tests::block::header_errors::block_building_fails_on_account_tree_root_mismatch went through several failure modes (advice underflow, commitment mismatch, mtree verify). The log has the full timeline. It should be re‑validated on the new branch.

Pitfalls and rules of thumb

• Prefer LE everywhere. BE helpers in kernel or account paths are almost always wrong.
• Keep stack word‑aligned when calling helpers that expect [WORD, ptr].
• Always padw before mem_loadw_le or loc_loadw_le unless you can prove the top word can be destroyed.
• Rebuild the kernel procedure table after big MASM changes.

Debug probes and cleanup

This branch had many debug markers and temporary prints. Remove all of these before finalizing:

• debug.stack.* and debug.adv_stack.* in MASM
• debug prints in Rust helpers and tests
• advice padding hacks

Suggested test loop for the next developer

1. Rebuild kernel procedures: BUILD_GENERATED_FILES_IN_SRC=1 cargo build -p miden-protocol
2. Run focused tests:
   • cargo nextest run -E 'test(test_create_fungible_asset_succeeds)' --no-capture
   • cargo nextest run -E 'test(block_building_fails_on_account_tree_root_mismatch)' --no-capture
   • cargo nextest run -E 'test_account_delta::fungible_asset_delta' --no-capture
3. When those pass, run the full suite:
   • make check
   • make test
   • make lint
   • scripts/check-features.sh

Files to scrutinize first

• crates/miden-protocol/asm/kernels/transaction/lib/epilogue.masm
• crates/miden-protocol/asm/kernels/transaction/lib/asset_vault.masm
• crates/miden-protocol/asm/kernels/transaction/lib/account_delta.masm
• crates/miden-protocol/asm/kernels/transaction/lib/prologue.masm
• crates/miden-protocol/src/transaction/kernel/procedures.rs

About UPGRADE_LOG.md

It is intentionally untracked. Use it as your scratchpad for assumptions, experiments, debug probes, and test results.