Loader V4 Redux

loader_v4_redux.md

Loader V4 Redux

After discussions on RFC-0423, it's clear that most contributors prefer to introduce the new BPF Loader account layout under a new program ID. Therefore, we plan to proceed with a BPF Loader V4.

I'd like to consider a new direction on the design for Loader V4. SIMD-0167 is well-specified and addresses several developer pain points. However, I believe it oversteps in some areas and falls short in others.

Issues with SIMD-0167

The core idea behind SIMD-0167's design is a Retract-Modify-Deploy workflow where developers can place deployed programs into "maintenance mode" to make any number of changes before redeploying. I believe instead the design should start from the first principle that deployments are all-or-nothing. Developers want their program disabled ("maintenance mode") for as little time as possible, and nobody wants to be stuck in a disabled state without an escape hatch.

A persistent request from program developers has been support for deployment failovers. If a program is upgraded and the new version has a vulnerability, the maintainers want a seamless way to quickly fail over to a previous stable version. This could prevent major exploits or errors that could be made during a manual rollback.

SIMD-0167 originally defined a standard for establishing a mapping between a deployed program and its superceding version. I personally nixed this idea at first, but it seems from talks with developers that this may actually be useful.

Takeaways from RFC-0423

We ironed out a solid story for a new account layout that accomplishes a few things:

• Solves current runtime inefficiencies:
  • ELF bytes should be in the program account (under the program ID)
  • Account containing ELF bytes should be 16-byte aligned, to comply with the verifier and VM.
• Makes metadata more flexible and extensible

This new layout is iterated on and included below under "Proposed New Design".

We also came to the conclusion - through instruction changes on Loader V3, specifically SIMD-0433 - that program account resizing should happen automatically on deployment and upgrade. There should be no standalone instruction for setting the length of the account.

Proposed New Design

Below is a new design that aims to address SIMD-0167's shortcomings and incorporate feedback from RFC-0423.

State

enum LoaderV4State {
	/// 0u32
	Uninitialized,
	
	/// 1u32
	Buffer {
		elf_bytes: [u8],
	},
	
	/// 2u32
	Metadata(ProgramMetadata)
	
	/// No serialized discriminator.
	/// Keyed on first 4 bytes of the ELF (ELF Magic): 0x7f 45 4c 46
	Program {
		elf_bytes: [u8],
	}
}

The Metadata account stores all information about a program besides its ELF bytes, and it lives at an adjacent PDA at the following derivation:

"metadata" + program_id/buffer_id

The metadata is also versioned, to allow for future customizations. Metadata can apply to any program type - Uninitialized, Buffer, or Program.

enum ProgramMetadata {
	/// 0u32
	/// V1: Current and only version to start.
	V1 {
		/// Slot of last deployment.
		deployment_slot: u64,

		/// Program or buffer authority.
		authority: Option<Pubkey>,

		/// Address of the program's next version.
		next_version: Option<Pubkey>,
	}
}

Instructions

struct LoaderV4Instruction {
	/// Initialize a program or buffer account.
	///
	/// Sets the state to `Buffer`.
	///
	/// 0. `[s]` Authority to set
	/// 1. `[w]` Program or buffer account to initialize
	/// 2. `[w]` Metadata account to initialize
	Initialize,

	/// Update a program or buffer's authority.
	///
	/// 0. `[s]` Current authority
	/// 1. `[s]` New authority
	/// 2. `[w]` Program or buffer account
	/// 3. `[w]` Metadata account
	SetAuthority,

	/// Write to a buffer account.
	///
	/// 0. `[s]` Authority
	/// 1. `[w]` Buffer account
	Write {
		/// Offset at which to write the given bytes.
		offset: u32,
		/// Bytes to write.
		bytes: [u8],
	},

	/// Deploy/upgrade a program using a buffer account.
	///
	/// Program account must have been pre-funded with rent-exempt lamports.
	///
	/// Program and metadata accounts must be initialized. Provided program
	/// account can be either `Buffer` or `Program`.
	///
	/// If Buffer account and Program account alias, the Buffer account is
	/// directly deployed (ie. changed from `Buffer` to `Program`).
	///
	/// If failover accounts are provided, the current program ELF and metadata
	/// are written to failover, to be used in case of emergency.
	///
	/// 0. `[s]` Authority
	/// 1. `[ ]` Buffer account
	/// 2. `[w]` Program account
	/// 3. `[w]` Metadata account
	/// 4. `[w]` (optional) Failover program account
	/// 5. `[w]` (optional) Failover metadata account
	Deploy,

	/// Trigger a failover to roll a deployment back to its previous version.
	///
	/// Authority must match on both metadata accounts.
	///
	/// 0. `[s]` Authority
	/// 1. `[w]` Program account
	/// 2. `[w]` Metadata account
	/// 3. `[w]` Failover program account
	/// 4. `[w]` Failover metadata account
	Failover,

	/// Withdraw excess lamports above the rent-exemption for any program
	/// account (`Uninitialized`|`Buffer`|`Metadata`|`Program`).
	///
	/// 0. `[s]` Authority
	/// 1. `[w]` Any program account
	/// 2. `[w]` Destination account
	WithdrawExcessLamports,

	/// Close a program or buffer account.
	///
	/// 0. `[s]` Authority
	/// 1. `[w]` Program or buffer account
	/// 2. `[w]` Metadata account
	/// 3. `[w]` Destination account
	Close {
		/// Whether or not to tombstone the closed program.
		/// * If `true`, assigns the program to itself.
		/// * If `false`, deallocates for garbage collection.
		tombstone: bool,
	},

	/// Finalize (freeze) a program by removing its authority. Irreversible.
	///
	/// 0. `[s]` Authority
	/// 1. `[w]` Program account
	/// 2. `[w]` Metadata account
	Finalize,
}

Workflows

Deploy a program for the very first time:

1. Initialize: Initializes program (Buffer) and metadata (Metadata) accounts.
2. Write (x N): Chunked writes the ELF to the program account (buffer).
3. Deploy:

• Provide the program account in both index 1 (buffer) and 2 (program).
• Directly deploys the program account, converting it from a Buffer to a Program.

Upgrade a program from a shared buffer account you do not own:

1. Initialize: Initializes program (Buffer) and metadata (Metadata) accounts.
2. Write (x N): Chunked writes the ELF to the program account (buffer).
3. Deploy:

• Provide the buffer account in index 1 and your program account in index 2.
• Copies the buffer's ELF into the program account.

Upgrade a program with failover and then trigger that failover:

1. Initialize: Initializes failover program (Buffer) and metadata (Metadata) accounts.
2. Deploy:

• Provide a buffer account in index 1 and your program account in index 2.
• Provide failover accounts in index 3 and 4.
• Copies the current program's ELF into the failover account.
• Copies the buffer's ELF into the program account.

3. Failover: Immediately rolls the program account back to the ELF in the failover account.