Skip to content

Instantly share code, notes, and snippets.

@njgheorghita

njgheorghita/devp2p.md

Last active June 12, 2020 23:36
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save njgheorghita/0b93371be85bcbef74476ba3fbe954e5 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save njgheorghita/0b93371be85bcbef74476ba3fbe954e5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
devp2p.md

devp2p

Components

  • Distributed Peer Table (DPT) / Node Discovery
  • RLPx Transport Protocol
  • Ethereum Wire Protocol (ETH) / Application Layer

RLPx v4

  • v4 is currently the most widely used - (besides LES which uses v5)
  • v5 adoption is coming soon (check felix lange's devcon video below)
  • This summary based on geth implementation (~ 70% of ethereum nodes)
  • Similar to Kademlia - except in ethereum p2p network is only used to discover new peers (vs. traditionally for sharing content) (though this might change in eth once sharding comes about)

Traditional Kademlia

  • Content (value) associated with a key
  • Stored only at peers whose nodeId is 'close' to associated peer
  • Closeness given by nodeId1 XOR nodeId2
  • Each node has b distinct buckets
  • Each bucket stores entwork information about k peers at a distance i
  • lookup(t) - t=target
    • Node finds nodeIds in her bucket closest to t
    • Send request for t or nodeIds in requested bucket that are closer
    • Repeat until t is found

RLPx v4 cont'd

nodeIds

  • Public key (64byte cryptographic ECDSA public key)
  • !NOT! tied to a unique network address (i.e. easily generate many nodeIds from one machine)

Network Connections

  • UDP

    • Only used to exchange information about the peer-to-peer network
    • No limit on connections except max 16 concurrent connections
    • 4 types of messages
      • ping solicits pong in return
      • findnode solicits neighbor
        • neighbor returns 16 nodes closer to t that have been seen by the responder
        • Node will ONLY respond to findnode if querying node is already in db
    • All UDP messages are timestamped and authentiacted w/ senders nodeId (i.e.public key)
    • To prevent replay attacks - UDP messages older than 20 seconds (relative to local time) are dropped
    • pongs also contain hash of ping it's responding to to prevent ip spoofing

  • TCP

    • Used to exchange all blockchain information
    • Encrypted and authenticated
    • Total number of TCP connections at any time is maxpeers (default = 25)
    • TO ECLIPSE A NODE - ATTACKER CONTINUOUSLY OCCUPIES ALL OF THESE CONNECTIONS
    • outgoing if initiated by client // incoming otherwise
    • Client can initiate MAX (1/2 * maxpeers) (default = 13)

Storing Network Information

  • db
    • Used for long-term storage of network information (i.e. nodes the client has seen)
      • Seen = client received a valid pong from node after sending valid ping
    • Stored on disk // persists across client reboots
    • db entry
      • nodeid
      • IP address
      • TCP port
      • UDP port
      • Time of last ping sent to node
      • Time of last pong received from node
      • Number of times node has failed to respond to findnode
    • Node's age = time elapsed since receiving pong from that node
    • Every hour client evicts all nodes from db older than 1 day
  • table
    • Used to select peers (i.e. outgoing tcp connections)
    • Short-term database
    • ALWAYS EMPTY WHEN THE CLIENT REBOOTS
    • Contains 256 buckets
      • Each bucket can hold up to 16 entries
      • Entry Info
        • nodeId
        • IP address
        • TCP port
        • UDP port
      • Entries sorted in order in which they were added to bucket
      • If new entry maps to full bucket
        • Oldest node in bucket is pinged
        • If node fails to respond with pong
          • Node is kicked out of bucket
        • If node responds with pong
          • New entry is not added
    • Nodes are removed from table if they fail to respond to findnode more than 4 times
    • Map (bucket => nodeId) via logdist()
      • This mapping is PUBLIC!!
        • Easy for adversary to predict which nodeId will map to which bucket in victims table
    • logdist() (ethereum's modification of XOR)
      • Measures the distance between two nodeIds
      • logdist(nodeIdA, nodeIdB) = r
        • dA = SHA3(nodeIdA)
        • dB = SHA3(nodeIdB)
        • r = # of most significant bits b/w dA and dB that are the same
          • i.e. r + 1 bit of dA and dB are different
      • nodeId with logdist() = r from client is mapped to bucket 256 - r in the client's table
      • logdist() results in skewed mapping of nodeIds to buckets
        • ~ 1/2 of all nodes map to bucket 256
        • ~ 1/4 of all nodes map to bucket 255
        • ~ 1/8 of all nodes map to bucket 254
        • ....
        • table has capacity of 256 * 16 = 4096 nodes
          • BUT a client with ~ all ethereum nodes will only store ~ 168 nodes in it's table

Populating the Data Structures

  • Bootstrap nodes
    • When a client is booted up for the first time - it's db is empty & only knows about 6 hardcoded bootstrap nodes
  • Bonding
    • Used to populate db and table
    • When client bods with a node - checks
      • Node exists in his db
      • db records 0 failed responses to findnode requests
      • db records that node has responded to pong in last 24 hours
    • If ^ checks pass - client immediately tries to add node to table (successfully if there's space in table)
    • Client sends ping to node
    • Successful bonding == node responds with pong
      • If successful - add node's entry to db and try to add to table
  • Unsolicited pings
    • Client receives unsolicited ping responds with pong and bonds to node
  • lookup(t)
    • Closeness to a target t (256-bit string)
      • dA = SHA3(nodeIdA) XOR t
      • dB = SHA3(nodeIdB) XOR t
      • nodeIdA is closer to t if dA < dB and vice versa
    • 1st. 16 nodes closest to t in client's table are selected
    • 2nd. query each of those 16 nodes with findnode msg that contains t
    • 3rd. upon receiving findnode msg - other node returns 16 nodes closest to t from it's own table in a neighbor msg
    • 4th. orig. client combines original nodes and all new nodes returned via neighbor msg
    • 5th. from up to (16 * 16) nodes - selects 16 nodes closest to t
    • Repeat from step 2
    • Continue repeating 2-5 until set of 16 closest node stabilizes (doesnt change) b/w iterations
    • Add the finalized 16 nodes to lookup_buffer (FIFO queue)    - If a node fails to respond to findnode request 5 times in a row - it's evicted from table.
  • Seeding
    • Triggered when
      • Node reboots
      • Every hour
      • lookup() called on empty table
    • 1st. checks if table is non-empty
      • if yes - nothing happens
    • 2nd. client bonds to each of the 6 bootstrap nodes
    • 3rd. bonds to min(30, l) seed nodes randomly selected from db and younger than 5 days
      • l = number of nodes in db younger than 5 days
    • 4th. client runs lookup(self)
      • Intended to populate other nodes' buckets with the client's newly-online nodeId
      • self = SHA3(nodeId) with client's own nodeId

Selecting Peers (i.e. outgoing TCP connections)

  • ~ 1/2 selected from lookup_buffer (result of lookup())
  • ~ 1/2 selected from table
  • When client starts - task runner is started and run continuously
  • Task runner
    • Populates client's db and table
    • Creates up to 1/2(1 + maxpeers) (default=13) outgoing TCP connections
    • Has MAX 16 concurrent tasks and a task queue
    • If < 16 tasks && task queue is empty => new tasks are created via task-creation algorithm (details below)
    • Two types of tasks
      • discover_task
        • Call of lookup(t) w/ t= 256-bit target string
      • dial_task
        • Attempt to make new TCP connection (aka dial) another node
        • Pre-dial checks - target node is . . .
          • Not currently being dailed
          • Not already a connected peer
          • Not itself
          • Not blacklisted
          • Not recently dialed
  • Resolving an unknown IP
    • Usually client knows ip address associated with nodeId
    • Except 2 cases . . .
      • nodeId statically configured by user w/o ip address
      • Ip address field is empty
    • If unknown IP - call lookup(t)
  • Task-Creation Algorithm
    • Initialize x = [1/2(1 + maxpeers)] (default=13)
    • Decrement x for each outgoing peer connection currently connected || currently being dialed
    • If client has no peers && 20+ seconds since client restarted => select bootstrap node
      • If bootstrap node passes 5 dial_task pre-checks => decrement x and create dial_task to boostrap node
    • Fill random_buffer with 1/2[1/2(1 + maxpeers)]] (default=6) random nodes from table
      • If random_buffer is not empty - overwrite what's there
      • Select first 1/2(x) nodes in buffer - decrement x - and create dial_task for all nodes that pass 5 pre-checks
    • While x > 0
      • Remove first node in lookup_buffer - and create dial_task if it passes checks
      • If len(lookup_buffer) < x and lookup() is not running => create discover_task

Resources

devp2p & Implementations

Devcon Videos

General

Eclipse Attacks

Ethereum Wiki

RLPX v3

RLPx v4

  • described above

RLPx v5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment