MICRODOT_FILE_TRANSFER_GUIDE.md

Microdot File Transfer Guide

Simple examples for implementing file uploads and downloads on MicroPython devices using Microdot.

Contents

• Device Setup
• Installing Microdot
• Upload Handling
  • PUT Upload (Raw Binary)
  • POST Multipart Upload (Form Data)
• Download Handling
  • GET Static Files
  • GET Dynamic Generation
  • POST Download with Parameters
• Testing
• Implementation Details
• Verified Hardware

---

Device Setup

Network Configuration (boot.py)

Configure network before Microdot starts.

WiFi Devices (Raspberry Pi Pico W, ESP32):

# boot.py
import network

sta_if = network.WLAN(network.STA_IF)
sta_if.active(True)
sta_if.connect('your-ssid', 'your-password')

# Wait for connection
import time
for i in range(10):
    if sta_if.isconnected():
        break
    time.sleep(0.5)

if sta_if.isconnected():
    print(f'WiFi connected: {sta_if.ifconfig()[0]}')
else:
    print('WiFi connection failed')

Ethernet Devices (STM32 NUCLEO):

# boot.py
import network

lan = network.LAN()
lan.active(True)

# Wait for link
import time
for i in range(10):
    if lan.isconnected():
        break
    time.sleep(0.5)

if lan.isconnected():
    print(f'Ethernet connected: {lan.ifconfig()[0]}')
else:
    print('Ethernet not connected')

Upload boot.py:

mpremote connect /dev/ttyACM0 fs cp boot.py :boot.py
mpremote connect /dev/ttyACM0 soft-reset

---

Installing Microdot

DEVICE=/dev/ttyACM0  # Adjust for your device

# Download Microdot from GitHub
git clone https://github.com/miguelgrinberg/microdot.git
cd microdot

# Upload required files
mpremote connect $DEVICE fs cp src/microdot/microdot.py :microdot.py
mpremote connect $DEVICE fs cp src/microdot/helpers.py :helpers.py

# For multipart upload support
mpremote connect $DEVICE fs cp src/microdot/multipart.py :multipart.py

---

Upload Handling

PUT Upload (Raw Binary)

Direct binary upload without multipart encoding.

Server Code:

from microdot import Microdot, Response, Request
import network

Request.max_content_length = 16 * 1024 * 1024
Request.max_body_length = 2048

app = Microdot()

@app.put('/upload/<path:filename>')
async def put_upload(request, filename):
    """Stream binary upload directly"""
    import binascii

    bytes_written = 0
    crc = 0
    chunk_size = 256
    content_length = request.content_length or 0

    # IMPORTANT: Respect Content-Length to avoid blocking
    while bytes_written < content_length:
        to_read = min(chunk_size, content_length - bytes_written)
        data = await request.stream.read(to_read)
        if not data:
            break

        crc = binascii.crc32(data, crc)
        bytes_written += len(data)

    crc = crc & 0xFFFFFFFF
    print(f"Complete: {bytes_written} bytes, CRC32: 0x{crc:08x}")

    return Response(
        body={'filename': filename, 'size': bytes_written, 'crc32': f'0x{crc:08x}'},
        status_code=201
    )

Client Usage:

# curl
curl -T myfile.bin http://192.168.0.10:8000/upload/myfile.bin

# httpie
http PUT http://192.168.0.10:8000/upload/myfile.bin < myfile.bin

Key Detail: Must respect Content-Length when reading request.stream to avoid blocking on keep-alive connections.

POST Multipart Upload (Form Data)

Standard HTML form upload with multipart encoding.

Server Code:

from microdot import Microdot, Response, Request
import network

Request.max_content_length = 16 * 1024 * 1024
Request.max_body_length = 2048
Request.max_readline = 8 * 1024

app = Microdot()

@app.post('/upload')
async def upload_file(request):
    """Handle multipart form upload"""
    from multipart import FormDataIter, FileUpload
    import binascii

    bytes_written = 0
    filename = None
    crc = 0

    async for name, value in FormDataIter(request):
        if name == 'file' and isinstance(value, FileUpload):
            filename = value.filename
            print(f"Receiving: {filename}")

            # Stream file in chunks
            while True:
                data = await value.read(256)
                if not data:
                    break

                crc = binascii.crc32(data, crc)
                bytes_written += len(data)

    crc = crc & 0xFFFFFFFF
    print(f"Complete: {bytes_written} bytes, CRC32: 0x{crc:08x}")

    return Response(
        body={'filename': filename, 'size': bytes_written, 'crc32': f'0x{crc:08x}'},
        status_code=201
    )

Client Usage:

# curl
curl -F 'file=@myfile.bin' http://192.168.0.10:8000/upload

# httpie
http --form POST http://192.168.0.10:8000/upload file@myfile.bin

---

Download Handling

GET Static Files

Serve files from device flash storage.

Server Code:

@app.get('/download/file/<path:filename>')
async def download_file(request, filename):
    """Serve static file"""
    try:
        return Response.send_file(filename)
    except OSError:
        return Response(body={'error': 'File not found'}, status_code=404)

Client Usage:

# curl - save with same filename
curl -O http://192.168.0.10:8000/download/file/data.bin

# curl - specify output name
curl -o myfile.bin http://192.168.0.10:8000/download/file/data.bin

# httpie
http --download GET http://192.168.0.10:8000/download/file/data.bin

GET Dynamic Generation

Generate data on-the-fly without using flash storage.

Server Code:

@app.get('/download/generate/<int:size>')
async def download_generate(request, size):
    """Generate data dynamically"""
    import binascii

    async def generate_data():
        """Async generator streams data"""
        crc = 0
        bytes_sent = 0
        chunk_size = 1024

        while bytes_sent < size:
            to_send = min(chunk_size, size - bytes_sent)
            # Generate deterministic pattern for verification
            chunk = bytes((i % 256) for i in range(bytes_sent, bytes_sent + to_send))
            crc = binascii.crc32(chunk, crc)
            bytes_sent += to_send
            yield chunk

        crc = crc & 0xFFFFFFFF
        print(f"Generated {bytes_sent} bytes, CRC32: 0x{crc:08x}")

    return Response(
        body=generate_data(),
        headers={
            'Content-Type': 'application/octet-stream',
            'Content-Length': str(size),
            'Content-Disposition': f'attachment; filename="generated_{size}.bin"'
        }
    )

Client Usage:

# curl - download 2MB
curl -o generated.bin http://192.168.0.10:8000/download/generate/2097152

# httpie
http --download GET http://192.168.0.10:8000/download/generate/2097152

Memory Efficiency: Generates multi-MB files using only ~1KB RAM (one chunk buffer).

POST Download with Parameters

Generate file based on request parameters.

Server Code:

@app.post('/download/report')
async def download_report(request):
    """Generate file from parameters"""
    import binascii

    params = request.json
    size = params.get('size', 1024)
    pattern = params.get('pattern', 'sequential')

    async def generate_report():
        crc = 0
        bytes_sent = 0
        chunk_size = 1024

        while bytes_sent < size:
            to_send = min(chunk_size, size - bytes_sent)

            if pattern == 'sequential':
                chunk = bytes((i % 256) for i in range(bytes_sent, bytes_sent + to_send))
            else:
                # Pseudo-random pattern
                chunk = bytes(((bytes_sent + i) * 37) % 256 for i in range(to_send))

            crc = binascii.crc32(chunk, crc)
            bytes_sent += to_send
            yield chunk

        crc = crc & 0xFFFFFFFF
        print(f"Report: {bytes_sent} bytes, CRC32: 0x{crc:08x}")

    return Response(
        body=generate_report(),
        headers={
            'Content-Type': 'application/octet-stream',
            'Content-Length': str(size),
            'Content-Disposition': f'attachment; filename="report_{pattern}.bin"'
        }
    )

Client Usage:

# curl
curl -H "Content-Type: application/json" \
  -d '{"size": 1048576, "pattern": "sequential"}' \
  http://192.168.0.10:8000/download/report \
  -o report.bin

# httpie
echo '{"size": 1048576, "pattern": "random"}' | \
  http POST http://192.168.0.10:8000/download/report \
  --download --output report.bin

---

Testing

Creating Test Files

# Create 2MB test file with random data
dd if=/dev/urandom of=testfile_2mb.bin bs=1024 count=2048

# Create 100KB test file
dd if=/dev/urandom of=testfile_100k.bin bs=1024 count=100

# Calculate expected CRC32 (install if needed: apt install libarchive-zip-perl)
crc32 testfile_2mb.bin
# Example output: 808d85f6

Testing Uploads

PUT Upload:

# Upload with curl
curl -v -T testfile_2mb.bin http://192.168.0.10:8000/upload/testfile.bin

# Upload with httpie
http -v PUT http://192.168.0.10:8000/upload/testfile.bin < testfile_2mb.bin

# Verify response contains matching CRC32

Multipart Upload:

# Upload with curl
curl -v -F 'file=@testfile_2mb.bin' http://192.168.0.10:8000/upload

# Upload with httpie
http -v --form POST http://192.168.0.10:8000/upload file@testfile_2mb.bin

# Verify response contains matching CRC32

Testing Downloads

Static File:

# Download with curl
curl -O http://192.168.0.10:8000/download/file/testfile.bin

# Download with httpie
http --download GET http://192.168.0.10:8000/download/file/testfile.bin

# Verify CRC32
crc32 testfile.bin
# Compare with server logs

Generated Data:

# Download 2MB
curl -o generated.bin http://192.168.0.10:8000/download/generate/2097152

# Verify CRC32
crc32 generated.bin
# Compare with server logs: "Generated 2097152 bytes, CRC32: 0x..."

POST Download:

# Sequential pattern
curl -H "Content-Type: application/json" \
  -d '{"size": 1048576, "pattern": "sequential"}' \
  http://192.168.0.10:8000/download/report \
  -o report_seq.bin

# Random pattern
echo '{"size": 524288, "pattern": "random"}' | \
  http POST http://192.168.0.10:8000/download/report \
  --download --output report_rand.bin

# Verify CRC32 values
crc32 report_*.bin

Complete Server Example

from microdot import Microdot, Response, Request
import network

Request.max_content_length = 16 * 1024 * 1024
Request.max_body_length = 2048
Request.max_readline = 8 * 1024

app = Microdot()

# Include all endpoint implementations from above...

# Network setup helper
def get_ip():
    # WiFi
    try:
        sta_if = network.WLAN(network.STA_IF)
        if sta_if.isconnected():
            return sta_if.ifconfig()[0]
    except:
        pass

    # Ethernet
    try:
        lan = network.LAN()
        if lan.isconnected():
            return lan.ifconfig()[0]
    except:
        pass

    return None

ip = get_ip()
if ip:
    print(f'Server: http://{ip}:8000')
    app.run(host='0.0.0.0', port=8000)
else:
    print('No network connection')

---

Implementation Details

Streaming Responses

Microdot supports three body types for streaming:

1. File-like objects: Auto-streams using send_file_buffer_size (default 1KB)

   return Response.send_file('myfile.bin')

2. Async generators: Custom streaming

   async def stream():
       for chunk in data:
           yield chunk
   return Response(body=stream())

3. Sync generators: Converted to async internally

   def stream():
       for chunk in data:
           yield chunk
   return Response(body=stream())

Important Headers

For Downloads:

• Content-Type: application/octet-stream - Binary data
• Content-Length: <size> - Total size (enables progress bars)
• Content-Disposition: attachment; filename="..." - Suggested filename

For Uploads:

• Must set Request.max_content_length for large files
• Set Request.max_body_length low (e.g., 2KB) to force streaming
• Set Request.max_readline for header parsing (e.g., 8KB)

Memory Management

Uploads:

• Stream in small chunks (256 bytes default for multipart)
• Don't buffer entire file in RAM
• For PUT uploads, track Content-Length to know when to stop reading

Downloads:

• Use async generators to stream large files
• Chunk size configurable via Response.send_file_buffer_size
• Device RAM usage: ~1KB per active transfer

Keep-Alive Connection Handling

Critical for uploads: Must respect Content-Length when reading from request.stream:

content_length = request.content_length or 0
while bytes_written < content_length:
    to_read = min(chunk_size, content_length - bytes_written)
    data = await request.stream.read(to_read)

Without this, stream.read() blocks indefinitely on keep-alive connections after all data is received.

CRC32 Verification

Calculate CRC32 during transfer for integrity verification:

import binascii

crc = 0
while reading_data:
    chunk = await read_chunk()
    crc = binascii.crc32(chunk, crc)

crc = crc & 0xFFFFFFFF  # Convert to unsigned 32-bit
print(f"CRC32: 0x{crc:08x}")

Client verifies:

crc32 downloaded_file.bin
# Compare with server log

---

Verified Hardware

All methods tested and verified on:

Device	Network	Upload (PUT)	Upload (Multipart)	Download (Static)	Download (Dynamic)
STM32 NUCLEO_H563ZI	Ethernet	✓ 2MB	✓ 2MB	✓ 100KB	✓ 2MB
Raspberry Pi Pico W	WiFi	✓ 2MB	✓ 2MB	✓ 100KB	✓ 2MB

Test Results:

• All transfers complete successfully
• CRC32 checksums verified
• No memory issues
• Both curl and httpie tested
• Keep-alive connections handled correctly

---

Troubleshooting

Upload hangs/timeouts:

• Check Request.max_content_length is large enough
• Verify Content-Length handling in PUT uploads
• Ensure multipart.py is installed for form uploads

Out of memory:

• Reduce chunk sizes
• Lower Request.max_body_length to force streaming
• Don't buffer entire files

Network issues:

• Verify boot.py network configuration
• Check IP address: mpremote connect $DEVICE exec "import network; ..."
• Test simple endpoint first: @app.get('/') async def index(request): return 'OK'

CRC32 mismatch:

• File corruption during transfer
• Incorrect streaming (missed chunks)
• Different data generated vs expected

---

License

Examples based on Microdot by Miguel Grinberg (MIT License).