Rendel turns any website into clean, LLM-ready Markdown so AI agents can read the web.
flowchart TD
AGENT[" <br/>AI Agent<br/>(ChatGPT, Perplexity, etc.)<br/> "]
AGENT --->|"request a webpage"| RENDEL
RENDEL[" <br/>Rendel<br/> "]
RENDEL --->|"render & extract content"| SITE
SITE[" <br/>Any Website<br/>(JS-heavy, SPAs, dynamic)<br/> "]
SITE --->|"raw HTML + JS"| RENDEL
RENDEL --->|"Markdown pages"| PAGES
RENDEL --->|"site navigation"| LLMS
PAGES[" <br/>PDP, PLP, Brand, etc.<br/> "]
LLMS[" <br/>llms.txt index<br/> "]
PAGES ---> AGENT
LLMS ---> AGENT
classDef agent fill:#f6f3ea,stroke:#9c7a2b,color:#2b2416,stroke-width:1.5px;
classDef platform fill:#e7f3e7,stroke:#4d7b52,color:#18311b,stroke-width:2px;
classDef site fill:#d9e8f5,stroke:#3f6f94,color:#132433,stroke-width:1.5px;
classDef output fill:#f3e4d7,stroke:#8a5a2b,color:#301d0f,stroke-width:1.5px;
class AGENT agent;
class RENDEL platform;
class SITE site;
class PAGES,LLMS output;
The core value: Websites are built for browsers. AI agents need structured text. Rendel bridges that gap — headless Chrome renders the page, Readability extracts the content, and a conversion pipeline produces Markdown with metadata frontmatter.
Two Lambdas, one Docker image. The API Lambda handles all inbound requests. Cached pages are served in <50ms. On a cache miss, the API Lambda renders the page on the spot (<5s).
flowchart TD
AGENT[" <br/>AI Agent<br/> "]
AGENT ---> APIGW
APIGW[" <br/>API Gateway<br/> "]
APIGW ---> API
API[" <br/>API Lambda<br/>512 MB · 30s timeout<br/> "]
API ---> CACHE
CACHE{" <br/>Cached?<br/> "}
CACHE -- "hit" ----> S3
S3[" <br/>S3<br/>Markdown cache<br/> "]
S3 --->|"< 50ms"| AGENT
CACHE -- "miss" ----> SITE
SITE[" <br/>Target Website<br/> "]
SITE --->|"raw HTML"| API
API --->|"render + extract + convert"| S3
classDef external fill:#f6f3ea,stroke:#9c7a2b,color:#2b2416,stroke-width:1.5px;
classDef compute fill:#e7f3e7,stroke:#4d7b52,color:#18311b,stroke-width:1.5px;
classDef state fill:#f3e4d7,stroke:#8a5a2b,color:#301d0f,stroke-width:1.5px;
classDef edge fill:#d9e8f5,stroke:#3f6f94,color:#132433,stroke-width:1.5px;
classDef decision fill:#fff7e6,stroke:#9c7a2b,color:#2b2416,stroke-width:1.5px;
class AGENT,SITE external;
class APIGW edge;
class API compute;
class S3 state;
class CACHE decision;
Customers submit URLs (or a sitemap) to refresh in bulk. The API Lambda fans out work to a queue, and Worker Lambdas render pages in parallel. Progress is tracked in DynamoDB. On completion, llms.txt discovery files are regenerated.
flowchart TD
CUSTOMER[" <br/>Customer<br/>(API call or dashboard)<br/> "]
CUSTOMER --->|"POST /ingest"| API
API[" <br/>API Lambda<br/> "]
API --->|"create job"| JOBS
API --->|"enqueue URL batches"| SQS
JOBS[" <br/>DynamoDB<br/>job tracking<br/> "]
SQS[" <br/>SQS Queue<br/> "]
SQS ---> WORKER
SQS -.->|"3 retries failed"| DLQ
DLQ[" <br/>Dead-Letter Queue<br/> "]
WORKER[" <br/>Worker Lambda<br/>1024 MB · 120s timeout<br/> "]
WORKER --->|"render pages"| SITE
SITE[" <br/>Target Website<br/> "]
SITE ---> WORKER
WORKER --->|"store markdown"| S3
WORKER --->|"update progress"| JOBS
S3[" <br/>S3<br/>Markdown + metadata<br/> "]
S3 --->|"on completion"| LLMS
LLMS[" <br/>llms.txt<br/>site index<br/> "]
classDef external fill:#f6f3ea,stroke:#9c7a2b,color:#2b2416,stroke-width:1.5px;
classDef compute fill:#e7f3e7,stroke:#4d7b52,color:#18311b,stroke-width:1.5px;
classDef state fill:#f3e4d7,stroke:#8a5a2b,color:#301d0f,stroke-width:1.5px;
classDef edge fill:#d9e8f5,stroke:#3f6f94,color:#132433,stroke-width:1.5px;
class CUSTOMER,SITE external;
class API,WORKER compute;
class JOBS,SQS,S3,LLMS,DLQ state;
Both Lambdas share one container image from ECR (Go + headless Chrome). Each Lambda includes a LambdaWatch layer that ships logs to Grafana Loki.
flowchart TD
AGENT[" <br/>AI Agents<br/>(ChatGPT, Perplexity, etc.)<br/> "]
CUSTOMER[" <br/>Customer<br/>(API / Dashboard)<br/> "]
AGENT ---> R53
CUSTOMER ---> R53
subgraph DNS[" DNS "]
R53[" <br/>Route 53<br/>api.rendel.app<br/> "]
end
R53 ---> CF
subgraph CDN[" CDN "]
CF[" <br/>CloudFront<br/>Accept header in cache key<br/> "]
ACM_CF[" <br/>ACM Certificate<br/>*.rendel.app<br/> "]
end
ACM_CF -.-> CF
CF ---> APIGW
subgraph GATEWAY[" API Gateway (HTTP API) "]
APIGW[" <br/>Routes<br/>GET / · POST /ingest · GET /ingest/{id}<br/>DELETE /ingest/{id} · DELETE /purge<br/>GET /usage · GET /health<br/>GET /llms.txt · GET /llms-full.txt<br/> "]
end
APIGW ---> API
subgraph APILAMBDA[" API Lambda · 512 MB · 30s · container image "]
API[" <br/>Go Handler<br/>auth · negotiate · cache check<br/>render on miss · metering<br/> "]
LW1[" <br/>LambdaWatch layer<br/> "]
end
API --->|"enqueue URL batches<br/>(10 URLs per message)"| SQS
subgraph MESSAGING[" Messaging "]
SQS[" <br/>SQS Standard Queue<br/>visibility timeout: 120s<br/> "]
DLQ[" <br/>SQS Dead-Letter Queue<br/>maxReceiveCount: 3<br/> "]
end
SQS ---> WORKER
SQS -.->|"3 receives failed"| DLQ
subgraph WORKERLAMBDA[" Worker Lambda · 1024 MB · 120s · container image "]
WORKER[" <br/>Go Handler<br/>Chrome render · Readability extract<br/>HTML→Markdown · frontmatter<br/> "]
CHROME[" <br/>headless-shell<br/>~200 MB · 3 concurrent tabs<br/> "]
LW2[" <br/>LambdaWatch layer<br/> "]
end
CHROME -.-> WORKER
API ---> SITE
WORKER ---> SITE
SITE[" <br/>Target Websites<br/>User-Agent: Rendel/1.0<br/>max 5 concurrent per domain · 1s crawl delay<br/> "]
API ---> S3_MD
WORKER ---> S3_MD
WORKER ---> S3_META
subgraph S3[" S3 Bucket "]
S3_MD[" <br/>cache/{domain}/{sha256}.md<br/>rendered Markdown<br/> "]
S3_META[" <br/>cache/{domain}/_meta/{sha256}.json<br/>title · description · word_count<br/> "]
S3_LLMS[" <br/>cache/{domain}/_llms.txt<br/>cache/{domain}/_llms-full.txt<br/> "]
end
S3_META ---> S3_LLMS
API ---> DDB_KEYS
API ---> DDB_METER
API ---> DDB_JOBS
WORKER ---> DDB_JOBS
subgraph DYNAMO[" DynamoDB "]
DDB_KEYS[" <br/>API Keys table<br/>PK: key_hash (SHA-256)<br/>plan tier · rate limit · active<br/>LRU cached 60s in Lambda<br/> "]
DDB_METER[" <br/>Metering table<br/>PK: api_key · SK: period (2026-03)<br/>renders_used · serves_used<br/>atomic ADD counters<br/> "]
DDB_JOBS[" <br/>Jobs table<br/>PK: job_id · GSI: api_key<br/>status · total · completed · failed<br/>atomic INCREMENT counters<br/> "]
end
LW1 -.->|"batch + gzip"| LOKI
LW2 -.->|"batch + gzip"| LOKI
subgraph OBS[" Observability "]
LOKI[" <br/>Grafana Loki<br/>labels: function_name · region · service_name<br/> "]
GRAFANA[" <br/>Grafana<br/>dashboards · alerts<br/> "]
CW[" <br/>CloudWatch<br/>Lambda metrics · API Gateway metrics<br/>SQS queue depth · DLQ alarm<br/> "]
end
LOKI ---> GRAFANA
APILAMBDA -.-> CW
WORKERLAMBDA -.-> CW
SQS -.-> CW
subgraph DEPLOY[" Deployment "]
ECR[" <br/>ECR<br/>rendel:latest<br/>Go binary + headless-shell<br/>multi-arch (arm64 / amd64)<br/> "]
SAM[" <br/>SAM / CloudFormation<br/>template.yaml<br/>infra-as-code<br/> "]
end
ECR -.->|"image source"| APILAMBDA
ECR -.->|"image source"| WORKERLAMBDA
SAM -.->|"provisions"| GATEWAY
SAM -.->|"provisions"| APILAMBDA
SAM -.->|"provisions"| WORKERLAMBDA
SAM -.->|"provisions"| MESSAGING
SAM -.->|"provisions"| DYNAMO
SAM -.->|"provisions"| S3
classDef edge fill:#d9e8f5,stroke:#3f6f94,color:#132433,stroke-width:1.5px;
classDef compute fill:#e7f3e7,stroke:#4d7b52,color:#18311b,stroke-width:1.5px;
classDef state fill:#f3e4d7,stroke:#8a5a2b,color:#301d0f,stroke-width:1.5px;
classDef external fill:#f6f3ea,stroke:#9c7a2b,color:#2b2416,stroke-width:1.5px;
classDef observability fill:#f0e6f6,stroke:#7b4f8a,color:#2d1638,stroke-width:1.5px;
classDef deploy fill:#e8eef5,stroke:#4a6785,color:#1a2a3d,stroke-width:1.5px;
class AGENT,CUSTOMER,SITE external;
class R53,CF,ACM_CF,APIGW edge;
class API,WORKER,CHROME,LW1,LW2 compute;
class S3_MD,S3_META,S3_LLMS,SQS,DLQ,DDB_KEYS,DDB_METER,DDB_JOBS state;
class LOKI,GRAFANA,CW observability;
class ECR,SAM deploy;