Created
August 19, 2025 18:35
-
-
Save charl-kruger/c3355fd1b029ed41de76976cf8e92c7e to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Ready to code? │ | |
| │ │ | |
| │ Here is Claude's plan: │ | |
| │ ╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ | |
| │ │ FASHN WhatsApp Agent with Cloudflare Agents & Scheduled Tasks │ │ | |
| │ │ │ │ | |
| │ │ Architecture Overview │ │ | |
| │ │ │ │ | |
| │ │ Integrate the existing agents-fashn app with WhatsApp Business API, utilizing Cloudflare Agents' Schedule tasks for efficient FASHN API │ │ | |
| │ │ polling. │ │ | |
| │ │ │ │ | |
| │ │ Core Components │ │ | |
| │ │ │ │ | |
| │ │ 1. WhatsApp Integration │ │ | |
| │ │ │ │ | |
| │ │ - Webhook Handler: Process incoming WhatsApp messages and media │ │ | |
| │ │ - Media Processing: Download images via WhatsApp Business API │ │ | |
| │ │ - Response System: Send text messages, images, and reactions back to users │ │ | |
| │ │ │ │ | |
| │ │ 2. User State Management │ │ | |
| │ │ │ │ | |
| │ │ - Profile Storage: Store user model images in Cloudflare R2 │ │ | |
| │ │ - Session State: Track user workflow (awaiting profile pic vs. processing garments) │ │ | |
| │ │ - Persistent Data: Use R2 for user preferences and history │ │ | |
| │ │ │ │ | |
| │ │ 3. AI-Powered Choice System │ │ | |
| │ │ │ │ | |
| │ │ When user sends an image, use OpenAI GPT-4 to: │ │ | |
| │ │ - Analyze the image type (person for profile pic vs. garment/background) │ │ | |
| │ │ - Suggest appropriate FASHN operation │ │ | |
| │ │ - Guide user through the three use cases │ │ | |
| │ │ │ │ | |
| │ │ 4. FASHN API Integration with Scheduled Polling │ │ | |
| │ │ │ │ | |
| │ │ Three Use Cases: │ │ | |
| │ │ 1. Profile Picture Storage: Store in R2 for Virtual Try-On model swaps │ │ | |
| │ │ 2. Model Variation: Create variations of existing fashion model images │ │ | |
| │ │ 3. Background Change: Replace image backgrounds while preserving subjects │ │ | |
| │ │ │ │ | |
| │ │ Polling Strategy with Schedule Tasks: │ │ | |
| │ │ - Start FASHN job → Get prediction ID │ │ | |
| │ │ - Schedule recurring checks: this.schedule(3, "pollFASHNStatus", {jobId, userId, attemptCount}) │ │ | |
| │ │ - Auto-reschedule until completion/failure (max 12 attempts = 36 seconds) │ │ | |
| │ │ - Handle rate limits: 50 status checks per 10 seconds │ │ | |
| │ │ │ │ | |
| │ │ 5. Enhanced Tools Architecture │ │ | |
| │ │ │ │ | |
| │ │ - uploadImageToR2: Store profile pics and garments │ │ | |
| │ │ - startVirtualTryOn: FASHN Try-On v1.6 API │ │ | |
| │ │ - startModelVariation: FASHN Model Variation API │ │ | |
| │ │ - startBackgroundChange: FASHN Background Change API │ │ | |
| │ │ - pollFASHNStatus: Check job status with exponential backoff │ │ | |
| │ │ - scheduleStatusCheck: Use Cloudflare Agents scheduling │ │ | |
| │ │ - sendWhatsAppMessage: Send responses back to users │ │ | |
| │ │ │ │ | |
| │ │ Implementation Steps │ │ | |
| │ │ │ │ | |
| │ │ Phase 1: Core Infrastructure │ │ | |
| │ │ │ │ | |
| │ │ 1. Merge WhatsApp webhook handling into agents-fashn │ │ | |
| │ │ 2. Add R2 bindings for image storage in wrangler.jsonc │ │ | |
| │ │ 3. Create FASHN API integration tools │ │ | |
| │ │ 4. Implement scheduled polling with this.schedule() │ │ | |
| │ │ │ │ | |
| │ │ Phase 2: AI-Powered Workflow │ │ | |
| │ │ │ │ | |
| │ │ 1. Add GPT-4 image analysis for use case determination │ │ | |
| │ │ 2. Create conversational flows for each FASHN operation │ │ | |
| │ │ 3. Implement user choice confirmation system │ │ | |
| │ │ 4. Add error handling and retry logic │ │ | |
| │ │ │ │ | |
| │ │ Phase 3: Enhanced Features │ │ | |
| │ │ │ │ | |
| │ │ 1. Add user preferences and history tracking │ │ | |
| │ │ 2. Implement advanced FASHN parameters (quality modes, variation strength) │ │ | |
| │ │ 3. Add batch processing capabilities │ │ | |
| │ │ 4. Create analytics and usage monitoring │ │ | |
| │ │ │ │ | |
| │ │ Key Technical Details │ │ | |
| │ │ │ │ | |
| │ │ Scheduled Polling Pattern │ │ | |
| │ │ │ │ | |
| │ │ async pollFASHNStatus(data: {jobId: string, userId: string, attempt: number}) { │ │ | |
| │ │ const result = await checkFASHNStatus(data.jobId); │ │ | |
| │ │ │ │ | |
| │ │ if (result.status === 'completed') { │ │ | |
| │ │ await sendResultToUser(data.userId, result.outputImages); │ │ | |
| │ │ } else if (result.status === 'failed') { │ │ | |
| │ │ await notifyUserOfFailure(data.userId); │ │ | |
| │ │ } else if (data.attempt < 12) { │ │ | |
| │ │ // Reschedule next check │ │ | |
| │ │ this.schedule(3, "pollFASHNStatus", { │ │ | |
| │ │ ...data, │ │ | |
| │ │ attempt: data.attempt + 1 │ │ | |
| │ │ }); │ │ | |
| │ │ } else { │ │ | |
| │ │ await notifyUserOfTimeout(data.userId); │ │ | |
| │ │ } │ │ | |
| │ │ } │ │ | |
| │ │ │ │ | |
| │ │ FASHN Rate Limit Management │ │ | |
| │ │ │ │ | |
| │ │ - /v1/run: 50 requests per 60 seconds │ │ | |
| │ │ - /v1/status: 50 requests per 10 seconds │ │ | |
| │ │ - Max 6 concurrent requests │ │ | |
| │ │ - Implement queue system with schedule tasks │ │ | |
| │ │ │ │ | |
| │ │ Benefits of This Approach │ │ | |
| │ │ │ │ | |
| │ │ 1. Asynchronous Processing: Non-blocking FASHN operations │ │ | |
| │ │ 2. Reliable Polling: Cloudflare Agents scheduling handles failures │ │ | |
| │ │ 3. Scalable: Each user session is an independent Durable Object │ │ | |
| │ │ 4. AI-Guided: GPT-4 provides intelligent workflow guidance │ │ | |
| │ │ 5. Extensible: Easy to add new FASHN capabilities |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment