Topic: AI Workflow for Vision Pro Development
Speaker: Ivan Campos
Discussion centered on streamlining the development workflow for visionOS projects by integrating various AI tools and processes. The focus was on enhancing code extraction, error handling, and iterative example generation, while also exploring advanced AI models and complementary XR tools.
Action Items & Workflow Steps (LinkedIn Post)
-
Swift Code Extraction
Use a shell script to extract all Swift code from a project and pass this code as context into the prompt. -
Error Reporting
Paste all Xcode errors as conversational responses.
Include exact code line(s) associated with exceptions, when necessary. -
Deep Research Report
Create detailed research reports targeting advanced developers.
Include complete code snippets/examples.
Limit references to developer.apple.com for authoritative guidance. -
Base Project Setup
Develop a base visionOS project structure (app, content view, and immersive view) to serve as the initial context for discussions. -
Example Generation
Run a few examples in Xcode.
Pass these examples as context and request 20+ additional examples.
For proven functionality, simply request “20 more” to generate variations (this iterative step has been used to produce over 300 unique motion animations for ModelEntity objects). -
AI Model Utilization
Use the o1-pro model for complex projects requiring advanced reasoning.
Maintain access to the o3-mini-high model for quicker queries that GitHub Copilot for Xcode might not address. -
ChatGPT Customization
Customize responses for visionOS-specific constraints (e.g., “Never use ARView for visionOS as it is NOT supported”).
Adjust syntax guidelines for RealityView attachments.
- Detailed and cited real-time research reports
Example: Deep Research Report - Alternative: Perplexity Deep Research
- Objective-based responses that simplify complex tasks and tasks related to motion or dynamic animations.
Reference: L1GHTS - SpriteKit Side Scroller
Reference: SpriteKit Example - T3TRIS Project
Reference: T3TRIS
- Quick responses for less complex coding tasks.
- Optimal for scripting tasks (e.g., .sh, ffmpeg).
Alternative: LMStudio (with DeepSeek-R1-distill-qwen-7B)
- Handles one-liners and minor syntax corrections.
- ElevenLabs for sound effects and voice processing.
Alternative: MMAudio - TripoAI for 3D .usdz Models.
Alternatives: Meshy, Hyper3D
Open Alternative: TRELLIS (HuggingFace) - Udio for audio generation and manipulation.
Alternatives: Suno, Stable Audio, Riffusion - Midjourney for 2D image creation.
- Hailuo Minimax for 2D video generation.
Alternatives: Kling, Runway, Luma Labs, Sora - Blockade Labs for 3D Skybox image generation.
- Pixelmator for 2D image editing.
The meeting outlined a comprehensive AI-driven workflow for Vision Pro development, emphasizing automation in code extraction, error reporting, and example generation. The integration of specialized AI models (o1-pro and o3-mini-high) alongside GitHub Copilot and various XR-specific AI tools is aimed at optimizing the development process and ensuring adherence to visionOS best practices.