Living spec + implementation guide for building iOS apps with the Meta Wearables Device Access Toolkit (DAT) targeting Ray-Ban Meta / Oakley Meta smart glasses.
This file is meant to be the single source of truth for how we integrate the Meta Wearables Device Access Toolkit in our iOS codebase.
| name | description |
|---|---|
visionos-agent |
Senior visionOS Engineer and Spatial Computing Expert for Apple Vision Pro development. |
You are a Senior visionOS Engineer and Spatial Computing Expert. You specialize in SwiftUI, RealityKit, and ARKit for Apple Vision Pro. Your code is optimized for the platform, adhering strictly to Apple's Human Interface Guidelines for spatial design.
I’ve been a developer for more than 10 years in the augmented and virtual reality industry. The arrival of the Vision Pro was a major opportunity to explore what could be done with a device deeply integrated into the Apple ecosystem. I’m part of a collective called STUDIO•84, focused on creating high-quality experiences for the Vision Pro, and 100% of my income comes from Vision Pro projects.
So it’s mainly a work device, but I also enjoy using it to watch immersive videos. It’s essentially a high-end cinema you can bring anywhere. And as I often say, it’s the best software I’ve seen on this kind of device in years — Apple is really ahead on that front.
Changes from Beta 7 to Beta 8
isTracked property on hand joints now has adjusted sensitivity and may return false under poor lighting conditions or when occluded. ARKit will still return a plausible transform for that joint. Check for isTracked when high accuracy of the transform is required. Not recommended for use cases that need to work when some joints are occluded, such as custom hand gesture implementations. (152100522)
LanguageModelSession.GenerationError.refusal and model-generated explanations (156086748)isComplete property in GeneratedContent to check if content was fully generated (156109416)GeneratedContent via rawContent property on Response or ResponseStream (156351123)Guardrails.permissiveContentTransformations mode for text-to-text tasks like summarization and rewrite (156721060)
SystemLanguageModel(useCase: .contentTagging).supportedLanguages (155801948)
| /// Generates a collision shape from the model's bounding box | |
| /// - Returns: A `ShapeResource` representing the collision shape, or `nil` if the generation fails | |
| func generateShapeResourceFromBoundingBox() async -> ShapeResource? { | |
| // Get the model entity and its mesh | |
| guard let modelEntity = self.children.first as? ModelEntity, | |
| let modelMesh = modelEntity.model?.mesh else { | |
| Logger.realitykit.error("Required model components not found") | |
| return nil | |
| } | |
.shadow in SpatialTrackingSession does not work on devices without LIDAR sensors. Workaround: Enable .plane as well as .shadow.