The Project

Every day, we are inundated with a flood of information—from university feedback and conversations with friends to everyday tasks. We store this information in various places—on computers, in apps, or notebooks—and often forget that we have it or where to find it. This is where Spot comes in. Spot is a tool that helps you consolidate your thoughts in one place, making them available whenever you need them. The vision is that, regardless of the form your thoughts take, Spot is the tool that allows you to access them.

This course was about discovering emerging technologies and how they can be used to solve problems. We focused on virtual reality and augmented reality, which are rapidly advancing fields. We defined a szenario that could be true in 5-10 years.

Ideation

How can we use spatial computing to store and access ideas? After extensive brainstorming, we developed the concept of storing ideas spatially at locations or on physical objects. In our case, the data storage unit is a Spot, essentially an animated particle sphere that can be digitally stored at a physical location. This data storage should be accessible using AR and VR technology.

Interface

After some exploration, we recognized the need for a unified interface to ensure users can effectively manage their Spots from a single dedicated location. The challenge was to create an interface that is both intuitive and easy to use, allowing users to remain fully engaged with the real world.

To address this, we developed an interface that can be controlled via gestures and voice commands. Users can manage all their Spots through one dedicated dashboardinterface. The creation process is designed to be as minimalistic as possible. Userssimply see their Spot along with a Prop-Bar, which enables them to customize their Spot. After applying specific props , these will appear next to the Spot. Users can then adjust values to tailor the Spot to their specific use case.

Prototyp

Because we live in a world where this technology is not yet available, we decided to create a prototype for the new Apple Vision Pro. This way we could make this vision experienceable for our users, in the best possible way.

Our aim was to create an interactive prototype which makes our idea of Spot touchable for users.

Hand/- and Collisiontracking

We used HandTracking and Collision Detection to create a 3D environment where the user can interact with the Spots.

  @MainActor class HandTrackingViewModel: ObservableObject {
    private let session = ARKitSession()
    private let handTracking =  HandTrackingProvider()
    private let sceneReconstruction = SceneReconstructionProvider()
    public var contentEntity = AnchorEntity()  // Initialize as AnchorEntity
    private var meshEntities = [UUID: ModelEntity]()
    
    private let fingerEntities: [HandAnchor.Chirality: ModelEntity] = [
        .left: .createFingertip(),
        .right: .createFingertip()
    ]
    
    private var lastCubePlacementTime: TimeInterval = 1
    
    func setupContentEntity() -> Entity {
        for entity in fingerEntities.values {
            contentEntity.addChild(entity)
        }
        return contentEntity
    }
    
    
    func runSession () async {
        do {
            try await session.run([sceneReconstruction, handTracking])
        } catch {
            print("failed to start session (error)")
        }
    }}

Result

Spot is an exploration of how thought, idea, and knowledge management of the future could look. It leverages the power of AR and VR to create a way to manage thinking in a more intuitive way. It is a digital second brain that allows users to store and access their thoughts and ideas in physical space with all of the benefits of a digital infrastructure, such as collaboration, editable content, and location-independence.