Project Ideas : Immerse The Bay 2024
Thing out loud about project ideas for the event this week: https://immersethebay.stanfordxr.org/
About Me
I work with startups in the US on mobile and embedded systems oriented projects, with a focus on IoT and Medical devices.
In Japan, I am looking to build products that intersect AR/AI/IoT
Ideas
Here is a short list without filter. I have hardware for the various project ideas.
Q-task AR - a task sprint overlay (rissoto timer style) / focus mode
Concept is to build a task timer to allow one to go into focus mode for a fixed amount of time, then switch to the next task, until all tasks are burned down. The unique aspect of this is to have it be "offline first", and just be a passthrough AR experience with some overlay for notifications and indicator of what to focus on.
Concept notes: https://1drv.ms/i/s!AuIL2jCvMCqRjKlHM9szpJfSZfeBZQ?e=1zLcBo
Book2.0 - a rethinking of books in XR
The concept is to go deep and explore alternative immersive experiences with books. Ideas:
- every book has its own server
- every page of a book has an AR marker
- immersive sound
- built in readers
- sky book liner "annotation"
Requires:
- XR skills
- design thinking mindset
- ability to ingest text content
- audio sound design
ConkyXR - a Conky experience available in XR
See: https://github.com/brndnmtthws/conky for the inspirations and background. Design approaches:
- use a raspi in the background
- serve the visuals in three.js or other, in a mix reality experience
- target client is XR compatible
Requires:
- designer or artist
- UI widget designer
- network protocols
- linux
PANKitXR - a personal area network for XR applications
The concept here is to take the great learning from ecosystems like Meshtastic, and build portable mesh networks that can work with any AR/XR capable device, allowing people to build:
- P2P communications servers, think an office network
- P2P gaming, ad-hoc
- P2P emergency services networks
Instead of using expensive ($40+) meshtastic devices, use low cost XiaoESP32C6 or M5NanoC6 devices (both around $7). These give us two tech stacks. We get Zigbee, and we get Wifi6.
What we can do with that:
- Zigbee gives us a mesh network design that is flexible, and secure enought, with support for a theoretical 254 nodes.
- the WIFI6 gives us an ultra fast standard protocol for communication
Design would involve:
- PanKit device: the MCU ESP32C6 will be designated as the PanRouter or PanController, and will serve a Web Server over Wifi, and and act as a router or controller for the Zigbee net
- each host device (PC or HMD or Phone) pairs up with the WIFI AP served on the PanKit device
- the host device subscribes to updates using some standard notification protocol (either web based, UDP, or other)
- host displays activity on the PANKit network in a HUD
Application ideas:
- a simple Team server (send pictures, messages, location sharing)
- a simple game server
- an AR art experience server, where every PANKit node can serve its data via its own web server, and also interact over IoT protocols with other zigbee devices (great for lighting)
Limitations
- high latency possible, so not great for a FPS
- range of Zigbee varies, the more nodes involved, the stronger the network becomes though. Somewhere between 0-100m is possible
- These devices don't have BLE
Challenges
- Zigbee protocol is fairly new on the ESP32 ecosystem, and a bit difficult
- The alternative that can be used is Matter on top of Thread. This is even newer and perhaps the most difficult to start up.
- There may not be enough PSRAM to store the server and a web server and AP code.
Requires:
- network protocols
- some basic XR skills based on web stack
- willingness to ideate
OpenWearXR
Wearable devices configurable via XR UI. Requires:
- ble knowledge
- ability to prototype simple but elegant UX in XR
- arduino skills
- hardware
I would like to be able to have pop up interfaces to configure all of my wearables without going into a phone app for each device.
Concept is being explored in open source, a write up here: https://github.com/IoTone/OpenWR/issues/7
My Approach
- Design thinking / ideation
- small starwman prototypes
- cross functional team
Skills Available
I can code in:
- swift
- flutter
- c/c++
- python
- node.js
Generally speaking, I can do the low level MCU coding, IoT protocol work, and mobile apps. I have done a limited amount of work on the AR side so am looking for help in this realm.
Skills Needed
See the projects list!
DM me on Linkedin, Discord, or email.
What's Next
I am looking for collaborators on AR/XR experiences to build in Japan. I am also looking to build products around Open Hardware and OSS with a focus on Japan as a market. Please follow up with me if interested.