A technician alone on a job with something they have never seen before. Sidekick puts an AI on the glasses they own, and calls a human only when it has to.
A field technician is standing in a plant room somewhere, alone, looking at a unit they have never serviced before. Something is wrong with it. The business is losing somewhere between three and five hundred dollars for every hour it stays wrong.
Their options, right now, are these. Phone a colleague and hope they pick up. Search YouTube one-handed while holding a torch. Or wait for a senior tech to finish their own job and drive out.
I have watched enough of this to be confident about the shape of the problem: none of those options are good, and the third one is the one that usually happens.
Sidekick is the thing I am building about it.
Be My Eyes, but the first responder is a machine
The clearest way to describe it: Be My Eyes, except the users are skilled trades workers and the AI answers first.
Four steps.
- Look at the problem. The glasses camera sees what the technician sees.
- AI responds instantly. Voice and display overlay. Hands stay on the job.
- A human expert is on standby. If the AI cannot handle it, a verified expert joins and sees exactly what the worker sees.
- Back to work. Solved on site.
Existing remote-assistance products are step 3 with nothing in front of it. They connect a worker to a human, which means the economics are the economics of human time, and human time does not scale down to the small stuff. Most of what stops a technician is not hard. It is unfamiliar. There is an enormous class of problem that is trivial for anyone who has seen it before and completely blocking for anyone who has not.
Putting AI first is what makes that class of problem addressable at all. The expert network then handles the genuinely hard remainder, which is a much smaller number of much more valuable calls.
The hardware decision, which is really a distribution decision
Sidekick runs on glasses the worker already owns.
That sounds like a product constraint. It is actually the whole strategy. The incumbents in this space sell proprietary headsets at $1,500 and up, which means every deal starts with a hardware purchase order and a procurement conversation, and that is why their sales cycles are measured in quarters.
Meta Ray-Bans are $300 and already on people's faces.
So the core is a web and Android application, deliberately hardware agnostic:
| Device | Status |
|---|---|
| Meta Ray-Ban | Primary development target |
| MentraOS devices | Roadmap |
| Xreal Air 2 | Roadmap |
| Jio Frames, Lenskart | Future |
Every new pair of glasses that ships becomes a distribution channel without me doing anything. Given that Indian manufacturers are about to put sub-$100 hardware into the market at volume, that seems like the right side of the bet to be on.
What it is built from
- Mobile app: Expo and React Native,
expo-camerafor live capture - AI layer: GPT-4o Vision for real-time image understanding
- Escalation: snapshot streaming from the phone to an expert dashboard
- Expert dashboard: a web interface where verified experts receive and handle calls
- Backend: Node.js and Socket.io, with long polling for low-latency signalling
The escalation path is the part I find most interesting to build, because the handoff has to carry context. An expert who joins cold and has to ask "so what am I looking at?" has already burned the advantage. They should arrive knowing what the worker asked, what the AI said, and what it was looking at when it said it.
Why this is the moment
Every piece of this was impossible two years ago, and all of them arrived at once:
- Meta Ray-Bans crossed into the mainstream at a consumer price
- MentraOS launched an open platform for glasses apps
- GPT-4o Vision made real-time visual understanding actually work
- Sub-$100 hardware is coming
None of these were things I made happen. The useful observation is just that the infrastructure landed, and the application layer on top of it is mostly empty.
There is also a demographic clock running. The average tradesperson is 45 or older. The knowledge that makes an experienced technician valuable is walking out of the industry every year, and almost none of it is written down anywhere. A system that captures what an expert says, in the moment, while looking at the thing, is a system that could hold on to some of it.
Where it actually is
Building, and in customer discovery. I am crossing the US in a campervan talking to the people who would use this, which is not a metaphor for anything: it is the cheapest way I could find to have a lot of conversations with technicians in the places they work.
The beachhead is HVAC and electrical in the US Northeast. If it works there, the same loop applies to anyone who needs expert guidance in real time, and that list gets long quickly: elderly care, field medicine, remote sites, eventually people at home.
I have been circling this problem for about a decade without meaning to. My PhD was on remote collaboration in mixed reality, sharing gaze and gesture and reconstructed scenes between people who are not in the same room. Wearable RemoteFusion, Bringing 3D to Zoom, the whole remote collaboration series: all of it is the same question of how you put one person's understanding inside another person's situation.
The research answer to that question was always beautiful and always required hardware nobody owned. The thing that changed is not the idea. It is that the hardware is now $300 and already on people's faces.
Hermes Glasses, a separate open-source project of mine, comes at the same hardware from the other end.