Post

Over December I’ll be posting 100 fun insights in this thread from working on @triplex.dev in my free time for two (!) years. It’s the longest and largest side project I’ve worked on. Keep an eye out, will be a fun one! #threadapalooza #threadapalooza2024

1. Triplex is a "visual IDE" for React/Three Fiber. For visual-first dev use transforms and prop controls to set prop values while leaning on features like undo and play controls. For code-first dev it's similar to Storybook but with inbuilt dev-time features like helpers and component controls.

2. So when clicking on scene objects Triplex takes a component-first approach, meaning it prioritizes components over host elements like <mesh>. Imagine having a <House> component in your scene and you want to change its position, but clicking on it you'd select its doors or windows. Weird right?

3. Okay easy, you might be thinking. But I found this to be a Hard Problem! It took two main revisions to get to a resilient solution but both leaned on one fundamental constraint: the need to inject metadata into the scene so we can tie scene objects back to elements and components. The first...

4. ...relied on injecting metadata by wrapping every element in a Three.js group. Think of it like a div with display: contents set. Each group would have data on where the element is in code and what component owns it and we could traverse up and down the scene tree to resolve it. This...

5. ...worked great, even for components from node_modules! When clicking on a scene object we'd grab this metadata and then traverse up the Three.js scene until we found an element that was inside the open component inside Triplex. But that architecture didn't last the test of time.

7. So we need a solution that doesn’t fundamentally change the Three.js object structure. What can we do? Well we need to zoom out. What levers do we have available to inject data? We have… (1) The React tree via context; and (2) The Three.js tree So these + leaning into the constraints above…

8. … we can get somewhere. So instead of storing all the metadata in a Three.js scene we only store against host elements as an extra field on the object, and for components we store in React context. Then it’s a matter of on raycast of an object look up the context tree for its component owner. 👌

9. The neat thing about this architecture is it works for host elements and custom components of any abstraction depth. And you can move in and out of components to change what you selects It works in node modules! All without needing any setup either internal or otherwise. Really happy about it.