Post

Over December I’ll be posting 100 fun insights in this thread from working on @triplex.dev in my free time for two (!) years. It’s the longest and largest side project I’ve worked on. Keep an eye out, will be a fun one! #threadapalooza #threadapalooza2024

1. Triplex is a "visual IDE" for React/Three Fiber. For visual-first dev use transforms and prop controls to set prop values while leaning on features like undo and play controls. For code-first dev it's similar to Storybook but with inbuilt dev-time features like helpers and component controls.

2. So when clicking on scene objects Triplex takes a component-first approach, meaning it prioritizes components over host elements like <mesh>. Imagine having a <House> component in your scene and you want to change its position, but clicking on it you'd select its doors or windows. Weird right?

3. Okay easy, you might be thinking. But I found this to be a Hard Problem! It took two main revisions to get to a resilient solution but both leaned on one fundamental constraint: the need to inject metadata into the scene so we can tie scene objects back to elements and components. The first...

4. ...relied on injecting metadata by wrapping every element in a Three.js group. Think of it like a div with display: contents set. Each group would have data on where the element is in code and what component owns it and we could traverse up and down the scene tree to resolve it. This...

5. ...worked great, even for components from node_modules! When clicking on a scene object we'd grab this metadata and then traverse up the Three.js scene until we found an element that was inside the open component inside Triplex. But that architecture didn't last the test of time.

6. It turns out wrapping ALL objects in groups isn't valid Three.js code so exceptions for objects that shouldn't be wrapped began. When users ran into more edge cases the errors were NOT helpful, and getting errors only inside Triplex was super confusing to everyone! Time to re-think the problem.

7. So we need a solution that doesn’t fundamentally change the Three.js object structure. What can we do? Well we need to zoom out. What levers do we have available to inject data? We have… (1) The React tree via context; and (2) The Three.js tree So these + leaning into the constraints above…

8. … we can get somewhere. So instead of storing all the metadata in a Three.js scene we only store against host elements as an extra field on the object, and for components we store in React context. Then it’s a matter of on raycast of an object look up the context tree for its component owner. 👌

10. Let’s talk about selection outlines. When an object is selected it has a yellow outline. It’s a custom post processing GLSL shader, not using the default Three.js pp nor pmndrs one. This prevents collisions. It’s less performant but it ensures if a user has pp enabled, it composes together!

11. What helped me crack the shader was this article io7m.com/documents/ou... Super useful if you're looking to do something yourself. During the frame loop when something is selected it is given a specific layer, I then exclusively render objects that have this layer only to a texture and use...

12. That as the mask texture. Then it's just a matter of, where edges are found, draw a pixel! It works well, especially for anything that has its object obstructed by another object. Have only had great feedback about how it! One tricky thing is antialiasing. For now the line doesn't have any.