Color Tracking CAD Viewer

Inspiration

Back in 2007, I was a young design engineer at Caterpillar, working on CAD models of engine parts. It was neat and all, but I hated having to use my mouse + some fancy keyboard work to spin / move / scale models. It was both hands on the table all the time.

I dreamed of something more Minority Report—an interface where I could just flick the air in front of me and the model would move. Something like this:

Careful, Chief. Dig up the past, all you get is dirty.

While things have gotten a lot better (touch screens), I'd still like a lean-back interface where I could contemplate the world while effortlessly spinning my models.

And with CAD Viewer, I think I'm halfway (ok, 10%) there.

What it does

Color Tracking CAD Viewer is a JavaScript web app that lets you spin and rotate 3D STL models using color tracking. Using a magenta token—like my purple Sharpie— I can lean back in my chair and spin models by waving my hand around in front of my webcam.

The pen really is mightier

How I built it

CAD Viewer is a mashup of a three.js STL loader demo, the tracking.js color tracking demo, 2 STL models (finger wrench & TSA master key 001), and some very hacked together JavaScript.

Yes, the model on the right is a TSA Master Key. And no, you're no longer in control.

Initialization

First, I used dat gui to create a small interface for selecting a model and setting it's initial position, rotation, and scale. This allows you to make sure the models looks good on screen before you start playing with it. Three.js & some custom loaders (not written by me) take care of all the shading, mesh generation, and 3D stuff.

Dat GUI tho!

Color tracking

Using tracking.js, I setup a color tracker that targeted any magenta objects seen by my webcam. Using the tracker's x/y position, I continually adjusted the camera position:

// objX & objY are coordinates set by the color tracker
// this camera positioning occurs within the render() function, which is in turn called by animate(), which is a continuously updates.

camera.position.x = (objX - camera.position.x) * 0.001;
camera.position.y = (- objY - camera.position.y) * 0.001;

Challenges I ran into

Blerg. Munging all this JS into one useable app was a bit of a pain. I ran into some trouble getting the app to run smoothly, which is probably because I basically copy & pasted two tutorials together.

Originally, I had planned to use both color & head tracking. Colors to rotate the object, and head tracking to move the overall scene. I couldn't figure out why, but I think my color & head trackers were overwriting each other. ¯\_(ツ)_/¯

Also, I was planning on adding a "upload STL model" button, but I ran out of time.

And of course, there's still tons of unused / outdated code left in the comments + a heap of extra JS from the examples folder that I haven't cleared out yet.