This project drew inspiration from a couple different sources. We knew we wanted to do something with graphics, maybe a game, and we knew we wanted to go after the multimedia category. Our first idea was a first person shooter written completely in ncurses for the command line. After a few more iterations, we somehow landed on the idea of doing stuff with whiteboard markings, be it sending it into a notetaking app, or constructing some kind of game. We landed somewhere in the middle of that, and decided to make a tool to interpret whiteboard markings as terrain in a voxel engine we are very fond of: Minetest (no, not Minecraft, https://github.com/minetest), it's FOSS!
What it does
DrawTown lets you transfer sketches from whiteboard to the digital realm. It enables you to scan a pic, and send it into a voxel engine called MineTest, where terrain and buildings are generated on the fly for you to explore and tweak. DrawTown is a great way to quickly prototype new worlds, or expand an existing one.
How we built it
Under the hood, DrawTown uses OpenCV in C++ to scan a whiteboard photo. It then uses tech like contours and Hough circles to interpret markings into shapes. The dimensions of the shapes are sent over a network socket to a server running alongside the Minetest server. Using some cleverly adapted Minetest modding API calls in Lua, the dimension data retrieved from the socket is sent as World Edit commands to alter the landscape in near-real time using Minetest's schematic system. Data sent is interpreted as commands to create and position schematics in the world. No reloading, no relogging, and with just a few strokes of an Expo marker. It's like magic!
Challenges we ran into
I didn't have a ton of experience with OpenCV's C++ API. I had worked with it in Python, and had done other C++ programming, but the API was different enough that it took some getting used to both using it and researching information with it. Additionally, the functions and math I used to interpret strokes into useful data were a bit finnicky to get working at 3 in the morning. Despite this, I managed to come up with a pretty robust solution for interpreting data from the whiteboard into the computer.
My partner handled the server side, and they had the opposite problem in that the documentation was spotty, and some of the existing code needed to be patched or worked around. In the end, this didn't slow them down a whole lot, and they managed to do some pretty incredible stuff with the data I provided.
Accomplishments that we're proud of
It builds! It runs!
Joking aside, this concept is something I'm exicted about and it has a really impressive effect when seen in action. Plus, this project has HUGE potential for extensibility, and being completely GPL 3, it is there for the community to take advantage of. There are all kinds of strokes and schematics that could be added to it for further variety. Our project is GPL 3, so it's there for the community to take and tinker with.
What we learned
I learned a ton about edge detection, contours, data management in OpenCV, and about OpenCV's C++ API in general. It was something I've always wanted to go really in depth with, so I'm glad I could have the opportunity now. I also learned that I need new whiteboard markers :)
My partner learned a ton of stuff about server-side and game engine Lua scripting, which, coming from a background of writing Lua for embedded platforms, was really neat to see. They also did more stuff with Unix sockets and a few interesting command line tools for working with them.
What's next for DrawTown
We have a few goals in mind, moving forward:
- Add more schematics to increase the variety of terrain you can generate - Improve shape detection and normalization - Optimize for speed - Add support for more arbitrary 2D shapes (like paths and blocks)