Inspiration

The proverbial "Canary in the Coal Mine" is more than just a myth. Throughout human history, birds have long been a bellwether for a wide range of environmental phenomena, ranging from everyday weather to brewing environmental catastrophe. With their wide-ranging habitats and lengthy migratory patterns, birds are often exposed to a much larger cross-section of the ecosystem than many other animals, and reside high enough in the food chain that they can be adversely impacted by environmental contamination in ways that other species are not.

Monitoring the everyday patterns and reproductive lifecycle of birds across the ecological spectrum can provide essential data for understanding the true health of our ecosystem.

Today, some efforts have been made to conduct prolonged, distributed studies of birds across the world, but these labor-intensive studies are naturally limited in scope and effectiveness, as they involve volunteers carefully walking or hiking to nests multiple times a day to make observations. Modern IoT technology makes it possible to automate and widely expand these essential studies in a manner that involves and educates local communities across the world--an opportunity that Canary was built to fulfill.

What it does

Canary is a low-cost automatic wildlife observation system, designed as an educational kit for students, bird enthusiasts, and researchers. Each Canary base station contains a high-definition camera with day and night vision capability (including infrared illuminators that are invisible to birds), an array of attachable sensors like thermocouples and strain gauges, and a web server that streams video and automatically gathered statistics to users on a local area network. Canary is intended to be mounted to the roof of a nesting box, birdhouse, or other observation site, and continuously captures video and other data from its feathered companions. Video and sensor data is processed on-board Canary in order to extract relevant events and summary statistics. At the micro scale, this means that users can play back amazing bird moments they may have missed, and view the history of the bird family building its life in their own yard. At the macro scale, distilling hundreds of hours of video into concise summary statistics and relevant video frames allows researchers to keep tabs on large populations of birds without consuming large amounts of bandwidth or cloud computing resources.

Canary is designed from the ground-up to be a plug-and-play solution that is extremely accessible to consumers and students.

The device runs on Power Over Ethernet (PoE), requiring only a single outdoor-rated Ethernet cable (easily had at your local Home Depot) for both its data and power connections. Once Canary is connected to a user's Local Area Network, no further configuration is required! Anyone on the network can view the video stream and other webpages in their internet browser, and if an Internet connection is available (and the user opts in), Canary will forward summary statistics and a small subset of images to the cloud to be added to public research data. With IP65 waterproofing on all of its components, Canary is built to last in an outdoor environment for years, in climate conditions across the globe.

Adding an IoT device to one's personal space can raise privacy concerns, even outdoors. Canary respects users' privacy by design. Canary base stations have no integrated GPS, and instead derive rough location data from their IP address, thereby limiting the ability to pinpoint users' locations from publicly available datasets. Additionally, Canary base stations are intended to be mounted to an enclosed observation space with only an internal field of view, thereby reducing the possibility of unwanted backyard video recordings.

How we built it

Canary's hardware is built around a Raspberry Pi 4 single board computer that runs all of its data acquisition, web services, and data processing. A CSI camera module with infrared illuminators was used as the video capture device, and two waterproof thermocouples allow continuous measurement of temperature inside and outside the attached birdhouse. A PoE hat provides power to the Raspberry pi over the attached Cat5e ethernet cable, which is routed through a window with a flat cable and attached to a PoE Ethernet switch indoors. Canary is assembled into an inexpensive and compact waterproof case with the use of a 3d-printed bracket that allows the camera module to be stacked beneath the other electronics. The whole enclosure is intended to fit atop a birdhouse we built with these plans from Lowe's, which requires only a single wooden board, nails, and some glue. External electrical connections are gasketed and properly strain relieved with the use of cable glands, ensuring environmental robustness and a long service life.

Canary's software stack is built on top of a data processing pipeline written in Python, which collects data and performs inference to capture relevant events and distill summary statistics. This pipeline writes video frames and sensor data into a REDIS key-value store, that acts as an inter-process communication layer between the Python backend and the web server, which is written in Node.js using the Express framework. While it adds some amount of overhead in inter-process communication requirements, utilizing a Python data pipeline allows the use of an unrivaled scientific computing ecosystem through packages like numpy, OpenCV, Pytorch, Tensorflow, scipy, and the like.

The Express server pulls video frames and data from the REDIS server and uses them to populate information in web pages that it serves to the user. Streamed, high-frequency data like the video feed and temperature recordings are emitted via SocketIO websockets, allowing users to see a live feed of values and images being recorded in real time.

Client-side webpages are coded in Node.js, with template generation using Jade/Pug and styling using CSS.

Canary makes extensive use of the Esri API and features in order to visualize huge amounts of real bird data. Although Canary has not yet gathered any real nesting or bird sighting data, efforts such as NestWatch and iNaturalist have made some datasets available to the public. These datasets were processed and integrated, and then combined with a hosted a custom feature layer on Esri in order to create a large, interactive map of bird sightings. Additionally, Esri's capability to do SQL querying on a feature layer was utilized to enable users to highlight specific bird species, allowing them to better understand which nesting species might be present in their area. This knowledge is essential for constructing a good nesting enclosure, as many different species of birds have different requirements for a healthy nesting enclosure.

Challenges we ran into

This was our group's first big project using Javascript, and there was a steep learning curve for setting up a webserver and getting it to serve content. We were more familiar with Python, but had not used REDIS before, and faced another set of hurdles in attempting to connect our backend data pipeline with our "frontend" Express server--and then yet another set of hurdles in attempting to connect our "frontend" server with the local web clients on users' machines. Many hours were spent publishing data to the wrong keys in REDIS, editing webpages without realizing that our local browsers hadn't emptied their caches and weren't reflecting changes, and combing through mountains of documentation to find the Node packages we needed. The Node.js package ecosystem is seemingly infinite, which is very much a double-edged sword. There is a package for everything, but sometimes adding a package just makes things more complicated.

Accomplishments that we're proud of

As of the writing of this devpost entry, everything works! We're very excited that our hardware is functioning, our processes are reliably glued together with a key-value store, and that we have a robust, low-latency livestream from the birdhouse. This project has extremely solid technical foundations, and we hope to build on them even further in the future.

What we learned

We learned how to build an Express server application from scratch, and how to glue a wide array of processes together both on a single device (using a publisher-subscriber key-value store) and across the web (using websockets). We learned how to display live-updating charts reflecting sensor values from our hardware, and do advanced visualizations using Esri querying. We're excited about all the new projects we'll be able to tackle with these skills!

What's next for Canary

Canary is getting installed in our backyard for the upcoming nesting season! We've port-forwarded our local webpage so that anybody can watch to see if we get some birds (the birdhouse is designed for House Finches). You can watch the live feed here!

Share this project:

Updates