Gravel

• 

  Hero Pitch

• 

• 

Inspiration

100% private computation is possible. When I discovered this 4 days ago, I knew I had to try and implement cyptographically private AI model inference. Everyone care's about data. People don't want to be spied on. Companies have proprietary information. The government has defense secrets. When everyone outsources their model inference to third parties, their data is in danger without extra precautions. Even if a company means well, this does not guarantee they aren't compromised. With Gravel's cryptographic protections, we can do model inference without leaking our data.

What it does

Gravel is a distributed system for parallel execution of homographic encryption secured programs. What the hell does that even mean? Image I have a super secret image. My life depends on the contents of this image and I need to be sure that this image does not get leaked. If I wanted to run a vision model on this image, I'd have to upload that image to a remote server. Consider my image compromised already. With Gravel, my image stays encrypted. From start to finish. The server running the AI model inference on my image can't even see my image. Yes, it's truly magical. Not only does Gravel support this private model inference, it accelerates it by providing the protocol for many machines to work together to parallelize the work, resulting in magnitudes of faster inference.

How I built it

Gravel is a conglomeration of MANY different technologies working together to make this possible. First, the example model is a bitwise neural network trained using pytorch and python. The parameters of this model are extracted and then executed in rust using inference I built on top of the "tfhe" homomorphic encryption library. These executions are coordinated via Deno scripts connected to Render web services as well Google's compute engine. Graphite is the combination of everything, the networking between servers, the secure inference, and the models trained for Gravel.

Challenges I ran into

Two major hurdles presented themselves over the course of this hackathon. Many libraries I tried to use for the homomorphic encryption were either non functional or too bare bones to support model inference. Only after about 18 hours did I finally manage to get the "tfhe" rust crate to work well enough that this project looked feasible. Not only was finding a library difficult, I've had to learn all about this technology on the fly as quickly as possible. The other major hurdle I faced was training a binary neural network for the first time. Many, many technical difficulties had to be overcome to figure out how to take this model trained in python and execute it in a secure rust runtime.

Accomplishments that I'm proud of

I am so proud that a real model is running privately on the Gravel platform. Truthfully I doubted and I would even get something close to working and many times I thought about how I would pivot if I couldn't keep pushing forward. Essentially I'm proud that I managed to pull of this ridiculous idea as a team of one.

What I learned

Of course I learned about homomorphic encryption this week, but I also learned how to orchestrate Google virtual machines. I learned how to deploy a web service on Render. I learned how bitwise nerual network work. It's truly remarkable what can be done in so short of a time.

What's next for Gravel

One of places where Gravel can be explored is speed. Right now all the model inference is on CPUs! Compared to the speedups we could get on GPUs or even specialized hardware for homomorphic encryption, Gravel has a lot of room to improve. I specifically want to research developing cryptographic accelerators on FPGAs. Homomorphic encryption is an active area of research and it's going to grow larger in the future as running these secured programs becomes more practical. Gravel is on it's way to explore this new paradigm of computing.

Built With

• fhe
• google-cloud
• python
• pytroch
• render
• rust
• typescript