Inspiration
Was finishing up my master and while learning about networking, I read about protocols like HTTP and gRPC and how they are layered on top of TCP. Most explanations focus on how to use these protocols rather than how they are implemented. Saw this hackathon listing and thought would be great opportunity to implement those theoretical knowledge I had. As per my knowledge, TCP only provides a byte stream with no message boundaries, I wanted to understand how higher-level protocols solve this problem by building one from scratch.
What it does
Custom TCP Protocol implements a simple binary application-level protocol directly over raw TCP. It defines its own message format, framing, parsing logic, state rules, acknowledgements, and error handling without relying on HTTP, gRPC, or any protocol libraries.
The protocol supports:
- A CONNECT handshake
- DATA messages with acknowledgements
- Explicit error messages for invalid protocol usage
- Per-connection state enforcement
How I built it
First I skimmed through multiple youtube videos and documents, took help of LLMs for understanding what can actually be built. Then I started with a minimal TCP client and server to observe how data arrives as arbitrary byte chunks. Based on this, I designed a binary wire format with a fixed-size header and a variable-length payload.
I implemented:
- A framing layer to construct protocol messages
- A parser that buffers incoming bytes and reconstructs complete messages safely
- State machines to enforce valid message ordering
- Structured ACK and ERROR responses for predictable behavior
All protocol logic is separated from the transport layer, with the client and server acting as thin wrappers around the core protocol implementation.
Challenges I ran into
Handling partial reads correctly was the biggest challenge. A single TCP read can contain part of a message or multiple messages, so buffering and parsing had to be carefully implemented. Designing the protocol to remain minimal while still supporting handshake, data transfer, and error handling also required careful tradeoffs.
Accomplishments that I'm proud of
- Built a complete binary protocol from scratch on top of raw TCP
- Correctly handled stream-based parsing and message reconstruction
- Implemented state machine enforcement instead of relying on assumptions
- Added explicit acknowledgements and structured error handling
- Kept the protocol logic cleanly separated from the transport layer
What I learned
This project helped me understand:
- TCP does not preserve message boundaries
- Buffering is required to reconstruct messages correctly
- Protocol correctness depends on state enforcement
- Explicit error handling improves reliability and debuggability
Implementing these concepts directly made their role and importance clear.
What's next for Custom TCP Protocol
Possible next steps include:
- Supporting multiple concurrent clients
- Building a small visualization or logging tool for debugging
Log in or sign up for Devpost to join the conversation.