hackillinoiseagle

• 

  System Design

Inspiration

We were excited to work with open source projects and we liked the idea of Apache Eagle where we pretty much understood the working of a Storm-based application.

What it does

Eagle uses Storm for streaming data and Kafka for messaging. The idea is to develop a security event correlation platform for user to easily onboard new metric (topic) and model metric correlation by sensing the changes in groups of topics dynamically.

How we built it

We have built a new Storm topology and a new API to fetch the events. We created the routing logic in Spout and dispatcher logic in Bolt for the system to enable correlation of topics belonging to a group in a bolt.

Challenges we ran into

We had hard time setting up kafka and zookeeper environment on our machines. Also, as Apache Eagle was a giant code base involving HBase and dynamic policy frameworks, we had to design a simpler and feasible one for the hackathon. Also, we spent more time understanding the flow of the system.

Accomplishments that we are proud of

We managed to add some pieces of code to Eagle, which is a storm and kafka-based application. We are happy that we set our footprints on an Apache project.

What we learned

We learnt the end-end working and implementation of a distributed system and our mentor was really helpful and patient enough to answer our dumb questions throughour the hackathon. He gave us constant feedback about our design, which was really motivating.

What's next for hackillinoiseagle

Built With

• html
• java
• javascript