StreetDex

StreetDex is an advanced dynamic search engine for financial securities, utilizing Burst-Suffix-Tree indexing, ML implemented by ARS, and Cross Entropy Loss to provide accurate and real-time search.

Comment

Inspiration

As a team with members having diverse interests including AI/ML, Algorithm Optimization and FinTech, StreetDex which built on the prompt given by Schonfeld seemed like the perfect fit for our team.

What it does

StreetDex is an advanced dynamic search engine which helps return search results for queries for Financial Securities and their Street Indexes. StreetDex utilizes a multitude of technologies including burst-suffix-tree indexing, machine learning utilizing ARS (Accelerated-Random-Search), and Cross-Entropy Loss (completely from scratch) for implementing the dynamic model to provide accurate and near real-time search results for massive datasets (containing more than 600,000 data points).

In essence, StreetDex is a search engine which not only takes into account all given initial parameters and provides fast search, but also adapts to user response for improving future results. Given the technology utilized by StreetDex and its base concept, it can also be modified and used as a general purpose search engine because of its underlying technology to provide fast search.

How we built it

Every team member had a different subject matter expertise and took ownership of different aspect of the project. This allowed all team members to work on the overall project and learn from each other. Team members took ownership of:

  1. Software Development
  2. Machine Learning
  3. Algorithm Development
  4. Designing

Challenges we ran into

  1. Utilizing a new programming language (Scala) to leverage its data structures and frameworks for this project.
  2. Co-ordinating merges and ensuring all aspects of the codebase were compatible
  3. Bug issues on the Frontend (React.JS) including promises
  4. Developing and optimizing algorithm repeatedly to fit use case
  5. Developing machine learning and apply it effectively to get meaningful improvements

Accomplishments that we're proud of

  1. Developing a dynamic search engine algorithm which improves over time.
  2. Developing and Implementing a Machine Learning model to dynamically improve the search results.
  3. Developing a Full-Stack Web-Application implementing the Search Engine
  4. Using a White Board

What we learned

  1. Use cases of special "Tree-based" data structures like "Burst-Suffix-Tree"
  2. How to develop Machine Learning algorithms from scratch based on use case
  3. How to use complex mathematical topics to optimize algorithms

What's next for StreetDex

  1. Implement Quadratic ARS for further optimization
  2. Deployment of the search engine and utilizing serverless technologies for storage
  3. Implement input interval based POST request to optimize search

Built With

Share this project:

Updates