CorporateMapper

• 

  Overview of the map taken form a sample scan

• 

  A demo employee profile

Inspiration

All of the team members have a strong background in cybersecurity, so we wanted to do something in that field. We started thinking about common "mapped" concepts in cybersecurity and figured organisational hierarchies would be pretty cool to map out. Org charts do almost always exist within companies, and it should be easy to do from an outside perspective as well, with all the data that companies publish. Simply making an external passive recon tool that maps companies seemed quite simple, and wouldn't have much practical purpose, so we linked it to the usual goal of passive reconnaissance/OSINT, i.e phishing. From our research, a holistic phishing attack surface map for companies doesn't yet exist, and would be a great way for companies to evaluate their social engineering security posture, and also be an invaluable tool for pentesters/red teamers.

What it does

Taking only the company website as input, CorporateMapper returns the full organisational hierarchy of the company including employee profiles, individual phishing risk and holistic company phishing risk.

When a query is submitted, the crawler automatically parses the sitemap of the website, ranks all URLs by OSINT significancy, and pulls the most significant sites. It then searches those sites for employee profiles and pulls them down into the neo4j database. When a profile is chosen to be enriched, several google dorks are run using their personal details, email and phone number. The most relevant results of the dork are pulled, and crawled with the same crawler, this time searching for mentions of the employee. Finally, all this information is collated and compared against pre-determined phishing risk factors to create a phishing risk score for the employee.

The corporate hierarchy is presented as a tree with each employee's profile visible in it. Clicking on an employee profile shows all of the personal details that were scraped including the sources from where they were pulled, allowing for easy information verification. If the profile lacks information from the initial crawl, the profile can be enriched using the secondary crawling previously described.

Besides the phishing risk score, we've also included a recent activity tab. This lists all significant events which employees of the company have have had, which could allow users to think of more phishing paths. For example, if an executive is currently in Europe for a work trip, executive impersonation phishing emails would be more successful as it is harder to contact the executive to verify legitimacy of emails.

It almost searches for leaks that the company and scraped emails have been in.

How we built it

Initially, we spent a lot of time planning out how the crawlers would work. We decided that the crawler methodology was the most important part of the project, so we spent time mapping it out on whiteboards. Then, we split up into three of us making the backend, and one making the frontend. None of us really had frontend experience, so most of the heavy lifting was done using AI on that end. As for the backend, most of the crawlers were done half-half, with AI providing the the syntax for web scraping libraries which we hadn't used, but with humans controlling the design of the program. As we reached the deadline, we started implementing more semantic things, such as the phishing risk factors, information synthesis from scraped sites, and employee profile collation. The definition of risk factors was almost entirely done by hand, with the data for it being taken from research papers on phishing, whilst most of the information collation and synthesis was delegated to the Gemini model with the highest free daily limits.

Challenges we ran into

We ran into numerous challenges throughout the span of the project. To start with most of use were unfamiliar with a graph based databased only having major experience with relational database languages like MySQL. Once becoming familiar we faced a lot of roadblocks from rate-limiting, to merge conflicts there were challenges almost at every turn. 2 of the most prominent roadblocks that needed to be overcome was AI safeguards and anti scraping measures. An important feature in our app was to give context to the data that we scraped, to do this an LLM was the most effective way to do it. However, due to ethical safeguard passing in the data directly wasn't yielding any results so we had to settle for proof of concept focused on fake data. This can be circumvented with a local model down the line. Another issue that we faced was during the scraping phase, google dorking would block after a few requested. to circumvent this we used google server API's, rate limiting, and pre-existing scraped databases.

Accomplishments that we're proud of

We are proud of building an application that can ingest information from multiple sources, process and structure that data, and represent relationships in a meaningful way through a graph-based system. A major accomplishment was designing the project so that different components, including data collection, risk scoring, backend processing, and the user interface, could be developed independently and then integrated into a single application. This forethought allowed us to all work independently on each module and was in keeping with good OOP design principles.

What we learned

Throughout this project we learnt a lot of valuable lessons. We learnt about circumnavigating rate limits, anti-botting protections and we learnt and we learnt about effective and efficient data ingestion. We also got experience with graph data-bases and strengthened out knowledge on systems design while designing this application. One of the main lessons we learn directly outside of our app is managing GitHub workflows and merges while working in a team. Separately from our application we also learn about phishing risks while researching risk factors for our risk matrix that used to categorise the likelihood an employee will be phished.

What's next for CorporateMapper

Next up are simply adding more features as the MVP works. This involves making the data ingestion more robust and ensuring data accuracy. For real world applications we would need to remove the safeguards on real world data and apply the demo pipeline through the real data pipeline. For overcoming safe guards local models should be used.

Other features partially implemented but not yet fleshed out are:

• robust linkedIn profiler
• 2 degree person finder using social media + google dorking

Features we haven't added yet are:

• Phishing type recommendation
• Training packs / alerts for employees at high risk

Built With

• neo4j
• python
• react
• vite