DomainLoom

The inspiration behind DomainLoom

Life is what happens while you're making plans

We've all been there. We finally decided on commiting some time to do something important, or to learn something new. It's important, and we want to do it right, so we plan.

We're commited, and we've set off to do it, just to be greeted by life demonstrating our plans aren't perfect, and more often than not, they're not even good.

The information we picked wasn't the best for us, but how can we know when there's so much of it out there? We're back to the drawing board.

Finding direction in a world cluttered with information

The old adage saying that you can't see the forest for the trees is still very relevant, but nobody would've expected it. Blazingly fast, we've traded having almost no access to any information, to being overwhelmed by it.

Access definitely isn't an issue anymore, but direction is. It is desperately needed to navigate the real world, following the compass of our dreams in the most expedient possible way.

DomainLoop is here to help.

Turning it around

The very abundance of information that's making our lives a bit overwhelming, along with some engineering and mathematical trickery, can be used to separate the wheat from the chaff, the knowledge from cluttered data, turning a hectic blind sprint into a calm adventure.

Machine learning, with its noticeable progress on a seemingly daily basis, provides an amazing opportunity to turn it back around. We could finally follow through with our plans, and even have fun exploring all the nooks and crannies along the way.

What it does

DomainLoom helps you figure out the best way to learn something, by asking and learning about you. It helps you visualise where you currently are, and all the steps you need to take to achieve the goal you've set for yourself using a web-like structure, just like a loom, showing you all the tiny fabrics of skill the domain is tailored from.

A key difference is that it also stays with you for the entire way, talking with you, allowing you to dive deeper and adjust that plan in the areas you're feeling unsure of, or are really interested in.

It will ask you questions, but it will provide better answers, with specific resources and recommendations, and the sources behind its recommendations.

We need expertise, in almost all fields, more than ever. Just like information, access to it should be abundant. DomainLoop allows you to lean on a domain expert, asking for their advice on how to approach your journey into their domain, democratising access to it.

How we built it

Using the engineering and mathematical trickery mentioned before! The project consisted of two main parts:

The Web page, built in React, visualises the learning journey as a skill tree, and allowing the user to continuously iterate on it, asking for learning resources for each step of the way.
The LLM/AI provides all the insights, interactively learning what the user's goals are, where they currently stand and how to get from one to the other in the best way possible.
To extend its knowledge with private, user-specific data, we've built a pipeline around it, allowing us to augment it (RAG), extend its capabilities (LoRA) and if necessary further specialise it (fine-tuning).
We also use agents to search for the best ways and resources to tackle the unknown; books, courses, documents, continuously analysing them and refining the skill tree.

Luckily, there were a lot of GPUs available:

Challenges we ran into

When we said we've all been there, we really mean it. We're regulars on life's corner of smiles and suprises. The things we prepared for, like the LLM implementation or the conceptual design of what we wanted to do went great(-ish)! And life was standing by the side, smiling and waiting to throw a bit of a wrench into it. :)

From small, but extremely annoying nuisances like being unable to open a single port on a GCP instance with the 20-something years of professional and research experience between us, to trying to find a way to describe and scope our idea without it sounding like a local supermarket chatbot or the inevitable AGI that's also smiling at us from the side.

All the while being exhausted from the 1000+km roadtrip we had just the day before the Hackathon, giving it our all up until the early mornings.

Accomplishments that we're proud of

We've actually used the tool ourselves, to help us build it! That is pretty cool. While we had some issues in places we would never expect, we also did really great on other, much harder stuff. There is no better way to learn than to tackle a real problem, and build around it.

We're proud of giving it our best, and as long as we continue to do so, we'll always be proud looking back.

What we learned

That the time is now, to build stuff like DomainLoom. We're lucky enough to have all the required skills among us to be able to work on something like this, and we strongly believe that work like this just scratches the surface of its potential.

What's next for DomainLoom

Exploring completely different market directions. Extending the infrastructure around the LLM to allow us to embark on that exploration journey. Optimisation and running the model not just locally, but on consumer hardware, on-edge.

We loved the fact that we could use it while building it, and all the possible avenues beside the small scope we've chosen as a demo for this hackathon, make us even more excited that we'll get the chance to continue building on it.