MMAI Integration

Inspiration

Many months of work on the MMAI GPU manager.

What it does

Deploys memmachine to a Kubernetes cluster via Helm, and then interfaces with memmachine via a new widget on the MMAI dashboard.

How we built it

Packaged memmachine and its associated databases via helm. Built out the frontend chatbot via Typescript.

Challenges we ran into

Properly configuring an old build of memmachine.

Accomplishments that we're proud of

Tunneling queries to memmachine through the Rancher service proxy API, among many other things...

What we learned

It's pretty easy to deploy memmachine and then tie it with both an on-prem vLLM and OpenAI's API.

What's next for MMAI Integration

Greater reliability, removing duct tape...

Built With

• go
• typescript
• yaml