Inspiration
Inspiration for Phystral3D came from the need to make physics modeling more interactive with LLMs. I wanted to create a tool that allows users to manipulate and explore physics concepts in a visual 3D environment.
What it does
Phystral3D is a 3D physics manipulation and simulation application. It allows users to input geometry and manipulate it using various prompts specifying physics concepts.
How we built it
Phystral3D was built using python and javascript: Mistral powers the LLM response, llama_index is the agent, and other common python libraries. The application uses a local file server to handle files, a Three.js visualization engine for rendering, and Streamlit for the app interface.
Challenges we ran into
The main challenge faced was parsing the coordinate information form the LLM response and applying the changes to the geometry. Ensuring realistic output to be rendered is also a challenge.
Accomplishments that we're proud of
Proud of developing a unique and interactive way to interact with 3D physics via prompting. Proud of the look and feel too!
What we learned
Structured prompting is key for using an LLM to render 3D physics. Also gained experience in using Mistral LLM, llama_index, and streamlit.
What's next for Phystral3D - Physics Agent
In the future, I plan to add more physics concepts to Phystral3D (example accurate wind blowing animation) and realistic output. The next few years of development will show large scale AI-based 3D physics engines and simulation!
Built With
- javascript
- llama-index
- mistral
- python
- three.js
Log in or sign up for Devpost to join the conversation.