MusicViz

Inspiration

The project was inspired by the rapid development of generat AI. I experiment with new creating workflows.

What it does

Music Visualizer It takes audio stream from microphone and renders realtime audioreactive 3D graphics. Visualizer is minted as NFT on Mantle blockchain. It can be used as music video or displayed at concerts or events. 3D AI editor Browser based 3D AI editor takes an image input and generates animated 3D objects. It uses Stable Diffusion AI via API and automates the process of controlled image generation.

How I built it

• visual programming - cables.gl (java script + WebGL)
• image generation - Stable Diffusion WebUI API
• 3D model editing - Blender

Challenges I ran into

• Using Stable Diffusion WebUI API
• Smart Contracts

Accomplishments that I'm proud of

Building 3D editor powered by AI in 2 days.

What I learned

• Smart Contracts
• Getting NFT metadata via API

What's next for MusicViz

• I plan to develop more user friendly web apps to create generative music videos with no technical skills
• Music NFTs + player with visuals.
• VJ mixer app - interactive music visuals NFTs.

Applying to bounty: Mantle - Best NFT or Gaming Project: $2,500 in MNT

Built With

• blender
• cables.gl
• chatgpt
• javascript
• lumaai
• mantle
• nftstorage
• stablediffusionwebui-api
• webgl