DreamerG

Inspiration

DreamerG was inspired by the growing demand for rapid, accessible, and efficient digital modeling tools that cater to various industries, from gaming to architecture. Recognizing the potential of A.I. to revolutionize 3D modeling, DreamerG harnesses Google's Gemini A.I. to provide an intuitive platform that transforms text descriptions into detailed 3D models.

What it does

DreamerG is revolutionizing digital creation with its state-of-the-art text-to-3D model generator. By harnessing the power of Google's Gemini A.I., DreamerG enables users—from game developers to architects—to transform simple text descriptions into detailed, accurate 3D models in seconds. This breakthrough technology not only speeds up production processes but also democratizes 3D modeling, making it accessible to non-experts. Whether you’re building virtual worlds or conceptualizing future products, DreamerG is your gateway to seamless, intuitive 3D model creation, redefining the limits of imagination and design.

Use case

DreamerG's text-to-3D model generator offers a range of applications across various industries, making it a versatile tool for both professionals and hobbyists. Here are some use cases for this innovative product:

Video Game Development: Game designers can quickly generate and iterate on 3D models of characters, environments, and objects simply by describing them in text. This accelerates the game development process, allowing for more creativity and experimentation with less dependence on specialized modeling skills.
Film and Animation: In the film and animation industries, DreamerG can be used to create detailed props, sets, and even background characters. This allows for rapid prototyping of visual ideas and helps storyboard artists and directors visualize scenes and settings without waiting for detailed renders from a graphics team.
Architecture and Interior Design: Architects and interior designers can use DreamerG to quickly turn conceptual descriptions into detailed 3D models of buildings, rooms, furniture, and decor items. This can drastically reduce the time spent in the initial design phase and help clients visualize and tweak projects in real-time during presentations.
Education and Research: In educational settings, instructors can utilize DreamerG to create detailed 3D models that help students visualize complex concepts in subjects like biology, engineering, and environmental science. Researchers can also use the tool to model experimental apparatus or visualize data in three dimensions, enhancing both understanding and presentation of scientific findings.

How we built it

This is the overall system architecture. The frontend is build on React use api directly from a backend build with FASTAPI framework. Inside the FASTAPI framework there are mutiple api called to specific model backend. The FASTAPI backend use api from Gemini Google A.I, Stable Diffusion API, and InstantMesh Huggingface API.

Model Architecture The above display the text-to-3D system architecture for the A.I model that generate text input into 3D model. It start with a simple user input int Gemini. Gemini will enhance the prompt so that it the prompt have enough detail and direction to allow a image generator to generate an image suitable for the 3D generator. Once the enhance prompt is generated from Gemini, the Gemini's generated prompt will be input into Stable Diffusion XL model to generate an image. The generated image then will be fed into InstantMesh(image-to-3D generator) to generate a 3D model.