MatGen

Inspiration :

Whenever I'm building 3D environments or indie games in Blender and Unity, making textures is honestly the most tedious part. You either have to pay for expensive texture libraries, download sketchy free assets, or spend an hour in dedicated software tweaking filters just to get your normal and roughness maps. I wanted to build a simple, instant web studio where I could just drop in any picture from my phone or the web and get game-ready 3D materials in seconds.

What it does :

MatGen PBR turns any standard 2D image into a full set of customizable PBR (Physically Based Rendering) texture maps:

PBR Map Generation: Automatically extracts Normal maps (with OpenGL and DirectX toggle), Roughness, Height/Displacement, Ambient Occlusion, and Metallic maps.

Smart Surface Analysis: Uses Gemini to scan the image, figure out what the material is (like polished marble, weathered wood, or stone), and automatically tune the physical parameters so you don't have to start from scratch. Photo Cleanup: Features a de-lighting slider to strip baked-in shadows from real-life photos, plus a seamless patch synthesis mode to tile textures without visible seams.

Interactive 3D Viewport: A real-time Three.js viewer letting you preview the material on a sphere, cube, shaderball, or cylinder, spin the light source, and check physical displacement live.

Engine-Ready Exports: Packages textures specifically for each engine, including channel- packed ORM maps for Unreal Engine 5, Metallic/Smoothness maps for Unity, and individual maps for Blender.

How we built it :

Frontend & 3D: Built with React 19, Tailwind CSS, and Motion for a clean dark-mode UI, paired with Three.js to run the real-time WebGL material renderer in the browser.

Backend & Al: A Node.js and Express server written in TypeScript. I prototyped the prompts in Google Al Studio and Antigravity 2.0, then hooked up the Gemini API to handle material detection and physical calibration.

Texture Pipeline: We used Sharp on the backend to manipulate and merge image channels at high speed, and JSZip to pack the output files for instant download. Challenges we ran into The trickiest part was definitely engine-specific channel packing. Unreal Engine, Unity, and Blender handle textures completely differently: Unreal expects AO, Roughness, and Metallic combined into the Red, Green, and Blue channels of a single image to save GPU memory, while DirectX needs the Normal map's green channel flipped. Getting Sharp to recombine raw pixel buffers correctly without slowing down the server took a lot of experimentation. Another challenge was keeping the Three.js viewport running smoothly at 60 FPS while updating displacement and lighting in real time.

Accomplishments that i'm proud of :

Building a full end-to-end workflow: you drop an image in, tweak what you want, and get a zip file that drops right into Unreal or Blender without any manual setup. Making the live 3D preview feel super responsive directly in the browser.

Using Al for a practical, concrete pipeline task rather than just generating a random picture.

What i learned :

I learned a ton about how PBR shaders and texture coordinates actually work under the hood, how game engines optimize memory with channel packing, and how to handle fast image processing streams on Node.js using Sharp.

What's next for MatGen :

Adding a text-to-texture generation feature to create textures directly from text prompts.

Supporting 4K texture upscaling and exports.

Creating quick add-ons for Blender and Unreal to import textures directly into a scene with one click.

Built With

• express.js
• gemini-api
• google-ai-studio
• jszip
• node.js
• react
• sharp
• tailwind-css
• three.js
• typescript
• vite