Real Time Updates:
- Created basic Figma design
- Got SD inpainting working from Replicate API
- SD 1.0 Inpainting not that great, so implemented Runway ML inpainting
- Image Segmentation next to try
- Got resnet-50 based panoptic segmentation working using Huggingface Transformers results not great for some objects. Not exact segmentation, overflow from object, holes not well covered
- multiple objects segmented, How to give users ability to choose from image?
- slow speeds - takes about 10-15 seconds
- Next to try - different segmentation apis from replicate
- Tried mask2former and dichotomous_image_segmentation from replicate mask2former - inaccurate and slow - 140 seconds dichotomous_image_segmentation - better accuracy and painfully slow - 150 seconds -Need to think of a better solution
- So instead of Segmentation, I tried using background removal APIs from Replicate, I can then - convert the background removed imaged into a mask
- API's were slowish - 15 second response
- Lets try a 3rd party bg removal service and see if accurate
- So tried Photoroom, Remove BG
- Photoroom is very fast & very accurate. 10 free credits
- Remove BG - fast and less accurate
- Lets go with Photoroom
- Now convert the bg_less image to mask image for stable diffusion inpainting
- Converted to Mask Image
- Resize Images to smaller size (stable diffsuion running out of memory)
- Okay, got the basic version running. Objects get mnodified, so need to cut the object from original and paste it in new image
- Got image copy paste working,
- The ML part is working reasonably well, now. The result image is in 512 pixel format, may need to upscale it later
- Next on to - deploy this using fast api on ec2 and build frontend
- Added basic fast API template, not working with current code yet
- Facing some issues with fast API, so moving to frontend first
- Added frontend skeleton code
- Added UI for NavBar, Upload Image component, Prompt text area & generate button
- Now need to add functionality
- Added background removal functionality (photoroom API) to frontend instead
- Since some outputs are not desired, we give multiple outputs and user can select the best, - Added that functionality
- Frontend mostly working well now
- Fast API still giving issue son EC2, so instead deployed to Banana.dev
- Added banana backend code
- Added Landing page and reset button functionality
Log in or sign up for Devpost to join the conversation.