Sanity Check

Inspiration

This project was born from a toxic work environment I spent a large portion of my corporate career in. It left me with issues that took a decade to come to grips with. Now when I look back understanding the physical output of an untreated stress disorder, I'm able to reflect on how terrible that place truly was and how strong I had to be to rise above and get tf out.

What it does

This was such an incredible amount of fun to make! I started editing by first downloading DaVinci and while I was moving forward, it was slow. Once I started editing in Capcut, my productivity accelerated with it's easily understandable UI and logical placement of tools right where I needed them. While I used OpenArt's Motion Sync for the character mo-cap, I was originally inspired to take this approach from Mago.AI, and the only reason I didn't use them was that my trial ran out before I had time to start.. I've learnt that you can get some ok results with most LipSync software but it takes too long to get the video generation the way I want, and the mouth movements made properly. Sync was the premium option here, but again my trial ran out LOL When my sub to OpenArt expires, I'll be using Mago for sure!

Seedream was my imageGen of choice, after creating the base images for each character in each scene, I would then use Seedream to strip the scenes of all people creating a blank environment to tell the story. I would then load the scene and the characters I wanted to use and instructed SeeDream to place them the way i wanted. This led to easily animatable scenes using Kling 2.1 for start and end frames, or 2.5 for the more "cinematic" sections. Seedream easily had the most useable output of all the image generators and once you figure it out, it's VERY powerful.

And as far as Kling goes, Wow! Using an method I obtained and altered from OpenArt, I was able to get exactly what I wanted out of the AI. For the most part, Klings AI video creation is outstanding! There are scenes with distortion and obvious AI errors but that's not Klings fault. If I was better at prompting then maybe it wouldn't be so bad, but honestly IDGAF as it's AI and should be seen to be obvious AI. I'm not wanting realism, I want an engaging video. The obvious nature of the twisting frames I think is hilarious and in the future there'll fixes for most of it anyway so I'm enjoying it for what it is now.

How we built it

Well I grabbed my tripod, turned all the lights on and recorded the motion and mouth movements. For the empty chair piece, I grabbed a stool I had handy and squatted down next to it. The same applies for the restaurant scene, I had the camera sitting across from me at my dining table! Every time just yelling along to the song making wild facial and eye movements. Then all of that was loaded into motion Sync and I ran a few generations of each recording. The best interpretations of that recording were then loaded into CapCut and edited into a cohesive song.

Challenges

Many! The biggest issue I have is getting 2 unique scenes to link where there isn't an obvious join and wouldn't make sense for a camera change given how close the end and start of each other are. Sometime I would have 30-45 of video to join, and I had to cram a solution in. Joining the scene where he steps in dog crap to the motion sync section immediately after was hard, as the photo model of the motion sync was also the end frame of the last scene, but the motion sync doesn't start on that as the first frame. So I had to use suuuuupppppeeeerrrrrr sloooooooooow motion prompting to get a 5sec video generation squished into a 1 sec movement. results may vary.... You'll see the same thing happen in the restaurant scene also. He's 'chatting' to his partner before jumping straight into the motion sync.

Accomplishments that we're proud of

The whole thing tbh. Some big life changes happened right in the middle of this so to get the story out of my head out and into a video, and the song mixed properly and the whole thing uploaded in time, Just so proud to get it done. I learned so much about so many new products in such a short time, when there's such a massive market available which can totally side track you completely! I stayed focused and got it done.

What we learned

Everything! I've only done some very basic editing so to come out with something like this is unbelievable. I don't have a history of finishing projects but I really believed in this and got it done. I learnt how to prompt Kling properly, How to get some more advanced editing done in Capcut, working on ideas and brining them to life. How to get characters and environments created properly in SeeDream, and how to get myself into the action and live out an intrusive thought using motion capture.

What's next for Sanity Check

Banned from entry LOL

Built With

• capcut
• kling
• openart
• seedream