Tingyu (Robert) Zhang's personal portfolio

Inspiration

This project stems from a vision to redefine online presence through a more engaging and human-centered approach. Leveraging the latest advancements in AI, I aimed to merge cutting-edge technology with personal branding, creating an authentic way to connect and communicate in the digital world.

What it does

The platform offers an AI-driven personal avatar that serves as your online representative. It engages visitors, shares your experiences, showcases your portfolio, and dynamically answers questions, effectively bridging the gap between technology and individuality.

How we built it

The system was built using Simli for video avatars and VAPI for voice-based WebRTC alongside seamless backend knowledge base integration. ElevenLabs was utilized for voice cloning, enabling a truly personalized audio experience. A modern front-end framework ensures an intuitive interface, while robust APIs facilitate real-time conversations and data flow for responsive user interactions.

Challenges we ran into • Crafting a conversational AI that accurately mirrors personal tone and style. • Integrating a variety of technologies into a cohesive and high-performing platform. • Ensuring scalability and security while maintaining a deeply personalized experience.

Accomplishments that we’re proud of • Creating a personal avatar that feels authentically human and aligns with individual personalities. • Successfully integrating multiple advanced AI technologies into a unified and seamless platform. • Delivering a transformative product that reimagines personal branding in the digital space.

What we learned • The intricacies of building AI systems that balance personalization and generalization effectively. • Strategies for integrating multiple APIs and frameworks to deliver seamless functionality. • The value of designing with user engagement, accessibility, and intuitive interaction in mind.

What’s next for Tingyu (Robert) Zhang’s personal portfolio • Connecting to the internet: Integrating tools like Tavily, SERP, and Perplexity to enable the avatar to retrieve real-time information. • Tool calling: Enabling the avatar to automatically schedule virtual appointments between viewers and the host. • Enhanced customization: Improving avatar features, including voice modulation, emotional expression, and visual personalization. • Real-time analytics: Implementing analytics tools like Coval to track user engagement and feedback effectively. • Reasoning models: Deploying advanced reasoning models in the background to handle complex questions seamlessly. • RL-based browsing agents: Developing reinforcement learning-based agents to provide in-depth project and repository walkthroughs, tailored for recruiters and professional reviews (e.g., Agent-S, browser-use, Claude-enhanced computing).

Built With

• elevenlabs
• next.js
• openai
• react
• simli
• typescript
• vapi