ShekarchAI

AI Powered Career Coach That Analyzes Millions of Job Posting and Delivers best career advies in the world.

Comment

Inspiration

The inspiration for ShekarchAI came from the growing need for personalized career guidance in today's rapidly evolving job market. Traditional career coaching is often expensive, time-consuming, and lacks access to real-time market data. We envisioned an AI-powered solution that could combine the intelligence of modern language models with live job market data to provide instant, personalized career advice at scale.

The project was born from recognizing that most career advice platforms rely on static data or generic guidance, while job seekers need insights based on current market trends, real job postings, and their specific professional background. ShekarchAI bridges this gap by creating an intelligent AI agent that can analyze LinkedIn profiles, scrape live job data, and provide contextual career guidance.

What it does

ShekarchAI is an intelligent career coaching platform that combines AI-powered conversations with real-time job market intelligence. The system acts as a comprehensive career advisor that can:

🧠 Intelligent AI Agent Capabilities:

  • Provides personalized career guidance through natural conversation
  • Analyzes LinkedIn profiles to understand professional background
  • Scrapes and processes live job postings from LinkedIn
  • Makes intelligent decisions about when to use job market data vs. general advice
  • Maintains conversation context for coherent, ongoing discussions

📊 Real-Time Market Intelligence:

  • Collects live job postings using Apify's LinkedIn scraping infrastructure
  • Stores and indexes job data in Elasticsearch for fast, intelligent search
  • Provides market statistics, salary insights, and company hiring trends
  • Identifies top skills, locations, and requirements for specific roles

🔍 Smart Search & Analysis:

  • Uses vector search and text search capabilities in Elasticsearch
  • Generates optimized search queries using AI to find relevant job data
  • Combines multiple data sources (profile + market data + conversation history)
  • Provides actionable insights based on current market conditions

How we built it

Frontend Architecture:

  • Next.js 15.5.6 with TypeScript for type-safe, server-side rendering
  • React 19 with modern hooks and state management
  • Tailwind CSS for responsive, modern UI design
  • Framer Motion for smooth animations and transitions
  • Lucide React for consistent iconography

Backend & AI Integration:

  • Google Gemini 2.0 Flash as the core AI agent for natural language processing
  • Elasticsearch 8.0 for advanced search, analytics, and data storage
  • Apify for reliable LinkedIn job and profile scraping
  • Next.js API Routes for serverless backend functionality

Elasticsearch Implementation: The Elasticsearch integration is sophisticated and multi-layered:

// Advanced field mapping with vector support
const mapping = {
  properties: {
    vector: { type: 'dense_vector', dims: 3 },
    title: { type: 'text', analyzer: 'standard' },
    company: { type: 'text', analyzer: 'standard' },
    description: { type: 'text', analyzer: 'standard' },
    // ... comprehensive job data fields
  }
};

// Bulk operations for efficient data ingestion
const bulkBody = jobs.map(job => [
  { index: { _index: 'linkedin-jobs-webhook' } },
  { ...job, vector: generateVector(job), text: combineText(job) }
]);

Gemini AI Agent Implementation: The AI agent uses a sophisticated multi-step decision process:

  1. Intelligent Decision Making: Gemini analyzes user questions to determine if job market data is needed
  2. Query Generation: Creates optimized search queries for Elasticsearch based on user intent
  3. Context Integration: Combines user profile, job data, and conversation history
  4. Response Generation: Produces personalized, actionable career advice
// Multi-step AI decision process
const needsSearch = await shouldSearchElasticsearch(userQuestion, userProfile);
if (needsSearch) {
  const searchQuery = await generateSearchQuery(userQuestion, userProfile);
  const searchResults = await textSearchJobs(searchQuery);
  const aiResponse = await generateFinalResponse(userQuestion, userProfile, searchResults, chatHistory);
}

Security & Performance:

  • Zod schemas for comprehensive input validation
  • Rate limiting with per-endpoint configurations
  • CORS protection and security headers
  • Input sanitization to prevent XSS attacks
  • Environment-based configuration for secure API key management

Challenges we ran into

1. LinkedIn Scraping Complexity:

  • LinkedIn's anti-bot measures made direct scraping unreliable
  • Solution: Integrated Apify's professional scraping infrastructure
  • Challenge: Managing rate limits and handling scraping failures gracefully

2. Elasticsearch Vector Search Implementation:

  • Initial difficulty with dense vector field configuration
  • Complex mapping requirements for job data with mixed content types
  • Solution: Implemented hybrid search combining text and vector search capabilities

3. AI Agent Decision Logic:

  • Teaching Gemini to intelligently decide when to use job market data vs. general advice
  • Balancing response quality with API call efficiency
  • Solution: Developed sophisticated prompt engineering with clear decision criteria

4. Real-time Data Synchronization:

  • Ensuring job data freshness while maintaining performance
  • Handling large-scale data ingestion from scraping operations
  • Solution: Implemented bulk operations and efficient indexing strategies

5. Security & Rate Limiting:

  • Protecting against API abuse while maintaining good user experience
  • Managing multiple API rate limits (Gemini, Apify, Elasticsearch)
  • Solution: Implemented comprehensive security middleware with per-endpoint rate limiting

6. Context Management:

  • Maintaining conversation context across multiple API calls
  • Balancing context length with API token limits
  • Solution: Implemented smart context truncation and conversation history management

Accomplishments that we're proud of

🚀 Technical Achievements:

  • Built a fully functional AI agent that can intelligently decide when to use different data sources
  • Implemented enterprise-grade security with comprehensive input validation and rate limiting
  • Created a scalable architecture that can handle thousands of job postings and user interactions
  • Achieved seamless integration between multiple complex APIs (Gemini, Elasticsearch, Apify)

🎯 User Experience:

  • Developed an intuitive onboarding flow that guides users through profile setup
  • Created a responsive, modern chat interface with smooth animations
  • Implemented intelligent conversation flow that feels natural and helpful
  • Built a system that provides genuinely useful career insights based on real market data

🔧 Innovation:

  • Pioneered the combination of AI conversation with real-time job market intelligence
  • Created an intelligent search system that understands user intent and generates optimal queries
  • Developed a context-aware AI agent that maintains conversation continuity
  • Built a system that can analyze LinkedIn profiles and provide personalized career guidance

What we learned

Technical Insights:

  • Elasticsearch Mastery: Learned advanced search techniques including vector search, aggregations, and bulk operations
  • AI Integration: Gained deep understanding of prompt engineering and context management for large language models
  • Security Best Practices: Implemented comprehensive security measures including input validation, rate limiting, and secure API key management
  • Performance Optimization: Learned to balance API efficiency with response quality in AI-powered applications

Architecture Lessons:

  • Microservices vs Monolith: Chose Next.js API routes for simplicity while maintaining scalability
  • Data Pipeline Design: Built efficient data ingestion and processing pipelines for real-time job data
  • Error Handling: Implemented graceful degradation when external services fail
  • State Management: Learned to manage complex application state across multiple data sources

AI Agent Development:

  • Prompt Engineering: Discovered the importance of clear, structured prompts for consistent AI behavior
  • Context Management: Learned to balance context length with API efficiency
  • Decision Logic: Developed sophisticated logic for when to use different data sources
  • User Experience: Understood how to make AI interactions feel natural and helpful

What's next for ShekarchAI

Short-term Enhancements (Next 3 months):

  • Advanced Analytics Dashboard: Real-time market trends visualization and career progression tracking
  • Resume Builder Integration: AI-powered resume optimization based on job market data
  • Interview Preparation: Mock interview sessions with AI feedback based on specific job requirements
  • Salary Negotiation Assistant: Data-driven salary recommendations and negotiation strategies

Medium-term Goals (3-6 months):

  • Multi-language Support: Expand to support career guidance in multiple languages
  • Industry-Specific Modules: Specialized career guidance for different industries (tech, finance, healthcare, etc.)
  • Company Culture Matching: AI analysis of company cultures and job fit recommendations
  • Skill Gap Analysis: Detailed analysis of skills needed vs. current skills with learning recommendations

Long-term Vision (6+ months):

  • Mobile App Development: Native iOS and Android applications for on-the-go career guidance
  • Enterprise Solutions: B2B platform for companies to provide career development to employees
  • AI Career Coach Training: Machine learning models trained on successful career transitions
  • Global Market Expansion: Support for job markets worldwide with localized insights
  • Integration Ecosystem: APIs and partnerships with HR platforms, learning management systems, and job boards

Technical Roadmap:

  • Advanced AI Models: Integration with more specialized AI models for different aspects of career guidance
  • Real-time Notifications: Push notifications for relevant job opportunities and career advice
  • Data Visualization: Advanced charts and graphs for career progression and market analysis
  • API Platform: Public APIs for third-party developers to build career-related applications

The future of ShekarchAI is focused on becoming the most comprehensive, intelligent, and personalized career guidance platform available, helping professionals worldwide navigate their career journeys with confidence and data-driven insights.

Built With

  • apify
  • autoprefixer
  • elasticsearch-8.0
  • eslint
  • framer-motion
  • google-gemini-2.0-flash
  • lucide-react
  • next.js
  • node.js
  • npm
  • postcss
  • react
  • react-dom
  • tailwind-css
  • typescript
  • vercel
  • zod
Share this project:

Updates