Cerebro Techies

• 

• 

• 

• 

Inspiration

Tech Buffalo’s mission to bridge the skill gap in the Western New York community inspired us deeply. We wanted to create a solution that could help identify the most relevant skills in the ever-evolving field of technology. Our goal is to bridge the communication gap between learners, educators, and employers by empowering them with data-driven insights and tools to make informed decisions

What it does

Our solution leverages Generative AI (GenAI) to serve three key groups: Learners, Educators, and Employers.

For Learners:

Career Pathways Search: Helps users find suitable career paths based on their current skills. Job Search: Allows learners to search for jobs tailored to their skill set. Skill Recommendations: Provides suggestions for the next skills to learn, based on the learner’s current abilities and job market trends. For Educators:

Trend Prediction: Displays future tech trends, helping educators tailor their courses to meet market demand. Skill Gap Insights: Shows the skill gap in the community, detailing what learners possess versus what employers require. For Employers:

Learner Matching: Recommends potential candidates for job postings based on their skills and career pathways. Skill Trends: Displays real-time skill trends in the Western New York area, helping employers understand the evolving tech landscape.

For Educators:

• Trend Prediction: Provides insights into future tech trends, helping educators align their courses with industry demand.
• Skill Gap Insights: Displays gaps between learners’ skills and the needs of employers, guiding curriculum development.
• Skills Overview: Allows educators to see the specific skills learners currently possess and what skills employers are seeking.
• Future Job Trends in Erie County: Highlights emerging job trends in the Erie County area, enabling educators to prepare learners accordingly.

For Employers:

• Learner Matching: Recommends potential candidates for job postings based on their skills and career pathways.
• Skill Trends: Shows real-time skill trends in the Western New York region, helping employers stay informed of market shifts.
• Skills Overview: Gives employers visibility into the skills learners are acquiring, aiding in targeted recruitment.
• Available Courses: Displays the educational courses offered, allowing employers to understand the training available in their region.

How we built it

We utilized several powerful technologies to build our solution. Here's a breakdown of the main components:

1. Streamlit for the Interface:

• We built an intuitive web interface using Streamlit, which allowed us to create a user-friendly platform that helps learners, educators, and employers interact seamlessly. It’s highly customizable and makes building data applications efficient and quick.

2. GenAI for Intelligent Responses:

• We incorporated Generative AI models like LLaMA (via Groq) to provide tailored responses based on user queries. For learners, this helps suggest career pathways, jobs, and skills they need to acquire, while educators get insights into skill gaps and emerging trends. Employers can also get recommendations on potential candidates and analyze job trends in Western New York.
• The chatbot is capable of handling a variety of requests, such as course recommendations, job roles, and even predicting trends in the tech industry.

3. Chroma for Document Vector Storage:

• We used Chroma as our vector database, which enables us to handle large sets of documents and retrieve relevant content efficiently. By splitting uploaded PDFs into smaller chunks, we indexed the data to quickly search and retrieve relevant information based on user questions.

4. LangChain for RAG (Retrieval-Augmented Generation):

• For answering questions that required specific document context, we used LangChain with a Retrieval-Augmented Generation (RAG) process. This approach allows us to search relevant documents and pass that context to the AI model, providing accurate and relevant responses. It also handles non-contextual questions by generating answers directly from the model.

5. Vector Embeddings and Similarity Matching:

• We employed FastEmbedEmbeddings to generate embeddings for user queries and stored them. These embeddings were used to compare and match previous similar questions using cosine similarity. If a related question was found, it was suggested to the user, enriching the experience by providing relevant, context-aware responses.

6. Custom Datasets:

• We integrated two custom datasets: one for courses and another for jobs available in Western New York. This was key to providing localized recommendations, focusing on available resources within the community.

7. AI for Skills Matching:

• For both courses and jobs, the AI matches the skills in user queries to available opportunities by assessing their input and ranking relevant results based on similarity scores. This helps learners discover fitting courses and jobs based on their current skills and preferences.

8. Dynamic Content:

• The platform dynamically updates visualizations, such as job trends and salary ranges, and displays growth trends for specific roles within the Western New York area using pre-processed data. This offers insight into real-time labor market demands.

Summary:

With these components working together, we created a tool that empowers the Western New York tech community by facilitating skill-building, improving education resources, and bridging communication between learners, educators, and employers.

Challenges we ran into

Challenges are a great indicator of a meaningful problem. For this project, we encountered a few key difficulties:

Data Access: While employment data for Erie County was available, much of it was outdated. After consulting with industry experts, we decided to mock data where necessary to ensure our recommendations reflected the current market trends. Scope Definition: With so many potential directions to explore, we had to focus on solving the most critical problems first, which led us to prioritize career recommendations, skill gaps, and employer-labor market alignment.

Accomplishments that we're proud of

Successfully extracting and displaying trends from Lightcast data, despite limitations in data availability. Building a functional web app in a short time, effectively bridging gaps between different entities and data sources. Collaborating from diverse perspectives, leading to innovative solutions through brainstorming and teamwork.

What we learned

Collaboration amplifies creativity—more ideas lead to more refined and comprehensive solutions.

What's next for Cerebro Techies

Our next step is to enhance this project by incorporating real-time data and fully integrating a robust database, turning it into a scalable web app.

Built With

• chroma
• figma
• lightcast
• llama
• pypdfloader
• python
• scikit-learn