ResearchEngine?

• 

  ResearchEngine? Homepage

• 

  ResearchEngine? Explorer Page

• 

  ResearchEngine? Insights Page

• 

  ResearchEngine? Saved Queries Page

Inspiration

The inspiration for this project stemmed from a common frustration experienced by many researchers and students: the challenge of finding truly novel research questions. While countless tools exist to find published papers, there's a significant void when it comes to identifying areas where research is sparse or entirely absent. This often leads to redundant studies or, more critically, a perpetuation of biases in research focus, leaving vital questions unanswered in certain demographics or geographical contexts.

When we learned about the Perplexity Sonar API, its ability to provide real-time, grounded answers from the web immediately clicked on us. We realized its potential wasn't just for answering questions, but for identifying the absence of answers. Could we leverage its comprehensive web understanding to systematically uncover what isn't being researched, rather than just what is? This core idea became the driving force behind the project.

What it does

• Intuitive Search: Easily input any topic to begin exploring its research landscape.
• Real-time Gap Analysis:
  • Research Timeline: Visualize the historical volume of research to identify periods of inactivity or decline.
  • Regional Distribution: Map where the research has (or hasn't) been conducted across different geographical areas.
  • Population Coverage: Highlight which demographic groups are underrepresented in studies related to your topic.
  • Subtopic Radar: Reveal related themes and their current research coverage.
• Actionable Insights: Automated summaries of identified gaps (e.g., "No significant studies found in Southeast Asia post-2020 on X topic").
• AI-Suggested Questions: Generate novel, impactful research questions directly derived from the identified gaps, fostering new avenues for inquiry.

How we built it

The project is built using Next.js, leveraging its App Router for a modern full-stack approach.

1. Frontend (React & Next.js): The user interface is crafted with React components, styled using Tailwind CSS for rapid development and responsiveness. ShadCN/UI components provide a polished look and feel, while Recharts handles the dynamic data visualizations.

2. Backend (Next.js API Routes): All sensitive operations, particularly calls to the Perplexity Sonar API, are handled securely within Next.js API routes (/api/search and /api/generate-insights). This ensures the PERPLEXITY_API_KEY remains server-side and never exposed to the client.

3. Perplexity Sonar API Integration: This is the brain of the operation. For a user's initial search, the /api/search route makes multiple, targeted calls to Perplexity Sonar. For example, it asks for estimated research counts by year, by specific regions (like "Africa," "Southeast Asia"), by population demographics (like "children," "elderly"), and for related subtopics.

   • Custom parsing logic then extracts these numerical estimates and textual summaries from Perplexity's natural language responses.
   • Simple heuristic algorithms compare these estimated counts to identify "gaps" (e.g., significantly lower research volume in recent years or specific regions).
   • A final Perplexity call is then made, fed with these identified gaps, to generate actionable insights and novel research questions.
   • The /api/generate-insights route provides a dedicated endpoint to regenerate these insights and questions on demand.

4. Data Flow: The processed research data, including visualizations, insights, and suggested questions, is sent back to the frontend to be rendered dynamically.

5. Deployment: The entire application is deployed seamlessly to Vercel via GitHub Actions for continuous integration and deployment, ensuring every push to main triggers automated testing and deployment.

Challenges we ran into

• LLM Response Variability & Parsing: The most significant challenge was reliably extracting structured data (numbers, specific entities) from Perplexity's natural language responses. While prompt engineering helped, the LLM could sometimes deviate slightly in format, requiring robust and flexible parsing logic to avoid breaking the visualizations. This was an iterative process of refining prompts and parsing regex.

Accomplishments that we're proud of

• Addressing a Real-World Problem: The project tackles a genuine pain point for researchers and academics, offering a valuable tool to foster more comprehensive and impactful research.
• Innovative Use of Perplexity Sonar API: We've gone beyond typical LLM applications by using Perplexity not just to find answers, but to identify the absence of information. This unique approach is a core differentiator and a clever application of the API's capabilities.

What we learned

• Perplexity Sonar API & Prompt Engineering: Gained hands-on experience in interacting with a cutting-edge LLM API. The biggest learning here was the art of prompt engineering to extract structured data (like counts per year/region) from natural language responses, and how to guide the model to generate specific types of output (insights, questions).
• LLM Output Parsing: Developed practical skills in parsing semi-structured text responses from an LLM into usable JSON-like data for visualizations. This involved crafting robust regex and string manipulation techniques.
• Data Visualization with Recharts & ShadCN/UI Chart: Successfully integrated Recharts via ShadCN/UI's chart component to create dynamic and visually appealing charts, moving beyond static mockups to real-time data representation.

What's next for ResearchEngine?

• Research Gap Dashboard: For power users to save queries, track discovered gaps over time, and export visualizations. This would require a database integration, such as using Drizzle ORM with a serverless Postgres/MySQL.
• User Accounts & Personalization: Allow users to create profiles, save their favorite research areas, and receive tailored gap alerts.
• Refined Gap Detection Algorithms: Incorporate more sophisticated NLP techniques, such as topic modeling, knowledge graph analysis to identify nuanced and complex interdisciplinary gaps.

Built With

• github-actions
• lucide
• next.js
• react
• recharts
• shadcn/ui
• tailwind
• typescript
• vercel
• zod