Neo linking - AI Powered Website Linking

Internal Linking Opportunities
Demo Loader
Demo

Inspiration

Imagine you have hundreds of blogs on your website—a situation that’s pretty standard, and some sites even have thousands. As an SEO expert or content writer, your job is to figure out where to place internal links. Every time a new blog is published, you need to go back and update hundreds of previous blogs to create internal links for the new content. It’s a nightmare for content writers and SEO or marketing teams.

I’m an AI expert by degree but a marketer at heart. I’m passionate about both and have been working at the intersection of marketing and AI for the past 5 years. I’ve collaborated with many SEO teams and even lead my own, all of whom face the constant struggle of inserting internal links into blogs.

What It Does

Imagine having an AI agent that automatically finds internal linking opportunities for you and suggests changes to your content to incorporate more internal links. Think about how much time that would save—time you could spend on more productive work.

With an AI-powered internal linking solution, you can also:

Improve site navigation
Boost SEO
Increase conversion rates
Drive more traffic
Enhance user experience

By strategically placing links, optimizing content connections, and effectively distributing link equity, this tool can transform the way you approach content management.

How We Built It

Our application consists of two main components: the frontend and the backend.

Frontend: Developed using Next.js
Backend: Powered by Python and deployed on AWS Lambda functions

The core of our product is the advanced algorithm we designed to identify internal linking opportunities on websites with hundreds of pages through vector search. Here's a quick overview of how our system works:

Web Scraping: We gather website content, including page text, metadata, and existing links.
LLM Integration: We utilize a Language Learning Model (LLM) to identify keywords and meta descriptions from the scraped pages. For pages that lack these elements, the LLM generates them automatically.
TiDB Database: The content from the websites and individual pages is stored in a MySQL-compatible table within TiDB. Keywords and meta descriptions for each page are stored in vector databases. We implemented three vector databases using Llama VectorStore, enabling efficient similarity searches across keywords, descriptions, and page content.

Our algorithm then extracts keywords, searches for related pages using vector similarity, and ranks the results to suggest the most relevant internal links. This approach not only optimizes internal linking but also significantly enhances the overall efficiency of SEO processes.

Tools We Used:

Llama Index: For handling LLM queries and creating AI agents
NPI.ai: Streamlined our workflow across multiple AI tools, making the creation of agent tools more efficient
TiDB: Used for storing website and page data in SQL tables, and keywords and meta descriptions in a vector database for similarity matching
AWS Lambda Functions: The backend is deployed using AWS Lambda for scalable and efficient serverless processing

Challenges We Ran Into

Creating an algorithm for internal linking presented several challenges:

Missing Keywords and Meta Descriptions: Some blogs lacked keywords and meta descriptions, so we had to generate these elements during the web scraping process.
Initial Algorithm Challenges: Our initial algorithm didn’t perform as expected. Despite making several changes, the results were still suboptimal.
Complete Overhaul After Multiple Iterations: After seven iterations, we decided to rethink our approach entirely. We started from scratch, reimagining both the problem and the solution.
Transition to Hybrid Search: Simple vector search wasn’t delivering satisfactory results, so we moved towards a hybrid search combined with a reranker, which significantly improved our outcomes.

Accomplishments That We're Proud Of

Creating an Internal Linking Algorithm: We developed an algorithm designed to help website owners identify optimal places to insert internal links, enhancing SEO and site structure.
Rapid Development: We built the entire product within one week, dedicating 14 hours a day to ensure its completion.

What We Learned

Creating AI agents on Llama Index
Developing a fully working MVP within a week
How to use LLM and vector search to solve complex problems
Working as a team to solve bigger challenges

What's Next for AI-Powered Internal Linking

Improving Internal Linking

Improve the internal linking algorithm for better results
Create capabilities for analyzing and suggesting internal links as content is being written
Implement a system that can update link suggestions as website content changes
Implement functionality to identify topics or areas where new content could be created to improve internal linking structure
Expand the tool to not only suggest links but also recommend content updates or new content creation to strengthen internal linking strategies

SEO Back Linking

As your platform grows, we can suggest backlinking opportunities to users, recommending they link to relevant content on your site. Implement automated backlink suggestions, track their performance, and offer incentives for effective linking. This enhances your platform's SEO and engages users by boosting their content’s visibility.