Inspiration
Broken documentation links are a small problem that can quickly become frustrating. A file gets renamed, a heading changes, or an external page disappears, and suddenly readers end up stuck. I built linkchecker-py because I wanted a simple way to catch those problems before documentation changes reach users.
What it does
linkchecker-py is an asynchronous Python command-line tool that checks links in Markdown files, HTML files, and small to medium-sized websites. It can find broken web links, missing local files, invalid fragments, and outdated heading anchors.
It can also crawl pages on the same website, respect robots.txt, cache previous results, and control request speed and concurrency. Results can be displayed in a clear terminal table or saved as Markdown or JSON, making the tool useful both locally and in CI workflows.
How I built it
I structured the project as a Python CLI with separate components for parsing documents, checking links, crawling websites, creating reports, and handling command-line options. I also added automated tests, continuous integration, examples, documentation, and PyPI packaging.
I built and tested the project in OpenAI Codex with help from GPT-5.5 and GPT-5.6. They supported different parts of the process, including planning the CLI, developing the checker and crawler, debugging problems, thinking through edge cases, writing tests, and improving the documentation. I reviewed the resulting code and ran the project’s tests in the repository.
Challenges and what I learned
One of the hardest parts was making sure relative links behaved consistently across local documents and crawled web pages. Fragment links were another challenge because different documentation systems can generate heading anchors differently.
I also had to balance speed with responsible website crawling. Features such as bounded concurrency, rate limiting, robots.txt support, and caching helped make remote checks faster without sending unnecessary requests.
This project taught me that a useful developer tool is about more than finding errors. It also needs predictable exit codes, understandable reports, sensible defaults, and a setup process that does not get in the user’s way.
Future improvements
Next, I would like to support more documentation generators, add a project-level configuration file, improve performance for larger repositories, and provide richer CI and pull-request annotations.
Log in or sign up for Devpost to join the conversation.