Inspiration
Keeping up with tech news manually was slow and unsustainable, and most existing AI-generated news tools run into the same wall: Google flags them as Automatically Generated Content and deranks them. The real problem worth solving was not just automating research and writing, but doing it in a way that survives that classification, rather than treating it as an afterthought.
What it does
Astin Tech News is an automation pipeline that:
- Sources news from RSS feeds and social media, including experimental data acquisition from Twitter through self-hosted Nitter instances
- Uses ChatGPT to research and draft articles based on the sourced material
- Applies structural safeguards intended to avoid AGC classification by Google
- Optimizes its sitemap for search engine indexing
- Publishes finished articles with minimal manual intervention
How we built it
The pipeline runs across three layers:
- Source acquisition layer: Pulls content from RSS feeds and explores social media as a secondary source, including self-hosted Nitter and custom tweet-fetching tools to work around official API restrictions.
- Processing layer: Uses ChatGPT to turn sourced material into written articles through a structured prompt system.
- Publishing layer: Manages sitemap generation and indexing optimization so content stays discoverable without tripping AGC signals.
The application is built on Next.js and Node.js, with PostgreSQL handling persistent storage.
Challenges we ran into
The most difficult issue was dead code sitting inside the prompt system. It never threw errors or crashed the pipeline, it just quietly produced weaker output that was easy to mistake for a model limitation rather than an implementation bug. Finding it required auditing the prompt pipeline component by component instead of trusting it as a black box.
The second, ongoing challenge is that Google has never published a definitive rulebook for what counts as AGC. That forced an iterative approach: adjust the pipeline, observe indexing behavior, refine again, rather than applying one fix and moving on.
Accomplishments that we're proud of
Getting source acquisition, AI-driven writing, and publishing to run as one coherent automated flow, instead of a set of disconnected scripts, is the accomplishment we value most.
Treating AGC avoidance as a design constraint from day one, rather than a patch applied after getting penalized, is the sharpest engineering decision behind the project. It shapes how content is generated before it ever reaches the sitemap, instead of fixing indexing problems after the fact.
What we learned
Automated content pipelines carry a built-in tension: pushing harder on automation speed tends to increase AGC risk, while pushing harder on AGC avoidance tends to reduce output volume. The real design problem was never automation itself, it was building a system that treats search engine trust as a first-class constraint rather than a metric to optimize around later.
What's next for Astin Tech News
Two directions define the next phase: expanding beyond tech news into multiple news categories to widen the audience, and building out monetization once that expansion is in place.
Built With
- chatgpt
- next.js
- nitter
- node.js
- openai-api
- postgresql
- rss
Log in or sign up for Devpost to join the conversation.