Screenshot of the frontpage
NY Times news index
NY Times article
We take a 45 minute train every morning and read news sites and blogs like Hacker News. We really like the feeling of the ebook and we thought it would be awesome to combine the best of both worlds. We though making it modular and open source would be perfect because anyone can add their favourite sources and everyone can benefit from it.
What it does
You subscribe to the news and blogs you want and get a link that lets you download a curated ebook generated from this sources. The news we generate are super ebook friendly.
How we built it
We get the content from the original website frontpages, follow article links and parse them to an adapted version which we convert to either
.mobi (for kindle). Every website/blog we support is a different parse module, and the app is built so that anyone can make new modules easily. The frontend is built in react Redux aiming to make the project as maintainable as possible. The styling follows material ui standards.
Challenges we ran into
Creating a modular backend and parseing websites to generate ebooks has taken most of the time and effort.
One big complication is that front pages only show previews, so we have to follow links and parse each article individually, then make an index so that the ebook is readable. We've tried two different approaches for getting the relevant context, using regex and virtual DOM manipulation. We haven't tested which one is better yet.
We also had to learn how the epub format is structured, as well as the mobi.
Accomplishments that we're proud of
Setting up a fully operational modular open source app with CI that serves both static files on S3 and runs a docker container on amazon ec2 that hosts the api, all in a nice looking domain with SSL encryption.
Parseing websites to extract only relevant data.
Being able to scape form callback hell by using promises.
What we learned
In everyone of the accomplishments that we are proud of we had to learn something. Be it promises, DevOps, Regex or dom manipulation or ebook file structure.
What's next for eInkNews
We would really like to carry on working on this because we will use it for our day to day lifes. We hope someone else will find it useful too.