Book Trigger Warnings API

Inspiration

I was delighted to find the Book Trigger Warnings website, which has an extensive list of trigger warnings for thousands of books. However, I was disappointed to find out that it had no API. I really wanted to use the information on the site to make an inclusive app that would recommend books while avoiding certain triggers, but I was discouraged because I knew it would be hard to create a database of all of the information on the site and to access that information quickly. In the end, I decided to just make the API myself.

What it does

The Book Trigger Warnings API has the same CRUD (Create, Read, Update, Delete) functions as any API does.

Create:

Users can add a book to the database by providing the URL to the book's page on the BTW Wiki. The API scrapes the webpage for the book's title, authors, genres, trigger warnings, main age group, publish date, and publisher and then adds it to the CockroachDB database.

Read:

Users can get information about a book (database ID, title, authors, genres, trigger warnings, main age group, publish date, and publisher) by providing either the book title or database ID.

Update:

Users can update book information in the database by providing the updated information.

Delete:

Users can delete books from the database by providing either the book title or the book's database ID.

How we built it

I used Flask (and therefore Python) in order to write the API. I used CockroachDB for the database, and I used SQL to code the commands used to access that database.

Acknowledgements

Code for connecting the Flask app to CockroachDB is based off of this repository and this tutorial.

Basic Flask boilerplate comes from this repository.

The information about triggers from certain books is from this website.

I used this tutorial to learn about how to web scrape Wiki articles and this tutorial in order to learn about how to build an API in Python with RapidAPI.

Challenges we ran into

I had never successfully connected to and used CockroachDB before, so that took me a while. I had also never built an API before, so I had to learn the basics through tutorials and a lot of trial and error. I had never used Python to web scrape before, so I had to learn that from scratch too. The Book Trigger Warnings Wiki is also full of inconsistencies, which made it difficult to scrape. Some pages on the Wiki still cannot be reliably scraped because there is no real standardized way of listing the book triggers on every page, since the Wiki was not written or created with scraping in mind.

Accomplishments that we're proud of

I'm so happy I was able to web scrape the wiki and to code all the CRUD functions for the API! Additionally, I'm glad that I learned how to connect to and to use CockroachDB because it's a powerful tool!

What we learned

I learned how to connect to and use CockroachDB, how to build an API, and how to web scrape using Python.

What's next for Book Trigger Warnings API

I need to add code so that the API can actually parse requests made using curl. At the moment, the CRUD functions are just being called in the main python file, so I'd like for people to be able to access them using the command line. I also want to publish the API so other people can use it easily.

Built With

• cockroachdb
• flask
• python
• sql
• sqlalchemy