openFDA for Splunk

Search recalls and events on prescription drugs, food, and medical devices

Comment

Inspiration

The inspiration behind openFDA for Splunk was the lack of an easy way for people to use the vast amount of data available through the openFDA data service. Not only is this freely available, but it can have high importance who want to know about medical products or prescription drugs they are using, or even the food they are consuming. The amount of available information on openFDA for food recalls in your location is pretty fascinating! Just try to search for 'ice cream' and 'Seattle' in the Food Recalls dashboard!

How it works

openFDA for Splunk works by using a node.js modular input to read data, by date, from the openFDA api for any or all of the following cobinations: drug (enforcement, event, label), device (enforcement, event), or food (enforcement). The application is smart enough to know what has already been indexed by report id, and also smart enough to skip report dates which have already been indexed.

Challenges I ran into

  1. Finding the right open dataset for something that I found interesting.
  2. Writing the modular input to be generic enough to work with all the openFDA data set types, and to be smart enough to not waste openFDA API credits for data that has already been indexed.
  3. A lot of the data on openFDA is only as good as it is reported to the FDA. For some types, you'll see that they are many future dates, even some centuries in the future!
  4. Getting the entry ready in time! I only found out about this competition two weeks ago. I started from scratch, finding an interesting data set, and coding up everything as fast as possible from there.
  5. Deciding where to focus on the application. With this data, there is SO much more that can be done!
  6. Some of the openFDA data is so far in the past that Splunk does not support the dates, so I had to restrict data to 1/1/2000 just to be safe (even though I'm sure that I could have stretched it back to the 70's).

Accomplishments that I'm proud of

  1. Getting the application ready from scratch in a very short period of time.
  2. Writing the modular input in node.js, a language I have not written modular inputs in before
  3. Creation of an application that I feed will be very beneficial to its users, giving them information they need on important subjects where the data is actionable.

What I learned

  • openFDA is a wonderful resource, but it full of inconsistencies and is only as good as the data reported to the FDA
  • openFDA has label information for a LOT of prescription drugs. I want to eventually link that information to the drug events and recalls.

What's next for openFDA for Splunk

  • More friendly dashboards and reports
  • Reports of companies with the most issues (events) being reported by medical device or drug
  • Linking drug events and prescriptions to their label information
  • Analysis of the drug label information, reports that show commonalities
  • Open source the application!

Built With

Share this project:

Updates