Internet fraud is evolving, the more you try to stop abuse, the more complex methods fraudsters use to trick you. If one really has to take action in real-time, there is a very small window available within which the decision has to be made on whether an online activity is fraudulent or not. The idea of a real-time fraud detector is not new,but very few solutions exist, primarily due to the complexity involved in making analysis within a few seconds and sometimes even in milliseconds. With Redis providing a highly scalable, performant and mature database that supports advanced data structures and extremely useful dynamic libraries called "modules", the idea was to build a real-time fraud detector with these features as the backbone of the application.
What it does
It detects fraud in real-time. For the project, we have specifically considered ad-network events to show how Redis data structures can be used to derive insights to label the click as fraudulent, based on which action is taken to disallow click monetization.
How we built it
The heart of the application is the Fraud Detection Module, which uses different data structures and Redis modules.
We needed a time series based data structure to identify if events were part of stacked ads(remember when you clicks one ad and multiple popups come over), for which Sorted Set is used. Redis Bloom is a probabilistic data structure that is fed with blacklisted IPs, the amount of memory saved here is incredible at the rate of very little and tuneable false positive rate. The bloom filter is looked up to evaluate if the event is from a malicious IP. Redis GeoSpatial is another extremely helpful module, which helps us in deriving location-based insights. Using services that can provide latitude and longitude values from IPs, we used the Geo-Spatial module to identify trends based on location. For example - the percentage of fraud in the 100km radius of the event location. Enriched data can be further used to derive more insights, and we use Redis streams to push the data downstream. Redis Gears is a subscriber to this stream which in our case is used for event transformations, map-reduce and micro batching use case. Redis Gears pushes the data to the Redis time series module, which powers the time series visualisation tool Grafana, on which trends can be analysed. And finally, the data can be persisted using Redis database . RedisInsight is a browser based tool to monitor Redis. Support for Redis Graph and Time Series makes it a handy visualization tool as well.
Challenges we ran into
Some of the modules we used weren't platform compatible, especially for Mac OS. We ended up using a redis-mod image which gave us all the modules in a single image since the module building process was failing. Since many of the modules were new or in beta, we did notice the lack of documentation (at the time of the hackathon).
Accomplishments that we're proud of
Redis has a proven track record when it comes to simplicity, extensibility and handling scale. Simplifying the application by using Redis stack alone makes it a lot easier to build and extend complex features. The application will be more stable and maintenance will be easier. Removing a lot of heterogeneous layers will help focus more on providing real business value, than having to manage interactions between components.
What we learned
Redis now is well-beyond just a cache or database. With advanced data structures and modules, Redis can be an important part of systems looking for time and memory optimizations.
What's next for Real-time Fraud Detection
-This application was built specifically with ad network fraud in mind for the hackathon, but internet fraud across domains have a similar pattern, which makes us think we can build a minimal functional internet fraud detector application with open chances to extend it for more domain-specific interests. -Artificial Intelligence plays a huge role in detecting internet frauds, modules such as neural-redis, RedisML, RedisAI can be used to handle this and some other complex use cases.