Quarkus Classify

Our goal is to shorten the development time of production-ready machine-learning applications

Comment

Situation:

On our team we work as an AGILE team with multidisciplinary skills. However, given its such a new field, machine learning applications are only developed by a restrict group of people. What this people have in knowledge of developing this specific type of applications, they sometimes lack in knowledge of building a highly-scalable application while addressing a wide variety of requirements such as security, testing, logging, observability, documentation and so on. For that reason we tried to use Quarkus framework to bridge both development pipelines.

Task:

Set up a full-fledged use case using Quarkus framework and its extensions with image classification as proof of concept since it's a well-known use of AI applications.

UI - Provide a frontend interface for users to create, read or delete image classifications;
BE - Provide a backend application with a REST API that handles UI requests and perform complex operations such as file upload, data persistence or messaging;
AI - Provide a machine-learning application that is capable of processing images using Python scripting;
INFRASTRUCTURE - Provide multiple technologies necessary for our use case such as Kafka, Prometheus & Jaeger.

Action:

We started by setting up our use case. We had two goals in our minds:

• Build a highly-scalable solution which aims to bring machine-learning applications to Java world;
• Show how easy is to use Quarkus and to integrate with other technologies by using its extensions.

Since machine-learning applications generally need a great ammount of time to perform, we decided to split up BE and AI concerns and communicate with each other using Kafka so it could outscale well. Our BE application aims to represent a more traditional Java application which runs on JVM and performs general operations using an imperative approach. On the other hand, our AI application aims to represent a new-world application combining Java and Python on same codebase, using reactive programming and acting as a serverless application by taking advantage of GraalVM native image and Knative resources. Finally, our UI application provides a simple way of interacting with our solution while also providing some neat notifications using Server-Sent Events (SSE).

Inspiration:

We were inspired by several blog posts and talks by Burr Sutter, Gunnar Morling, Edson Yanaga, Emmanuel Bernard & other great engineers from RedHat. Since we first heard about Quarkus we started thinking how could we use it in order to move towards the next wave on software development innovation. This hackaton served the purpose of exploring not only how can we use Quarkus ecossystem, but also on how can we take advantage of GraalVM and its native image and polyglot capabilities.

Learnt:

We learnt a lot. First and foremost, it was the first time we used Qute and its templating capabilities. Although it is on its early stages, it was quite pleasant to use. Second of all, we work on an environment where Vert.x is our first-nature approach to every single challenge. What we learnt is that with Quarkus we can unify imperative programming with reactive programming and use abstractions like verticles and event bus. Last, but not least, we learnt that embedding Python in Java using GraalVM is REALLY on its initial stages, so it's very unlikely that any Python program that requires imports at all will work using this approach.

How we built:

We wanted to showcase an end to end use case to reinforce how useful Quarkus can be in different situations. For that reason, we mixed and matched with lots of other technologies:
Java, Python, Kafka, Debezium, Docker, Prometheus, Jaeger, RESTEasy, Jackson, Qute, Microprofile (Health, Metrics, OpenAPI, Tracing), SwaggerUI, Hibernate, Panache, PostgreSQL, JUnit, RESTAssured, Testcontainers, AWS S3, Vertx, GraalVM, Qute, Maven, Flyway, K8S and so on.

These were the requirements that we tried to address:

FE:
• Create image classification; ✓
• Read one image classification; ✓
• Read all image classifications; ✓
• Delete one image classification; ✓
• Consume notifications (SSE); ✓

BE:
• Build REST API: ✓
• Create image classification; ✓
• Read one image classification; ✓
• Read all image classifications; ✓
• Delete one image classification; ✓
• Read and persist data from database (PostgreSQL, Panache, Hibernate); ✓
• Upload files to external repository (AWS S3); ✓
• Asynchronous messaging (Microprofile, Kafka, Kafka Connect & Debezium); ✓
• Produce notifications (SSE); ✓
• Logging; ✓
• Tracing (Jaeger); ✓
• Health (Microprofile); ✓
• Documentation (Microprofile, OpenAPI, SwaggerUI); ✓
• Testing (JUnit 5, TestContainers, Flyway, RestAssured); ✓
• Metrics (OpenMetrics); ✓
• Run on Docker with JVM. ✓

AI:
• Perform image classification: ✓
• Asynchronous messaging (Microprofile & Kafka); ✓
• Execute Python script (GraalVM Polyglot API); ✓
• Run on Native image (GraalVM Native Image). ✓

INFRASTRUCTURE:
• Setup Kafka on Docker; ✓
• Setup Prometheus & Jaeger on Docker; ✓
• Setup UI, BE & AI applications on Docker; ✓
• Setup BE DB on Docker; ✓
• Apply Debezium connector on Kafka Connect cluster; ✓
• Register all images on Docker Hub; ✓
• Setup EC2 instance; x
• Setup K8S cluster; x
• Setup Kafka cluster on K8S; x
• Setup Prometheus & Jaeger on K8S; x
• Setup UI and BE as standard deployment on K8S; x
• Setup AI as a serverless deployment on K8S. x

Challenges:
We came across lots of challenges. We had to learn how to implement tons of requirements and know how to break through frustration when it occured. The two things that created the most discomfort were:

• Trying to setup a kubernetes environment with our solution (we had success running our Quarkus applications using Knative, but we didn't had time to deploy the remaining infrastructure such as Kafka);
• Execute Python scripts embedded on Java (we wanted to provide a complex script which could actually prove that we can shorten development time of production-ready machine-learning applications, but it's not possible using GraalVM polyglot and native image at this given time).

Improvements in the future:
• Move infrastructure to Kubernetes;
• Perform static code analysis and code coverage with Sonar.

Result:

We ended up with a very neat solution. It presents a well-implemented use case using a wide variety of technologies while showcasing Quarkus features such as:

• Container First (usage of Docker & K8s components, but also JVM and native image options);
• Unifies imperative and reactive (usage of both programming models in different applications);
• Developer joy (usage of unified configuration and native executable generation);
• Best of breed libraries and standards (mix and match with lots of technologies).

Built With

  • debezium
  • docker
  • graalvm
  • hibernate
  • jackson
  • jaeger
  • java
  • junit
  • kafkaconnect
  • maven
  • metrics
  • microprofile
  • openapi
  • openmetrics
  • opentracing
  • panache
  • postgresql
  • prometheus
  • python
  • qute
  • restassured
  • resteasy
  • s3
  • swagger
  • testcontainers
  • tracing)
  • vertx
Share this project:

Updates