Inspiration

"Training Deep CRNN with Determined AI for Text Recognition" - As an AI enthusiast, I was always fascinated by scene text recognition and its role in various applications. The ability to recognize and extract text from an image has wide-ranging implications in document scanning, OCR systems, and even self-driving cars. This inspired me to take up the challenge and develop my own scene text recognition model. I used the Determined AI platform to train and optimize my model, which allowed me to experiment with different hyperparameters and architectures more efficiently. Determined AI will enable me to train and work efficiently on such a huge and complex dataset Working with Determined AI was an exciting and rewarding experience that helped me push the boundaries of what I thought was possible in my project.

What it does

Scene text recognition is a deep learning project that involves recognizing text in images of natural scenes using Convolutional Recurrent Neural Networks (CRNN) trained on synthetic data. The project implements an end-to-end trainable neural network for image-based sequence recognition.

The codebase is built using PyTorch and includes several important files, such as the CRNN structure in the crnn_structure.py file, which defines the architecture of the neural network. The predict.py file can be used to make predictions on new images, while the train.py file is used to train the CRNN model on new data. The project also includes a pre-trained model that has been trained on the Synth90k dataset, which is stored in the checkpoints/crnn_synth90k.pt file.

The project has been enhanced with the use of Determined AI to improve its training and prediction capabilities. Determined AI provides a scalable platform for training deep learning models, with support for distributed training and hyperparameter tuning. By integrating Determined AI into the project, users can take advantage of its advanced capabilities to accelerate model training and improve prediction accuracy. The config.py file has been modified to include Determined AI-specific settings and hyperparameters, allowing users to easily configure the training process to take advantage of Determined AI's advanced features.

How we built it

The Scene Text Recognition project was built using PyTorch and consists of several key components. The first step was to download and preprocess the Synth90k dataset, which was done using the download_synth90k.sh script. This script downloads the dataset and generates training and testing data in the form of images and text files.

Next, the src/config.py file was used to define various hyperparameters for the model training process. These include settings for the learning rate, batch size, number of epochs, and other important parameters.

The src/models.py file contains the architecture of the CRNN (Convolutional Recurrent Neural Network) model, which is based on the paper "An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition (2016), Baoguang Shi et al.". The model consists of convolutional layers to extract image features, followed by a recurrent layer to capture sequence information and a fully connected layer for classification.

The src/train.py script is used to train the CRNN model using the Synth90k dataset. During training, the model is optimized using the CTC (Connectionist Temporal Classification) loss function.

Once the model is trained, the src/evaluate.py script can be used to evaluate its performance on the Synth90k test dataset. This script uses various decoding methods, including greedy decoding, beam search, and prefix beam search, to obtain predictions from the trained model.

Finally, the src/predict.py script can be used to perform text recognition on new images using the trained model. The script takes a list of image paths as input and outputs the predicted text for each image.

To integrate Determined AI into the project, the determined.yaml configuration file was created to define the hyperparameters and training settings for the model. The src/train_determined.py script was then created to allow the model to be trained within the Determined AI platform. This script uses the determined_pytorch.TorchTrial class to define the model architecture and training process, and the determined_pytorch.DataLoader class to handle data loading.

Overall, the Scene Text Recognition project was built using a combination of PyTorch and Determined AI, and consists of several key components including data preprocessing, model architecture, training and evaluation scripts, and integration with the Determined AI platform.

Challenges we ran into

There were several challenges that I ran into while working on this project. One of the main challenges was to find an appropriate dataset for scene text recognition. I needed a large and diverse dataset that included images of various fonts, sizes, and orientations, and it was difficult to find such a dataset that was also free to use.

Another challenge was optimizing the hyperparameters of the deep neural network. This process involved a lot of trial and error, and it was time-consuming to run multiple experiments to find the optimal configuration.

Finally, deploying the trained model was also a challenge. I needed to ensure that the model was able to handle new input images and that the output was accurate and meaningful. This involved testing the model on various real-world scenarios, and tweaking the parameters and architecture as needed.

Accomplishments that we're proud of

Throughout the development of this project, I faced various challenges but was able to overcome them with the help of the Determined AI platform. I am particularly proud of successfully training the CRNN model on the Synth90k dataset using the platform, which would have been a daunting task without it.

One of the biggest benefits of using Determined AI was its ability to efficiently manage and distribute the training process across multiple GPUs, resulting in faster training times and ultimately a better-performing model. Additionally, the platform provided useful insights and visualizations that helped me optimize the hyperparameters and fine-tune the model.

Overall, the combination of PyTorch and Determined AI made it possible for me to develop a robust scene text recognition system and showcase its capabilities in the demo. I am confident that this project can be used in various applications such as automatic license plate recognition, document digitization, and more, thanks to the successful integration of Determined AI.

What we learned

Through this project, I learned about the capabilities of the Determined AI platform and how it can be leveraged to simplify the training and deployment of deep learning models. I was able to integrate Determined AI into my project and utilize its features such as distributed training, hyperparameter tuning, and experiment tracking.

I also learned about scene text recognition and the challenges that come with it, such as dealing with different fonts, sizes, and orientations of text in various backgrounds. I gained a better understanding of the CRNN architecture and its use for sequence recognition tasks.

Overall, I am proud to have successfully integrated Determined AI into my project and used it to train a model that achieved high accuracy on the Synth90k dataset. I have also gained valuable experience in scene text recognition and deep learning, which can be applied to future projects.

What's next for Training Deep CRNN with Determined AI for Text Recognition?

The next step for training deep CRNN with Determined AI for text recognition would be to explore more advanced models and techniques to improve the accuracy and performance of the current model. This can be achieved by experimenting with different types of neural network architectures, such as attention-based models, and using more diverse and larger datasets for training. Additionally, hyperparameter optimization with the help of Determined AI can be performed to further improve the performance of the model. Finally, the trained model can be deployed in real-world applications for text recognition, such as optical character recognition (OCR) systems for digitizing printed text.

Built With

Share this project:

Updates

Submission history