Silly hacks is the inspiration, as the main theme suggests that you need to make something silly 😋 and something which brings smile on the person.
What it does
it grabs the image screen and the screen is fed into a neural network which predicts the output(i.e left,right, forward) and the output is used to drive a vehicle inside the game.
How we built it
first we collected the training data and we uploaded the training data into s3 and with the amazon sage maker we trained this on alexnet's tflearn model and stored back the trained model into s3. then from their we downloaded the model into local host and tested the trained model in our machine.
Challenges we ran into
we faced most of the difficulty in screen capture as it is game which requires high frame rates we need to capture screen as fast as possible and preprocess it and send it to the network so we spend a lot of time in reasearching on how to grab screen as fast as possible. later we faced difficulty in capturing dimensions of the screen as we need only particluar screen image not the complete image and we need to make sure this screen can be fed into a neural network.
Accomplishments that we're proud of
we are really astonished by the result of the trained model, it predicts exactly in which direction it need to move in frame even though it is not accurate in sharp curves but in a road with less traffic it performes amazingly well, we are really proud of our logic which is used in collecting frames and keys and also all this result is just because of our training data.
What we learned
we learned on how sagemaker can be useful in training a model easily on large training datasets, and how s3 can be used to store files, we learned about alexnets tflearn model, and learned new ways to preprocess the data before training the model.
What's next for GTAV automating
we believe if we make large training data and train the model then results will be much more accurate and this can be really applied on real self driving cars like we can build an arduino car and test the model on the arduino car and improve the models performance.