Inspiration
Inspiration for this project was one of Whatsapps image editing tool. It is quite simple tool and there are already thousands of image filters/editors are out-there. Yes! the simple doodle that we can create in many social media platforms like Whatsapp, Instagram...etc
It was recently that one of my friend asked me to solve a Logical(mathematical) problem and also to explain it to him so at first i sent him this picture:
After few minutes he again sends me a message saying he is having trouble understanding it. So i again captured it drew some circles and other marks and also sent him a voice message explaining the whole concept of the solution.
It was this moment that it clicked in my mind that i had never seen such platform that would allow me to perform this specific function, meaning a platform that would allow students or teachres to create boxes on images and recording small audio clips for students to clear there confusions. Further brainstorming made me realize that this was something that had the ability to maintain 2 way flow of information rather than a single way like many Pakistani based sites do (ilm ki dunya, sabaq.pk, tutoria.pk) and keeping it Book allianced at the same time.
What it does
Initial prototype that we have developed has working functionality of what I have just mentioned above. In other words, an uploader(teacher/student) can:
- Upload Maximum 2 images
- For each image He/She can create 4 resizeable Annotations/Labels anywhere on the image.
- Similarly in each Annotation/Label He/She can record and upload upto 5 min of audio recording.
How I built it
For this project I decided to make the most out of the simple tool that were available. I didn't opt for a framework like REACT right off the bat, i got creative with
- Javascript.
- Php.
- Ajax.
- MySQL.
Since this was a fast-pace Hackathon I had to go with the Inline Scripting technique and break the logical code into small modules as much as i could, this decision helped me in process of debugging and bring changes to the code without any hassle but now there are over 80 files 😨.
One of the crutial parts was to make the Annotations/Labels or boxes, for this purpose I did first attempted to code a javascript but it was way to complex for it to be done in a small period of time. So i searched the internet and came across a very old plugin with poor documentation and full of bugs, btw now its completly deprecated and i was lucky enough that with some help from my Discord Friends i was able to fix it and even add new functionalities to it, about the plugin the name is Annotorious and the authors have allowed programmers to modify and redistribute the code without having to change the credits which are in the files.
By the time i was fixing the bugs i got enough knowledge that I am now able to easily add new functionalities to it. It does sounds simple after it has been created but the process of recording and keeping it sync with the annotation was very complex process.
Another crutial part was the image itself.. in other words the optimization of the images of which i will talk in detail in next section Challenges i ran into
Challenges I ran into
If you had been following by now you may have realized that this project consisted of 3 main pillars:
sequenceDiagram
Images ->> Annotation : Optimizing image in browser
Annotation -->> Images: Deleting if user wants to reupload
Annotation -->>Recording: Display recording button if there is Annotation!
Annotation -x Recording: Do not allow recording of there is no Annotation!
Note right of Recording: Once in editing state<br/>everything is saved<br/>in realtime meaning <br/>after each operation<br/> our system will keep<br/> track of everything<br/> and allow user to edit aswell
There were 2 main challenges:
- How could i reduce the uploading time for the images?
- How could i reduce the uploading time for the audio?
I know what you are thinking... why didn't i write in single point
- "How could I reduce uploading time for images & audio ?"
well it may sound simple and relatable for both the modules but its not.
Cameras on average these day can take upto 10 MB per image and if we are allowing 2 images then it becomes 20 MB only for uploading images 😨. On the other-hand, if we use the simple mic recording plugins that are easily available in the internet then it creates a file of size 100MB for only 1 minute recording in WAV format 🤔 so if we are allowing 4 labels per image and we have 2 images it totals upto 8 recordable audio files.. and it may be around atleat 500-600MB!!! 🤯🤯🤯...
Don't worry i did managed to solve it.
for the first challenge which was of images i managed to code a javascript function that allowed me to optimize the image by 70% on the users own laptop before even uploading.. the optimization process occurs when the user selects the image when the user clicks the upload button the upload function waits for the optimization to complete and hence 10MB file is converted into 200kb or so...
for the second challenge i embeded an encoder, meaning when the user completes the recording it it starts to convert the WAV to MP3 format this way the size of the files is compress without having to loose the audio quality or the content itself... However this process does takes some time, when the user stops recording the user may have to wait 5 to 10 seconds for the convertion to take place and for the upload option to show up
Accomplishments that I'm proud of
- I have failed.
- I have failed.
- I have succeeded.
- Bring this idea to life.
- having to participated in this Hackathon.
- I have made Problem Solving my second nature.
- I have made something that is completely Student-Centric
What I learned
During the event days the mentoring sessiong have been really amazing for me the whole process made me realize that entrepreneurship & Problem Solving:
- Are iterative Process
- Always have to think outside the box
- Always have a Design thinking mind and rational mind.
- Problems of any size can be broken down into small portions.
- How the Bussiness Eco -system works.
- How much sustainability is important
- How to monatize your idea.
What's next for JATAQ - "Teaching & Learning"
As I am student of COMSATS University Islamabad and my majors are AI. I have quite hands-on experience on image processing and NLP aswell. We can build:
- Noise removers.
- Image Enhancers.
- Voice detection based searched.
- Voice to Text and launch it in USSD platforms.
- To automatically complite a yearly class-wise notes.
Log in or sign up for Devpost to join the conversation.