TABLE 26: due to the nature of our code, we are unable to submit it through GitHub and thus would like to request that the judges come to our table.
As a team of two Mechanical Engineers, an Electrical Engineering and a Computer Scientist, we came to the hackathon driven to explore an interdisciplinary problem. With an eagerness to explore natural language processing through this project, we wanted to convert a user audio into text. However, with 3 Hardware Engineers on the team, we wanted to take this a step further by making this a printer. We felt that something like this would be useful for those who are blind, as braille printers are not only not readily available in the general public but are extremely expensive for private use.
What it does
The Braille printer is activated when the user says "start recording." The printer then records the user's speech until the user says "stop recording". The recording is then relayed back that information to the user to confirm the correct information has been recorded. This confirmation is done through sentiment analysis of the user's response to whether or not they confirm that the information is accurate. The text is then extracted from the audio file into a string. That string is then encoded and sent to the arduino. Each character is parsed in a switch case in the arduino, thus determining its corresponding braille character. This information is then translated into three degrees of motion (x,y,z) in the printer, which determines the location of and creates the depressions required to document the braille character.
How we built it
The speech to text code was written in Python using libraries such as the Natural Language Processing Toolkit (NLTK) and speech recognition. To do the keyword matching, we use multiple threads, one to perform the recording and one to match the keyword. The threads communicate using kill signals.
The arduino takes the encoded string sent from the python script, character by character, and was used to encode the motor instructions corresponding to each letter. For example, if a "b" was passed in, the motors were coordinated so that it would press once, move right and press again. We created functions for spaces, moving to new lines and moving on to the next letter to make the final product resemble braille.
Finally, the printer itself is made of 3 separate motor-axle systems that allow movement with 3-degrees of freedom: the first motor system moves along the paper to specify the line on the page where we print, the second system moves along the line to the position where the braille character must be imprinted, the final system moves up and down along the z-axis and makes the incisions for the braille characters. The main task for designing each system was to come up with a way to convert the rotational motion of servo motors into translatory motion.
For the first motor system, we used a scotch yoke system: the main shaft (constructed out of pen body tubes constrained together) contained a slot-shaped out of key rings, and the motor is mounted separately with an arm that fits through the slot in the main shaft. The arm rotates inside our slot thus converting the rotational motion of the motor into translatory motion of the main shaft. The main shaft itself is made out of two beams constrained together like an L: while one beam acts as an axle for the first motor system, the second beam (constrained orthogonally to the first), acts as a frame to mount the second motor system.
To move to the position of the character on the line, we use a horizontal pulley system, that would allow us to move horizontally along a page. The rope for the pulley is made from a lanyard and the pulley itself is a pop socket which is attached to the motor. This pulley system has the piece used for making braille incisions attached to it: the horizontal pulley moves to bring the incision piece into position. The pulley (attached with the motor) is mounted onto the first rod of the frame (constrained onto the second beam of the main shaft) and the other end of the pulley system is attached to the third motor system that governs the up-down motion of the incision piece.
Finally, the mechanism for the z-axis has the motor connected to an L-shaped bar made from a wooden stick and a paper clip that causes a push along the cap of a mints box which is attached to our x-axis pulley system and thus causing the pressing motion for our braille imprints.
Challenges we ran into
The primary challenge that we faced was working with the materials we were given and trying to make the most out of limited supplies. Further, due to the lack of access to gearboxes or stronger motors, we could not achieve a strong enough torque to cause the depressions we were aiming for the paper. Thus we experimented with different types of foam surfaces that could be depressed, however, those also did not work. Hence, in order to have a final product that acts as a proof of concept, we decided to use ink prints.
Accomplishments that we're proud of
With Off the Brailles, we built something that is a coherent whole, and fits together well. There is a definite use case for our product, which is what we hoped to accomplish at this hackathon. We're really proud of the design and construction of the printer, as well as the integration between the hardware and speech-to-text software.
What we learned
We learnt that we have the ability to design and construct something useful out of materials available to us in a short amount of time. We also learnt how to design good software and hardware that integrate well together and work as a system.
What's next for Off The Brailles
See you at the next BRH!