SignSpeak was built at uOttaHack, where our team ended up winning 1st Place. The project started from a simple problem — most sign language learning tools felt inaccessible, expensive, or passive. A lot of existing platforms relied on static videos with no actual feedback, so learners had no way of knowing whether they were signing correctly.
what we built
We wanted to build something more interactive. SignSpeak uses computer vision to analyze hand movements in real time and help users practice sign language directly from their browser. The platform combines lessons, quizzes, and live gesture recognition to create a more engaging learning experience instead of just memorization.
tech stack
Built with React, Next.js, TensorFlow, and MediaPipe. MediaPipe tracks 21 hand landmarks in real time through the webcam, while our TensorFlow model compares those landmarks against trained gesture patterns to evaluate whether a user is signing correctly.
technical challenges
One of the hardest parts was making the recognition system accurate across different users and environments. We had to think about lighting conditions, camera quality, hand sizes, skin tones, left-handed signing, and subtle gesture differences between signs. Our team spent hours collecting training data ourselves, recording gestures from different angles and conditions to improve the model's accuracy.
Running TensorFlow models in the browser forced us to think carefully about performance. We had to optimize model loading, prevent UI blocking, throttle predictions, and balance responsiveness with accuracy so the app still felt smooth in real time.
learning curve
The project also became a huge learning experience technically. Most of us came into the hackathon with very limited experience in Next.js, TensorFlow, and real-time computer vision systems. We were learning things like client-side rendering, model optimization, browser-based ML inference, and webcam processing while actively building the product.
what i learned
More importantly, SignSpeak changed the way I think about accessibility and human-centered design. Before this project, accessibility felt like something people added at the end of development. Building SignSpeak made me realize accessibility needs to shape the product from the beginning. Every design decision became a question of who could use the platform comfortably and who might be excluded unintentionally.
What stayed with me most wasn't winning the hackathon, it was the feedback from people in the deaf community after our demo. That was the moment I realized technology becomes meaningful when it removes barriers and helps people connect with each other.
SignSpeak was one of the first projects that made me see programming as more than just building software. It showed me how technology can create genuinely inclusive experiences when it's designed with empathy first.