Filipino Student Innovator Builds AI Robot That Gives Sign Language a Voice, Wins International Award

SANTA ROSA, LAGUNA, Philippines — Driven by a vision to make daily communication accessible and inclusive, a student researcher has developed an AI-powered robotic arm capable of translating sign language into real-time speech for speech- and hearing-impaired communities.

The project, officially titled "DUM-E: Prototyping an AI-Powered Robotic Arm Assistant Integrated with Sign Language Translation using ESP32-S3 to Create an Interactive Assistant Technology," earned 2nd Place in the Creative Innovation Category at the Electronics & Robotics International Competition (ERIC) 2025, hosted by the Engineering Faculty of Universitas Negeri Jakarta, Indonesia, on September 24–25, 2025.

Spearheaded by student innovator and team lead Alexander Mathew R. De Leon, the project began as an ambitious personal initiative funded out of his own daily allowance savings in March 2025. Aiming to go beyond voice-only smart assistants like Siri or Alexa, De Leon set out to build an interactive physical assistant equipped with a robotic pick-up arm to bridge communication barriers faced by deaf and non-verbal individuals.

The system utilizes an ESP32-S3 microcontroller to drive a custom 3D-printed robotic arm, servomotors, and a TFT touch display. The microcontroller connects wirelessly over Wi-Fi to a processing system running OpenCV, TensorFlow, and PyTorch to analyze live camera feeds, recognize American Sign Language (ASL) gestures, and convert them into spoken words. Additionally, DUM-E incorporates an emotion-aware conversational model that detects context and emotional cues, adjusting its mood and responses to deliver empathetic interactions. While the prototype is not yet perfect, its core concept and essential functionalities are already fully operational.

While De Leon built and programmed the system independently with his team, he credits the technical skills that enabled the build to the hands-on hardware and engineering training he received from Evtech Research Group, where he currently serves as a Product Developer.

"Evtech gave me the technical knowledge and hands-on training to assemble electronics, create 3D models, 3D printing, and develop projects from scratch," De Leon shared. "Through this training, I became capable of turning an idea into a working prototype and completing projects such as DUM-E. Without them, I may not have realized that these kinds of projects were even possible. Them and the many setbacks also taught me that leadership means carrying a team forward, taking responsibility for the result, and continuing even when the situation feels impossible."

Beyond foundational technical training, bringing DUM-E to the competition in Jakarta proved to be an intense test of technical agility and perseverance. After pulling an all-nighter to submit their qualifying research paper just before the deadline, the team secured an invitation to compete but faced severe financial delays regarding plane tickets and component procurement.

The situation escalated just 24 hours before their flight when the robot's primary control computer, a Raspberry Pi, short-circuited during pre-departure testing. As the lead developer responsible for the setback, De Leon owned the mistake and took full accountability for resolving the crisis. At the critical moment, he improvised an emergency setup: re-routing hardware control to a spare Arduino Uno, offloading AI processing to a laptop, and recording video demonstrations of the robot's full feature set as a fallback.

Upon arriving in Jakarta after receiving last-minute travel assistance from city officials, the team discovered that transit vibration had damaged the 3D-printed chassis, disabling one arm, while crucial connector cables had been left behind. Assisted by competition organizers who lent a replacement cable, the team presented the surviving functional features alongside recorded video feeds demonstrating the complete system capabilities.

Despite the setbacks, including a short-circuited primary control computer, transit chassis damage, missing cables, and an improvised fallback setup, DUM-E still won 2nd Place for the Philippines, falling behind by just five points in system reliability given the severe circumstances.

"After the presentation, I felt certain that we had no chance of placing because of all the setbacks," De Leon recalled. "The experience was a roller coaster of frustration, sadness, anxiety, joy, and relief, but in the end, we were able to present the project and proudly earned second place for the Philippines."

Eager to improve the prototype's performance and fix its initial hardware limitations, the team reinvested their remaining resources post-competition to completely overhaul DUM-E. De Leon rebuilt the system architecture with an ESP32 primary controller, a larger touchscreen interface, an updated AI recognition pipeline, and a dedicated mobile app controller. The upgraded prototype subsequently secured Champion Overall at the 4th Regional Robolution held at Don Bosco Technical Institute of Makati, Champion in the Health & Wellness Category at the Pilipinas Robosports Tournament 2025, and a 3rd Best Research Project institutional recognition from local school and city officials.

Reflecting on the journey from short-circuited hardware to champion status, De Leon emphasized that engineering resilience is built through facing failure head-on.

"Those engineering memes are not jokes, they are warnings," De Leon noted. "Still, if you find this field fulfilling, no hardship should stop you. Expect difficulties, do not be afraid to fail, and learn from every failure. Most importantly, keep learning, and have fun!"