saga--speech-and-gesture-assistant

SAGA is a new innovation for using speech recognition and gesture commands in space with the help of a smartphone that consists the necessary hardware such as an Accelerator and runs on any android operating system. SAGA Features two modes : 1. Gesture Mode 2. Speech Mode

An astronaut on his venture to space can use these two modes inside and outside the space shuttle.

Outside the shuttle he can use the gesture mode which works on three different gestures :

Gesture Mode :

As the name suggests in command mode a used when the user is outside the shuttle. He can strap his phone to a glove or take it outside with proper casing as an external accessory.

Functions of Gesture Mode :

In Gesture mode there are three functions :

1. Astronauts can Send Emergency Texts to fellow companions using the Shake Phone Gesture.
2. Capture VIDEO using the TILT PHONE Gesture.
3. Capture PICTURES using the wave Gesture.

The Following must be done before the user goes outside the space shuttle :

When sending emergency texts , first the user has to type the Mobile Number to which the SMS has to sent and click on the save button in the app. The mobile number is saved in TinyDB and then the user can select the message from the list picker. Some examples of the messages are "Hello , All OK! and Mission Complete!"

When the user is outside the shuttle : If the Smartphone is attached to a glove he can shake his hands to send the message , else he perform the shake gesture to send the text.

Features of Speech Mode :

In speech mode the user can control the full app using his voice , it can be used both outside and inside the shuttle :

Features :

1. Control the whole app with voice.
2. Search Internet with voice.
3. Send emails and make phone calls with speech recognition. THE MOST EXCITING FEATURE :
4. LIVE Translation

Live Translation :

In live translation , I used the Yandex Translation API , Google Speech Recognition API and the Google Text-To-Speech API (TTS).

Uses Of Live Translation Feature :

During space ventures sometimes both astronauts and cosmonauts travel to space. And there can be times when it is difficult to understand each others language. Thus , I created a live translation feature in the app that can translate English to Russian or any host language and vice-versa. This will effectively help improve communication , and break barriers of communication between colleagues who speak different languages.

How does it work?

Let us take my example :

I am astronaut travelling to space with my colleague who is a cosmonaut is knows Russian and some English. There can be times when we cant understand each others language , thus we can use the Live Translation Feature in the app that does Back to Back Translation from My Language to Host Language and Vice-Versa. Currently it supports :

1.English 2.Chinese 3.Russian 4.Spanish 5.French 6.German

*Please Refer the videos for getting to know more and in detail about the project.

Project Information

Resources