valkyrie

Valkyrie is innovative approach for gesture and voice recognition using a "smart" glove and mobile device(Android, iOS or smartwatch). You can use it for home automation as well as controlling devices such as robo arm, robots etc. Valkyrie is connected to the IOT using WiFi, BLE, Bluetooth Classic, UHF RF Transceiver and XBee, using universal protocol called V-API. This is Valkyrie the future way of gesture and voice control.

This project is solving the SpaceGloVe: Spacecraft Gesture and Voice Commanding challenge.

Description

Video link - https://www.youtube.com/watch?v=72cvLqnaD0c ValkyrieLogo

Slogan Features of project Valkyrie:

  • Innovative real-time motion processing
  • Voice control service
  • Smart gesture control detection
  • Smartphone applications
  • Wellness functionality
  • Open API for developers
  • Various communication interfaces

Glove: Project Valkyrie is aimed to complete the challenge "SpaceGlove". For the 2 days that we have worked on that project we have made a glove from leather and elastic material that is comfortable for everyday use and works about 70 hours on full power transmitting mode.

With that glove we made experiments to control many of the standard home devices(light bulbs, lights, fan heater, LED light strip, TV, setup box) with based gestures as slide, up-down, down-up etc. After that we made a service that is transmitting data from the glove to an iPhone, and we have modified the gesture data base, and added custom gestures. With 10% precision we have successfully distinguished 16 different gestures and added support for more than 60.

Control center: For controlling many interfaces we need a control center that is repeating signal received from any of the communication interfaces and send it to the other interfaces. The following interfaces are supported: Wi-Fi, Bluetooth Low Energy, Bluetooth Classic, XBee(ZigBee), 433 MHz radio transceiver and USB communication. We have integrated many libraries and unique and universal communication and implemented it in a V-API(Valkyrie API). The idea in the V-API is that is can be used with any kind of devices and using the universal protocol you can easily write the codes for communication. The V-API is explained in detail on slide 17 of the Hardware and firmware documentation included at the resources.

Glove sensor positions and descriptions: Alt text

Alt text

Alt text

Mobile applications: We have developed 2 applications for interfacing with the command center(and the glove) - one for Android and one for iOS. The Android application is only voice detection service using the google now search and if command is recognized it sends http post request to our online service(electric imp). The Android service is working on phone as well as on Android watch. The iOS application support custom made Voice recognition service, home automation control using voice or direct input in the application, weather station information.Our wellness service is checking your heart rate automatically every 30 minutes or continuously while your are training. It can also count you steps and detect you sleep quality. The application can also be used for uploading new firmware to the glove and to create more gestures to be recognized by the glove and bind them to execute a variety of actions. Alt text

Low level design, High level design, different diagrams and 3D files are available in the resources of the project below this text. All of the information about the project is available in our google drive and bitbucket repository.

At the moment we have fully functional glove, iOS application with all the properties that we need, remote controls for computer(glove acting like HID device), phone control though BLE HID service, smart home control with custom and already implemented gestures, robo arm control, computer game control, rover/car gesture control, Smart TV remote control and many more because of the universal V-API protocol which can be used at many places.

Q&A:

  • There is no gravity in the open space - we can use a magnetic transmitter to determine the 'zero' point in the space and use it to calibrate and sense pseudo gravity and angle velocity.

  • It is hard to use most of the GHz communication devices in the space - This is why we have 433 MHz transceiver on the Control Center. The VHF and UHF bands are able to penetrate the earth atmosphere and transmit low noise communication from the earth to space. This is used by the Radio amateurs to make QSOs using the International space stations as repeater.

  • Low data speed if the controlled device is far away from earth - there is not much that we can do about that, so we decided to make custom macros(for example if you are controlling robotic arm), you make the movement once with you hand then this is send to the robotic arm, it memorizes the movement and do it by command. For example on the video we have used this system to grab the box and move it few inches behind.

  • Connection drops while controlling something - If there is no connection for 300 ms, the operation stops until the connection is not resumed.

  • Charging - we are considering wireless charging, because with the new technologies in the wireless charging it is possible to charge 800 mAh battery for less than 40 min.

  • Power consumption - it is 13 - 15 mA in transmitting mode.

  • Can you use external devices (such as android phones, bluetooth joystic) for controlling Control Center Peripheral devices - Yes you can V-API is universal protocol for communication, all of the received reports are repeated to the other communication interfaces.

Images from the glove made on the 2 days of the space apps challenge: Real pressure sensor system. Alt text

Real heart rate sensor position. Alt text

Future development

  • To make Fully functional Android Application
  • To add more sensors to the control station
  • To improve and make clearly readable V-API libraries for C, C++, C#, Java, Java Script, Objective C
  • To cooperate with many universities and other science institutions all over the worlд
  • To make flexible PCB board for the glove
  • To develop custom Bluetooth Low Energy Services to be easy to use by Android/iOS developers
  • To add more documentation and establish support center for supporting developers using our V-API
  • To make few variations of the glove for example business, sport, casual and make the hardware to be easy changeable
  • Official Testing
  • Release of the project (If we start at 1.06.2015 until the end of september '15 we will have a product)

Team Behind Valkyrie

  • Martin Kuvandzhiev - Team Leader; Android, Firmware and Hardware developer - Technical University of Sofia
  • Peter Lazarov - iOS Developer - Technical University of Sofia
  • Borislav Fillipov - ERP developer - Technical University of Sofia
  • Phillip Yankov - Android, Web developer - University of Sofia
  • Georgi Velev - Hardware developer - Technical University of Sofia
  • Miroslav Dobrev - University of Sheffield
  • Violeta Kaneva - Glove design - Technical University of Sofia
  • Konstantin Zhlebinkov - QA - High school of natural science - Burgas, BG
  • Evgeni Sabev - Database research and QA - Technical University of Sofia
  • Krisi Ilieva - PR - Technical University of Sofia
  • Teodora Malashevska - Application Design - Technical University of Sofia

Project Information


License: MIT license (MIT)


Source Code/Project URL: https://bitbucket.org/rapiddevelopmentcrew/valkyrie/src


Resources


Electric imp data site - https://agent.electricimp.com/K5cBLKohWns7
Pulse sensor - http://pulsesensor.com/
Accelerometer & Gyro - http://playground.arduino.cc/Main/MPU-6050
Gesture recognition algorithm - ftp://ftp.tik.ee.ethz.ch/pub/students/2014-FS/SA-2014-21.pdf
Arduino Boards Site - http://Arduino.cc/
Electric imp(WiFi Module) - http://electricimp.com/
Valkyrie Documentation Directory - https://drive.google.com/folderview?id=0B8wUW9gi5NSjcGFGY2pFazFkdEU&usp=sharing
Vaklyrie Presentation - https://drive.google.com/file/d/0B8wUW9gi5NSjRXVrWGItdDFFUW8/view?usp=sharing
Valkyrie Hardware Firmware High&Low level design - https://drive.google.com/file/d/0B8wUW9gi5NSjTEhwTjZzUmxiQWM/view?usp=sharing
Valkyrie Communication Schematic - https://drive.google.com/file/d/0B8wUW9gi5NSjdjRhNjJyZmNlV00/view?usp=sharing
Video link if not working - https://www.youtube.com/watch?v=72cvLqnaD0c
Android Voice Control Demo - https://www.youtube.com/watch?v=0oupbX9aW2c&index=5&list=PLUC8LNkun3qo5jChBofkrYDxjCIO2YCcL
iOS App Demo - https://www.youtube.com/watch?v=6BTaMg41n6w&index=4&list=PLUC8LNkun3qo5jChBofkrYDxjCIO2YCcL
Gesture controlled presentation demo(Valkyrie presentation in Bulgaria) - https://www.youtube.com/watch?v=b-86kRnJe2M&index=3&list=PLUC8LNkun3qo5jChBofkrYDxjCIO2YCcL
Robotic Arm Test Demo - https://www.youtube.com/watch?v=Ynp7WAY0ONA&index=2&list=PLUC8LNkun3qo5jChBofkrYDxjCIO2YCcL
Control center & weather station demo - https://www.youtube.com/watch?v=GIg0eo2PfYA&index=6&list=PLUC8LNkun3qo5jChBofkrYDxjCIO2YCcL

Team

  • Miroslav Dobrev
  • Konstantin Zhlebinkov
  • Teodora Malashevska
  • Krisi Ilieva
  • Philip Yankov
  • Borislav Filipov
  • Petar Lazarov
  • Violeta Kaneva
  • Georgi Velev
  • Evgeni Sabev
  • Martin Kuvandzhiev


Loading...
×
Loading...
×