Writing in the air
Although computer technology would be almost inconceivable without traditional input devices such as the mouse or the keyboard, developers are working on modern solutions that could replace this hardware – such as camera-based gesture control or language commands.
State of the art
These new forms of interaction could above all provide considerable benefits in industrial areas with high hygiene standards or in unclean environments, since touchless control of computers is possible. As yet, the first camera- and language-based systems have only been used as auxiliaries. The reason for this is that the considerable data processing effort and dependence on environmental conditions such as lighting or noises lead to stability problems.
Scientists at the KIT Cognitive Systems Lab have come up with a new approach to hands-free interaction between humans and machines. They have developed a character recognition system that identifies hand movements with sensors and transforms them into a digital signal – so-called airwriting. With the aid of hardware and software, writing movements are turned into text inputs or commands in the air. All the user has to do is wear a sort of motion recorder attached to the wrist, for example a bracelet with sensors or a commercial smart-watch. When something is written in the air, the pattern of motions is detected with the aid of motion and acceleration sensors and transmitted wireless to the PC or the software. Here, the motion data is evaluated or compared with the models that have been stored. Following the principle of machine learning, models have been stored stroke by stroke for the individual letters and characters as a word catalogue, and the software has been trained to use them. They correspond to a statistical model of the motion process when writing is in progress. This also enables imaging of more complex systems of writing such as Chinese. In the matching process, the models are compared with the sensor data, and the result presented is the most probable sequence of words or characters.
Modelling inside the system allows simple adjusting to different languages, writing systems and gestures. The technology can be applied with any computer system and independently of the conditions in the surroundings of where it is used.
Options for companies
KIT is looking for partners to advance and apply it in co-operation with the KINEMIC spinoff.
Your contact person for this offer
Karlsruhe Institute of Technology (KIT)
Innovation Manager New Materials, Climate and Environment, Innovation and Relations Management (IRM)
Phone: +49 721 608-26107