Interaction by motion gestures on Android – Modified Gallery3D Project
by Avdo Hanjalic and Erik Bakker
The first idea for interaction by motion gestures on Android was to create a service that maps motion gestures (accelerometer input) to system events, e.g. zooming, scrolling etc. This we hoped to accomplish by running the service in the background, that records accelerometer values, and triggers zooming/scrolling events as done by 'pinch-to-zoom' and swiping the screen. One of the innovative components we wanted to implement was zooming by displacement (accelerating the phone in some direction).
Initially we have started on experimenting with the accelerometer values by creating an 'ordinary' application. After a while we were able to use the tilting angle of the phone as input to the scrolling/zooming function. However, zooming by acceleration without tilting, i.e. displacement over 1 axis, appeared too hard to implement smoothly; in practice, when accelerating the phone, it appears that the phone is always tilted, which makes it impossible to distinguish acceleration from tilting for low acceleration values. High acceleration values did give the results we aimed for, but this is useless in practice because people do not feel like moving the phone as if they were retarded monkeys. Because of this we have decided to skip zooming by acceleration (displacement).
Returning to the initial idea, implementing the solution as a system service, we ran into Android limitations; It is not possible to trigger events without rooting the phone, modifying and recompiling the source code for every phone apart. The Android SDK does not provide any tools for doing this, in the contrary, it blocks external event triggering.
After this second setback we have taken a look at the possibilities to implement something useful with the code we had already written. Gallery3D appeared to be a very interesting project, which was modifiable too. Finally we have added our motion navigation support to the application for navigation through albums and pictures. We have published the results on xda-developers, where the application was received with quite some interest. It appears that the final choice, with usefulness as main criterion, was quite a good one. The application has been downloaded more than 500 times within the 3 first days.
We are planning to add some functionality in the future, in the hope that our additions will be included in one of the future 'stock' versions. For more information on the planned improvements we refer to the topic, where all future work will be announced and published.
Our conclusion is that the Android SDK is too limited to add innovative applications to the OS. The only way is to modify the source code. Funny is that Google seems to have implemented the initially targeted functionality in Android 2.3. However, at the moment of writing Android 2.3 still is only available for the Google Nexus S. Furthermore, the accelerometer can be seen as a orientation sensor, instead of a true acceleration meter. To be able to use acceleration for smooth interaction with the phone, one should add a dedicated acceleration sensor, which does not depend on orientation changes.