2011-2012-S Ho QRS

SHoQRS (formerly known as FLOP, formerly known as SHoQRs)

by Gabe van der Weijde and Stefan de Ruiter

The problem

We wanted to create a software product which would help in solving one of the biggest problems in a human life: laziness! We all know the drill, you’re sitting behind your computer, working hard to make a deadline or maybe you are in the middle of a serious world changing game that you must win. And then it happens: your coffee mug is empty and you can’t or won’t make time to walk to the kitchen to make fresh coffee and later grab that coffee. You would want to avoid unnecessary work.

It would be nice if you could just show your empty coffee mug to your webcam and your coffee machine would then turn on. Then you only have to walk to the kitchen once!

The solution

We at first wanted to make a program that would to QR code recognition and map those codes to corresponding actions. So then if you would print a QR code on an item (for instance a coffee mug) and then show it to your webcam, it would turn on your coffee machine. We called it SHoQRS which stands for Smart Home QR Scanner.

Later on we decided that QR codes aren’t actually that innovative and that it would be nicer to be able to recognize objects themselves and then map them to actions. So then we renamed our project to FLOP which stands for Fast Lazy Object Processing.

At the end of the project we weren’t able to implement working object recognition so we decided to change back to the original QR code scanning, which works.

The process of getting there

At first we started out prototyping. This in our case meant that we made a paper simulation of the pop-up messages that would be displayed in the lower right corner of the computer. Three different kinds of messages would appear for respectively recognition, confirmation and processing. The library used for the QR-Code recognition is called Zebra Crossing (or ZebraXing for short). It's the primary QR code library used on mobile phones (in various applications) and proved very easy to integrate into our (java) code base. We made use of the Civil library to acquire images from our webcam.

The algorithms

To implement object recognition, we made use of the OpenCV set of libraries (implemented through JavaCV). We tried the to use the SIFT, SURF and (briefly) ORB algorithms to detect objects, but all proved to be extremely unstable, slow and prone to crashes. This, along with the very poor documentation of OpenCV, eventually lead us to give up on object recognition and focus on making the QR-code recognition better.