Report by Yves Degoyon, workshop leader (along with Lluis Gomez i Bigorda), OpenCV workshop at BALTAN Laboratories, September 21-27, 2009
After 2 years of playing around with computer vision in various contexts and installations, working mainly with performers and choregraphers, we were eager to share these experiments with other people interested in the subject and very interested to test our recently published library: ‘OpenCV for Pure Data’ in a wider context and with people involved in different kinds of applications using
This opportunity was given to us during the workshop organized by BALTAN Laboratories in Eindhoven, from the 21st to the 27th of September 2009. A variety of people showed up with different interests and needs and we tried to provide solutions for their projects using the toolbox we are developing, based on OpenCV.
Niko Knappe and Tara Pattenden (TAIK Media Lab, Helsinki) work in new media installations and were curious to learn new interaction techniques using a simple camera. After inspecting the different algorithms available, they opted for some interaction based on shape recognition and realized a first prototype of a ‘Fruit Detector’, working in the direction of making playful environments.
Mateu Batle and Alberto de Rodrigo Aparacio (IBIT Foundation, Mallorca) work on rehabilitation and were very interested in gesture recognition. They showed us a very impressive program already made with OpenCV that encourages the patients to make certain gestures and thus practicing also by playing games. Although their application was very specific, they were interested in the modular approach of pure data that enables them to prototype a solution that can be later be coded in a specific program.
Qi Wu (Volkswagen, Germany) was interested in a very specific use of computer vision applied to an industrial context, the face recognition in the context of images taken from the street, from someone driving a car. This application seemed very difficult as the conditions of lightning and of camera angles can be very changing, and face recognition works well when the context is less moving
and people are facing the camera. He experimented with OpenCV and he was finally glad that the face recognition has been improved in OpenCV version 2.0, which he could install on his machine.
Michael Aschauer (artist, Vienna) just came the first day to have a overview of the state of the art in computer vision for PD. He obviously had studied and practiced before and, after one day, he already had a working prototype of tracking cars from a camera set up on a highway.
Wendy Ann Mansilla and Jordi Puig (NTNU, Trondheim, Norway) were here to improve their knowledge in computer vision, putting the focus on the tracking of a skeleton in order to make an installation that combines OpenCV for PD and Blender and that will be an immersive installation that will be developed during their residency at BALTAN Laboratories and shown later at Piksel.
Francesco Bruno (Italy) is working on developing a gesture recognition engine that can be installed on machines for aerobics training in a gym. He experimented with various techniques and finally opted for the technique of markers and color filters that would enable people to command some computer program without touching any screen or keyboard.
The group of students from the TU/e Industrial Design department were here also to make a prototype of a music browser commanded by gesture recognition, a project where they are all involved working in groups. They already had a prototype working in MAX/MSP, but for obvious reasons of distribution rights of MAX/MSP, they were looking for an Open Source equivalent, so that the application could be distributed easily to a wide number of people. We gave them the hints for transforming their MAX/MASP patch into a Pure Data patch, that could do the same or maybe more. It’s funny to note they were using a python external in MAX/MSP that was first written for Pure Data.
And Geert Mul and Erik Overmeire (BALTAN tracker project) came to see what was now available in Open Source as they are doing their project up to now using MAX/MSP. After experimenting with OpenCV for PD, they also agreed that the toolkit we proposed was equivalent to what exists in MAX/Jitter, with the additional advantage to be open and extensible.
From the variety of projects we have seen in this workshop, this was a very interesting workshop that enabled us also to locate some flaws in our library and we spent the last two days fixing these issues that led us to release a new version of OpenCV for PD, version 0.2-rc4, codename BALTANIK: http://www.hangar.org/wikis/lab/doku.php?id=start:puredata_opencv
Some videos of the most advanced prototypes made in the workshop are available here: http://giss.tv/dmmdb/index.php?channel=pdopencv