Scanpaths

Funny how things happen. I've been working on my implementation of scanpath comparison for what seems like years now. Then I came across an interesting experimental protocol that I've never heard of before that suggested a very simple and hopefully effective means for collecting eye movements to test my implementation. So above you see my eye movements as I twice look at the sequence 1-2-3-4-5-A-B-C-D-E. The key to scanpath comparison, IMO, is getting character labels assigned properly to clusters of fixations. This yields a string for each scanpath, in this case I have s₁ = abcdefaghijkllm and s₂ = nbecefaghkijollllp. Using string-editing that is used by both DNA sequencing practitioners as well as the diff program, my program produces S_s = 0.61 and S_p = 0.79, which are sequence and position similarity indeces, respectively, in this case "repetitive". There's more to this, but it's going into a paper that I'm working on. My REU students are collecting more eye movement data for me to compare a larger set of scanpaths. I think it should be a decent paper when I'm done with it.

libraw1394, libdc1394, OpenCV, and OpenGL

This summer with a bunch of visiting undergrads interning in my eye tracking lab, I thought as a side project we'd explore real-time face and eye detection. Ever since I visited UBC to attend a PhD candidate's defense (I was the external reviewer), I became interested in cameras from Point Grey Research (or PGR). They're pretty well-known as good "Digital Cameras", particularly for computer vision applications. I put the term in quotes because although your run-of-the-mill store-bought camera is also digital, the DC cameras differ from consumer-level electronics such as digital video recorders (DVRs) or webcams. The PGR cameras are supposed to adhere to the IEEE 1394 bus specs, and thus are to some extent software-controllable. To do this in code, one has to learn a collection of new Application Program Interfaces, or APIs, namely APIs to libraw1394, libdc1394, OpenCV, and OpenGL. libraw1394 is the low-level library that is used to actually talk to the hardware (1394 controller card that puts commands on the bus). The higher-level library libdc1394 is what is mainly used in C/C++ code to open a connection and start video streaming. The latter relies on the former, being a kind of wrapper that abstracts away the really nitty-gritty stuff at the bus level. OpenCV is a higher-level abstraction still, allowing the programmer to apply various known computer vision algorithms to the video frames pulled off the camera. (Technically speaking, one can just use OpenCV and let it act as a wrapper for libdc1394, but libdc1394 is supposed to offer various useful techniques for camera synchronization and hardware pixel culling that may or may not be in OpenCV, so it's probably a good thing to learn anyway.) If you google for "OpenCV face detection" or "OpenCV eye detection" you'll come across code examples of how to do what's pictured above. The only hitch is that those examples all draw the resulting face and eye rectangles directly onto the image frames that they process. Coming from a graphics background, however, I of course would rather use OpenGL to draw those rectangles. Sounds easy enough, but the problem is keeping all the coordinate frames straight. Face detection, as given by the examples, uses a shrunken image to do its work (for speed). This implies a coordinate scaling operation. Meanwhile, eye detection uses image segmentation (stipulation of a Region Of Interest, or ROI) within which to search for the eyes. This implies a coordinate translation, or offset. Meanwhile, two other transformations are needed to normalize the frame coordinates and then to scale them to the display dimensions, remembering to flip the y-coordinate to re-orient the origin to the lower-left (computer vision types think it is at the upper-left, whereas computer graphics types think it's in the lower-left). It took me a couple of days to get all this squared away, but I was finally rewarded with the code being able to locate my face and eyes—at least when you're looking at the camera, I think the algorithms used are therefore not rotationally invariant. Anyway, it turns out that face detection is fairly popular these days. The latest version of Apple's iPhoto uses it and lets you label faces in your pics. Having done so, it will go and search your library for other pics where it thinks that person may also be in. It works fairly well.

Are we having duck?

Delicious! A couple of weekends ago mein bruder visited the area and came over for our version of bistecca fiorentine. I don't remember now whether any photos were taken or whether I wrote about that, but we "invested" in a 3 lb, 3 in hunk of T-bone (with the tenderloin part no less) and prepared it a la Mario Batali's grilling recipe. It turned out excellent, warmed to the perfect 120 F interior that is medium rare. Mein bruder only stayed for one night, but we had actually prepared for two, having bought a pair of ducks for the night before the Italian fare. Since he didn't show up, we were left with two ducks in the freezer. So, consulting Batali's grilling book once again, I found a recipe there for "spit-roasted duck" that sounded fairly appetizing and relatively simple. We decided to try it after a day of soaking in the sun on our deck. The only trouble was our lack of rotating spit. However, I knew our BBQ (Weber model 2000LX—I need to record its model number for reference) had these little grooves made in its metal casting to support a spit, but of course, it was an extra accessory that I needed to find.

Saturday I went out in search of a rotisserie attachment to our barbie. I thought the same hardware store from where we bought the grill some 10 (?) years ago might actually have the right thing, but like in the movie Color of Money, when we rolled up to the store, we found an empty hulk of a building. Guess it went out of business. So on to ACE hardware. I found a kind of el-cheapo universal rotisserie ($40 vs. the "right" Weber type that runs about $80) that I thought would fit. The trouble with this thing was that the motor attaches to the shelf, which is really too low for proper operation. As a result the motor was raised halfway out of its mounting bracket. It would pivot precariously upon every rotation. Meanwhile, fixing the duck onto the spit was an adventure in itself. Those little metal prongs that are meant to hold the bird in place didn't tighten very well and my technique for skewering the duck wasn't good in the first place. We ended up really man-handling the bird onto those prongs to get it to stay. I was afraid we'd find the duck had fallen off at some point.

Eventually it turned out ok. One had to be somewhat vigilant because the spit would recoil out of the motor socket and would therefore stop spinning. If the motor had been fixed to the iron casting I suppose this would not have been a problem. But with this cheapo system the "fix" was to walk up to it and squeeze it together to re-insert the spit rod into the motor housing and watch it spin up again. After about 1 1/4 hrs the duck was done. The sauce for it was sort of Asian, based on orange marmalade, containing rosemary, garlic, and balsamic vinegar. Pretty tasty. The burnt parts were a result of the spit not spinning, but tasty nonetheless. We still have one more duck in the freezer. Not sure what to do with it, but I do like the idea of spit roasting: all the fat renders off so that the result is a less greasy duck than what you get out of the oven (at least that was my perception of it). We ate the whole thing with me getting the lion's share, leaving the bones and carcass for the makings of duck stock.

Shiny New Laptop

I just received a new MacBook Pro, courtesy of the School of Computing (thanks!). As it happens with new machines, one has to get through the pain of installing all types of software onto the new box. Everything from Apple stuff (iLife, iWork) to free but useful software such as Adobe Reader, Fink, and yes, even Microsoft software as well. I use Office on the Macs just to be compatible with Windows (l)users. I thought it was a bit funny that while some things installed ok, Microsoft's Office installation just sat there for about half an hour (I took a screenshot out of boredom and amazement). I think I was supposed to press some Ok button that got obscured by the Installer, but as the screenshot shows, how was I supposed to guess that? It just looked like it got stuck. Anyway, tonight the software install process continues. Now it's Fink complaining that it can't add ptex. WTF? I NEED latex/pdflatex, that's one of my bread and butter pieces of software. Not sure why it's compaining, but in the meantime I have to perform various reboots due to other things going in...we'll see how it goes.