Real-time heatmaps

Some of you already know what the above images are about, but I'm so happy about getting this to work, that I thought I'd post a blog about it. It happens to coincide with the end of this year's REU (Research Experience for Undergraduates) program during which I had 5 undergrads in the lab working on several eye tracking projects. This year it was all about video, which prompted me to develop the program responsible for drawing the above images. Collecting eye movement data (x,y,t) over video is what I worked on in Barcelona. It took me pretty much most of those 6 weeks to get enough C/C++ code together to be able to display video while recording gaze data. Once that was done, I handed the program over to the REUs who then ran four studies and who also extended the program to do various other things. Meanwhile, the whole effort motivated me to figure out how to display the captured data atop the video frames as a means to visualize the recorded gaze data. The algorithm for generating the above heatmaps is pretty straightforward and is well-known. Step 1 involves dropping a Gaussian point-spread function at each gaze location, growing the resultant heightfield with as many gaze points as collected per each video frame. Step 2 requires finding the maximum value in the heightmap. Sounds easy, but for an NxN image, it takes O(N^2) operations. Step 3 then requires normalization of the heightfield (division by the max value). Step 4 then recolors the height (luminance) by mapping it to the rainbow color palette. The last two steps, which can be combined into one, together take another O(N^2) steps. The image above at left was created this way for a data set of 24 scanpaths (sequence of gaze points) on the CPU. Looks good but it's slow (took about a minute). The image at right took only a fraction of a second and looks almost identical. The trick here is to use the GPU to reduce the number of operations form order O(N^2) to O(log(N)) for the max value localization and O(1) for the recoloring. On one particular workstation with a decent graphics card I observed a 700-fold speedup due to these reductions. That just blew me away, which is why I'm so excited about this development. I recently moved that bit of GPU code onto my video playing code and sure enough, even for a fairly large data set (oh, about 8 people or so), the code appears to play the video at real-time (30 Hz) rates. I suppose I should take timings of this just to confirm how long it takes...this could make a nice little paper someplace. Other eye tracking types might like to know how the whole thing is put together...

Reedy River Gig

The pic is back from our gig on July 7th. Right in the middle of a heat wave, when it went up to 98F. If I look like I'm trying to concentrate, I was. I don't know whether it was the heat, the Gretsch drums, or something else or a combination of factors, but I was not having a good first set. I think it played ok, but I know I made a couple of flubs here and there. I pulled out my old Gretsch drums but I haven't played on them in a very long time. Lately I've been playing on the Tama rockstar kit I traded for my old Pintech electronic set. I like the Tama's larger toms and I think its compactness—one less floor tom. The Gretsch kit has two floor toms and maybe because of that seems more spread out. I remember missing the right crash symbol because it was out of my immediate reach (had to stretch for it). So after the gig I transferred the bass drum's resonant head (with the band name on it) onto the Tama kit and bagged the Tama kit. I don't think I'll be using the Gretsch kit any more, so it may be time to start looking for a new owner for that kit. Right during setup I busted a resonant head on the small 10" tom...it sounded kinda crappy the rest of the night, another reason for a lackluster first set...once I got into the habit of going for the second tom, and the temperature went down, the second set was much better.

Greenville Gig at Reedy River

Tomorrow the Hoodoo Hounds play at the Reedy River Nighttime Concert Series. Just ahead of this gig we got a story on us printed in the Greenville News, our local city paper, as you see above. The full story can be found here.

4th of July

As soon as I got back from Barcelona, it was straight back to work. I was already a week behind on my meetings with this year's REU students, so it's been pretty hectic. Today is the last day of the 4th of July long weekend, and since I'm stuck at home on the couch, I thought I'd update the blog. Somehow I managed to pull my left pec muscle a couple of days ago, which is why I'm sitting here coding and not out on the boat. I guess it must have been lugging the new sailboat battery that did it, but I can't be sure. All I know is my left pec is so painful that it's difficult to breathe. I wanted to get out on the sailboat, give it a nice cleaning, and putt around on it, but I'm having trouble moving around much. It's nice to have caught up to some badly needed coding, but it's really nice outside, too... Hopefully this goes away by tomorrow, because that's when I need to load up the truck with my drum gear for Wednesday's Greenville gig. Tonight is band practice, when I'll see how I can play in this condition. Major annoyance. Anyway, above is a short video of the fireworks that the city of Clemson puts on every year at the YMCA beach. We get out there on the motorboat and sit and watch (a major reason for getting the powerboat back in '03 I think it was—the idea was to have it on the water by July 4th—I think we got it in the water just in time). This is the first video upload I've done on the blog, hopefully it turns out ok. I shot the vid on my iPhone, so it may be a little blurry, although I think it managed to focus itself fairly well.

BCN-AMS-DTW-YOW

Although I had the option of taking the train to Barcelona's BCN airport, in the end I decided to cab it. It was either spend 6 hours at the airport or get 6 hours worth of sleep. I went with the sleep and hit the sack at 9pm. Got up at 3am, hailed a cab (lots of them around), paid the 26 euros, and checked-in. The airport was surprisingly busy for 5am—I pissed off some American ladies by cutting in front of them in the queue, but then jumped back out when I saw the "elite" line empty. Due to my frequent flyer miles I am privy to this service and so it took me 5 mins to check in vs. standing in line for what looked like it would take 30 mins–1 hr. Security was fairly smooth throughout the trip, including Amsterdam's Schiphol. The plane closest to the window in the pic at right is at the gate from where I departed and on a similar plane, the Airbus A330.

The AMS-DTW flight was its usual 7-8 hour duration, this time I watched the Academy winning Jeff Bridges in Crazy Heart, and it was about the only thing left in the selection that I hadn't seen that I was interested in. I'd already seen Sherlock Holmes (very good, btw, if you like Guy Ritchie movies, as I do) and The Book of Eli with Denzel (also pretty decent). Oh I forgot! I hadn't yet seen The Damned United about British football ca. 1967—1974 and about Brian Clough, a coach of that era. It starred the guy who played Frost in Nixon/Frost, and it was surprisingly very good. After that flight, I had sushi at DTW then hopped on to the flight to YOW where I am now. I decided to stay a few nights in the University of Ottawa student dorms to save a few bucks—46 a night is hard to beat, although I have to use the common washroom and I now miss having my own kitchen! Even though I thought that the little kitchen in the Barcelona apartment was pretty dinky, it's particularly good for breakfast items, something my stomach is now demanding :)