Home Sweet Home: Encode THIS!
Phwew! What an eventful month it has been! I’ve been lucky enough to hit three conferences and four countries (not counting airport layovers), seeing new things, meeting new people, learning new facts, and gaining news ideas and inspiration along the way!
As far as “Positive Brain” matters go, I had an excellent experience at the 2010 Human Brain Mapping conference in Barcelona. Not that popular media is by any means the end all/be all metric of a good scientist, but there’s something cool about coming home and seeing people who you met appearing in the New York Times.
The buzzword (for me) from the 2010 HBM conference was: encoding. The number of top-notch researchers doing exciting work in this domain are too many to enumerate. If I have to drop names, though, I was extremely impressed by the flashy work of Jack Gallant that he presented at the advanced fMRI educational course that kicked off the conference. Gallant essentially gave an intro into his work with the bling-bling software package ‘STRFPak’ (not to be confused with ‘STFUp’) and demonstrated how it can be used to reconstruct sensory stimuli from encoded images of neural activity. It sounds like science fiction: basically, what Gallant and his team did was acquire fMRI images for 18,000,000 seconds of YouTube videos. Then, when a subject is shown a new video that they had never seen before, the software algorithms pull snippets from the 18,000,000 seconds of recorded videos and blend them together in proportion to how similar the brain activity pattern for each 1 second of cataloged video is to the 1 second of the new video that is being presented. For example, imagine that you catalog the brain activity pattern when someone is looking at a cat and a dog, respectively. Then they are shown the picture of a fox. The brain activity pattern corresponding to the picture of the fox might be fairly similar to the brain activation pattern from the cat, and moderately similar to the brain activation pattern from the dog. This is the “encoding” process–to create a library of neural responses to an assortment of stimuli, then to register how much similarity there is between the responses to known stimuli compared with the brain’s activation response to a new stimulus. The decoding part of the process is to take all of those assessments of similarity and to recreate, or in other words, to predict what stimulus the person was exposed to, based solely upon their activation pattern at the neural level when they were exposed to the novel stimulus. So in the example of seeing a picture of a fox, if your brain activation is somewhat similar to the brain’s response to a cat, and somewhat similar to the brain’s response to a dog, then the reconstructed (or decoded) image would be a blend of the dog image and the cat image.
Ok–so the fox example isn’t exactly the best one. But you can begin to imagine how if you are averaging the similarities between a novel stimulus and 18,000,000 recorded brain response patterns, you could start to get cool reconstructions of what the person is actually seeing…based solely on the activity we read in the visual cortex. This is cool stuff! It’s the fast-track to strong mind reading.
After the conference, I was able to spend a little bit of extra time in Europe before coming home. One of the highlights for me was going to the open-forest monkey park in Alsace, France. It was so delightful!–about 200 Macaques roam the park as visitors sit on benches next to them, watch them in a semi-natural habitat, and hold out their hand to share treats with the Macaques (given to the visitors by the park management upon entry). Their eyes were stunning to me–some may argue I am projecting, but I could see the sentience and cognizance behind those eyes. Monkeys have now moved up to the top of my wish-list for future pets. Completely intriguing, charming animals.
It’s back in the saddle now. Hopefully there will be good things to report about journal submissions before too, too long. In the meantime, I’m happy to be settling back in at home and catching up on emails and blogging! I love hearing from you, so don’t hesitate to say hello!