Neuromagicology: At the Intersection of Art and Science

We all know how the cameras in our phones are only so good. The photos look grainy and the colors washed out. Compared to the naked eye, phone cameras don't seem to compare.

Well actually, the camera in your smartphone is 2 1/2 times better than your eye! In other words, if the resolution on your phone camera is 5 megapixels, the processing power of your eyes roughly equates to about 2 megapixels. But then, how is it the world around you looks so much sharper, richer, and full of color than the photos on your camera roll? It's because you have something your phone can't even begin to emulate, the brain.

Don't believe me? Hold your arms straight out in front of you. Put the tips of your thumbs together with your index fingers pointed up towards the ceiling, so you're making mirrored L shapes, or one big U shape. Now close your left eye and look at the tip of your left index finger with your right eye. While looking at your left finger tip, focus your attention on your right finger tip. Did it disappear? If it didn't, wiggle it around a bit, and you'll see what I mean; you'll notice that it suddenly vanishes from sight.

That's because your right finger is sitting squarely in the blind spot of your right eye. There are no light-sensing photoreceptors there because that's where all the fibers that make up your optic nerve converge. It has been there all your life, yet you don't notice it until an illusion forces you to. You might have noticed that instead of your finger where it should have been, you just saw the wall or the computer screen or whatever your finger was in front of. What's going on here is the brain is "filling in" that blindspot with the stuff around it. Kind of like the clone stamp tool does in Photoshop.

Illusions reveal the "supreme achievement of the brain"

For a long time, illusions have been thought to be the tools to reveal the limitations of the visual system and show where the brain "got it wrong." Neuroscientists Stephen Macknik and Susana Martinez-Conde see it differently. They think illusions really reveal something special, magical even, about the brain. "This is one of the supreme achievements of the brain," says Stephen, "The brain has actually evolved these processes that are illusory for the purpose of improving vision."

Not only do these illusions show us the nature of our visual experience, but they can also tell us something about consciousness. Consciousness is the first person experience of your life in the world, and it is home grown in your brain. Your senses interact with the outside world and send electrical signals to your brain to make sense of them, but when you look at these sensory systems, "you realize that the information going [to the brain] is really quite deprived." When so little information goes in, the brain has to fill in the details. The so-called conscious part of your brain comes from a separate group of neurons that takes information from your sensory and cognitive systems, your memories, your attention and other systems, and cobbles it together to make a simulation of reality. As Stephen puts it,

That simulation of reality is the only thing you've ever interacted with, it's not that the real world isn't out there--it is--but you've never been there. You've only ever interacted with this simulation of reality that's put together from sparse information from the outside world and the rest is essentially confabulated, just like that blindspot is a confabulation of sorts.

From illusions to magic.

Stephen and Susana are very interested in how our attention and awareness, through the visual system, can be manipulated and what that manipulation says about the process--or confabulation--of consciousness. Illusions can certainly help, but they really only pertain to vision, not awareness and attention in particular. But while organizing a conference for the Association for the Scientific Study of Consciousness in Las Vegas, a little magic happened for Stephen and Susana.

They were brainstorming on how to generate public interest in the topic of consciousness, and they realized that they needed to study the artists of attention and awareness. But who would that be? "Finally it got through to us, Las Vegas spoke to us directly. It said, 'Magicians are the performance artists of attention and awareness.'"

Things took off from there. Stephen and Susana have worked with some great names in magic, like James Randi, Penn & Teller, and Apollo Robbins. Magicians, in the pursuit of bettering their art, have come up with some great theories about how the brain works that neuroscientists have yet to test in the lab. Having these theories before you start to research can also take years off the research process, and really help advance the field of awareness, cognition, and consciousness.

The intersection of art and science

Susana focuses her research on eye movements. There are two different types of eye movements, saccades (which I talked about before) and smooth pursuit. To see the difference. hold out your thumbs in front of you and look at your right thumb. Now try to move your eyes in a line from your right to your left thumb, and you'll notice that you can't do it. Your eyes seem to "skip" along a line to your left thumb. That skipping from point A to point B is called a saccade. Now look at your right thumb as you move slowly to your left, and now you can follow it smoothly, hence "smooth pursuit." So we've just demonstrated to ourselves that smooth pursuit eye movements are involuntary. Now for the Magic.

Apollo Robbins is a professional thief. His act involves very close-range sleight of hand where he pick-pockets from people right before their eyes. Through his art, he noticed that if he moves his hand in a straight line from someone's pocket, people will look at where the hand is going to go, and then immediately back to the pocket through saccadic movement, and that this is a good way to distract someone, or trick them into thinking that he stole something from that pocket when he really didn't. But when he moves his hand in an arch people have to use smooth pursuit to follow his hand, and they don't look back to the pocket at the end.

Apollo's observations led Susana and Stephen to think that perhaps smooth pursuit and saccadic movements affect attention differently, and prompted them to do a study. They found that with straight arm movement away from the pocket from which an item was "stolen," the attention of the thief-ee is directed through saccadic motions from the pocket to Apollo's hand, making the pocket the last place where the thief-ee had their attention, and thus they look back at it. But smooth pursuit eye movement directs attention to the hand as it moves away from the pocket, and there's enough time in between that the pocket isn't the next logical point of attention anymore.

And that isn't the only example, either. Magicians will "use humor in order to, basically, get away with magical murder. If they get people to laugh, their attention is suppressed." When you think about it, this might seem obvious, but there actually isn't any literature in neuroscience on the emotional modulation of attention. Studies on PTSD and anxiety get at the idea, few have looked at the effect of emotions other than fear on attention.

Sleights of Mind

In their book, Sleights of Mind: What The Neuroscience of Magic Reveals About Our Every Day Deceptions, Stephen Macknik and Susana Martinez-Conde explore just that. They look at how magicians intrinsically understand the mechanisms of our attention and awareness and what their manipulation of those mechanisms can tell us about how our brain constructs our sense of reality from sensory stimuli.

Not only is this book very educational, but it's fun! Sleights of Mind is just as much about magic as it is about neuroscience. It's a great read for anyone, regardless of their background in science, who wants to know more about the brain and how it can be hacked. To order the book, and see some really awesome videos, illusions, and more, visit sleightsofmind.com!

The Advantage of Being Cute

Most people would agree that babies are pretty damn cute. Put a grown man or woman in a room with an infant, and all bets are off, that baby is getting 100% of that man or woman's attention. Wild horses could not stop most people from cooing a baby, yet we don't usually question why. The truth?

Carolyn McGraw

That baby is pretty much helpless on its own, so if it's going to survive, it needs lots of attention from adults. Until it gets to the point where it can walk and talk on its own (and then some), the baby is going to lure you into caring for it with those adorable chubby cheeks and wide eyes full of wonder.

Ethologist Konrad Lorenz first put forth the idea that the typical cute baby, with a large head, round eyes, small nose and mouth, elicits a caregiving response from adults, and even suppresses aggressive behavior.  These features, known as a baby schema or a Kindchenschema, are a pretty useful thing for a baby to have. However, until recently this hypothesis was a bit shaky. Most studies that looked at people's responses to infantile adorableness used line drawings or unmanipulated photos of babies that could not control for other aspects known to affect emotional responses such as facial symmetry.

With the help of Photoshop, researcher Melanie Glocker and her team at the University of Pennsylvania created a situation where participants would only see differences in the baby schema of infants and be able to rate their cuteness and how much they desired to care for the babies. They took pictures of babies and created three photos of each baby: one undoctored, one changed to maximize the baby's cuteness, and one to minimize it.

In the center, the unmanipulated photo. With the less cute manipulated photo to the left, and the cute one to the right.

Then the researchers got a group of 122 undergraduates and split them into two groups: one group would rate each photo's cuteness, and the other would rate how much they wanted take care of the baby in each photo, both on a 1-5 scale.

As expected, participants rated the "high cuteness" manipulated photo as being significantly more cute than both the undoctored and "low cuteness" manipulated photos. Participants in the caregiving group also rated themselves as having a stronger desire to take care of the cuter babies.

Other research has suggested that the emotional impact of cuteness is influenced by female sex hormones, so the researchers hypothesized that women would be more strongly affected than men by cuteness. In fact, men and women rated the cuteness of babies pretty equally, and both men and women had a stronger desire to take care of cuter babies than less cute babies. However, women rated their desire to take care of babies of all cuteness higher than the men did. The researchers suggested that this could be a cultural as well as biological predisposition as historically in many cultures, women have generally been the primary caregivers of children.

The researchers also mentioned that before the rise of the nuclear family, childrearing was often done with the cooperation of friends and extended family. This could explain why both men and women have a strong desire to take care of babies, and why this desire extends to babies that they are not related to. After all, we are social animals, so we have to look out for each other!

Reference:
Glocker, M.L.; Langleben, D.D.; Ruparel, K.; Loughead, J.W.; Gur, R.C.; Sachser, N. (2009). Baby Schema in Infant Faces Induces Cuteness Perception and Motivation for Caretaking in Adults. Ethology 115(3): 257-263.

Special thanks to my wonderful and wonderfully talented friend, Carolyn McGraw, for making that awesome Admiral Ackbar drawing! You can see more of her stuff here and here.

Blindsight and Consciousness, what can we learn from the blindsighted?

If there were ever a perfect example of an oxymoron, the term blindsight would be it.

Other than the best oxymoron ever, what is blindsight?  Alan Cowey, in his 2010 review article, The blindsight saga, describes it as such:

It is the ability of patients with absolute, clinically established, visual field defects caused by occipital cortical damage to detect, localize, and discriminate visual stimuli despite being phenomenally visually unaware of them.

In simpler terms, it's the ability to sense the presence of objects in one's visual field without consciously seeing them the way normal sighted people do.  While it would be a stretch to call blindsight a superpower, the visual capabilities of a person with blindsight could be considered akin to those of Dare Devil.  Blindsighted people often are able to identify visual stimuli, although they deny having a conscious experience of actually seeing anything.

In this video, a blindsighted patient is able to navigate a field of obstacles successfully, even though he can't see them.

How does blindsight happen?  It is a fairly unique condition; not all people who are blind possess blindsight.  It occurs in patients who become blind in part or all of their visual field after suffering damage to the primary visual cortex, known as V1.

The primary visual cortex (V1) highlighted in yellow.  The bottom view is from a mid-section of the brain, the top view is from the outside.  In both views, your eyes would be on the left.  Source.

Becoming totally "cortically blind," as is the case for the patient in the video above, is actually pretty rare (thankfully), so most patients with blindsight are only blind in part of their visual field, while the rest of  the field remains normally sighted.

A controversial subject

Blindsight has drawn a lot of controversy among researchers.  Since the hallmark trait of blindsight is responsiveness to visual stimuli without the conscious experience of perceiving it, it makes animal studies a little difficult.  You can observe where a cortically blind animal turns its attention when presented a stimulus, but you can't ask it if it actually saw anything.  So researchers interested in getting at the question of what people with blindsight actually do or do not experience have a relatively small pool of subjects they can perform research on.

Then, of course, there's the fact that whatever blindsighted patients report as their experiences of blindsight must be taken with the grain of salt as all experience is subjective.  Some patients report that, even though they didn't see anything, they have a feeling that "something happened."  The question, then is, is this feeling essentially visual in nature?  Did it come about due to light scattering from the stimulus into their seeing field?  Or did the subject report a feeling of something happening because that's what he or she felt that the researchers were looking for?  The answers to these questions have proven to be extremely difficult to tease apart.  Regardless, the fact that some patients have "feelings" when presented a stimulus and others do not requires that blindsight be split into two categories: type 1, no awareness at all; and type 2, awareness without visual experience.

Even in patients with type 2 blindsight, the awareness of the stimuli isn't always consistent.  But we can still learn something here.  In one task, patient GY was asked to discriminate between the presence or absence of a stimulus and then wager money on his answer.  Incorrect wagers would be subtracted from his winnings, thus prompting him to only wager high when he was very confident of his answer.  When he felt that he was aware of the stimulus, despite not seeing anything, he wagered high and was correct more than 90% of the time.  When he didn't report any awareness, he wagered low, as would be expected.  But he was correct more often than could be explained by chance, suggesting that even without awareness, he was still able to perceive visual stimuli.

Total cortical blindness

Let's re-visit the patient in the video, TN.  After successive strokes, TN's primary visual cortex was completely destroyed, as was later verified by both structural and functional MRIs.  Something helpful at least came of TN's misfortune, in that he gave researchers De Gelder et al a rare chance to study a person with total cortical blindness.  The video above is truly astonishing because, at first glance, it looks like he is able to avoid obstacles without the use of a cane or any outside guidance.

Nevertheless, this video is not without its critics.  The first that springs to mind is echolocation, a capability that has been shown in other blind people.*  The researchers recognized that this could not be ruled out, but many critics of the research suggested that not enough attention was paid to the possibility of echolocation anyway.  Other critics say that TN could have been unconsciously processing auditory signals from the researcher shadowing him (to make sure he didn't stumble or fall), and that those aided in his navigation.

TN's contributions to blindsight research don't just end with that video.  TN also demonstrated affective blindsight, or the ability to discriminate between emotional stimuli in a physiological sense.  Researchers Gonzalez-Andino et al showed TN pictures of various facial expressions and monitored his brain activity via an EEG.  Without any awareness of the stimuli whatsoever, the researchers were able to localize changes in the electrical activity of parts of the brain associated with emotional stimuli, such as the right amygdala, in TN.  In these trials, TN was also not asked to guess at the nature of the stimuli, either, so the results suggest some perceptual ability without any conscious awareness at all.

Conclusions: what blindsight can tell us about consciousness

It is extremely important to note that all studies on blindsight and consciousness are done with very small sample sizes.  Most studies are case studies that focus on the abilities of only one patient.  Because of this, everything that we can say about blindsight must be taken with a grain of salt.  There simply aren't enough subjects with blindsight to tell us very much with any certainty about the nature of consciousness and vision.  Nevertheless, some results are so provocative that they can at least give us clues and ideas about where consciousness in vision lies in the brain, and can give us leads for further and more focused studies in the future.

It has become clear that blindsighted patients' vision is certainly unlike normal vision, and obviously unlike total blindness.  There is definitely some ability in patients to perceive objects, even if the true nature of that ability remains murky.  And it's the murkiness of that ability that makes blindsight so tantalizing to researchers interested in the neural mechanisms of consciousness.  How can one see without actually seeing?  That is the big question.  The visual pathway is certainly very complex, but the blindsighted just may be able to tell us what aspects of that pathway give rise to the conscious experience of sight, and wouldn't that be so cool?

*While looking for the video about Ben Underwood, I found out that just a few years after the original news spot was filmed, he sadly died of cancer.  You can read more about him and his life here.

References:

Cowey, A.  (2010).  The blindsight saga.  Experimental Brain Research 200:3-24.

de Gelder, B.; Tamietto, M.; van Boxtel, G.; Goebel, R.; Sahraie, A.; van den Stock, J.; Stienen, B.; Weiskrantz, L.; Pegna, A.  (2008).  Intact navigation skills after bilateral loss of striate cortex.  Current Biology 18,24:1128-1129.

Gonzalez-Andino, S.L.; de Perlata Menendez, R.G.; Khateb, A.; Landis, T.; Pegna, A. (2009). Electrophysiological correlates of affective blindsight.  NeuroImage 44:581-589.

How film makers are using your own imagination to scare you

Happy Halloween!  It's a time of costumes, candy, and for those more thrill-seeking types, horror movies.

Personally, I'm a total wimp when it comes to scary movies.  Show me anything that's even trying and failing to be scary, and it will still scare me.  So that got me thinking, why is it you can walk into a movie feeling like this:

See this:

And suddenly feel like this:

After all, there's nothing inherently scary about that image.  It's just a forest at night.  But it's the fact that it's a forest at night in a scary movie that has me sitting on the edge of my seat.  This is a phenomenon called priming, wherein the fact that I know that this is a scary movie and scary things will happen will make me more likely to "fill in the blanks" of that scene with my imagination.  In other words, since I know something scary will probably happen soon after being presented the visual stimulus of the dark forest, I will begin to look for something to scare me in the scene when nothing inherently scary is there at all, while still proceeding to scare the living daylights out of myself.

Movie Magic!

Film makers are very, very aware of this phenomenon and they love to exploit it.  After all, it makes their job a lot easier.  If you can scare yourself by just imagining what's happening off-screen, then the film makers don't have to go through all the trouble and expense of actually showing you, and your imagination will almost always come up with something more horrifying than what they can show you on screen, anyway.  Movies like Paranormal Activity and The Blair Witch Project capitalize on this by showing you shaky, home-movie style shots and lots and lots of scenery without ever showing you the source of the threat throughout most of the movie.  The idea is that by presenting these otherwise neutral scenes with the implication of a threat intensifies the emotional reaction of the audience.

The Science

So what's actually going on in your brain while this is happening?  A recent study in Social Cognitive and Affective Neuroscience aimed to figure that out.  Researchers observed subjects' brain activity in a fMRI as they read two different types of sentences; one which implied a fearful situation, and another which was neutral.  In the fearful type of sentence, none of the individual words themselves were inherently fearful--that is to say that there were no words like "threat" or "hurt" or the like.  An example of one of the fearful sentences used in the study is "The boy was never found again."

In the next trial, the researchers showed the subjects neutral images (like a boy on a beach) and paired them with the fearful and non-fearful sentences.  In a final trial, the subjects were shown the images again, but without the sentences to see if the emotional memory of the images with the sentences would carry over without the presentation of the sentence.

The Results

When subjects were presented a fearful or non-fearful sentence with and without a picture, there were higher levels of activation in subjects presented a fearful sentence with and without a picture than in subjects shown non-fearful sentences with and without pictures.  These areas of activation were the middle temporal gyri, the temporal poles, and the left inferior frontal gyrus, which are associated with language processing and understanding.

Brain activity in people presented with fearful sentences

The researchers also found an additive effect in the right temporal pole when subjects were shown a fearful sentence with a picture (the right black bar) than when subjects were shown a fearful sentence alone (the left black bar).

The temporal poles, which are the front-most projections of the temporal lobe, are still poorly understood in their function.  However, they are connected to many structures in the brains emotional, or limbic, system and they have been implicated in the processing of emotion, and binding emotions to linguistic and visual stimuli (such as associating a fearful looking face with feeling fearful yourself).

So far, one brain structure has been conspicuously absent from this study on fear: the amygdala.  The amygdala is often referred to as the "fear center" of the brain, so why has it been so quiet up till now?  It would appear that the visual stimulus is necessary in this case to cause the amygdala to react to the fearful sentence.  Subjects who were shown fearful sentences with images had higher levels of activation in the right amygdala, whereas subjects who were shown fearful sentences without images had no activity above baseline in the amygdala.  This would imply that the amygdala does not necessarily interpret emotional salience from language alone, leading the researchers to conjecture that perhaps the amygdala can only be activated in this context with linguistic-emotional binding input from the temporal poles.

The amygdala also has an interesting role in the third trial of this experiment.  Researchers showed subjects pictures that were either previously paired with a fearful sentence, a non-fearful sentence, or no sentence at all, and monitored activity in their right and left amygdalae.  Pictures that had been previously shown with a fearful sentence led to a higher level of activation in the subjects' amygdalae than did pictures that were previously shown with a non-fearful sentence or no sentence at all.  This is in line with other evidence to show the role of the amygdala in emotional memory.  That is to say that when the image was previously shown with the fearful sentence, it was "tagged" by the brain as emotionally salient.  The presentation of the image again, even without the fearful component, will bring up the emotional flavor of the image, and lead to higher levels of activation in the amygdala.

Our brains are excellent at drawing connections between various stimuli in our environment.  We can take inherently un-emotional words, phrases, and images and combine them to form context and illicit emotion.  So the next time you go see a horror movie, take a moment to observe how artfully (or perhaps artlessly) the movie is taking advantage of and manipulating your own imagination to scare you even more.

Images courtesy of ragemaker, we <3 it, and Social Cognitive and Affective Neuroscience
Reference:
Willems, R.M.; Clevis, K.; Hagoort, P.  (2011) Add a picture for suspense: neural correlates of the interaction between language and visual information in the perception of fear.  Social Cognitive and Affective Neuroscience, 6(4), 404-416.

The Stopped Clock Illusion

Everyone can relate to this. You're in class, at work, waiting for something... you look at the clock and that first second that goes by seems to take forever. Then every second after that appears to progress normally. Why is that?

This is due to a phenomenon called saccadic masking or saccadic suppression. A saccade is the rapid movement of your eyes from one point of attention to another. To demonstrate this to yourself, hold out your two thumbs in front of you, and try to move your eyes smoothly from your right thumb to your left. You'll notice that your eyes don't move smoothly. Instead, they jump to points between your two thumbs along the way.

Saccades allow us to make a mental map of our surroundings and all the points of interest within it.  This is because the part of your retina directly behind your pupil, the fovea is packed with receptors that enhance visual acuity.

Cross-section of the eye, Wikimedia.

When you look around, your eyes don't span smoothly across the scene in front of you. Instead they move in saccades, quickly directing your fovea from one object of interest (the sharp corner of that table that you don't want to walk into) to another (that cute guy/girl waiting for you at the other end of the room). Saccades also occur when focusing on the details of a single image, as demonstrated by these eye movement traces of subjects as they examined the bust of Nefertiti.

Source: MIT

What you don't notice in between these saccades is well, anything.  The saccade itself is so fast that your brain doesn't have enough time to process the information coming to it to make a clear image.  A blurry image isn't too helpful and would probably just give you motion sickness.  In fact, the shaky "hand-held" camera effects in the movie Cloverfield did just that to many of its viewers.

So then what does your brain do with the information sent to it during a saccade?  Nothing; this is what is meant by the terms, saccadic suppression and saccadic masking mentioned earlier.  Even though the eyes are sending information to the brain, the brain does not process the information, leaving you effectively blind during a saccade. However, saccades are not so fast that you wouldn't notice the lapse in vision. What's going on? You don't actually perceive being blind during a saccade but you also don't see the blurry image, so what are you seeing?

It turns out that once you've fixed your fovea on an object, your brain actually tells you that you've been looking at it from the beginning of the saccade.  You don't notice this difference in timing at all, unless of course that object of your gaze actually keeps time.  So even though you're focused on the clock for only a second, your brain is telling you that you've been focusing on the clock for the 1,000 milliseconds it takes the second hand to move plus the time it took for your eyes to move to the clock.

On average, a saccade takes about 100 milliseconds, or about 10% of one second. So if you happen to look at a clock right at the beginning of a new second, it will appear to take 10% longer than normal, resulting in the famous illusion known as Chronostasis, or the stopped clock illusion.