Caltech researchers developed the illusion above to illustrate postdiction, a sensory phenomenon “in which a stimulus that occurs later can retroactively affect our perceptions of an earlier event.” From Caltech Matters:
“Illusions are a really interesting window into the brain,” says first author Noelle Stiles (PhD ’15), a visitor in biology and biological engineering and a postdoctoral scholar–research associate at USC. “By investigating illusions, we can study the brain’s decision-making process. For example, how does the brain determine reality with information from multiple senses that is at times noisy and conflicting? The brain uses assumptions about the environment to solve this problem. When these assumptions happen to be wrong, illusions can occur as the brain tries to make the best sense of a confusing situation. We can use these illusions to unveil the underlying inferences that the brain makes….”
Postdictive processing has been demonstrated within individual senses, but this work focuses on how the phenomenon can bridge multiple senses. The key to both of the new illusions is that the audio and visual stimuli occur rapidly, in under 200 milliseconds (one-fifth of a second). The brain, trying to make sense of this barrage of information, synthesizes the stimuli from both senses to determine the experience, using postdiction to do so.
Read more in the researchers’ scientific paper: “What you saw is what you will hear: Two new illusions with audiovisual postdictive effects” (PLoS ONE)