ART & ORGANISM
notes on
FILLING-IN
completing perception, even at the cost of illusion
____________________________
FILLING-IN
eLife digest To make sense of the world around us, we must combine information from multiple sources while taking into account how reliable they are. When crossing the street, for example, we usually rely more on input from our eyes than our ears. However, we can reassess the reliability of the information: on a foggy day with poor visibility, we might prioritize listening for traffic instead. But how do we assess the reliability of information generated within the brain itself? We are able to see because the brain constructs an image based on the patterns of activity of light-sensitive proteins in a part of the eye called the retina. However, there is a point on the retina where the presence of the optic nerve leaves no space for light-sensitive receptors. This means there is a corresponding point in our visual field where the brain receives no visual input from the outside world. To prevent us from perceiving this gap, known as the visual blind spot, the brain fills in the blank space based on the contents of the surrounding areas. While this is usually accurate enough, it means that our perception in the blind spot is objectively unreliable.
To find out whether we are aware of the unreliable nature of stimuli in the blind spot, Ehinger et al. presented volunteers with two striped stimuli, one on each side of the screen. The center of some of the stimuli were covered by a patch that broke up the stripes. The volunteers’ task was to select the stimulus with uninterrupted stripes. The key to the experiment is that if the central patch appears in the blind spot, the brain will fill in the stripes so that they appear to be continuous. This means that the volunteers will have to choose between two stimuli that both appear to have continuous stripes. If they have no awareness of their blind spot, we might expect them to simply guess. Alternatively, if they are subconsciously aware that the stimulus in the blind spot is unreliable, they should choose the other one.
In reality, exactly the opposite happened: the volunteers chose the blind spot stimulus more often than not. This suggests that information generated by the brain itself is sometimes treated as more reliable than sensory information from the outside world. Future experiments should examine whether the tendency to favor information generated within the brain over external sensory inputs is unique to the visual blind spot, or whether it also occurs elsewhere.” DOI: 10.7554/eLife.21761.002
.
https://en.wikipedia.org/wiki/Filling-in
then
READ “Mind the Gap” in Scientific American (2005)
REMEMBER: TELLING THE BEST STORY YOU CAN INVOLVES CONNECTING THE DOTS (=THE BEST EVIDENCE YOU HAVE)
Neurophysiology: filling in (perceptions of things not present)
Dots are connected with more or less confidence to create a “narrative” … The neurophysiological ability to “fill in” the gaps is an example of this at the level of organization of aggregates of neurons (tissues) in in specific areas of the brain.
Visual.
- At the level of the brain, in vision for example, filling in is “a perceptual phenomenon in which a visual attribute such as colour, brightness, texture or motion is perceived in a region of the visual field even though such an attribute exists only in the surround. Filling-in dramatically reveals the dissociation between the retinal input and the percept, and raises fundamental questions about how these two relate to each other. Filling-in is observed in various situations, and is an essential part of our normal surface perception.” In a review, Hidehiko Komatsu (2006)[i] provides evidence for the tissue-level mechanisms responsible.
- The most famous filling-in phenomenon is that of the eye’s “blind spot, the region in the peripheral visual field that is blind owing to the absence of photoreceptors at the site where the optic nerve exits the eye. “Perceptual filling-in occurs when structures of the visual system interpolate information across regions of visual space where that information is physically absent. It is a ubiquitous and heterogeneous phenomenon, which takes place in different forms almost every time we view the world around us, such as when objects are occluded by other objects or when they fall behind the blind spot.” Weil RS1, Rees G. (2011)[ii] “Yet, to date, there is no clear framework for relating these various forms of perceptual filling-in. Similarly, whether these and other forms of filling-in share common mechanisms is not yet known.” The authors develop a “a new taxonomy to categorize the different forms of perceptual filling-in” to unify the literature and then “examine experimental evidence for the processes involved in each type of perceptual filling-in.” They find that “the presence of boundaries in determining the phenomenal experience of perceptual filling-in” is important.
- Auditory. “… New data suggests that these insertions are processed as if the brain had really heard the parts of the word that are missing. … We’ve known since the 1970s that the brain can “fill in” inaudible sections of speech, but understanding how it achieves this phenomenon – termed perceptual restoration – has been difficult. … [using people who already had hundreds of electrodes implanted into their brain to monitor their epilepsy… “ They heard words with the middle pafrt obscured and found ] ”… one region of the brain, called the inferior frontal cortex, predicts what word someone is likely to hear – and it does this two-tenths of a second before the superior temporal gyrus starts processing the sounds a person has heard.” (2017)[iii]
Dreams
- The creation of a story from isolated, even random facts, made The Activation- Synthesis Hypothesis of dreaming very alluring. This hypothesis proposes that dreams are generated as an artifact of a state of consciousness during the REM stage of sleep (Hobson and McCarley 1977).[iv] Although the brain is active during REM sleep, the activation of specific areas is significantly different, resulting in the unique REM patterns of influence on consciousness: information from the senses and physical movements of the body are largely blocked. Experience is limited to random activation within the brain and at some point, possibly the moment of waking, the brain assembles the random inner experiences into a narrative, trying to make the best sense of it possible.
Constellations …
· Groupings of stars (Wikipedia)
EXAMPLES:
- PHYSIOLOGICAL FILLING-IN of the human BLIND SPOT in the retina: that naturally occuring scotoma leads to a deficit in detail that is compensated for by filling-in. Look at WEBMD’s page on scotoma and read there about finding your own blindspot. Look here for details of visual filling-in.
- COSMIC level FILLING-INis required for any account of our universe: We kmow about 5% of the universe. “And everything else? The other 95% of literally everything? That’s a combination of two different things: dark matter and dark energy.” Read Amelia Settembre‘s brief blog account of dark energy.
- KINTSUGI: making the seams between the fragments beautiful: YouTube from Asian culture
WAYS of FILLING IN
interpolation, extrapolation
Small facts … big theories:
· “Modern Cosmology: Science or Folktale?” (Current cosmological theory rests on a disturbingly small number of independent observations) by Michael J. Disney
Overconfidence in the FACTS that underly NEUROBEHAVIORAL theory.
· Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron’s causality? Hendrikje Nienborg and Bruce Cumming Curr Opin Neurobiol. Author manuscript; available in PMC 2011 Nov 1. Published in final edited form as: Curr Opin Neurobiol. 2010 Jun; 20(3): 376–381. doi: 10.1016/j.conb.2010.05.002 PMCID: PMC2952283 NIHMSID: NIHMS210154
Abstract. How the activity of sensory neurons elicits perceptions and guides behavior is central to our understanding of the brain and is a subject of intense investigation in neuroscience. Correlations between the activity of sensory neurons and behavior have been widely observed and are sometimes used to infer how neurons are used to guide a certain behavior. This view is challenged by 1) theoretical considerations that these correlations rely on the existence of correlated noise and its structure, and 2) recent empirical observations suggesting that such correlated noise is not a fixed network property but that it depends on various sources, and varies with a subject’s mental state.
Introduction. A core question neuroscientists are venturing to resolve is how the information carried by sensory neurons is used by the brain to create perceptions and guide behavior. A direct approach is to perturb the activity of sensory neurons and quantify the effect on behavior. As such direct measurements of the causal effect of sensory neurons have traditionally lacked single cell resolution (1–6, but see 7), and knowledge of the tuning properties of the individual manipulated neurons, an alternative route has been to observe the activity of individual sensory neurons without perturbing the system, while an animal is performing a sensory task. For such tasks, weak but consistent trial-to-trial correlations between the activity of individual sensory neurons and the animal’s behavior have been found for different sensory modalities, brain areas and perceptual tasks8–20. Frequently, these correlations have been quantified as ‘choice-probabilities’, which measure the probability with which one could predict an animal’s choice on each trial, from the spike counts on a given trial (see box 1). Choice-probabilities have often been –implicitly or explicitly-interpreted as reflecting the causal effect of these sensory neurons in the particular sensory task. In this interpretation, larger CPs in a neuron imply that a neuron is given more weight in the decision-process, and hence CPs can be used to infer how neuronal activity is “read out” 15,21. However, theoretical work has long recognized that this interpretation is complicated by the structure of correlated activity in the population of sensory neurons 22,23. Recent theoretical and physiological work on these correlations, reviewed below, has revealed even greater complexity, so that the extent to which CP reflects the “read out” of neuronal activity is still poorly understood.
[i] The neural mechanisms of perceptual filling-in. by Hidehiko Komatsu (2006) Nature Reviews Neuroscience 7, 220-231 (March 2006) | doi:10.1038/nrn1869 : “Filling-in is a perceptual phenomenon in which a visual attribute such as colour, brightness, texture or motion is perceived in a region of the visual field even though such an attribute exists only in the surround. Filling-in dramatically reveals the dissociation between the retinal input and the percept, and raises fundamental questions about how these two relate to each other. Filling-in is observed in various situations, and is an essential part of our normal surface perception. Here, I review recent experiments examining brain activities associated with filling-in, and discuss possible neural mechanisms underlying this remarkable perceptual phenomenon. The evidence shows that neuronal activities in early visual cortical areas are involved in filling-in, providing new insights into visual cortical functions.”
· Published: 01 March 2006 The neural mechanisms of perceptual filling-in. Hidehiko Komatsu
Nature Reviews Neuroscience volume 7, pages220–231(2006) Cite this article.
Key Points
· Filling-in is a remarkable perceptual phenomenon in which visual features such as colour, brightness, texture and motion of the surrounding area are perceived in a certain part of the visual field even though these features are not physically present.
· One extreme possibility is that our visual system simply ignores the lack of visual input and that filling-in is a passive outcome of this. However, various psychophysical experiments suggest that some active processes are involved in the occurrence of filling-in, and that some neural computation occurs in the brain when filling-in occurs.
· In the past decade, several single-unit recording experiments in monkeys and functional MRI experiments in humans have examined neural activities related to filling-in in early visual cortical areas. Many of these studies found that neurons are activated in the region of the retinotopic map of the early visual areas that represents the interior of the surface where filling-in occurs.
· Neural mechanisms of filling-in investigated in the above-mentioned studies must be distinguished from topographic remapping induced by binocular retinal scotoma. When retinal lesions are made at corresponding positions in both eyes (binocular retinal scotoma), reorganization of the retinotopic map of the visual cortex occurs, but this differs from situations in which other types of filling-in occur.
· Traditionally, ‘symbolic’ and ‘isomorphic’ theory have been proposed as neural mechanisms of filling-in. Symbolic theory assumes that early visual areas extract only the contrast information, but this contradicts the results of most of the recent neurophysiological and neuroimaging experiments.
· Isomorphic theory assumes that when perceptual filling-in occurs, a two-dimensional array of neurons with a point-by-point representation of visual features is activated in the early visual cortex. Although neurons are activated in early visual areas during filling-in, neural responses recorded during filling-in at the blind spot differ from those predicted by isomorphic theory in several important ways.
· Selective activation of neurons in deep layers of the visual cortex that represent a particular spatial scale and that are selective for particular features might be involved in the process that mediates perceptual filling-in.
· This research is still at an early stage, and many questions remain to be answered about the neural mechanisms of filling-in. Understanding the details of these mechanisms is important, because it might provide answers as to where and how subjective visual experience emerges, a fundamental question about visual perception.
Abstract
Filling-in is a perceptual phenomenon in which a visual attribute such as colour, brightness, texture or motion is perceived in a region of the visual field even though such an attribute exists only in the surround. Filling-in dramatically reveals the dissociation between the retinal input and the percept, and raises fundamental questions about how these two relate to each other. Filling-in is observed in various situations, and is an essential part of our normal surface perception. Here, I review recent experiments examining brain activities associated with filling-in, and discuss possible neural mechanisms underlying this remarkable perceptual phenomenon. The evidence shows that neuronal activities in early visual cortical areas are involved in filling-in, providing new insights into visual cortical functions.
http://www.nature.com/nrn/journal/v7/n3/full/nrn1869.html
[ii] A new taxonomy for perceptual filling-in. Weil RS1, Rees G. Brain Res Rev. 2011 Jun 24;67(1-2):40-55. doi: 10.1016/j.brainresrev.2010.10.004. Epub 2010 Nov 5.
Perceptual filling-in occurs when structures of the visual system interpolate information across regions of visual space where that information is physically absent. It is a ubiquitous and heterogeneous phenomenon, which takes place in different forms almost every time we view the world around us, such as when objects are occluded by other objects or when they fall behind the blind spot. Yet, to date, there is no clear framework for relating these various forms of perceptual filling-in. Similarly, whether these and other forms of filling-in share common mechanisms is not yet known. Here we present a new taxonomy to categorize the different forms of perceptual filling-in. We then examine experimental evidence for the processes involved in each type of perceptual filling-in. Finally, we use established theories of general surface perception to show how contextualizing filling-in using this framework broadens our understanding of the possible shared mechanisms underlying perceptual filling-in. In particular, we consider the importance of the presence of boundaries in determining the phenomenal experience of perceptual filling-in. PMID: 21059374 PMCID: PMC3119792 DOI: 10.1016/j.brainresrev.2010.10.004 Free PMC Article
[iii] Your brain fills gaps in your hearing without you realizing. By Aylin Woodward NS 18 March 2017:19.
Noise is everywhere, but that’s OK. Your brain can still keep track of a conversation in the face of revving motorcycles, noisy cocktail parties or screaming children – in part by predicting what’s coming next and filling in any blanks.
New data suggests that these insertions are processed as if the brain had really heard the parts of the word that are missing.
“The brain has evolved a way to overcome interruptions that happen in the real world,” says Matthew Leonard at the University of California, San Francisco.
We’ve known since the 1970s that the brain can “fill in” inaudible sections of speech, but understanding how it achieves this phenomenon – termed perceptual restoration – has been difficult. To investigate, Leonard’s team played volunteers words that were partially obscured or inaudible to see how their brains responded.
The experiment involved people who already had hundreds of electrodes implanted into their brain to monitor their epilepsy. These electrodes detect seizures, but can also be used to record other types of brain activity.
Faster/Factor
The team played the volunteers recordings of a word that could either be “faster” or “factor”, with the middle sound replaced by noise. Data from the electrodes showed that their brains responded as if they had actually heard the missing “s” or “c” sound.
This seems to be because one region of the brain, called the inferior frontal cortex, predicts what word someone is likely to hear – and it does this two-tenths of a second before the superior temporal gyrus starts processing the sounds a person has heard.
“They took a well-known phenomenon and showed, undoubtedly, that the brain puts in the acoustics that are missing,” says David Poeppel at New York University.
But although this prediction might seem clever, the team found that it has its limitations. The brain doesn’t seem to use the context of a conversation to improve the accuracy of its guesses. When they primed people to hear a particular word – for example, preceding the obscured word with “I drove my car” – they were just as likely to hear the word “factor” as “faster”.
Journal reference: Nature Communications, DOI: 10.1038/ncomms13619 Read more: ‘Cocktail party effect’ identified in the brain A shorter version of this article was published in New Scientist magazine on 18 March 2017
[iv] Hobson JA, McCarley RW. (1977) Am J Psychiatry. 1977 Dec;134(12):1335-48. The brain as a dream state generator: an activation-synthesis hypothesis of the dream process.
Abstract. Recent research in the neurobiology of dreaming sleep provides new evidence for possible structural and functional substrates of formal aspects of the dream process. The data suggest that dreaming sleep is physiologically determined and shaped by a brain stem neuronal mechanism that can be modeled physiologically and mathematically. Formal features of the generator processes with strong implications for dream theory include periodicity and automaticity of forebrain activation, suggesting a preprogrammed neural basis for dream mentation in sleep; intense and sporadic activation of brain stem sensorimotor circuits including reticular, oculomotor, and vestibular neurons, possibly determining spatiotemporal aspects of dream imagery; and shifts in transmitter ratios, possibly accounting for dream amnesia. The authors suggest that the automatically activated forebrain synthesizes the dream by comparing information generated in specific brain stem circuits with information stored in memory.
PMID:21570 DOI: 10.1176/ajp.134.12.1335
For example, “Illusion Reveals that the Brain Fills in Peripheral Vision” reporting on research of Otten et al. (2016)[1]
What we see in the periphery, just outside the direct focus of the eye, may sometimes be a visual illusion, according to new findings published in Psychological Science, a journal of the Association for Psychological Science. The findings suggest that even though our peripheral vision is less accurate and detailed than what we see in the center of the visual field, we may not notice a qualitative difference because our visual processing system actually fills in some of what we “see” in the periphery.
“Our findings show that, under the right circumstances, a large part of the periphery may become a visual illusion,” says psychology researcher Marte Otten from the University of Amsterdam, lead author on the new research. “This effect seems to hold for many basic visual features, indicating that this ‘filling in’ is a general, and fundamental, perceptual mechanism.”
As we go about daily life, we generally operate under the assumption that our perception of the world directly and accurately represents the outside world. But visual illusions of various kinds show us that this isn’t always the case. As the brain processes incoming information about an external stimulus, we come to learn, it creates a representation of the outside world that can diverge from reality in noticeable ways.
Otten and colleagues wondered whether this same process might explain why we usually feel as though our peripheral vision is detailed and robust when it isn’t.
“Perhaps our brain fills in what we see when the physical stimulus is not rich enough,” she explains. “The brain represents peripheral vision with less detail, and these representations degrade faster than central vision. Therefore, we expected that peripheral vision should be very susceptible to illusory visual experiences, for many stimuli and large parts of the visual field.”
Over a series of experiments, the researchers presented a total of 20 participants with a series of images. The participants focused on the center of the screen — a central image appeared and then a different peripheral image gradually faded in. Participants were supposed to click the mouse as soon as the difference between the central patch and the periphery disappeared and the entire screen appeared to be uniform.
Otten and colleagues changed the defining characteristic of the central image in different experiments, varying its shape, orientation, luminance, shade, or motion.
The results showed that all of these characteristics were vulnerable to a uniformity illusion – that is, participants incorrectly reported seeing a uniform image when the center and periphery were actually different.
The illusion was less likely to occur when the difference between the center and periphery was large; when the illusion did occur on these trials, it took longer to emerge.
Participants indicated that they felt roughly equally sure about their experience of uniformity when it actually did exist as when it was illusory. This suggests that the illusory experiences are similar to a visual experience based on a physical visual stimulus.
“The fun thing about this illusion is that you can to test this out for yourself,” Otten says. “If you look up the illusion on www.uniformillusion.com you can find out just how real the illusory experience feels for you.”
Now read about GASLIGHTING: attempting to impose an alternate (likely false) perception of reality[2]:
“One problem with any bid to hold on to your own reality in the face of a gaslighter is that, whether we like it or not, none of us sees the world as it is. Our brains only process a fraction of the incoming sensory information we detect. The gaps are filled in by the brain, which constantly makes predictions based on previous experience and then updates them in light of new information. “The perceptual world is a dialogue between what we’re receiving from our senses combined with our experiences of weaving together a narrative reality that makes sense to us,” says Mazviita Chirimuuta, a philosopher of neuroscience at the University of Pittsburgh in Pennsylvania.
Given that our experiences and memories are unique, that adds up to a bespoke personal reality that differs from everyone else’s. It is no wonder that we sometimes disagree on how to interpret events. On top of this, memory can be notoriously unreliable. For example, studies of eyewitness testimony have shown that two people can form different memories of the same incident, including what was said and who did what. Our memories are far from a faithful recording of events.”
[1] The Uniformity Illusion: Central Stimuli Can Determine Peripheral Perception Marte Otten, Yair Pinto, Chris L. E. Paffen, Anil K. Seth, Ryota Kanai (2016) Psychol Sci 2017; 28(1):56-68. Find in PubMed or https://doi.org/10.1177/0956797616672270 (copy/paste URL into browser)
[2] Caroline Williams reporting in New Scientist (2021) [LINK]
NEXUS: Filling-in is how we create narratives: