Dr. Christof Koch: Sure.

David Boulton: Good. I’d like to begin with you sharing a brief thumbnail on who you are and what’s driving you’re exploration of consciousness.

Dr. Christof Koch: I’m a forty-year-old neuroscience professor in biology and engineering at Caltech. My background is in physics and philosophy. One of the questions that is key in philosophy is the mind-body problem: how is it that consciousness and sensation, the feelings of pain or pleasure or of ‘being me’ – how do they arrive out of a physical object like my brain? This problem has been explored for the last 2,000 years using philosophy, but for the last fifty years it has begun to be looked at from a scientific point of view, an empirical point of view, and over the last twenty years, through neuroscience.

David Boulton: This is a good segue into the work we’re interested in, which has to do with the construction of the virtual language experience we call reading. We’re taking information and instructions from an external artifact — this code — and assembling according to its instructions and content an internal experience of the recognition of words.

Dr. Christof Koch: Uh-huh.

David Boulton: And as you point out in your article, on the one hand it would take about thirty milliseconds for a good reader to recognize a word, on the other hand, it takes almost a quarter of a second to comprehend that word. There is a faster than conscious assembly process which happens in order to take this code and transform it into an internal representation of language we can comprehend.

Dr. Christof Koch: Uh-huh.

David Boulton: During that processing…

Dr. Christof Koch: Most of which is unconscious, of course.

David Boulton: Yes. It’s unconscious.

We’ve talked to some people who study eye movements to deduce brain processing time relative to the average time it takes to convert letters into phonemes.

Dr. Christof Koch: Uh-huh.

David Boulton: And we’re talking about twenty-five milliseconds roughly.

Dr. Christof Koch: Very fast.

David Boulton: Very fast.

David Boulton: In this country, here in the U.S., there are almost 100 million people whose lives are diminished or constrained because they never broke through to transparency with literacy.

Dr. Christof Koch: 100 million? There are only 300 million Americans total.

David Boulton: Yes, I understand. There are ninety…

Dr. Christof Koch: Are you saying one in three?

David Boulton: Yeah, in terms of the last adult literacy survey and the current state of reading proficiency in school. This isn’t to say, they can’t read at all, but their lack of literacy proficiency is causing them not to engage in literacy practices in ways which are constraining their lives, to various degrees. There’s a spectrum. There are about thirty to forty million people who fall into a relatively severe difficulty category, and another fifty plus million who are operating below an 8^th grade reading level.

Dr. Christof Koch: Good God. Forty to fifty million, that’s still one out of five!

David Boulton: Well, people who are really highly literate don’t have a lot of opportunity to engage much with people who are not.

Dr. Christof Koch: Yeah. Yeah, I mean, you’re right. I don’t think I know a single person who is not literate. Maybe I do, but I don’t think so, but anyway, I understand.

David Boulton: We’ve talked to neuroscientists who are studying the brain at different stages in the reading process. And they’re basically saying variations of what you said in your article regarding the limitations of current brain imagining technology. We aren’t yet able to peer into the levels of details and at the rate of processing that this construction is occurring in.

Dr. Christof Koch: Yes, we have EEG that of course can resolve the timing, but that doesn’t allow us to really pinpoint anything, because it doesn’t really allow us to pinpoint the signals in terms of particular parts of the brain. We’ve got an fMRI, which allows us to pinpoint, this part of the brain or that part of the brain, but it’s very sluggish because it works on a time scale of many seconds.

David Boulton: Right. I make the analogy; it’s like a satellite looking at the aggregate light patterns of Los Angeles and Houston at night. We can see the difference in the intensity of light between the two cities, but we can’t see the federal express messages going back and forth between them.

Dr. Christof Koch: No, no.

David Boulton: Well…

Dr. Christof Koch: That’s what we’re stuck with right now. We have more options in the study of animals, but of course, no animal is able to read.

David Boulton: Some neuroscientists we’ve talked to believe that reading is a particularly unique opportunity for exploring the construction of consciousness (Tallal). Like the way you described the movie projector, reading is a projected fabrication from one dimension of activity in the brain into another dimension of activity in the brain which is assembled according to this external code.

Dr. Christof Koch: And, what I’ve found most amazing about reading.

I am on sabbatical, my family is back home, so I spend a lot of time just reading endless books. When you read, it’s totally transparent, right? I mean, you never see the letters and you don’t see the phonemes. All you see are these final constructs that whiz by at a very high speed.

David Boulton: Right.

Dr. Christof Koch: And you’re totally oblivious of all of the processing that goes on. And of course, you miss — that’s the trouble when editing the text, right?, because you miss all of these mistakes, because you just pass over them.

David Boulton: Right. But this is precisely where the obstacle wall is for children coming into this because they don’t experience it the same way. They experience a very confusing relationship between the automatic unconscious oral language facilities they’ve developed, and this external, very ambiguous code that has a very rough correspondence to that system.

Dr. Christof Koch: Okay.

David Boulton: So, back to your sense of the brain, and in particular, the timing. So do you concur from your work, that we don’t have the ability at this stage to peer into the dimensions of constructing this virtual reality that reading or other high-level comprehension processes require?

Dr. Christof Koch: Well, in order to understand things such as, reading or memory or consciousness, or anything of this nature, it’s important to understand, it is all based on neurons, on nerve cells. For example, if you want to understand chemistry, it’s not good enough to understand what the alchemists did, or to know about gold, or about sweet and sour; you need to understand about molecules and molecular interactions. Right? Otherwise, you would never be able to understand chemistry. Likewise, if you want to understand nuclear physics, you need to understand protons, electrons and neutrons. So, to understand reading or other related abilities, you need to understand nerve cells.

Now the trouble is, with humans, the techniques we usually use are things like, an EEG or an fMRI, and they all have severe limitations. Even fMRI, which has great resolution from a clinical point of view, if you do a reading experiment, for example, or a visual experiment, with fMRI your typical focal size is three-by-three-by-four millimeters. That size includes on the order of two million neurons, two million nerve cells. And unlike molecules, they’re not all alike; they’re highly, highly differentiated.

It’s a little bit like molecular biology. We now know there are a few hundred thousand different proteins in the brain, and in order to study any one molecular disease, you really need to know which types of molecules are involved.

So likewise, to understand something as complicated as reading you need to understand neurons. However, we really need to have access to the individual neurons, and unfortunately we only have access to them in very rare conditions/cases. Because there’s no animal model of reading, we can only study it in humans, and very rarely do we ever have access to individual neurons in humans, with the exception of a small sample of patients, so we are somewhat stuck at this point in our research.

David Boulton: So what do you think is on the horizon, as far as gaining better tools to peer into this window?

Dr. Christof Koch: Well, all sorts of people are trying invasive techniques, but the trouble with this is, sure, we’ll have more brain prosthetic devices, but these devices are only for patients. So ultimately, all of the techniques we have available right now, which can look at and query individual neurons, ALL devices require invasive surgery; which means, these techniques require you to drill holes into the human skull. So of course, we’re only going to use these techniques as a last recourse in patients, who are paralyzed and such. Therefore, for the foreseeable future, i.e., for the next five to ten years, we’re stuck with the techniques we have. We have to do what we’ve done in the past, which is to just try to find better techniques, better EEG, better resolution fMRI, and on the basis of these imperfect tools, try to decipher as much as we can about reading.

David Boulton: Have you ever explored the correlation between the triggering of different affective states or emotional states and the turbulence or effects of such triggering on cognitive activities?

Dr. Christof Koch: No. We study vision.

It’s a little bit like the scientists in the ’40s and ’50s who studied heredity, not in humans, but in flies or worms or in bacteria. The belief was that there was an underlying universal mechanism — and these animals are much easier to study than humans, so let’s first study these simple things first. So likewise, when most people right now study consciousness, most people in neuroscience, we study the simple forms of consciousness like vision or pain, because they are relatively easy to manipulate and we are able to do it with great precision. Just like a magician, we can make things appear and disappear quite precisely in front of your eyes. So, we can manipulate your visual consciousness in very exact ways.

Now, with emotion, it’s more difficult to do. And it’s often an ethical problem because we can’t make you really angry or really cause you pain or something for obvious ethical reasons. So therefore, most people who study consciousness, study visual consciousness, which is what I do.

David Boulton: I see. Okay. There are some scientists we’ve talked with who are using evoked potential methods to monitor people as they read texts that contain emotionally charged words…

Dr. Christof Koch: Right.

David Boulton: …so they can see the brain’s response to something which triggers a significant emotional shift.

Dr. Christof Koch: Yes, we also do that. We study, for example, angry faces. And you can see the different parts of the brain which typically lights up when you do things or make sounds that induce fear or anger in people.

Dr. Christof Koch: The other thing we’ve done is to record individual neurons, sometimes neurons that respond to text. The only findings of that work with text that I am aware of came out in Nature. We looked at the individual neuron level while patients were shown things — images of famous people, like Jennifer Anniston, Brad Pitt, Halle Berry, or other celebrities, and we could see individual neurons firing. We showed people different pictures of celebrities. But then also showed the text of their name to some patients. “Halle Berry,” for example, we spelled out. And we would watch for some neurons, which are called “invariants” and that respond both to different pictures of the actor, as well as the text if we just spell out the actor’s name.

So there’s quite a bit of neuronal activity when we flash the image, or flash the text for one second. And it typically takes 300 milliseconds for the nerve cells to respond. In fact, in the current issue of Scientific American, there is a small article about it, if you want to read more about it.

David Boulton: Yes.

Dr. Christof Koch: I just feel like it’s relevant to you. There are extremely few cases where scientists record activities of individual nerve cells in humans. And this is one of the few ones, and I believe it is the only case where text has been used or one of the only cases, where text has been used, so you may want to look at that.

David Boulton: Okay, I will. Yes, thank you. I’m particularly interested in that.

The reason I brought up the emotions earlier is because it’s very clear that emotions are critical to children learning to read. Just like reading itself is an automatic, unconscious assembly process…

Dr. Christof Koch: Once you do it at the expert level.

David Boulton: Right, right, exactly.

Dr. Christof Koch: Otherwise it’s not.

David Boulton: Yes, so the purpose of learning here is to create the automational machinery.

Dr. Christof Koch: Yes, exactly — which we don’t really understand. We don’t understand right now, which is true for anything. It’s true for reading, it’s true for piano playing, it’s true for learning how to ride a bicycle, and it’s true for playing tennis or climbing. Early on, you have to be exquisitely conscious of everything you do and say, and how you hold your hand and so on. But after a while, it becomes second nature, and you don’t even think about it. In fact, when you do think about it, very often your performance deteriorates because thinking about it evokes, once again, a conscious activity, which usually takes longer than the unconscious processing you do at the expert level.

David Boulton: So it inserts a kind of serial delay, and it also consumes processing bandwidth…

Dr. Christof Koch: Yes, exactly. You should try it with tennis. Next time, you play a game with somebody; compliment them on their backhand and say, “Wow, that’s cool. That’s pretty nifty. How did you do that?” And the next time they hit the ball, they will probably be conscious of their backhand and it will interfere with their optimum performance.

David Boulton: Yes, exactly. That’s the point we’re trying to make with reading. It is an extraordinarily complex and difficult skill to learn, particularly for children who haven’t had sufficiently rich language environments to develop the language processing reading requires when first learning to read.

It is very confusing, we have forty-plus sounds, and only twenty-six letters to represent them with. There’s not a direct match, there’s not a simple correspondence. Therefore, the brain has to do something in reading, it has never had to do in evolutionary history, which is to disambiguate the relationships between this tacitly learned natural language system and this explicit mechanical technological process of using this code to assemble a simulation that the natural language system can comprehend.

In the course of learning to do that, if learners develop a protracted difficulty with it, they very often develop an emotional aversion to wanting to do it, which seems to get into the automation itself, so much so, that most struggling readers develop an automatic preconscious aversion to the feeling of the confusion in the reading.

The stories of reading shame are legendary.

Dr. Christof Koch: About reading what? Reading shame?

David Boulton: Reading shame, yes. The lengths and degrees to which people go in order to hide their illiteracy is just amazing. We have collected hundreds of stories detailing what happens to people, some of which are on our website. There are governors and actors and famous people out there who have managed to survive it (illiteracy), yet who nonetheless when they reflect on what it felt like to not be able to read or what it still feels like, still feel great shame. Reading has so much social importance, in part because it’s tied into language and in part because learning to read tends to happen in social settings. So, it’s incredibly embarrassing when one can’t read and it elicits enormous shame. And humans don’t like to do things that cause them shame.

Dr. Christof Koch: Yeah, true.

David Boulton: Therefore, we develop this preconscious aversion to the shame, which starts to interscript with the reading stream construction process. And just as you were saying about tennis, the shame bifurcates consciousness and reduces the brain’s ability to do the code work.

Dr. Christof Koch: Okay, I’ll look at them. Two ex-governors?

David Boulton: Yeah, two ex-governors, doctors. I mean, you would not believe — it’s hard to…

Dr. Christof Koch: Yeah, I mean, I know many doctors and I can’t believe it. There’s no way, you can survive medical school without knowing…

David Boulton: Yes. I’ll let you read about what some of the other people have to say about that.

Dr. Christof Koch: These stories are real stories?

David Boulton: They are real stories from governors, and from children, which you’ll see interspersed in our collection. Some of the stories are second hand and some of them we have directly collected. I think the thing you might find most interesting is, the cognitive shock of shame conversation, and the conversation with Russ Whitehurst, who’s the head of the Institute of Education Science for the United States Government. The point is, all of these things are happening inside the field you’re exploring. You have to say at some level, we’re talking about an unnatural and artificial module of the brain forming which can produce simulated language from the instructions and information contained in an external code. And yet when reading works well, just like you were saying, there’s this transparency, just as there is in spoken language, that goes right to meaning, without getting trapped in the forms.

Dr. Christof Koch: Yeah. I agree it’s a wonderful property. It’s really amazing how the brain evolved in evolution making it have this magic — almost magical property that sends this signal, telling it to go directly to meaning, bypassing all intermediate stages.

David Boulton: Except that though we’re able to do that in natural environments where we have proprioceptive feedback loops guiding our perceptual systems, here we’re talking about trying to get to meaning through an invention which has enormous confusion and gives no natural feedback.

Dr. Christof Koch: But the — well let’s see, I guess, driving is something most adults can do, and I guess there’s no symbolic processing involved with it.

David Boulton: And there is real-time, immediate, sensory feedback informing it.

Dr. Christof Koch: Yeah, yeah.

David Boulton: Reading is a kind of virtual processing which has to happen in relation to this code – this technology.

I don’t know if you’ve ever studied the history of it, but the collision between the Roman writing system and the English language shaped the written system which became English. No one was minding the store.

So, we’re talking about a neglected technological artifact that is causing an unnatural and unprecedented form of brain confusion-challenge that must be resolved at incredible speed in order to create this virtual inner experience of language.

Dr. Christof Koch: But I guess just as with the QWERTY keyboard, we are stuck with it. We’re not going to invent a new language.

David Boulton: I’m not so much saying we should, but I do think it’s important to understand the challenge in a different way than we have before.

Dr. Christof Koch: Yeah. Yeah, it’s a very valid point. It’s a very valid point. We have these programs now like Paula Tallal’s …

David Boulton: Right. Fast Forward. There’s an in-depth interview with Paula Tallal on our website as well as with Mike Merzenich.

Dr. Christof Koch: Yeah, yeah.

David Boulton: I’ve also talked with Sally Shaywitz, and a number of other neuroscientists. One of my favorite conversations is with, Zvia Breznitz, who’s working on asynchrony problems relative to reading in Israel. And she’s exploring the timing related problems of the different sub processes that have to flow together just in time or the virtual reality construction of the reading stream breaks down in the brain.

Dr. Christof Koch: Zvia Breznitz, yes.

David Boulton: She’s doing some really remarkable work with the EEG and the ERP to show the asynchrony problem that is at the heart of what we at COTC call the “reading stutter.”

Dr. Christof Koch: Yeah. Yeah, I’ve talked with Paula about that. The issue is this pattern of universal processing time. It probably differs genetically, and you can probably train it from person to person. But, if you don’t get things within this processing time, you process it as a separate chunk, then it’s much more difficult to integrate. Is it possible to show there is such a universal integration time?

David Boulton: I don’t know, but that’s only one of the issues. Another issue is how a person’s overall language abilities have prepared them for the unnatural challenges involved in reading. For example, in order to learn to hear and speak language our brain’s must differentiate phonemes but in order to engage in print the level of granularity required to map speech to print is of a much different order. And the ability of the brain to do that is, for most people, developed according to the qualities of the language exercise environment that their brain developed in. So, for example, there have been studies which shows that the difference between taciturn parents and talkative parents, across the SES spectrum, in America, results in four-year-olds who have differences in language exposure in the neighborhood of thirty million word exposures. In other words, the difference between…

Dr. Christof Koch: Thirty million words? Well, this must be an extremely taciturn parent, who only grunts every day, or what?

David Boulton: No, no, no.

Dr. Christof Koch: Thirty million seems like a lot.

David Boulton: Well, there are studies that compare language exposure rates, like Todd Risley’s work did. This study put a person in the home once a week for an hour from the time the children were nine months old until they were four and then aggregated the results across sixty-something families selected to represent a sampling across the SES spectrum. It’s a highly regarded educational study. They found that differences in talk and engagement across the spectrum of children over this longitude, strongly correlated to Peabody Picture Vocabulary tests, to reading levels in the third grade, and to IQ.

Dr. Christof Koch: Yes.

David Boulton: So, that kind of a correspondence, between language exercise and those kinds of test performances, seems perfectly natural to me.

Dr. Christof Koch: Yeah. I wasn’t quite aware of these large differences of thirty million.

David Boulton: Is there anything else from your explorations which might shed some light upon this process of creating an artificial stream in consciousness from this code?

Dr. Christof Koch: Well, we study how consciousness is integrated in simple perception, which is relatively natural to us, for example, color and motion. So, I’m not sure I can add anything above and beyond which is specific to reading, except perhaps, that there may be some universal time we use for all processing. Whether it’s speaking or whether it’s reading, or whether it’s looking at moving things, it’s essentially the same integration process our brains have to deal with. And if it’s true that all of conscious perception is chunked in time, then of course it will also apply to reading…

David Boulton: Right.

Dr. Christof Koch: Paula Tallal has also done experiments where she shows children who have difficulty distinguishing two nearby sounds as being different, or two nearby flashes of light, may have a sensory processing deficit. So if I flash you two flashes very quickly, if I do it too quickly, you perceive them as one. Or if I give you a headphone, and I put two sounds in your left and your right ear separately and simultaneously, you hear the sound coming from the midpoint of your head. If I give the left speaker a little bit of an advance compared to right, the sounds shift inside your head to the left or to the right. Some children can’t track these kinds of changes, and Tallal says, the children who have trouble differentiating the sounds fast enough are more likely to become dyslexic later on.

So that would imply, there’s a central sensory processing deficit. This is a different study population, right? This is probably not the same group you are concerned with, maybe only partially overlapping the larger group of people who have reading difficulties. But these are dyslexic individuals who may have a central sensory processing deficit that leads, among other things, to a reading deficit. However, it’s not unique to reading deficits.

David Boulton: Right, right. But it is an important avenue of thinking. However, rather than thinking of it in terms of some kind of neurobiological structural deficit, it may have been a environmental-developmental trajectory difference in the efficiency of processing language.

Dr. Christof Koch: Yeah. So the old question: it’s nature versus nurture. In this case of the dyslexic to what extent is this genetically preconditioned behavior and to what extent does it arise through the environment?

David Boulton: Right.

Dr. Christof Koch: Meaning, I’m not just talking specifically about dyslexia.

David Boulton: No, I understand.

Dr. Christof Koch: I don’t know what the current literature says about the concordance in identical twins. My gut feeling is, it’s probably going to be like many of these things, there are genetic predispositions, which if you compound them with parents, or not having parents, or parents who don’t talk, or who don’t give you the proper feedback, then probably you do get the symptoms of dyslexia. But if you have the genetic condition and you do get the right training at the appropriate age — early intervention if you will, then you can probably compensate for having the condition. At least that’s what Paula’s work seems to suggest, right?

David Boulton: Right, right.

Dr. Christof Koch: Which would be very promising.

David Boulton: Yes. The work in neuroplasticity, relative to holding people on the edge of making the differentiations they need to make in order to build up the granularity and speed of phonemic processing, is very promising. Well, I really appreciate the opportunity to talk with you and I really respect what you’re doing, and believe you’re on the leading edge of the future in exploring how consciousness works.

Dr. Christof Koch: It’s great fun. It’s great fun. It’s really challenging. Let me look around your website a little bit, it’s pretty interesting.

Dr. Christof Koch: Has there been studies to compare children — for example, in a country like China, which has a totally different script, right?…

David Boulton: Right.

Dr. Christof Koch: …which is iconic, and the difficulties children there have with reading, as compared to countries that have an alphabet, a small alphabet like we do?

David Boulton: Right. There has not been good cross comparisons. There are reading score comparisons, but nobody has actually built a matrix which shows the different layers of ambiguity and complexity in the different orthographies and then indexed the amount of time which is spent teaching and so forth, in order to show how reading is functioning in different language systems based on differences in orthography and different investments in teaching. We haven’t got there yet.

Dr. Christof Koch: Okay.

David Boulton: It is very clear, though, that within the alphabetic languages, English is the most difficult. There is a direct correlation between the orthographic depth, opaqueness or ambiguity in the letter sound relationships, and the difficulty associated with reading. For example, the relationship between the word live (long i) and live (short i), can’t be derived from the code itself, it has to be contextually disambiguated…

Dr. Christof Koch: Yes, yes.

David Boulton: …through attention to words down the road a word or two. That means that a lot more information must be buffered up in the brain and some contextual processing has to participate in the iterations that resolve the ambiguity about the letter sounds.

Dr. Christof Koch: Yes. I guess you need your short-term memory, acoustic memory to deal with that.

David Boulton: Right. So in some sense, this kind of ambiguity may be partly responsible for expanding or exercising, or pushing the limits of our abstract, memory processing.

Dr. Christof Koch: Yeah, yeah.

David Boulton: One of the main points of our series is, “we are children of this code“, just like Francis Crick’s work showed we are children of the DNA code.

Dr. Christof Koch: Yeah, yeah.

David Boulton: …our civilization and our minds, in some respects, are reflections of how this code works and what it makes possible.

Dr. Christof Koch: One thing I’ve wanted to understand better… I grew up in a German household, not actually in Germany. And of course, what you have in German — if you’ve ever read, for example, Thomas Mann, who is, of course a very high functioning human being, you will find sentences which contain 200 words, with a verb at the very, very end.

David Boulton: (Laughs).

Dr. Christof Koch: It can literally run on for two pages. I’ve always wondered if there is a correlation between the average between lengths of sentences in a language and the average short-term memory span of its users.

Could we surmise that people who grew up speaking German should have a better short-term memory because they constantly need to use it compared to English folks? I mean, typically, if you read the New York Times compared to the typical similar German paper like the Frankfurter Allgemeine, which is for roughly the same reading audience — I mean, if you just look at it, English sentences are much, much shorter.

David Boulton: Right.

Dr. Christof Koch: Again, I wonder whether anybody has correlated that, along the lines which you suggested before, to see whether this also has implications for short-term memory duration.

David Boulton: I think it would be a great thing to do. I had a conversation last week with Guy Deutscher. I don’t know if you’ve seen his book, Unfolding Language, an Evolutionary Tour of Mankind’s Greatest Invention.

Dr. Christof Koch: No.

David Boulton: He’s a linguist in Holland. Anyway, these are questions linguists haven’t got to yet, but they’re intrigued by them. And just as you were saying, the sentence structure and verb placement in the structure, and what is required in terms of memory stretching and so forth, is interesting. We are what we’re exercised to be, in a certain way, and this must be affecting our cognitive processing, if you will.

Dr. Christof Koch: Sure, oh, yeah, it must.

David Boulton: And in the case of English, what’s so interesting is, again, we have twenty-six letters making forty-four sounds in hundreds of different combinations.

Dr. Christof Koch: (Laughs) It’s a lot.

David Boulton: So, we’re talking about having to disambiguate a code at high speeds. Throughout the emergence of literacy in the western world through the Greeks and the Romans, the writing systems were phonetic, one letter, one sound. Reading was code-cued speech ‘see the letter – say the sound’ blend as you go – you’re reading. But now with the confusion caused by the collision of the Roman alphabet and the sound systems in the romance languages, there’s a different requirement for the brain to read and that’s affecting us.

Dr. Christof Koch: You know, that’s an interesting point. That’s a very good point. All right.

David Boulton: All right, well, thank you so much for your time. I would like to talk more, so if at any time you encounter any insight or idea relative to reading or any of these dimensions we’ve talked about please send me an email, I’d like to continue our conversation.

Dr. Christof Koch: Sure, sure. And, I’ll poke around your website.

David Boulton: Thank you so much.