Langston, Psychology of Language, Notes 5 -- Fonts, Visual Perception, and Reading
I.  Goals:
A.  Evolution of written language.
B.  Fonts.
C.  Orthography.
D.  Components of reading.
E.  Models of reading.
II.  Evolution of written language.  Spoken language evolved as part of the human condition.  In spite of the debate about whether the ability to perceive speech is innate or learned in childhood, one thing is clear:  All children who acquire language acquire speech with no formal instruction, no children acquire reading without formal instruction.  Written language first emerged around 5,000 years ago, and independent forms of written language emerged in only a few parts of the world.
A.  Three classes of writing systems:  All of these have arbitrary symbol-sound mappings.  They differ in whether or not they have arbitrary symbol-meaning mappings.
1.  Logographic:  Each sign represents an entire word.  You rarely have inflections (like plurals).  Ex.:  House = <a picture of a house>
2.  Syllabic:  Each sign represents a syllable.  This cuts down the number of signs needed to a limited set of syllables.  Ex.:  Japanese:  car = kuruma (<symbol> = ku, <symbol> = ru, <symbol> = ma).
3.  Alphabetic:  Each symbol represents a phoneme (sort of).  English.
B.  Evolutionary progression:  (1st three are all logographic).
1.  Pictograms:  A picture communicates an idea.  A picture is what it stands for, no clue to pronunciation in the symbol.  Ex:  Mesopotamian <a picture of an arrow> = arrow.
2.  Ideograms:  Pictograms are hard to use for abstract ideas and grammatical information.  If you extend pictograms to abstract ideas, you get ideograms.  They represent ideas rather than objects.  This is a big step because you introduce arbitrary mappings between the symbol and the meaning.  Ex: <foot symbol> = to go.
3.  Logograms:  Each symbol is a word.  Chinese is an example of this.  It works well for China because wherever you go you can read, even though pronunciation is radically different in different regions.  If the language was alphabetic, and the spellings were based on sound, different dialects would look and sound different, and reading another dialect would be difficult if you didn't speak it.  The problem with logograms is that they're memorization and symbol intensive.  You need thousands just to read a newspaper.
a.  Hieroglyphics:  Started as pictograms and developed into a logographic system.
b.  Cuneiform:  Signs are read for sound value: <waves> = sea or see.  Use rebus sentences to say what you want.  Ex.: <eye> (I) <waves> (see) <sun> (son) = "I see my son."  This is moving in the direction of having symbol-sound correspondence.
4.  Syllabary:  Each symbol = one syllable.  Advantages:  A relatively small number of symbols to learn; you know how to say it based on the symbols.  But, it's still a lot to memorize.
5.  Alphabetic:  Each symbol = one phoneme.  Ideally, one sign for one sound. Advantages:  Efficient:  Each language has only a few sounds (usually around 50), so you can get by with 20-40 symbols to memorize; Flexible:  Any language can be characterized by an alphabet.  Simply list the sounds and then assign them symbols.
a.  Phoenicians:  Adapted hieroglyphics into 22 signs (all consonants).  They traveled a lot, encouraging the spread of alphabetic systems.
b.  Greeks:  Adapted Phoenician alphabet and added vowels.  First true alphabet:  Each sign = one sound, every sound has a sign.
III.  Font development (formal printing).  Most of the reading that you do is from printed material, so no survey is complete without discussing it.
A.  Evaluating fonts:  There are two (related) dimensions along which fonts are evaluated:
1.  Legibility:  How easily can the letters in the font be perceived?  The easier it is to make them out, the more legible they are.
2.  Readability:  How fatigued do you get when you read them?  Poorly designed fonts make you more tired than well-designed fonts.
Note that these characteristics were rarely considered as fonts developed.  Instead, you have a history of technological improvements tempered by a desire to preserve what was done in the past.  Sometimes, this resulted in happy accidents that improve legibility and readability.  But, it was rarely done on purpose.
B.  Font characteristics:
A list of some basic properties Best option
a.  Serif vs. sans-serif:  A serif is the little horizontal mark on the tops and bottoms of vertical lines in fonts (the line on the bottom of this f).  Originally, it was used when cutting letters in stone to prevent the stone from cracking, but it was preserved out of tradition.  Luckily, serif is better. 
b.  Weight difference:  Some lines in a character are thicker than others (e for example). 
c.  Bias:  The fonts can be on a bias or they can be vertical. 
d.  x-height:  How much height is devoted to the body of the characters (how tall the x is). 
e.  Spacing:  Some characters are wider than others (i vs. e).  Typewriters force these to take up the same amount of space, but it's not required in printing (proportional is when they take up only the required space) (piece vs. piece). 
f.  Proportions:  How big is the x-height relative to the heights of ascenders and descenders (parts going above and below the body of the letter).  There is an optimal proportion for each font.
More is better 
C.  Why are we discussing this?  Because the fonts can have a huge influence on reading.  We have to know what's being read before we can look at the process of reading.  It all comes down to constraint satisfaction.  We need to understand the constraints on fonts (what are the parts, what aren't) so that we can understand how they constrain reading.
As an example of the importance of features on letter identification, consider an experiment by Neisser on visual search:  You scan for an X in a field of Z's and N's or a field of O's and P's.  X is easier to see in letters with different features than letters with similar features.  Try it:
N N Z N Z N Z N Z   
Z N Z Z N Z Z N N   
N N N Z N X N Z N   
N N Z N Z N Z N Z   
Z N Z Z N Z Z N N 
O O P O P O P O P  
P O P P O P P P O  
O O P P O X P O P  
O O P O P O P O P  
The X should "pop-out" of the grid with dissimilar features.  A similar process of analysis-by-features probably takes place in reading, making it very important to understand the features.
IV.  Orthography.  What are the spelling rules?  Again, it comes down to constraints.  If there are rules that govern spelling patterns, then knowledge of these rules can help the process of identifying letters and words.  Additionally, a lot of theories of reading require a mapping from spelling to sound.  If you understand how these correspondences work, then you can go a long way as a reader towards knowing what's being said.  The system for English was worked out by Vinetsky.
A.  Some of the big rules:
1.  Avoid letter doubling.
2.  V C V,  V C C V,  V C:  A vowel before a consonant-vowel is long, a vowel before a C-C-V is short, a vowel before a consonant is short.
a.  To override V C, add a dummy 'e' to get V C V.  Examples:  "fin", "can" vs. "fine", "cane."
b.  To override V C V, you have to double.  Example:  "cunning."
3.  Especially avoid doubling at the beginnings and endings of words.
a.  Except for ff, ll, ss.
b.  Except for 3-letter words (egg, inn, add, ebb).  Why?  18th century editors decided to reserve 2-letter words for function words (to, in).
B.  Why does this matter?
1.  It gives you hints at pronunciation even if you've never seen the word before ("mabe", "mab", "mabing", "mabbing").
2.  It helps you to know what letters to expect in a particular situation.  The interactive activation model was designed to take advantage of this.  By letting letter features and letter identification be influenced by regularities at the word level, you increase the sources of constraint and make it easier to identify the stuff on the page.
C.  A bit on English spelling:
There are a lot of irregularities in English spelling.  Part of this is due to the great vowel shift where pronunciation changed and spelling didn't.  Part is due to English's propensity for borrowing.  A lot of borrowed words contain spellings from other languages that make little sense in English.  Here are some of the problems:
1.  Some letters in a word may not represent any sound (through, sign, and give).
2.  A group of two or more letters can represent a single sound (think).
3.  A single letter can be two or more sounds (saxophone).
4.  The same letter can represent different sounds in different words (on, bone, one).
5.  The same sound can be different letters in different words (rude, loop, soup).
The long poem at the end of the notes has a lot of examples of weird English spelling-sound correspondence.
Why is the English spelling system good (even though it violates one-sound one-symbol allowing "ghoti" to be an acceptable spelling of "fish")?
1.  Because the meaning is more transparent.  We have to work harder to learn pronunciation, but we can then figure out word derivations based on spelling, helping us to get the meaning.  Some examples:
electric -- electricity /k/ and /s/ are both covered by 'c'
insert -- insertion /t/ and /  / are both covered by 't'
right -- righteous /t/ and /  / are both covered by 't'
bomb -- bombard ø and /b/ are both covered by 'b'
damn -- damnation ø and /n/ are both covered by 'n'
produce -- production /  / and /  / are both covered by 'u'
2.  Dialects.  In parts of the country, "car" would be spelled "ca" to stick with the one-sound-one-symbol rule.  Different speakers would have different spelling systems.
3.  Because we can distinguish homophones by different spellings.  Some examples:
to, too, two
bare, bear
no, know
flea, flee
sore, soar
Without spelling, readers would have to figure this out on their own.  That's not as easy because there's more room for ambiguity.  Of course, irregular spelling is a different sort of ambiguity, but it's easier to handle.
So, crazy English spelling is actually good.  In a sense, English orthography is not based on pronunciation.  So, it's not really trying for one-sound-one-symbol.  Instead, it's preserving the same spelling for all variants of a morpheme.  In other words, it's a morphological system, more sensitive to morphological differences than phonemic differences.
D.  Examples of orthography constraints:
1.  Word superiority effect:  Letters are perceived better in words than alone.  This is probably due to information from the orthography (surrounding letters).  The interactive activation model of reading uses the extra information from the bottom (more letters) interacting with information from the top (knowledge of words) to arrive at an identification for a given word.
2.  Effects on reading:
a.  People with Broca's aphasia don't suffer as much disruption if their writing is logographic because logographic systems use a different part of the brain.  For example, Japanese Broca's patients can have difficulty with hiragana or katakana (syllabaries), but be OK with kanji (logographic).
b.  Congenitally deaf people have an easier time learning logographic systems than alphabets or syllabaries (probably due to trouble with sound-spelling correspondences).
c.  Learning logographic systems is more memorization intensive, but reading is easier right from the start (for the symbols you know).  Alphabets are hard to master.  Syllabaries aren't as bad because children seem to be better at identifying syllables than phonemes.  Even (phonologically naive) adults can have a hard time segmenting words into phonemes but can do syllables with ease.  Note that a syllabary would be impractical for English.
V.  Components of reading.  What are the basic processes?
A.  From eye-tracking studies:  In eye-tracking experiments, subjects sit in front of a computer monitor to read some text.  A small camera records their eye position, and this information is used to determine exactly where on the screen the subject is looking.  Based on this, the amount of time spent on each word can be computed.  From observational studies of readers in eye-tracking studies we know that reading consists of:
1.  Gazes/fixations:  These are pauses on words.  This is where information is accumulated about words.  Some facts:
a.  Length is a function of the reader and the material.  Some people have a different time constant than others.  Generally, a fixation lasts around 250 ms (1/4 second).
b.  Approximately 83% of content words get fixated and 38% of function words.
c.  A fixation generally contains 2-3 characters to the left and 7-12 characters to the right (for a language like English which is read left to right).
d.  In the periphery, readers are affected by the physical characteristics of the text about 12 characters ahead.  They're affected by the semantic content of the text about 6 characters ahead.
2.  Saccades:  If you try to observe yourself reading, you'll probably feel that your eyes travel smoothly across the page.  But, what really happens are periods of relative stability (fixations) separated by rapid jumps (saccades).
a.  They take around 10 ms, and generally move forward 10 characters.  The exact size will depend on the reader and the text.
b.  They're not uniform.  You generally jump over function words and fixate about 2 characters into content words.
c.  The saccades allow for processing and erase your sensory registers to stop perceptual processing of the last fixation and prepare for the next fixation (masking).
3.  Regressions:  About 15% of saccades are back to some earlier portion of the text.  The number of regressions depends on the reader and the text.
B.  Additional processes/factors:  We've been structuring things as ecological survey -> theoretical explanation.  The reason is that you can't explain something until you've described it properly.  This is a survey of the processes and factors involved in reading.  Any models that we discuss will have to consider these components:
1.  External factors:
a.  Physical stimulation:  The lighting and the text characteristics (that font stuff we talked about).
b.  Word frequency:  Some words appear in print more frequently than others, meaning that some words are encountered more frequently when reading.  This frequency has an effect on reading (you read more frequent words faster).
c.  Sentence construction:
1)  Some sentence forms are read faster (active sentences are easier to process than passive sentences).
2)  Length:  Generally, shorter sentences improve comprehension.
3)  Anaphors (part of the text that refers back to an earlier part):  More make it worse.
d.  Density of propositions (idea units):  The more you have per sentence, the harder it is to comprehend.
e.  Text construction:
1)  More coherence between propositions is better (the propositions in consecutive sentences should be related).
2)  The more it corresponds to story grammars the better (for instance, fairy tales have a particular sequencing, following that improves comprehension).
3)  Organization:  The more headers, labels, and structure you provide the better.
f.  Content area:  Some kinds of texts (newspapers) and some content areas are easier to understand.
g.  External goal set:  Why are you reading?  It's different if you're memorizing, summarizing, or just understanding.
2.  Internal factors:
a.  Schema knowledge and application:  Most events have a regular structure, and most articles and books follow a structure.  The more familiar you are with an area the more of this you can apply.  For example, me reading a paper for this class works very differently than you reading it.
b.  Procedural knowledge:  Knowledge about reading.  For example, knowing how to identify main points.
c.  Decoding ability:  Getting from the letters to a representation of meaning.  This is an important individual difference factor.  My decoding ability for Japanese is very bad, middling for Romanian, and good for English.
d.  Viewpoint/purpose:  An internal goal set for reading (what's my motivation).
e.  Cognitive resources:  How much mental energy you have to devote to the task.  This is another important individual difference.  Two points:
1)  More is better.  There doesn't seem to be any way to change a person's amount of capacity.
2)  Some people use it better than others.  This is something you can influence.
VI.  Models of reading.  I'm presenting one example of each of the two major classes of models.  Obviously there are many more approaches.
A.  Production systems:  Just and Carpenter (1980):  A production system is a list of condition -> action pairs.  Usually, it's like an if-then statement.  So, "if I see a green light, then I start to drive".  The action is only carried out if the condition is met, and the action will be carried out automatically if the condition is met.  They string a bunch of these together to model reading.  Kinds of productions:
1.  Get next input:  A set of productions to initiate a saccade to get more information
2.  Feature analysis:  Pull out the physical features of the words.
3.  Word encoding and lexical access:  Use the features from 2 to access the meaning of a word.  Access can come from:
a.  Perceptual information (the way the word looks).
b.  Spreading activation from related words already read.
c.  Serial productions:  Information that's read goes into a limited capacity working memory, and anything that's in here gets refreshed periodically.  As word meanings become less important, they're replaced by other meanings, and this source stops providing activation.
This process is influenced by:
a.  The number of syllables in the word.
b.  Word frequency (how often it appears in print).
c.  Whether or not it's a novel word.
4.  Case role assignment:  A case role is the role a word plays in a sentence.  For example, in "John pounded the nail", John is the agent, nail is the patient.  Knowing the case role can help you narrow down what the next word might be (more constraints).  For example, if I start the sentence "The man was interrogated by the", you're probably expecting an agent.  You should be surprised to hear "window", even though it is acceptable.
5.  Interclause integration:  You have to relate the components of a text via some kind of structure if you're going to understand it, and this process builds that structure.  This process is affected by:
a.  Ends of sentences.
b.  Ends of paragraphs.
6.  Sentence wrap-up:  At the end of a sentence you have a good opportunity to make sure that all ambiguities have been cleared up, all meanings have been accessed, and that everything fits the structure.  It's like clearing the table for the next course.
Some notes about the model:
1.  They used eye-tracking data to test the predictions of the model.  The model accounts for 72% of the variability.  This is really good.
2.  This is between the stage approach and parallel models.  Each production is stage-like in the sense that it only does one task, but the productions can fire in any order, and in parallel.
3.  This model makes predictions about poor readers.  They should have two problems:
a.  They should have less working memory capacity.  (Software to measure capacity is on my software page.)
b.  They should have trouble suppressing inappropriate information.  Why's that bad?  You don't have much room to begin with.  Keeping stuff you don't need makes the problem worse.  Also, if you're poor at suppressing irrelevant information, it can cause you to shift your representation more often than necessary, making it hard to follow the text.
Gernsbacher, Varner, and Faust (1990) looked at suppression skills of poor comprehenders.  They presented sentences like the following:
He dug with the spade.
"Spade" is ambiguous because it could be the card suit (ace of spades) or a kind of shovel.  The shovel meaning is appropriate in this context, the card suit is not.  Right after reading an ambiguous word, all readers show evidence of activating both the appropriate meaning and the inappropriate meaning.  It's as if activating all meanings and then suppressing the wrong one is easier than trying to search for just one meaning to begin with.  How is this assessed?  Immediately after the sentence, present a word about which people make a judgment (is this word related to the sentence).  For example:
The correct response is "no."  Compare that to deciding about "ace" after:
He dug with the shovel.
"Ace" should never be activated if you read "shovel," so it should be easy to say "no."  If your read "spade," it should be hard to say "no."  100 ms after the sentence was presented, good and poor comprehenders (measured using a comprehension battery) were both slower in the "spade" version than the "shovel" version.
The real question is how do they do later?  If good comprehenders suppress information better, then they should get rid of the inappropriate meaning faster than poor comprehenders.  So, the critical condition was to wait until 300 ms after "spade" and then test with "ace."  Good comprehenders had no problem with that.  Poor comprehenders were still thinking about "ace."  So, it looks like poor comprehenders are slow to get rid of irrelevant information.
B.  Interactive activation:  McClelland and Rumelhart (1981):  (We discussed this before, but here's something a little more formal.)  A mostly parallel model.  You have two components:
1.  Nodes:  Each node holds a piece of information.  You have feature nodes (that detect things like / - ) ( | ), letter nodes (that detect things like a f e o p), and word nodes (that detect things like words).  The nodes are arranged into layers (a feature layer, a letter layer, and a word layer).
2.  Connections:  They connect nodes.  They're like dendrites and axons on neurons.  The information that the model learns is stored in the connections.  The connections can be either excitatory or inhibitory (make a node more or less likely to fire).
How it works:  The feature level receives inputs from (theoretically) your eyes.  Each node detects a particular feature.  If the feature is present, any letter with that feature receives an excitatory input along the connection.  If a feature is absent, inhibitory inputs go up to letters.  As letters become more excited, they can feed down to excite some features even more.  Letters also inhibit each other.  Eventually, this part stabilizes, and some features and some letters are really excited, the others are suppressed.  At this point, it's mostly driven by the physical characteristics of the word (the features).
The letters also feed up to words.  So, any word with a particular letter that's excited will also be excited.  Words can feed down to excite other letters.  Words can also inhibit each other.
All of this trading excitement and inhibition will rapidly settle on a particular word.  Then you can move your eyes to get a new word.  Some notes:
1.  This accounts for word superiority because of the additional information coming from other letters in the word.  If the word is "work", the features excite letters, which excite the word, which in turn excites the letters even more.  This activation coming down from the word isn't present when the letter is presented alone, and that's why the word makes letter identification easier.
2.  Think about scaling this up.  Once you get past physical words you still need meaning.  Then you need sentence nodes, text nodes, etc.  Where is the understanding?
That concludes the formal part of the unit.  Here's a bonus poem that highlights the irregularities of the English orthographic system.  Try to read it fast, out loud.

Dearest creature in creation,
Study English pronunciation.
I will teach you in my verse
Sounds like corpse, corps, horse, and worse.
I will keep you, Suzy, busy,
Make your head with heat grow dizzy.
Tear in eye, your dress will tear.
So shall I!  Oh hear my prayer.
Just compare heart, beard, and heard,
Dies and diet, lord and word,
Sword and sward, retain and Britain.
(Mind the latter, how it's written.)
Now I surely will not plague you
With such words as plaque and ague.
But be careful how you speak:
Say break and steak, but bleak and streak;
Cloven, oven, how and low,
Script, receipt, show, poem, and toe.
Hear me say, devoid of trickery,
Daughter, laughter, and Terpsichore,
Typhoid, measles, topsails, aisles,
Exiles, similes, and reviles;
Scholar, vicar, and cigar,
Solar, mica, war and far;
One, anemone, Balmoral,
Kitchen, lichen, laundry, laurel;
Gertrude, German, wind and mind,
Scene, Melpomene, mankind.
Billet does not rhyme with ballet,
Bouquet, wallet, mallet, chalet.
Blood and flood are not like food,
Nor is mould like should and would.
Viscous, viscount, load and broad,
Toward, to forward, to reward.
And your pronunciation's OK
When you correctly say croquet,
Rounded, wounded, grieve and sieve,
Friend and fiend, alive and live.
Ivy, privy, famous; clamour
And enamour rhyme with hammer.
River, rival, tomb, bomb, comb,
Doll and roll and some and home.
Stranger does not rhyme with anger,
Neither does devour with clangour.
Souls but foul, haunt but aunt,
Font, front, wont, want, grand, and grant,
Shoes, goes, does.  Now first say finger,
And then singer, ginger, linger,
Real, zeal, mauve, gauze, gouge and gauge,
Marriage, foliage, mirage, and age.
Query does not rhyme with very,
Nor does fury sound like bury.
Dost, lost, post and doth, cloth, loth.
Job, nob, bosom, transom, oath.
Though the differences seem little,
We say actual but victual.
Refer does not rhyme with deafer.
Foeffer does, and zephyr, heifer.
Mint, pint, senate and sedate;
Dull, bull, and George ate late.
Scenic, Arabic, Pacific,
Science, conscience, scientific.
Liberty, library, heave and heaven,
Rachel, ache, moustache, eleven.
We say hallowed, but allowed,
People, leopard, towed, but vowed.
Mark the differences, moreover,
Between mover, cover, clover;
Leeches, breeches, wise, precise,
Chalice, but police and lice;
Camel, constable, unstable,
Principle, disciple, label.
Petal, panel, and canal,
Wait, surprise, plait, promise, pal.
Worm and storm, chaise, chaos, chair,
Senator, spectator, mayor.
Tour, but our and succour, four.
Gas, alas, and Arkansas.
Sea, idea, Korea, area,
Psalm, Maria, but malaria.
Youth, south, southern, cleanse and clean.
Doctrine, turpentine, marine.
Compare alien with Italian,
Dandelion and battalion.
Sally with ally, yea, ye,
Eye, I, ay, aye, whey, and key.
Say aver, but ever, fever,
Neither, leisure, skein, deceiver.
Heron, granary, canary.
Crevice and device and aerie.
Face, but preface, not efface.
Phlegm, phlegmatic, ass, glass, bass.
Large, but target, gin, give, verging,
Ought, out, joust and scour, scourging.
Ear, but earn and wear and tear
Do not rhyme with here but ere.
Seven is right, but so is even,
Hyphen, roughen, nephew Stephen,
Monkey, donkey, Turk and jerk,
Ask, grasp, wasp, and cork and work.
Pronunciation -- think of Psyche!
Is a paling stout and spikey?
Won't it make you lose your wits,
Writing groats and saying grits?
It's a dark abyss or tunnel:
Strewn with stones, stowed, solace, gunwale,
Islington and Isle of Wight,
Housewife, verdict and indict.
Finally, which rhymes with enough --
Though, through, plough, or dough, or cough?
Hiccough has the sound of cup.
My advice is to give up!!!
-- Author Unknown

Psychology of Language Notes 5
Will Langston

 Back to Langston's Psychology of Language Page