Chris Roberts Thesis 2

Empirical Research/Previous Comparative Research

While the current body of empirical research on comprehensible-input based instruction such as TPRS is low, it is growing.  Studies have been conducted that show that the use of comprehensible input-based methods is effective in language acquisition.

A study conducted by Beniko Mason found that story-telling is as effective as more traditional methods (Mason, 2004).  In this study, 58 first year Japanese female students at a college in Osaka who had very little exposure to English were split into two different classes.  One class was a story only group and the other class was a story plus supplementary activities group (Story-Plus Group).  The story-only group’s treatment consisted of a vocabulary pretest and then a listening to a story which contained the target words.  Then, a posttest on the same list of words was taken.  The story-plus group also took a vocabulary pretest and listened to a story.  However, this group also experienced an extra 10 minutes of the teacher asking oral comprehension questions that used the target words.  The students also read a written version of the story for 10 minutes.  After the reading the participants told the story to their study partner, using the target words which were on the board.  The students then took the same posttest as the story-only group.  The story-plus group performed better than the story-only group, however the story-plus group spent 85 minutes on hearing the story and doing different activities.  The story-only group listened to the story for 15 minutes.  It was found that despite the higher scores, when time is taken into account the story-plus group acquired 0.13 words per minute while the story-only group acquired 0.15 words per minute, nearly identical results.  This findings in this study show that simply by hearing stories, vocabulary development occurs just as efficiently as an approach that is supplemented with vocabulary learning activities.

In a study conducted by Joseph Dziedzic, it was found that students in a TPRS level 1 classroom performed significantly better than traditional students in a level 1 classroom in speaking and writing performance assessments.  The measurement tool used in the study was the Denver Public Schools Proficiency Assessment which is given at the end of the academic year.  The test measures proficiency in the four skill areas of listening, reading, speaking and writing.  The teacher who conducted the research teaches four level 1 Spanish classes.  He taught 2 classes using comprehensible input-based methods (TPR, TPRS and SSR) and two classes using traditional, output-based instruction with the Buen Viaje textbook.  The Denver Public Schools Proficiency Assessment was administered at the end of the school year.  The reliability of this proficiency assessment has been estimated to be .82 for the fall assessment and .84 for the spring assessment, indicating that the test is “stable”.  On this assessment students are assessed on listening, reading, speaking and writing: • Listening: students listen to short dialogues or narratives and answer questions • Reading: students read short texts in target language and answer questions • Writing: students write 10 or more sentences based on a scene of pictures • Speaking: students produce between 4-6 sentences about a series of pictures According to the results of the proficiency assessment, the students in the TPRS class scored significantly better than the traditional students on tests of output, speaking and writing, and were equivalent to traditional students on tests of input, reading and listening (Dziedzic, 2012).

A comparison study of TPRS and traditional language instruction was also performed by Barbara J. Watson, showing similar results (Watson, 2009).  The study was conducted in first year Spanish classes in which students in a TPRS class were compared with students in a traditional class.  The TPRS class focused on unconscious language acquisition by providing comprehensible input through storytelling and not forcing students to speak nor engage in heavy grammar instruction or error correction.  The traditional class focused on conscious learning, “with a great deal of the instructional time dedicated to helping students understand grammatical concepts through explanations” (Watson, 2009, p. 21).  Techniques used in the traditional class included short interviews, student-to-student question and answer exercises and review games to identify and review vocabulary.  This class also incorporated photo and video projects, technology into the lessons.  Both of these groups read every week and each group read the Blaine Ray novel, Pobre Ana, a first year Spanish reader that used typical first year Spanish vocabulary.  An analysis of audiotapes of the lessons showed that the TPRS class was teacher-fronted 68% of the time while the traditional class was teacher-fronted 29% of the time, giving students opportunities to practice output activities with partners.  Seventy three students enrolled in first year Spanish were split into three classes: 23 in the traditionally taught class and 50 were enrolled in two sections of a TPRS class.  The demographics of each class were very similar socioeconomically but it is important to mention that in the TPRS classes 4% of the students spoke Spanish at home or outside of school and in the traditional class 15% of the students spoke Spanish at home or outside of school.  Two measures of assessment were given at the end of the school year, a final exam and an oral exam.   The final exam consisted of listening, vocabulary and grammar and reading.  The results of the study show that the TPRS students outperformed the traditional students on both the final and oral examinations, scoring about one standard deviation higher.  There were no extraneous variables in this study that might have had an effect on the outcome of the study.  Other than the methodology used, there were no reasons for the TPRS group to perform better than the traditionally taught group.  In fact, a larger percentage of the traditional students had exposure to Spanish outside of the class, which suggests that the TPRS groups’ performance superiority was larger than reported from the assessments (Watson, 2009).   A similar study conducted by Kelly Varquez also showed that TPRS students outperformed students in a traditional foreign language class on a standardized test, the University of the State of New York’s standardized Spanish Language Proficiency Examination in Spanish (Varquez, 2009).  In this study, four different teachers from four different schools participated.  The teachers were chosen on the basis of three factors: reputable recommendations, survey score, and personal description of typical classroom activities.  Of the four chosen teachers, two were labeled traditional and two were labeled TPRS instructors according to the scores on their survey.  The traditional teachers typically elicited practice of reading, writing, listening and speaking skills in their classrooms.  Grammar instruction and drills played a role in instruction as well as the study of vocabulary lists and the practice of speaking and writing.  The TPRS teachers typically focused on language comprehension activities such as storytelling, teacher-led class conversations, and reading.  Both of the traditional groups (groups A and B) and one of the TPRS groups (group D) were schools that educated students with more socioeconomic advantage and the other TPRS group (group C) was in a school of lower socio-economic status as well as high rates of student mobility, 38.90% (Varquez, 2009).  This lower socio-economic school also had a student teacher experienced in TPRS as the instructor for a portion of the school year.  The University of the State of New York’s standardized Spanish Language Proficiency Examination in Spanish was the test given to students to test the reading and listening comprehension skills of the groups at the end of their first year of study.  This test had a reliability coefficient of .75.  The results of the study show that the TPRS group, from the school with more socioeconomic advantage far outperformed the two traditional groups.  The two traditional groups’ combined mean score for the exam was a 23.45 out of 35 while the TPRS group’s score was 32 out of 35.  The TPRS group from the socioeconomically disadvantaged school had a mean score of 22.30 out of 35, not significantly different from the traditional students’ scores.  This study shows that when demographic factors are similar, TPRS students heavily outperform students in more traditional classes that emphasize output and grammar instruction.  The study also shows that the students who come from the lower SES school and also had a student teacher for a portion of the year performed just as well as the students in the traditional class.

In a study by VanPatten and Cadierno, students in an input-based classroom were compared with students in a traditional classroom.  In the input-based classroom, students at no point were required to produce the target structures.  In the traditional instruction classroom, the students were instructed with the more dominant approach for instruction in grammar: presentation, and then output practice moving from mechanical to more meaningful communicative kinds of exercises.  VanPatten and Cadierno used a standard pretest/posttest design with several delayed posttests.  The students in the input-based class made significant gains on both interpretation and production tasks while the traditionally taught students only made gains in production.  Van Patten and Cadierno concluded that an input-based approach was superior to an output-centered traditional model of language teaching because of the “two for one” gains made by the students in the input-based class (VanPatten, Farmer & Clardy, 2008).

Beniko Mason found similar results to VanPatten’s in a study.  Mason found that comprehensible input alone through reading was twice as efficient as traditional instruction involving output and grammar instruction in score improvement on the TOEIC, Test of English for International Communication.  The subject for this study, Mr. Tanaka, was a 42-year-old man in Osaka, Japan who wanted to improve his English. Over the course of two years, Mr. Tanaka had read most of the Macmillan, Penguin and Oxford graded readers for pleasure.  In addition to these readers, he had also some authentic books such as those by Judy Blume as well as best-sellers such as Twilight.  Through reading and attending an English class that focused on listening comprehension, giving him 30 hours of listening and hearing stories in English, Mr. Tanaka raised his score from 475 initially  to 655 a year later.  He improved .73 points for each hour spent on reading or listening, or 180 points of improvement through 247 hours of receiving input in the language.  Mr. Tanaka’s results were compared to college students of English as a foreign language in Japan.  These students, in four semesters, or 500 hours of instruction that combined traditional instruction with reading, gained about 135 points on the TOEIC.  This is .27 points per hour.  Mr. Tanaka, with 15 points per month, or .73 points per hour, was nearly three times as efficient.  This study, while limited due to such a small sample size, does demonstrate that comprehensible input alone can result in significant improvement in another language (Mason, 2011).

In a second, similar study, Mason finds similar results with a 75-year-old student (Mason, 2013).  In this study, Mr. Nanako, a 75-year-old male in Osaka, Japan, took a reading and listening class at an adult education program offered by a university in Osaka.  This course was a comprehension-based course that met once a week and focused on listening to folk tales from around the world, told by the teacher.  For homework, students read from graded readers, beginning at the 200-word level and advancing to more advanced books.  After completing the course, he took the TOEIC test and scored 495.  He then independently reading and listening, not attending any English classes, but he kept a log of what he read and the time invested in reading.  In five months, he spent 253 hours in reading and listening.  He took the TOEIC test again and scored 580, gain of 85 points in five months.  Like in Mason’s previous study (Mason, 2011), the student in this study made gains on the TOEIC that exceeded the gains by college students in Japan majoring in English.

Rodrigo, Krashen and Gribbons (2004) found that students in a comprehensible input based class outperformed traditionally taught students on a variety of assessments.  In this study, the researchers examined the impact of two different approaches on comprehensible input and compared them with a more traditional approach to language teaching.  This study was done in intermediate level Spanish classes at the university level.  The study consisted of an initial study as well as a follow-up or replication, each taking one semester.  Both parts included three groups: • Experimental Group I: Reading – This group experienced an extensive reading approach.  The students read as much as they could during the semester, reading a combination of assigned and self-selected readings.  After reading one of the graded readers, the students would write a short report, including their reaction to the book.  The reports could be written in either Spanish or English.  There was no direct grammar teaching or error correction in this course. • Experimental Group 2: Reading-Discussion – This group read and participated in discussions about the readings.  The class did the same readings as Experimental Group 1, but they did not do self-selected reading.  The readings were the basis of the in-class discussions, which students were expected to participate in.  There were no exams, grammatical explanations, error correction or explicit teaching of vocabulary in this class. • Comparison Group: traditional grammar and composition – This group was an Intermediate Grammar and Composition course.  This course emphasized explicit instruction of Spanish grammar and vocabulary.  Each chapter of the textbook had a required reading passage.  Students wrote compositions that were corrected for linguistic accuracy and took several quizzes, a midterm and a final exam.

There were three tests given to compare the groups.  All three assessments were given as a pre-test and as a post-test to measure growth over the semester.  The first assessment, a vocabulary checklist test, was given to measure vocabulary acquisition.  This test had a high reliability coefficient of 0.90.  The gains of the two experimental reading groups were significantly higher than the gains achieved by the Traditional group.  The second assessment was a grammar test published by the Ministry of Education and Science of Spain and had a reliability coefficient of 0.86.  Interestingly, on this test, the two experimental reading groups showed improvement over the course of the semester but the results of the Traditional group got worse from the pre-test to the post-test.  There is research that helps explain why the group that was explicitly taught grammar was not able to perform as well as the other groups.  According to Ponniah (2007), the reason why students are not able to correctly use grammar is because they have not internalized the application part of the rules of grammar due to relying on rote memorization of grammar rules.  Ponniah says that “memorization is the process of establishing information in memory.  Relying on it for acquiring language skills will not give productive results.” (Ponniah, 2007, p. 2).  What Ponniah is arguing is that rote learning and memorization of grammar rules is a constraint to actually acquiring language skills, such as internalizing grammar.  Ponniah recommends giving a lot of listening and reading input in order to acquire implicit knowledge.  Because the Reading and Reading-Discussion groups focused on input, they were able to internalize grammar, developing a “feeling of correctness” (Ray and Seely, 2012).    The third test was a cloze test, consisting of a paragraph in which every fifth word was deleted.  There was a total of 50 blanks, yet the first and last sentences of the paragraph were left intact to provide context.  This test was a placement test used by the Spanish department of a major university and had a reliability coefficient of 0.91.  The traditional group made gains of 3.33 while the Reading-Discussion group made gains of 10.00.  The results of the study found that the comprehensible input-based classes, the Reading and Reading-Discussion groups outperformed the traditionally taught group on a vocabulary test and a grammar test.  On the cloze test, the Reading-Discussion group outperformed the Traditional group and the Reading group and Traditional group did not differ.  The results of the study support the hypothesis that comprehensible input-based methods are more effective than more output-based, traditional approaches.

A 2009 doctoral study compared Communicative Language Teaching (CLT) and Teaching Proficiency through Reading and Storytelling (TPRS) (Spangler, 2009).  Spangler studied the effects of these two teaching methodologies and their effects on beginning-level students’ achievement, fluency, and anxiety.  Communicative Language Teaching is a communication-based method that encourages both L2 input and L2 output.  This study used 162 public school students and two public school teachers, one in California and one in Rode Island.  In California, the study was done in a high school and in Rhode Island in a middle school.  Each teacher was familiar with and trained in the skills involved in CLT and TPRS.  Each teacher taught one class using CLT and one class using TPRS.  Each group of students experienced 14 weeks of language learning with their respective method being used.  In order to measure student anxiety, the researcher used the Horwitz Foreign Language Classroom Anxiety Scale (FLCAS).  In order to measure student achievement and fluency, the researcher used the Standards-based Measurement of Proficiency (STAMP) test.  The STAMP test “is a web-based assessment tool characterizing proficiency levels that are tied to the ACTFL Performance Guidelines” (Spangler, 2009).  The test measures student achievement and proficiency in reading, writing and speaking.    In the classes using Communicative Language Teaching, some of the strategies used were: real-life situations that necessitate communication, information-gap activities, structured language output activities, verbal and written role-play, communicative classroom activities and communicative games.  In the classes using TPRS, strategies used were: chapter stories, personalized mini-stories, personalized question and answer, free voluntary reading, TPR gestures, student actors, circling question techniques and songs or chants.  On the 12th week of the 14 week study, participants took the FLCAS to evaluate student anxiety.  Then, on week 14, all students took the STAMP test.  The results of the study in regard to reading achievement indicated that there was not a significant difference between the mean reading scores for students who received CLT instruction and those who received TPRS instruction.  In writing fluency, the results of the study showed that there was no significant difference between the writing of the CLT students and that of the TPRS students.  However, there was a statistically significant difference between the mean speaking scores for students who received CLT instruction and those who received TPRS instruction.  The TPRS methodology of instruction resulted in higher levels of students’ speaking fluency than with the CLT methodology.  On the anxiety scale, there were no significant differences between the two teaching methodologies.  What is interesting about this study is that the Communicative Language Teaching approach focuses on communication between students and between teacher and students.  In CLT, the focus is on student output, giving students frequent opportunities to produce language.  CLT focuses on the use of pair work activities, role plays, group work activities and project work (Richards, 2006).  A typical lesson in Communicative Language Teaching follows the P-P-P approach, Presentation, Practice, Production.  Having students practice and produce new structure in different contexts is the goal of the typical CLT lesson (Richards, 2006, p. 8).  The idea driving CLT is that we acquire language by practicing it.   In the TPRS classroom, the Comprehension Hypothesis is adhered to, which says that we acquire language when we understand messages.  In the TPRS classroom, speech is allowed to emerge, it is not forced.  According to language acquisition theory, speech is a result of language acquisition, not its cause (Krashen, 2013).  Spangler’s study supports the previous research done on comprehensible input-based teaching.  The students in the input-based classroom (TPRS) outperformed the students in the Communicative Language Teaching classroom on the speaking assessment.  This supports the argument that output produced by the language acquirer does not contribute to language acquisition (Krashen, 1998).

By listening to and discussing stories, receiving comprehensible input, students perform better than their traditionally taught peers.  Storytelling can develop interpretive abilities, even at very early stages of acquisition (Shrum and Glisan, 2005).  Even in beginning level classes students can successfully interpret what they hear in a story “especially when the story (1) is highly predictable or familiar to children from their native language, (2) is repetitive, (3) lends itself to dramatization and pantomime, and (4) lends itself to use of visuals and realia to illustrate meaning” (Shrum & Glisan).  These factors are almost always present in TPRS stories.

Despite the studies showing the effectiveness of an input-based method such as TPRS, many still think that explicit vocabulary and grammar instruction is needed.  There is still research out there presenting a theory of direct instruction being more effective than incidental vocabulary instruction (Mason & Krashen, 2004).  Mason and Krashen addressed this in a study that showed that students simply receiving comprehensible input through stories learned more words per minute than students in a more intensive class involving stories and supplementary, traditional instruction.  The results of their study show that the additional focus on form in traditional vocabulary exercises is not as efficient as hearing words in the context of stories (Mason & Krashen, 2004).

Present Study Research Question My preliminary research question is: 1. How does student achievement in a TPRS classroom compare with student achievement in a traditional classroom on a nationally standardized exam?



9 thoughts on “Chris Roberts Thesis 2”

  1. Chris, I’ve read every word and I’m very impressed. You are putting numbers and facts to the deep in the gut feeling so many experienced teachers have felt when they started doing TPRS. Keep up the good work and keep us posted. I’m sending this to a certain Inspecteur Pédagogique Régional who vetoed Blaine Ray giving a demonstration to teachers here in France. Thank you for handing me the ammunition I needed.

  2. I’ve been busy with a 5 yr old, teaching as well as Master’s classes. I don’t post as often as I would like, but I do lurk occasionally when I have time. I don’t know what I would do without this blog.

  3. Well Chris we are going to have to do more with this than we thought. You have built something very strong here. Maybe we could make a hard link for it.

    Think about how we can keep it alive and not like most feces, I mean theses, which disappear down the turlet with time and even the writer forgets what it looked like, its make up and what was in it.

    I know that my master’s thesis had something to do with depth psychology, Bachelard and the Jungian concept of interior space in the 19th c. French novel, but I couldn’t tell you exactly what. Time flushed it.

    What you have done here is alive and valuable and can actually address and clear up the current confusion in the minds of many, many teachers who have heard of comprehension based instruction but don’t yet really grasp what it is.

    I’m thinking this can be a hard link. It will take a year for me to get my new people training vids up, but slowly we will build a strong set of hard links across the top here, and most of them will be directed at new people.

    I want this on it if you will allow that. Really well done. Plus you get the big buck salary increase, right?

  4. Very true about the thesis/feces connection. It’s very Freudian. In fact, I had a professor whose thesis was about feces, namely ancient Greek male “birthing” rituals. I think the title, which had the phrase “turd child” in it actually made some list of the 100 most ridiculous thesis titles.

    What’s that saying about youth wasted on the young? It’s sad how such a wonderful opportunity to really research a topic in a rigorous and careful way so often turns into a load of crap, a burden to be unloaded. Let’s try not to lose sight of the love of research and scholarship which inspired us to want to take on the challenge of writing a thesis in the first place. Now I have the drive and the ideas, but not the time, which is very frustrating.

  5. I agree John but who reads that stuff? Where can we best be used? In classrooms, of course.

    We are important and our work is important. The ivory tower stuff is too easy. Going into a classroom with a bunch of young cyborg like humans who have never been taught how to function in a social setting, THAT is the hard work, the work that is real. The work that Krashen readily has shared with us that he cannot do.

    Plus, a few younger teachers come up with real shit, as it were. Chris has, and we are proud of him. But notice he never left the classroom in doing this and back he goes in the fall, to the real work, the hard work, the work that the intellectuals cannot do. Chris stays in the classroom, although he might be able to extend this M.A. into a PhD. – who knows?

    But alas for all of our inner Jeremys (Yellow Submarine), we’ll have to save all the research and in depth research for another lifetime. Although I love it dearly, this work seems more real. There is much more suffering, yes, but when more heart involved more happiness can be there.

    We really do have other work to do now. We have to reach kids, and there is no time to lose. For twenty years, Krahsen researched. Now we work. It’s that way.

  6. Chris this is pure gold. Thank you!!!
    My former colleague who was a traditional teacher, and was RIFed because our numbers were dropping, is now going to be THE trainer in our county for the new State Standards. We had a conversation last night: she said that “we” (our school) REALLY has to start writing unit plans – they cannot be “SWL Preterite” “SWL conditional” etc. etc. There has to be a performance task as a summative assessment. NO MORE worksheets/textbook drills. No more pen and paper tests as summative assess. BUT…the ACTFL people who were at her training were saying that CI, TPR and TPRS are not really good ways to teach the kids. They NEED to read authentic texts (huh? they do!) not just stories written by kids (huh? what are you talking about? and if that is not good, then why is having pair communicative activities OK?)
    She said that the units HAVE to be written via UbD and no more focus on grammar (yeah! ? what have I been doing and trying to explain to you for the past 2 years?)
    So, when I reiterated the value and importance of CI (ACTFL does say 90% CI, so why are you saying that they say it isn’t good?) and she said that “we have to have the kids outputting because otherwise they will NOT be able to when the Proficiency Diploma is mandated for Class of 2018” I then asked, “oh, so the state is mandating that the Class of 2018 has to be proficient in a WL?” and she said “yes”, so I asked, “OK, what LEVEL of ACTFL proficiency do they expect them to be at?” she responded that it is up to each district to decide that, based on the amount of time that they have with students. AND each district comes up with their own performance tasks (and she warned me that we should make them do-able for OUR students because we will have to send them in to the State and will be held accountable!) I then asked if she was SURE that the state was going to mandate WL instruction, because that was the scuttlebutt 10 years ago and they ended up having to nix that because they could not come up with the funds to train and hire and guarantee a supply of qualified WL teachers! She said that 1. she was told that “people will doubt this, but it IS coming down the pike!” 2. but, she knows the state doesn’t have money!!! 3. BUT, everyone must still believe it bc the state people assured that it IS happening!!!
    This conversation raised SO many questions for me — to the point where I am just so sick and tired of the runaround!!! But, bottom line (I know, it took me a while), I told her that forcing kids to output will not make them better speakers. She argued that it will, and that the ACTFL people at the training, who are “high up in ACTFL with multiple degrees” even said that it has to be done! So, I told her that they might be high up in ACTFL with multiple degrees, but they certainly have not done any research on SLA; because the research is out there and there is data to prove that students in CI-based instruction outperform Traditionally-taught students. She then said that she is unaware of any kind of research like that, so if I could please find it and send it on to her she would greatly appreciate it, (and that our other colleague that is staying with me is “definitely going to be demanding it!”)
    So, Chris, thank you SO much for this Empirical Evidence that I can provide to her! I am also going to supply her with my Masters Research project that had the same “problem” – it’s not as lengthy or detailed as yours, but it also claims that CI is more beneficial than Traditional. Thanks again!!!!

Leave a Comment

Your email address will not be published. Required fields are marked *

  • Search

Get The Latest Updates

Subscribe to Our Mailing List

No spam, notifications only about new products, updates.

Related Posts

CI and the Research (cont.)

Admins don’t actually read the research. They don’t have time. If or when they do read it, they do not really grasp it. How could

Research Question

I got a question: “Hi Ben, I am preparing some documents that support CI teaching to show my administrators. I looked through the blog and

We Have the Research

A teacher contacted me awhile back. She had been attacked about using CI from a team leader. I told her to get some research from

The Research

We don’t need any more research. In academia that would be a frivolous comment, but as a classroom teacher in languages I support it. Yes,



Subscribe to be a patron and get additional posts by Ben, along with live-streams, and monthly patron meetings!

Also each month, you will get a special coupon code to save 20% on any product once a month.

  • 20% coupon to anything in the store once a month
  • Access to monthly meetings with Ben
  • Access to exclusive Patreon posts by Ben
  • Access to livestreams by Ben