Data -1

I got an email from someone who received this question from a colleague and sent it on to me:

Are there any stats re just how much the TPRS method is being used in schools in the U.S.? What are universities thinking about it? Are any adopting it? Does Krashen or anyone have a handle on this nationwide? Do you know if any nearby schools that are using this? I know I asked you this several years ago, and there was not much evidence to support it as “the real thing”. I am just wondering if there are new developments in tracking this. I would really appreciate it if you could find out this info for me. When we meet, I can fill you in as to why I would like it.

This pushed a few of my buttons so I responded with a bit of a rant. I apologize for the bitchy edge in it. No, I don’t. I am tired of people thinking that a set of comparison data gathered in the old ways of the last century has much value in this new world of language teaching that is finally here. When are people going to get that data is not the only way to determine if something is effective? Here is my rant:

This is rather an awkward question. Why? It is awkward because we don’t have tests developed and in place to test what our CI kids know. Even the DPS tests are flawed, in my opinion, but I don’t want to get into that here. Don’t tell Diana I said that.

I would tell your friend that, until tests written for and by traditional instructors are eliminated from the landscape, we cannot really show what our CI kids know. We are talking about giving a CI kid a test designed by a set of traditional teachers. What will that prove? What can it prove? Are those tests aligned with standards? Uh, no they are not. How could they reflect anything about what our CI kids know, because the instruction we offer them is in fact based strongly and rooted firmly in the standards.

We are in a whole new world, and until the testing area catches up with our new instructional practices, the question is moot – it doesn’t work as a question. My own question is why can’t people go in and see our classes, see what our kids can do with the language in the classroom, see the interaction there, the real stuff going on, put their hands up by the sides of their heads to stop the spinning, and leave and no longer ask such hubris-ridden questions.

Of course, this would mean that they have to wrap their minds around an uncomfortable new truth, that if a kid does well on a traditional test, it means nothing if the child cannot understand the language. It also would require that they accept that input must precede output in the form of both writing and speaking for years before anything real can be measured. Do not miss the sublety of that last point. YEARS of input must precede output, so how can one test what our first and second year kids even know? What is the hurry? I will say it a third time – years of input must precede any measurement of what our kids know.

I will say it a fourth time. The process is unconscious for years, just as it is in small children. We have so little, just a fraction, of the time needed to demonstrate any gains, and yet the data folks want to measure what our first and second, and third and fourth for that matter, can do with the language. Do we do that with four year old kids, who have been exposed to far more language in those four years than our fourth year kids have? As Robert Harrell has said, learning a language is a really hard thing to do that requires lots of time and effort, and to measure what a person, especially a non-motivated person like so many of our students are, can do with a language after only 400 or so hours of instruction time, is really an absurd thing to do.

It is this long silent period when the input is happening in the form of listening and reading that may in fact be a factor in our not having any big time test scores that destroy those of privileged white kids who make it to the upper levels in traditional classes. The role of time and the role of the unconscious mind and how so much time is required for anything real to be measured must be studied and I nominate the same people who ask these questions about what data we have gathered on our behalf to start that research, as if we need to defend what we do.

However, with certain kids, those same kids who form the entire foundation of testing “success” for traditional teachers in traditional classrooms, we do have strong proof. So what if it’s all anecdotal and field based? I am so weary of the claim that anecdotal and field based research is somehow not valid. And Blaine did try to collect data whenever he heard it starting in 2001, and there were some impressive scores especially on the AP exam that we would share with each other each summer at the conferences (Laurie can speak to this) but he never completed the project and everybody else was busy doing things that actually had some importance like learning how to get better at teaching our kids, weird stuff like that.

So our success is all hearsay. People hear that we are doing fantastic things (because we certainly are and in spite of great opposition on a daily basis). But they don’t believe it because they only heard it – it is not put in the form of data somewhere. That’s just so lame. I personally had frequent national scores from TPRS kids until I decided to no longer play that pathetic game of standardized testing. I had to sever my relationship with those tests, all of them, when I realized that I was only doing it to make myself look good.

Want some scores? One of my CI kids got a perfect score on the National French exam, a great many of them had scores in the 60 out of 70 range. Scores below 50 of 70 were rare, except in the kids with hard hearts and difficult home lives. One time my 8th graders got 14 of the 15 scores in CO on the level 1 exam. The kid with the perfect score got a 4 on the AP French exam after only two years of French with no prior background.

But it’s all just hearsay. I would have to prove it. I would have to go back into some dingy file somewhere in some school and try to find those results, right? My perception that those kids rocked the house every day and brought me so much happiness that I decided to start a blog about how cool this way of teaching was is not enough. All of the incredible success stories that we have shared with each other here over the years, all those great stories, are not enough. That is where we have arrived in our society. What a teacher knows about her students and experiences with them is not enough. It must be proven by numbers, which are always flawed.

We seem to have reversed things. What we say to each other doesn’t count. Numbers count. Explain that to me.

I just don’t think that your friend is going to get the information she wants. Why? Because what we are doing is so radical that no one can keep up with it. No one can really test what happens in the unconscious mind where language acquisition really occurs, can they? I would advise your friend to just go into a classroom where CI is being done well and draw her own conclusions. Is a rocking classroom with plenty of laughter and obvious communication through happy eyes vs. worksheets being done in tomb-like silence enough proof? Is it enough proof?

Sorry to get pissy about this, but the fact is that we who are doing this work are changing enough right now. We are working hard enough to bring what Krashen and others have shown us to be true about how people acquire languages to fruition. We should not have, added to the burden of the massive amount of work we are currently doing, another chore, that of proving that what we do is far superior to anything ever done. It is far superior to anything ever done. People don’t know that now, but they will one day.

Those stuck in wondering whether what we do has any value clearly have some reading of their own to do. Let them do their own research if it’s that important to them. But beware of those at the university level doing any research. They don’t even get what is happening. No, they don’t. They don’t.

You can send her some of the Primer articles especially the one by Robert. Really, isn’t the question of whether what we are doing has any value in the world of collected data up to this point really kind of a worn out question? Can somebody ask us another question please? Like is there a map they can use to get to our classrooms? So they can see for real, without the need for any fricking data? Jeez Louise.

Share:

Facebook
Twitter
Pinterest
LinkedIn

10 thoughts on “Data -1”

  1. As far as how many teachers are using TCI. I don’t know. Way too few of us are active here and on moreTPRS. And there are degrees to which TCI is applied.

    We are the radicals – lightyears ahead. Krashen is still a radical 30+ years later.

    I’ve included links to the evidence in applied research in a Primer linked on this site.
    https://docs.google.com/presentation/d/1GUvGA29n89B36ju9iEuK5oO26Wnp2YY19x43-vHxX7U/edit?usp=sharing

    In that PPT, I include this link to comparison studies:
    http://skrashen.blogspot.com/2014/08/comprensible-input-based-methods-vs.html

    and this link to TPRS studies:
    http://ntprs.org/2013/tprs-research.pdf

    See slide 14 for what I call the Insanity Argument. Rather than just look at whether TPRS works, we should also be looking at whether the mainstream approach has worked. It hasn’t. Low student enrollment, low student proficiency, and high teacher turnover.

    I agree that it’s “case closed” when you see a good TCI class in action. Seeing is believing. Like Ben said, we’ve traditionally not tested fluency nor proficiency. The bulk of FL teachers and SLA researchers have measured language gains in terms of “learning.” (explicit learning).

    There’s the iceberg model – a lot is not testable and it may only be possible to test the tip.
    https://drive.google.com/file/d/0B6n9VA2R4h9IRTNGX2djNXNDRWc/edit?usp=sharing

    100 instruction hours in a class of 15+ students is nothing – you’re trying to look at the microscopic difference between minuscule SLA gains between TCI and traditional approaches. Despite this realization, I acknowledge that if we had data in support of TCI, then it could bring us back to the future.

    There aren’t any good comprehension-based, proficiency tests that I know of. That’s why I tried to write my own with guidance of Dr. Beniko Mason. I discussed the details in a forum post:
    https://benslavic.com/blog/forum/general-discussion-1/ci-friendly-test/
    Here’s the English version:
    https://drive.google.com/folderview?id=0B6n9VA2R4h9IRHlRSnhoT2h0VGc&usp=sharing
    And the Spanish version:
    https://drive.google.com/folderview?id=0B6n9VA2R4h9IaTVEMHNhZm5iYTA&usp=sharing

    I wanted a way to show what my TCI kids have accomplished, what a traditional test doesn’t measure. I recommend you find a reading at the level you desire your students to reach. For each of the 4 skills, they can read* and/or listen to a story in L2 and then summarize in L1 , read* and/or listen in L2 and then rewrite or retell in L2.
    *take away the reading while they are summarizing

    I think you can use lextutor.ca to analyze the word frequency levels of the reading – the higher the frequency, the more valid – the more you show that comprehension of that language will be useful to real world communication.

    If forced to give a common assessment, ALSO give them a comprehension-based test in order to better show all that the kids can do. Anyone with common sense should be able to see from the way you tested the kids that it is closer to measuring real language ability.

    Seeing is not believing for everyone. Maybe, a teacher so used to the traditional paradigm may not be able to grasp the magic of a good CI class. Maybe because they’re looking for the wrong things. They have on their “grammar goggles.” I’ve had observers. I’ve had an admin get it immediately and ask me right after to go spread it to the high school (unfortunately, she moved the following year). My fellow non-CI teaching colleagues were the ones that were the least enthusiastic when I showed them video of my classes and demoed for them.

    Parents have been the easiest sell of all. They’ve probably all suffered a grammar-oriented classroom, so they see the value in what I’m doing. Then, if you demo for them, then another “case closed.” I had Open House last night and I received wonderful comments from parents “My kid said I had to meet you” and “I wish I had had teachers like you.”

    I’ve probably shared this before, but worth sharing again. This puts some numbazz on the sea that is SLA. . .

    50 function words = 60% unsimplified speech
    1,000 high frequency word families = 85% unsimplified speech
    3,000 word families = 95% unsimplified speech
    7,000 word families = 98% unsimplified speech

    Research shows that to ensure comprehension you need 98% knowledge of the words (defined as a 70%+ on comprehension tests). If you want moderate comprehension (55% on tests), then you need 95% comprehension.

    The average native speaker acquires 1,000 word families per year from ages 3-25.

    Now, it should be obvious that with our limited time, we can’t expect 1,000 word families to be acquired in 1 year. We probably can’t even expect a quarter of that amount! (Remember: a word family includes inflected forms and many derivations). This trivializes the actual differences between a level 1 student with 100 hours of FL instruction and level 2 with 200 hours. The difference is minute.

    So, if we can’t get a kid to acquire 7,000 word families, then they won’t have great comprehension of the real world. Thus, only the fastest processors would ever have a chance at achieving that lightspeed of acquisition in 4 years of a FL program.

    That brings up another point: the ultimate goal of education in general is to produce a life-long student. In the case of SLA, if you don’t produce a life-long student, then that person will never continue to acquire and can never handle the outside world. In order to develop a life-long student, the study of the language must be pleasurable. Unfortunately, “rigor” has been confused with ideas such as “they must do a lot of homework” and “it should be hard” and “not everyone should be able to successfully acquire a foreign language.”

  2. Hi, All,

    Hope everyone’s year is great.

    Ben — BTW — I just got a French I and II spot in a much better situation, same district, different high school. Praise the Lord!

    Anyway, I think it is important to note in all of these discussions about the efficacy of TPRS, along with searching for data and peer-reviewed journal articles in respected professional publications —- there is some stuff out there.

    It may fall under ’emergent curricula,’ ‘implicit learning,’ or ‘developing tacit knowledge,’ but there are quite a few articles out there that point to the intrinsic value of CI. :-)))) That’s the way to go, IMO.

    Be well, everybody…

  3. Congrats on that job Leigh Anne – you certainly deserve it. I appreciate that there are some valid studies out there in our favor, no doubt. But I, unlike just about everybody in this group if I am accurate in saying that based on past similar discussions on this topic here, stick to my point that gathering data about an unconscious process that requires vast amounts of time to show any truly and authentically measurable results is a waste of time and will never be accurate; it will always be flawed simply because too many factors fail to get built into whatever research design is suggested.

    In my opinion, and that’s what’s good about this group – we all get to express our opinions – those people who clamor for more data are part of a mindset that in my view is going to disappear in future years, when we get healthier mentally, and lose some of the fear that saturates our professional lives these days. We know that what we do rocks the house; then why do we look for the approval of people who don’t understand what we do? Why would we seek data to prove that something works when we already know it works? How can people who design data collection mechanisms design good ones when they are unclear about how comprehensible input really works?

    People have collected enough data, the university people have led the way, and to what end? What has been accomplished by this obsession? Not much, and we know it’s not much because we are having this discussion, even though our “not much” is far more than what the textbook side has.

    I say again that nothing can really be measured accurately in TPRS/CI instruction because language acquisition occurs in the domain of the unconscious mind where data collectors cannot go. On top of that is the irrefutable fact that input precedes output, and that years and years and more years are needed for the acquisition mechanisms to show any results of any consequence.

    It is as if the data people are trying to measure the growth of a seed under the ground without being able to see it, and then pronounce the soil that the seed is in as good or bad without waiting for the flower to bloom. These people need to be sat down and kindly told that their work in gathering data will certainly run into a road block when it comes to measuring the relative merits of instruction based on comprehensible input, which can only be measured in actual output years, not months, after the seed has been planted.

    what’s wrong with just observing some good TPRS/CI classes and drawing our own conclusions. What’s wrong with just asking students how they feel when they learn in our classes? Doesn’t that count for anything? It’s not a mystery when you see it. Let’s just see if a student can understand or read the language when they have had many years, far more than we offer in secondary schools, to do so, at a different pace for each student. What is the obsession with gathering writing speaking and writing data on babies? Let’s also only test motivated students. The others skew data unmistakably, and yet no one talks about that.

    To me, in my lonely world on this topic, I think that it makes a lot more sense to just wait the necessary years, after many thousands of hours of meaningful context have been presented to the student, and to measure only motivated students, and to avoid measuring things that the data gatherer cannot yet see because so much time is needed, far more time than we have in our schools.

  4. I believe, and continue to say to people who inquire about research, that it’s not our job to defend what we do. Rather, it’s incumbent upon those who use “legacy” methods, as Terry Walz has called them (and I love that term!) to prove that what they’re doing produces people who can speak the language. People who represent the beautiful demographic kaleidoscope that is our country, not just primarily those who are high achieving, white, college-bound females.

    Ben’s right. You must see it to believe it. To really let it sink into your head. Seeing the student-athletes, the special ed kids, the C students who sit with their hoodies up in other classes but are not just interacting, but fully engaged with their mind and body. You can’t #evaluatethat.

  5. How about the fact that some of our students (ok, well mine anyhow) can actually understand and somewhat function in the language to a decent degree even in the first few weeks of level 2 better than our non CI colleagues?

    Why are people so obsessed with numbers and quantitative bullshit, when in reality the quality of instruction has gotten so much better? Who gives a flying fart if my student can’t conjugate the verb etre 6 ways from Sunday if he can’t actually use it in context? You know what? My students in French 2 can’t tell time yet. You know why? Because I didn’t get to it last year and I don’t think it’s important to a French 2 student to be able to tell the damn time in French yet. What can they do? A hell of a lot more than most of the other level 2 students in the 5 other languages we offer on campus because I’m the only CI teacher. There is no question in my mind that what we are doing is far superior to the old ways because students can actually recall stuff without thinking about it as opposed to having to search the deepest recesses of their mind for a useless conjugation of a verb they’ll never use anyhow.

    Ugh…ok, rant over for me. I just get so tired of hearing stuff like, “Oh these students can’t do double object pronouns in the imperative, they just don’t get it! ” Yeah, no shit they don’t get it. They’ve had less than 500 hours of exposure in the language and you’re expecting stuff that is ridiculous!

  6. I see two problems here:
    1. Our society is addicted to data.
    2. Data is not information; information is not knowledge; knowledge is not understanding; understanding is not wisdom. -Clifford Stoll

    Since a lot of people fail to realize #2, they substitute data for knowledge and understanding and think they are wise. Unfortunately, when these people hold positions of power, it has an impact on all of us.

  7. I’ve always thought that the most amazing proof of the power of CI/TPRS is the number of experienced teachers who tried it and never turned back. Those that I’ve met are almost never new teachers. They know all about teaching with grammar and workbooks and text books and they know that it doesn’t work. Those who have tried it both ways have no doubts. I’m not talking about people who went to one workshop because someone bent their arm. I’m talking about teachers who gave it an honest try. I know that my first efforts were fumbling, but I immediately saw my students ‘results improve. I could never again justify assigning grammar exercises for homework.

Leave a Comment

Your email address will not be published. Required fields are marked *

  • Search

Get The Latest Updates

Subscribe to Our Mailing List

No spam, notifications only about new products, updates.

Related Posts

CI and the Research (cont.)

Admins don’t actually read the research. They don’t have time. If or when they do read it, they do not really grasp it. How could

Research Question

I got a question: “Hi Ben, I am preparing some documents that support CI teaching to show my administrators. I looked through the blog and

We Have the Research

A teacher contacted me awhile back. She had been attacked about using CI from a team leader. I told her to get some research from

The Research

We don’t need any more research. In academia that would be a frivolous comment, but as a classroom teacher in languages I support it. Yes,

$10

~PER MONTH

Subscribe to be a patron and get additional posts by Ben, along with live-streams, and monthly patron meetings!

Also each month, you will get a special coupon code to save 20% on any product once a month.

  • 20% coupon to anything in the store once a month
  • Access to monthly meetings with Ben
  • Access to exclusive Patreon posts by Ben
  • Access to livestreams by Ben