Data – 4

Here is what Mark says about it:

I like what Eric wrote about lifelong learners. We do need to be able to see which programs are producing life-long learners, and this should be the very first thing we try to measure, however we go about it.

I’m seeing some frightening drops in enrollments at the university level. Why is that happening? Arne Duncan himself is aware that 82% of all Americans are monolingual. I asked a colleague today whose wife is from West Africa, where educational resources are dwarfed by Northern industrialized societies, what the percentage of monolinguals was there. 20% was his estimate. Why do we think that pouring more resources into testing will solve our problem? Is that what they do in West Africa? Do we believe that more testing in our universities will up the enrollments there?

Eric says, “Unfortunately, “rigor” has been confused with ideas such as “they must do a lot of homework” and “it should be hard” and “not everyone should be able to successfully acquire a foreign language.” It may be an unfortunate, anti-intellectual, and tough truth to swallow, but the trappings of academia have done little to advance the causes of second language acquisition in the US.

Share:

Facebook
Twitter
Pinterest
LinkedIn

15 thoughts on “Data – 4”

  1. I have to chime in here. I agree with Ben on the ethereal level. However, I also accept that “data” drives the train right now. So, here’s what I’ve done.

    1 I worked to get our district to give a proficiency based listening exam at every level. It’s ACTFL normed. It’s designed to identify Intermediate Low. It’s 1 component of a battery that includes the other modalities also. Why listening? Comprehension precedes production.

    2. I’ve kept track of my scores and requested all district scores from our data person for the purpose of presenting at NTPRS, IFLT, Central States, and our local MN conference. Still, after 3 years, some people refuse to give it. Guess who? Those who know that their kids can’t understand language. They say it’s just guessing. I say, “Well, isn’t that really what interpretive communication is about? Doing what you can with what you know?”

    3. I’ve compiled the data, graphed results over 3 years, and compared my TCI class (8th year) with another TPRS class (3rd year teacher who has been using TPRS to varying degrees for all three years) and a third class (veteran teacher who, in her words, “teaches acquisition and grammar”)

    4. I’ve compared my level 1 results on this test to the results from level 3 traditionally taught French classes and a TPRS taught level 3 German class.

    In both cases 3 and 4 the results are not just significant, but astoundingly significant.

    GIVEN all that Ben has been saying in Data-1 thread, the results I can show indicate that SOMETHING IS DIFFERENT about the TPRS/TCI classroom. They indicate higher achievement for more kids. They indicate a bell curve doesn’t apply. They indicate that normal kids can achieve higher than typically high achieving kids.

    Like I said on the first thread, it’s on THEM to prove that they can get the same results.

    Not everyone will be able to pull this off. And the benefits are probably minimal anyway. Let’s face it, data is ultimately able to be manipulated. People know that and have a deeply rooted skepticism for any type of data, as they should have.
    Still , she who comes and observes will believe and he who is determined that it can’t be true will continue to find a reason to believe the story he hears in his head.

    1. Do you have a sample test that you could post? I agree with you that data drives right now and that having a listening test is better than the alternative. If I didn’t have to test I wouldn’t, but my district is going to test no matter what! If I can give them some sort of alternative summative listen test they are more likely to not push so hard for a production test.

      1. Sorry Paul, I didn’t even mention the test. It was developed at the Center for Advanced Research on Language Acquisition (CARLA) at the U of MN. It’s been sold in the interim to EMC. But we hold an older version and CARLA gives us permission to use it as long as it runs on our hardware.

        You can read about it here:
        http://www.carla.umn.edu/assessment/MLPA.html

        There are sample items there too along with these beautiful quotes:
        “The assessments have been used to certify that students have met designated levels of language proficiency, and facilitate the process of articulating expectations of student performance at the end of secondary studies and the beginning of post-secondary studies.”

        Passing the test is: “a reasonable benchmark for students completing their secondary studies and entering college”

        “the assessments were created to ensure that students were able to demonstrate their ability to use the language of study at the appropriate level instead of just putting in the proper amount of “seat-time.””

        1. I looked at the link. It’s designed to measure Intermediate-Low, and as it says, to measure the results at the end of a secondary school program. And you give it at every level?

          I’m always wary of these “ACTFL-aligned” tests, because that usually means “authentic” texts (read: incomprehensible) – and that means there is a lot of guessing and not real reading – and these tests often depend on thematic vocabulary. Preparing kids for “real-life communication” sounds good, but that often means the non-CI students are practicing and rehearsing dialogues of survival situations, the same “real-life” situations presented on the tests. Therefore, it’s “performance-based,” not proficiency-based – I don’t think what has been practiced indicates “acquisition” very well.

          This was written by CARLA of the reading assessment: “Authentic readings on diverse topics taken from a variety of sources are united within a theme to motivate readers to interact with L2 texts.”
          Authentic. Theme. Motivate. I don’t think “motivate” belongs in the same utterance. Just like a teacher saying she teaches grammar and acquisition.

          All that said, these ACTFL-aligned tests are definitely improvements.

          Another issue: are the ACTFL levels sensitive enough to measure differences between level 1 and 2? It takes tons of time to advance a sub level, so grouping kids into ACTFL proficiency levels we end up with most everyone falling in 1 or 2 sub levels – that doesn’t distinguish between those students at those levels.

          1. …it takes tons of time to advance a sub level….

            This is so important. It takes so much more time than we appreciate. It’s like measuring speech in a child between 24 and 36 months. Why can’t we wait until the child can say something, which is when they want to? What’s the rush to collect data going to do for our teaching? Has the collection of data ever caused a teacher to see the light and suddenly make the changes needed, with profuse thanks and bowing down given to the data gurus who unearthed their flaws and please may we have another battery of tests in the fall? Teaching is not like that. It is a slow and intensely personal inner dialogue over years that causes the very best of us to wonder why we are even in the field. Teaching is heartbreaking in so many ways.

          2. Keep the faith. Thirty-plus years in, and thanks to Facebook, former students check in from time to time and it blows me away. Here is a message I received from a student who graduated about 10 years ago…

            “There aren’t too many high school memories that don’t involve your Spanish class! You are an amazing teacher and when I look back on it you were one of my favorites…. You always included everyone- there was no “back seat” in your class. I hope you know how many students you have touched, as a teacher and as a person”

            You ALL have students who feel the same way about YOU, I promise!!!!

            Notice there is nothing in there about Spanish…….

            I hope that I have been a good Spanish teacher, but in the end, it’s YOU that makes the difference!!

            with love,
            Laurie

          3. Thanks Laurie! I just had a previously depressed student thank me for being her teacher. Watching her laugh her head off in class yesterday
            will carry me for a long time! Language learning was incidental.

          4. This is beautiful.

            You know I always thought that I knew my students but this last year after training and this PLC I realize I missed out on so much. I know my students better and can be exactly where they need me to be because of it. Before, and I didn’t realize it, I was concerned with curriculum too much. Now they are my curriculum.

            One thing that I did this summer was to study the faces and names on our grade program of my future students. Every few days I would go on and shuffle the faces to see if I knew the names. Learning names has always been a difficult thing for me but this year I feel that this has allowed me to already make students feel welcomed and safe in my classroom. When I was able to call a student by their name right off I could see their response in their eyes. Thank you to all of you. It makes all the difference and our students are the winners.

          5. Melissa said:

            …before, and I didn’t realize it, I was concerned with curriculum too much. Now they are my curriculum….

            This makes me think of the discussion about targeting vocabulary. I don’t think that the laughter than Charlotte got from that depressed kid (what is better than that in life but that, to help a kid smile in spite of what they are enduring in life?) was because she was focusing on teaching certain words. She said herself that in that class where they laughed so much

            …language learning was incidental….

            I just keep saying that if we let go of the reins a bit, not the reins of control over the behavior in the classroom which we must do, but letting go of the reins of wanting to teach the words we think they need to know but rather encouraging those words that want to come up by themselves, we will see more success. And definitely more laughter. That has been my own experience.

          6. That is AMAZING to read Laurie!!! Makes me feel good and it was a comment to you, not me! haha. You are so right, that there are many students in our classes who appreciate us in the same way! It may take some a couple years of maturity to realize it. We teach humans, not robots! That is why what we are doing is also lightyears ahead of our time!

    2. This jumped out at me, and I have also found this to be true. It is one of the main things that fuels my passion for this, and it is also one of the most difficult things for the “high achievers” to grapple with.

      “They indicate that normal kids can achieve higher than typically high achieving kids.”

      It’s such a huge lesson in humility and acceptance, the reality that everyone has innate capacity to acquire. I know this is off topic re: assessment, but I have found this to be a touchy dynamic with certain people: students, parents, teacherz and adminz. It can be threatening to them and it requires lots of educating. Their perception is that we are watering down or teaching “baby Spanish” (guess what, kids ARE babies in a language they were not reared speaking!).

      1. Hey everybody: Have you ever seen Jim Trelease’s free-for-download brochures?
        http://www.trelease-on-reading.com/brochures.html

        In that first brochure: See the chart of the 32-million words heard difference between low-income 4-year olds and high-income 4-year olds.

        And see the graph of the effects of poverty on literacy which can be overcome entirely by reading. This is related to the above discussion that everyone can succeed. The amount of CI can mediate success in a FL.

  2. You are thus pre-empting the others. It’s an excellent plan. You can tell them why listening should be tested, as a 21st century standards-aligned skill, and gently point out that the skills they value and test for truly are 20th century skills. You thus invite them into the 21st century. Can you challenge them to give the test themselves, explaining about input and output? They will dig their heals in on that, but the question will have an impact. Can you imagine the results if they accepted. We speak to our kids about 125-150 hours in the TL per year and theirs hear a fraction of that. That would be bad for them. As language teachers they should know and respect the role of input over output in the beginning of study, of course, but many clearly don’t. Maybe this is the way to go because it is simple. Diana is a tremendous leader and has spent at least $500,000 over the years on producing the tests we have designed, at the very least that amount, and we see now we are not going to have that kind of funding in the future. We will not get to that mythic level of an “ideal” instrument in DPS. Most districts don’t have it in the first place. This looks real do-able, Grant.

  3. Eric has interesting and valid points. The test is designed from the get-go to measure achievement in grammar-centric, themed classrooms.

    But there’s no getting around that at this point in time.

    Also, I’m going to consent that there is a lot of guessing. So, as Eric says, it doesn’t truly measure what kids KNOW.

    BUT BUT BUT it DOES measure what kids CAN DO when faced with language that is BEYOND THEIR LEVEL. In other words, when they’re interacting with non-sympathetic people or texts.

    The results over a few years have shown that CI kids are able to outperform the non-CI kids. Period. They can navigate the waters of language beyond their level with greater success. Period.

    So, Eric is getting at the question of “what’s an adequate or the perfect assessment for CI kids?” I’m getting at: “how can we use what we have to demonstrate that what’s happening in our classes makes a difference”

    MLPA has an Int. Low and Int. Mid version of this test. We use the Int. Low. We extrapolated backwards from the “Pass” score of 22. CARLA , in conjunction with tons of professionals, established 22 as the score to indicate whether a student has reached Intermediate Low. Their number. Not mine. Presumably, they should be able to reach 22 or higher after a high school program.

    We extrapolated backwards and decided to expect 12-15 for a Level 1 range, 16-20 for a Lvl 2 range and over 20 for a Level 3 range. Then we said, “let’s see how our kids do and revisit these ranges after a few years”.

    12-15 appears to be an accurate range for level 1 teachers who are beginning in TCI or who teach with legacy methods. in the legacy teaching class I referneced in my first comment, 30-40% don’t reach 12. 45% or so fall into the range. and about 10% achieve over 15 (between 15-20 typically) with 1 kid scoring at 21 or higher. One. The beginning TPRS teacher I referenced, with only 3 years under her belt, has better results than this.

    Alternately, in my classes, for 3 years running, 90% score 12 or higher. That’s a big difference in how they deal with Spanish beyond their level. Of the 90%, 30% scored 21 or higher – they PASSED at Intermediate Low according to a viable, ACTFL-aligned exam. Not an ideal exam, by any means. But it’s a result that has repeated itself 3 years running. It’s not a fluke.

    So, while this test or other “proficiency-based” tests may not be perfect, our kids will out-perform the traditional kids. And if we can show that, we can have an impact on the adminz because we’re speaking their language… kind of.

  4. Exactly, Grant! This test actually underestimates the impact of TCI!!!!! So cool.

    There could be even better tests of what our kids can actually do. BUT this test is still an improvement on more traditional forms and it still shows the superiority of TCI.

Leave a Comment

Your email address will not be published. Required fields are marked *

  • Search

Get The Latest Updates

Subscribe to Our Mailing List

No spam, notifications only about new products, updates.

Related Posts

CI and the Research (cont.)

Admins don’t actually read the research. They don’t have time. If or when they do read it, they do not really grasp it. How could

Research Question

I got a question: “Hi Ben, I am preparing some documents that support CI teaching to show my administrators. I looked through the blog and

We Have the Research

A teacher contacted me awhile back. She had been attacked about using CI from a team leader. I told her to get some research from

The Research

We don’t need any more research. In academia that would be a frivolous comment, but as a classroom teacher in languages I support it. Yes,

$10

~PER MONTH

Subscribe to be a patron and get additional posts by Ben, along with live-streams, and monthly patron meetings!

Also each month, you will get a special coupon code to save 20% on any product once a month.

  • 20% coupon to anything in the store once a month
  • Access to monthly meetings with Ben
  • Access to exclusive Patreon posts by Ben
  • Access to livestreams by Ben