My New Experiment Comparing LingQ Levels vs. CEFR Tests

In follow up to my previous posts about clearing up definitions about CEFR ratings like B-2, C-2 etc., I’ve decided to run a little experiment to compare how LingQ’s levels from Beginner 1 to Advanced 2 might compare to the vocabulary requirements on CEFR tests from A-1 to C-2.

I also wanted to find out what results casual self study through reading on LingQ might get you in the academic testing environment.

The experiment:
1.) Spanish: Having recently completed Intermediate 2 level in Spanish on LingQ, I’ve decided to take a sample test of B-2 reading comprehension from Instituto Cervantes. I will be simulating a test environment of reading on printed paper and finishing in the allotted time of 70 min for four exercises.

2.) German: I’ve been at Advanced 2 in German for a while so I’ll take a sample C-2 reading comprehension test from the Goethe Institute also following allotted time requirements.

(I also really wanted to do a French C-2 reading test, but they administer those differently, more on that later.)

Caveats:
– B-2 and C-2 mean reading, listening, writing, and speaking at a certain level. I’m only testing the reading comprehension part of the available sample tests, because that is really the one thing LingQ can best measure with their levels based on known words and passive vocabulary.
– I’m a casual learner and I did not prepare for the tests in any specific way, I just did them on a whim to see how I would currently do.
– CEFR tests are “Pass / Fail” with 60% of answers needing to be correct across all four parts of the test (read, write, speak, listen). Since I’m only doing the reading section, I’ll give myself a “Pass” if I can get 60% of the questions correct just for this one section. I’m figuring that will work for the purpose of this test.

Results:
1.) Spanish: I passed the B-2 reading test – but it was a squeaker – I managed to get the minimum of 22 out of 36 questions correct for a 60% score. Overall, I felt I did quite well with the comprehension of the articles presented. Some of the multiple choice questions did get tricky, but most of my wrong answers actually came from the fill in the blank grammar questions that are part of the reading test. I don’t study grammar as I hope to just absorb it over time, but if I was to take an actual B-2 test, I’d have to brush up on it for sure. But then again, I absorbed just enough of it so far to squeak by on this one, so that’s not bad.

2.) German: I passed, scoring 76 out of a 100 points on the C-2 reading test. This test followed the same model as the Spanish one, but they score differently, using a point system giving a little more weight to the article reading part which benefited me here.

The C-2 tests for French follow a different model where reading and writing are combined into one test, therefore there is not really a way to do a practice test without an actual evaluator involved. You’re supposed to read the articles provided and respond to them in writing. I did read the sample articles of a C-2 test and I had no difficulties understanding them. But my lack of French writing practice probably would’ve had me fail the written part of this test pretty fast.

My conclusions:
Obviously my experiment was imperfect and limited, but from what I can tell, it seems that LingQ’s levels based on known word counts seem to have a decent correspondence with at least the minimum vocabulary requirements for their equivalent CEFR levels. So, if you complete Intermediate 2 on LingQ, your reading is probably around the B-2 level, and so on. My Spanish result is more interesting here, because I’m only about 400 words over the Intermediate 2 threshold and I just managed to pass the B-2 test by one question. That could be a coincidence of course, but still.

With German, I’m well past the Advanced 2 line, and correspondingly, my margin of passing the C-2 test was wider.

It seems also, that learning through self study and casual reading alone can be effective enough to pass certain aspects of a CEFR test, but of course anyone who’s seriously considering taking a full test would be advised to study towards the specific requirements, especially when writing, speaking, and grammar are concerned.

Sample tests for many languages are available online for anyone to test themselves – with the reading comprehension part being the easiest to self test – so if you’re wondering where you might stand with your language levels, taking one of these is a good way to ascertain that.

26 Likes

Great post man. Awesome work.

2 Likes

Thank you for doing this, I was really curious about the transition from informal to formal knowledge in languages

1 Like

Very cool. How would you say your listening and reading skills compare to each other?

2 Likes

I’d say my listening is around the same level for the most part. In Spanish, I’m probably a little behind in my recommended listening hours. But with German and French I listen to a lot of audiobooks and can watch tv shows without subs. I thought about taking the listening parts of these tests as well, but I feel like it would be harder to qualify the results as far as how they relate to LingQ progress, since I don’t keep track of my listening on here. Whereas reading and known word counts are more of a quantifiable aspect and our progress between levels here are measured by this factor, so it’s easier to draw a line between reading comprehension results and. LingQ levels.

2 Likes

Any plans to try a sample TOPIK test and see how your Korean level measures up? :slight_smile:

I would, but my Korean studies have been very sporadic of late, though I try to maintain it by listening to podcasts and watching some shows here and there, I haven’t been very active with it for the past year. I plan on getting back to Korean at some point and maybe I can play with some testing at that time.

Excellent work. Thank you good sir!

1 Like