[Analysis] Is LingQ Intermediate 1 equivalent to CEFR B1 comprehension? A case study in Italian
Is LingQ's Intermediate 1 equivalent to the comprehension level of CEFR B1? Questions similar to this are common on the forums. I will seek to see if my experience in Italian can provide a case study to help answer this question.
Recently I reached Intermediate 1 in Italian, which is classified as 6,700 known words. This took me about 8.5 months to reach. I have predominately used LingQ as my means of study. I know we've had some serious statistics tracking issues, but I've best adjusted my data to be as accurate as possible.
Known words 6,700
LingQs learnt 4,958
Hours of listening 227.54
Words of Reading 737,109
On top of this, I've done the following activities outside of LingQ:
- Approximately 18k words of reading offline, bringing the total up to 755k words read.
- Some pre-made Anki decks when I got bored sometimes. At 789 matured cards, this was 10h 24m.
- Some speaking practice where I was repeating out loud some beginner Anki sentences. With the listening and speaking, this accounted for 7h 49m.
The listening is a little bit inflated, because I used to listen to a lot of repetitive audios on increased speed. This is because the beginner content had very slow audio, much slower than the normal speaking speed (eg. the LingQ Mini Stories are about 75 wpm, so half normal speed. At 15x each, that's 35 hours over-recorded). So to make up for this, I guess it's probably around 150 hours of listening.
Adding this all up, I imagine I totaled between 300-400 hours to reach Intermediate 1, of which 90% was on LingQ. This would turn out to be somewhere between an hour and an hour and a half per day on average. Some days were practically the whole day, while others were only the token number of minutes to keep my streak. I even took a two week break somewhere in there too.
EDIT: Sorry. I forgot to mention, but, as a disclaimer, I studied a little bit of Italian in primary school (a long time ago). I didn't remember much, hence why I forgot to mention it, but this journey was starting from a vocabulary of remembering how to count, the days week, the months of the year, and a few words that in English you just know are Italian like "ciao," "bella," "pizza," and "pasta." All up this would've been <100 LingQ words.
To analyse if LingQ’s level of Intermediate 1 is equivalent to the CEFR level of B1 comprehension (that is, reading and listening) I will look at it from four perspectives:
- Self-assessment based on CEFR descriptions
- Hours studied
- Size of vocabulary
- B1 exam
1. SELF-ASSESSMENT BASED ON CEFR DESCRIPTIONS
B1 LISTENING: I can understand the main points of clear standard speech on familiar matters regularly encountered in work, school, leisure, etc. I can understand the main point of many radio or TV programmes on current affairs or topics of personal or professional interest when the delivery is relatively slow and clear.
Yes, I can understand clear standard speech on topics I am familiar with. Some complicated topics are too challenging, but some topics are very understandable.
B1 READING: I can understand texts that consist mainly of high frequency everyday or job-related language. I can understand the description of events, feelings and wishes in personal letters.
I have not read any personal letters, but I am confident that I know a lot of high frequency vocabulary and I can understand such texts. I can read transcripts of podcasts aimed at A2/B1 students, know most words, and can understand generally what's going on. I can also read the news on certain topics.
2. HOURS STUDIED
'Time with the language' is arguably the most reliable predictor of competence in the language, at least in the beginner and intermediate stages of the language learning journey. We will compare how long it took me to get to Intermediate 1 with how long other students take to get to B1.
One way to do this is by looking at how long it takes to go through a language school. The following Italian language school has an intensive 'standard' language course with 15 hours per week of class. Their course from beginner up to B1 is 28 weeks. Assuming 30 minutes of homework per weekday, this intensive language school requires 490 hours from beginner to complete B1.
If we assume some inefficiency by being in a class with up to 13 other students and time dedicated to practising speaking and writing, the 300-400 hours it took me to get to Intermediate 1 is a reasonable amount of time to expect a level of B1.
3. SIZE OF VOCABULARY
It was hard to find a decent, free frequency list of lemmas (dictionary head words) in Italian, which didn't require exorbant editing to be able to use, so I settled on an Anki deck, sourced from a Quizlet deck, sourced from I don't know where. But going through the list, aside from the rare word, it seemed reasonably accurate based on my experience with Italian.
I went through this list, marking if I could correctly state with reasonable confidence at least one definition of the word. If I got the word wrong, which happened occassionally, and I checked the definition and then remembered that I actually knew the word, I left it as wrong. I treated this more like an exam in a particular moment in time.
There were words, which I guessed wrongly, because I have not seen them outside of a specific context and other verbs, which I had never encountered the infinitive before and so sometimes got wrong. On the other hand there were several words, which I have never met before in my studies, of which some I could correctly guess the meaning due to the knowledge of the cognate, eg. vendetta.
Of the 4,998 high-frequency lemmas in this list, I knew and got correct 3,227 words.
Note: There are some very high-frequency words (words which are in the top 1,000), which I still don't know! There are some prepositions, conjunctions, and interjections, which I have encountered hundreds if not thousands of times and I still have to look up the word almost every time. Eg.'dunque' which means 'therefore', 'so', 'then'.
However, considering I successfully recognised 3,227 of the top 4,998 high-frequency head words, this means I have a decently sized vocabulary. There are many mid- and low-frequency words not in this list, which I also know and have been marked as Known on LingQ.
Even though there is no definite, official vocabulary size recommendation for Italian, we can guess based on various sources, such as the following research paper, that, yes, this vocabulary size is wholly adequate for someone at a B1 level.
4. B1 EXAM
I found a sample B1 exam online. I completed it as per the instructions and self-marked it.
READING: Giving ½ points to two answers, which were half correct, I scored 18/20 (10/10 for multiple choice and 8/10 for short answer). So, offically a strong pass, but honestly, it felt like guessing. I only half understood it. I knew generally what the two texts were about, but there were definitely a few important words, which I didn't understand.
LISTENING: In total I scored 14/20 (7/10 multiple choice and 7/10 short answer). It felt a little challenging too, but the second audio was a bit easier than the first one, especially on the second listen.
TL;DR After spending 8.5 months on LingQ learning Italian, averaging a bit over an hour per day, I reached Intermediate 1. On all four measures – CEFR descriptions, hours studied, size of vocabulary, and the B1 exam – I would confidently say my progress is, yes, around the comprehension level of CEFR B1. However, even though B1 sounds like you are actually partially competent in the language (“understand clear standard speech”), it's such an inadequate level. Having communicated with several Italians just after I reached this level, I can say that there were consistently large amounts of misunderstanding.