Is "Anki" a necessity?
I recently started learning Korean.
I am trying to use flashcards of the most common words, and also do short children stories here on LingQ. And I can say that flashcards alone are not working for me at all.
I got a high quality deck from Refold, btw.
It was fine for the first 150 cards because I already knew many words from watching tv series. And then new words started coming up. The huge problem appeared: I could not, for the life of me, make them stick, forgeting not just the next day, the next 5 minutes. The reason is, there is no context, nothing I can connect the word to.
They also promise to have sentences in that deck but so far it's 98% just single words "to build the foundation vocab".
Then I go to LingQ and read and listen a short story. It has 200 words, and I do not know 90% of them. It is not painful, it is fun. I learn how words connect, how they sound together, I put them in context, I see them again in different context, and hey, here they are, in my memory.
I feel that using the same time reading and listening is a lot more effective.
Unless maybe your flashcards provide proper context for each word. But to make it yourself, it is very labourious and no guarantee that it is correct.
Hello just a quick note, with LinkQ you can export to Anki. So you would have both options to learn in LinkQ and also in Anki. No vocabulary needs to be created and you can start right away.
"The huge problem appeared: I could not, for the life of me, make them stick, forgeting not just the next day, the next 5 minutes. The reason is, there is no context, nothing I can connect the word to."
I have gone through exactly the same experience with Russian.
However, it is inaccurate to say "the reason is...".
What is more accurate is to say "one of the reasons is..."
(long) explanation follows....
TLDR: Long story short: anki does work for distant languages. But distant languages are harder due to their nature of being distant. This is not anki's fault.
We'll get your reason out of the way first (this is only ONE of the reasons, not THE reason), then we'll dig into some of the others..
If you've done anki or another SRS before with a language that is close to your own then you will have noticed that the words that are similar to ones from your native language are DEAD EASY to remember. They click very quickly. They are typically similar looking, similar sounding and with a similar or identical meaning.
There are, however, some words that are hard to remember. Typically they are further away or completely different sounding or have a different meaning.
These are non-cognates and are up to 10X as difficult to get them to "stick" as for cognates or close-to-cognate. That is the reason for that.
As you have correctly pointed out, Korean is a distant language from your native language. This means with no shadow of a doubt that most of the vocabulary is non-cognate, which makes them by definition more difficult to stick.
Compounding that could be different pronunciation of vowels and consonants, clusters of unfamiliar consonants combined with perhaps tones.
If you have all of those things together your brain will not even comprehend what it's hearing never mind be able to match it to something. Your brain needs to be able to recall something to make the link. If there is no representation you will need to build one by repeated exposure over and over and over and over over and over and over and over.
What this means, as you have correctly pointed out is that in the beginning "it just doesn't stick".
However. There is no way round this. Regardless of which method you use, you will have to learn from scratch the tongue placement, fricative sounds, palatal sounds, consonant clusters, tones and generally try to remember the gibberish.
It is not anki that is at fault, it is that the language is distant.
For reference: I had the exact same experience with Russian and even worse when I tried Mandarin for a month.
For French or Spanish I was able to get about an 85% retention rate on average for new words over the entire six month period I did anki.
For Russian, the first six months I got about a 30% retention rate for new words.
When I tried Mandarin for one month I got about an 8% retention rate for new words.
With some research into phonetics I eventually figured out that I was trying to layer on my understanding of English consonants and vowels onto Russian and English consonant clusters. It didn't work.
After some months, however, I started to get used to the consonant clusters and also figured out (by discovering "IPA alphabet") that several of the consonants had entirely different pronunciations that I was listening for. Initially I could not hear the difference between for example ssss-Zh and just Zh. My brain heared just Zh. I also could not hear the difference between Ш and Щ which both initially sounded like "sh" to me. Now, nearly a year and a half later (and close to 10,000 audio words memorized with the help of anki) I can hear the differences between the wierd consonants and the wierd consonant clusters.
Anyhow. Anki still works.
i love anki and i think because of it i've gotten hyper fluent in portuguese. For example i just listened to a podcast that used the phrase "fazer alarde" without anki i would've never remembered that phrase. And now that i've seen it in real context it will solidify more in my brain.
That being said i have a retirement setting on anki so i don't spend forever on them, when it reaches a 2 year interval i retire them.
If i wanted to become fluent in a language as quick as possible id use it.
Also i really recommend adding audio, and making reverse anki cards to practice speaking.
I couldn't reply to your original post.
I wouldn't say that I disqualify SRS as a practical learning tool to some extent. I may use it as a beginner to overcome hurdles, such as an unfamiliar writing system. Nonetheless, I have other activities and focus on the language besides reading to help me acquire vocabulary at a mass scale.
SRS only focuses on the forgetting curve. I emphasize fundamental language skills more as they significantly improve one's efficiency in acquiring vocabulary. There's an idiom in Chinese that applies to all learning: 磨刀不误砍柴工. （Sharpening the axe won't delay the cutting of firewood.) I acquired a significant amount of words on the first encounter, provided I have a good mastery of these essential skills. A good question regarding the efficiency of a particular method for learning a language is whether it allows you to absorb the content or vocabulary at an exponential rate over time.
I recommend memorizing words always in the context unless it's mainly to familiarize the writing system and the pronunciation in the first two or three weeks of learning a language. I do some sentence mining when I study grammar, and I even read dictionaries to review word usage in different contexts from time to time. These references can be TL-NL, NL-TL, and monolingual dictionaries. As for special colocations and set phrases, I would read the books dedicated to the particular topic as they provide more detail with sample sentences and the origin of the words with historical anecdotes. These can be grammar books, books on slang, idioms, set phrases, and any books on a particular subject.
The learning methods complement other activities that a learner engages in the language as long as one's comfortable with a particular approach with significant progress over time. It will be such a hassle and unnecessary to argue over which preferred method to use and the nonexistent mutual exclusiveness in our approach in our language learning journey.
Right. Different strokes for different folks.
We do what works for each of us.
I know my method works because I've successfully used it three times now.
But it's painful and boring for some folks so they do something else.
Consistency is key to be honest. You need to keep grinding it out.
Look up Matt vs Japan's videos on Anki, especially his later ones where he temporarily swears off Anki.
I think the principle is repetition and any way that works for the individual learner works. But as one becomes more advanced it should be in context and word families matter more than vocab words, and sentences that have associated memories.
I'm pretty taken by the idea that it's really about seeding recognition in immersion. Not a way to learn a language in and of itself by any means.
I got pretty far in my beginner and low intermediate stages with Olly Richard's Spanish books, where I could repeat one chunk of a story over and over again, with the audio books. Like how the short stories and dialogues work in lingQ
I don't think Anki's spaced repetition algorithm is anything special. In order to use it correctly you need to use the hard/easy buttons as well and yet most people say not to use those for language learning. I don't know if Memrise lets you do sentences as opposed to just vocabulary. If you can stand it, I think studying with sentence cards is a big step up because you're doing immersion while you review. I say if you can stand it because if you're a beginner or your language has a foreign script then reading a sentence card or a lot of sentence cards could drain the life out of you. I'm actually using a totally different app called Flashcards Deluxe which is a lot like Anki but more basic.
I think flash card apps or similar focused review is a big help to learners since words often don't repeat often enough just through reading to stick. The more advanced you are in a language of course the more you can read etc but even then focused review just seems so beneficial. The trick is to make it into an activity you enjoy.
"The trick is to make it into an activity you enjoy."
No, wrong approach and wrong attitude.
The trick is to make it a habit that sticks so that learners are independent of mental volatility (i.e., motivational ups and downs, "feeling like it", joy, etc.) and say good-bye to "avoidance behavior".
See BJ Fogg's "Tiny Habits": https://tinyhabits.com/
"since words often don't repeat often enough just through reading to stick."
No, just read more because reading is a "natural SRS in itself". And this works very well for higher frequency words.
BTW, learners should (almost) never focus on individual words because native speakers tend to use tens of thousands of word groups (esp. collocations), not single words in communication processes.
"could drain the life out of you"
Just reduce the number of repetitions in Anki. 25 per day, for example, works very well in my experience (I've been using Anki for more than 7 years).
Maybe we need a "course" on how to use artificial SRS and how not to use them...
I am not using Anki at all, but maybe you could make a case for using it to remember phrases. For instance, I asked ChatGPT to write 10 example sentences for "are opposed to each other" in mandarin:
- 男人和女人是相互对立的。 (Men and women are opposed to each other.)
- 公司和竞争对手是相互对立的。 (A company and its competitor are opposed to each other.)
- 阴天和晴天是相互对立的。 (Rainy days and sunny days are opposed to each other.)
- 健康和疾病是相互对立的。 (Health and disease are opposed to each other.)
- 正义和邪恶是相互对立的。 (Justice and evil are opposed to each other.)
- 爱和恨是相互对立的。 (Love and hate are opposed to each other.)
- 学生和老师是相互对立的。 (Students and teachers are opposed to each other.)
- 成功和失败是相互对立的。 (Success and failure are opposed to each other.)
- 信仰和怀疑是相互对立的。 (Belief and doubt are opposed to each other.)
- 和平与战争是相互对立的。 (Peace and war are opposed to each other.)
I could imagine loading them into Anki or other flashcard systems and review them (of course shuffled and with 100s of other phrases unrelated to "are opposed to each other")...
Excellent idea, Jan!
I like it.
It's an excellent addition to sentence mining using a dictionary. Something missing from SRS using sample sentences is the absence of context clues in some cases. That contextual clue may come from preceding sentences in a paragraph.
(Two football teams are opposed to each other in a tournament.)
(Eventually, the red team gained victory in the final match by helping each other among teammates.)
Another thing that needs to be added to SRS using sample sentences is the historical clue we get from the context to derive a more accurate sentence meaning and thus memorize the words with less effort.
He doesn't see the shadow of a deer, not even to mention governing a place.
Actually, "鹿" comes from "逐鹿天下" in the context of contending for the supreme power to govern the world/land (天下) among warlords in ancient China.
Another thing that helps memorize the vocabulary in reading is the natural flow of the text, which connects the intriguing plots of a compelling storyline. The topics-focused text also offers some valuable clues for the meanings of some words. Sometimes it's better to focus on the level of our impression of the vocabulary than the frequency of repetition.
My feeling is Anki is useful at beginner level to build up a base of vocab and practice basic grammar but is better used sparingly beyond this.
It's easy to use all your spare time just revising old words in Anki. I feel you will develop quicker if you use most of your available time reading and listening to new content rather than revising old content. This reading/listening method is a type of spaced repetition because the same words come up again and again so you keep getting refreshed.
I am lower intermediate. I spend 10-15 minutes per day on Anki. I have to be careful I don't add too many words each day or my review time will get too long. Most of the time I don't add any words or a max of 5 a day. Nowadays i only add words/phrases that are not intuitive or easy to remember. Basically: Anki for 10-15 minutes is a delight. Over 20 minutes gruelling! haha.
15 minutes per day / 60 minutes per hour * 365 days per year = 91 hours per year. This is no small amount of time.
Here's one suggestion that's an alternative to the SRS:
Randomly read a page in a book or graphic novel in your target language that you've read before.
And randomly read over a page or two in LingQ of a lesson that you've completed.
Some random repetition of a whole chunk of something you already have memory of can be as powerful as an SRS
Interesting discussion! Myself, I've been a convinced Anki user for a long time. Not only for language learning but for almost anything.
However, since I started here on LingQ (not long ago), I experience really quick progress in Italian. I also have been using Memrise since I started with the language, but soon after starting here, Memrise felt like a waste of time. It just feels slow, and intuitively I think it's more effective to invest that time in immersing instead.
As for Anki, I removed the deck with the first 1500 vocabulary today as I spent daily almost an hour, going through vocabulary and short phrases, I actually know already. - Again, feels like a waste of time.
What I highly enjoy is going through mined sentences from movie or video game trailers. But creating these cards take quite long in comparison. I guess I will still keep creating them at times, I want to slow down a bit.
The slow pace is welcome though in other languages which are a bit more difficult to learn. At least for now.
About the SRS - I like that using LingQ's built-in SRS saves a lot of time. However, I only activated multiple choice. As many words have various meanings depending on their context, I think, the other options just complicate things and I believe, with time, the needed vocabulary will eventually be learned anyway, as we keep encountering them.
Maybe if I wanted to build some vocabulary in a specific area, I would use Anki again.
whether Anki or any SRS or flash card app is a good tool for you, you have to find out for yourself. Everyone learns differently.
I like to work with a SRS-system and I use it (on and off) for about 11 years now. Personally I tried Anki once (about 11 years ago) but then I chose to use the app "Flashcards Deluxe" which fits me and my learning style more and the app served me well for all this time because I spend a lot of time in trains and busses, and I always have my phone with me. Actually I liked the app a lot as a beginner for French, because there is the possibility to use text to speech for French and I could listen to the pronunciation of each expression while learning it. That helped a lot with my own pronunciation. And there is the option to write my answer down as a drawing option. I always use that tool when I am somewhere where I can't use pen and paper, because I think that writing the words down helps learning them faster.
The advantage of an SRS app itself is that the app reminds you when you have to revisit the vocab. But people tend to feel pressured after the app tells them they HAVE to study 345 flashcards now... and sometimes people just swipe "known" without really knowing a word, just to get rid of the heap of flashcards that they "have to" learn that day. That's not how SRS works...
Before you use an app you should consider that it can take a lot of time to create flashcards in an app. That depends on the app and the source of your learning material.
BUT, when you already have great cards on paper... just use them the same way!
Actually I switched to an app because I had a looooot of paper flash cards "sitting around" as the result of learning three languages: more than 30.000 flash cards.... and I don't want to know how many hours (or rather days!) it took to transfer the cards to the app.
I love both content-flexible audio readers à la LingQ and artificial SRS such as Anki because they "complement" each other quite nicely, esp. via LingQ's export function for Anki.
"read that Anki is a program that works wonders for learning vocabulary, its algorithm is unique."
I'd say that's magical thinking without any basis in the realities of second language acquisition.
The important aspect in this context is "spacing vs non-spacing", but the quality differences of the spacing algorithms (SuperMemo, Duolingo, Anki, etc.) seem to have minor or no effects.
"Repeated retrieval with long intervals between each test produced a 200% improvement in long-term retention relative to repeated retrieval with no spacing between tests. However, there was no evidence that a particular relative spacing schedule (expanding, equal, or contracting) was inherently superior to another. Although expanding schedules afforded a pattern of increasing retrieval difficulty across repeated tests, this did not translate into gains in long-term retention. Repeated spaced retrieval had powerful effects on retention, but the relative schedule of repeated tests had no discernible impact." (https://www.semanticscholar.org/paper/Spaced-retrieval%3A-absolute-spacing-enhances-of-Karpicke-Bauernschmidt/23c01da059b9eb8be667930bddddc2033e719e31 . highlighting by me).
That said, the "main problem" for becoming fluent in our L2s is in my experience neither the spacing algorithm nor the distinction "artificial vs natural SRS", but the quality of the language learning material.
In other words, what language learners need, esp. at the intermediate language levels, is a lot of information about tens of thousands of collocations (including their use cases, i.e., oral vs written, slang vs informal vs neutral vs formal language registers, positive vs negative connotations, etc.).
Just randomly reading / listening and / or doing flashcards seems to be a "bad", i.e. "time-inefficient" SLA strategy... so I'd say we need much better tools based on, e.g., collocation training, AIs (say "hello" to ChatGPT, etc.), corpus linguistics, etc.
A 'natural SRS' would be defined as having 'random' or 'disordered' intervals. If the relative internal has "no discernible impact," as this study may imply, why not just use the 'random'/'disordered' intervals of a 'natural SRS'? This way you are also exposing yourself more texts and a wider variety of collocations.
(I can't comment too much on the details of the paper, as I don't have a subscription to the journal.)
There's a misunderstanding here. By the expression "random," I was referring to the nature of the learning material.
I "love" the subject of "Muslim Spain" in the Middle Ages (https://www.britannica.com/place/Spain/The-Visigothic-kingdom#ref70359).
I never get tired of listening to audiobooks / reading books about this period, esp.
in Spanish and English.
Does this help me improve my everyday English or Spanish? Not really. It's simply the wrong learning material for this purpose.
It's the same for audiobooks in general where speakers have a slow and clear pronunciation. This doesn't help at all when confronted with natives who are the complete opposite: fast-paced, sloppy, using a lot of slang and contractions...
And even Netflix series, etc. aren't good enough in this context because intermediate learners need more information about the degrees of (in-)formality, the social and temporal aspects of frequent collocations, etc.
ESLPOD, for instance, does an "outstanding" job by offering more than 2000 dialogues (including a lot of helpful explanations of what to use or not to use in what context!) related to everyday and cultural American English on their website.
Unfortunately, I didn't find something similar in other languages such as Br. Portuguese.
I looked at the ESLPOD site (or maybe it's one of many sites) and I could see they have conversations about various topics. I think the aformentioned chatGPT can definitely help with this. I can ask it to give me dialogues for various scenarios and I can continue and steer the dialogue in anyway I want. Kind of like those "Choose your own adventure" books when I was young, except now you have unlimited choices.
Or I can simply converse with it. Won't help with the listening aspect, unless I was to have TTS read it, which would be helpful, but for listening I think one needs the authentic conversations...like in Easy German/Spanish/etc. channel. I've found those very helpful for natural speech to listen to.
It's not about the "conversations" (reg. a myriad of topics) per se, but how these dialogues are "dissected" by the ESLPod team (https://secure3.eslpod.com/about-us/).
Native speakers usually resort to highly conventionalized multi-word combos (tens of thousands of them) when speaking / writing. And they know the associated nuances (i.e., the language registers, the connotations / associations, the typical social contexts, etc.).
Language learners, even advanced ones, have "enormous problems" with this.
To give you an example in German that I discussed with my American colleague at work a few weeks ago:
Before leaving for the weekend, I texted in Teams: "Wir sehen uns am Montag in alter Frische!"
Then I thought to myself: "Hm, how could I say that in English?" And I didn't come up with a satisfactory translation.
When I consulted Deepl, it translated the German sentence literally: "See you on Monday in old freshness!" However, nobody says that in (American) English.
Thanks to Google-Fu, I came across the idiom "bright-eyed and bushy-tailed", which I had never heard of before. Therefore, I asked my American colleague. And he said that was something people used more in the 1950s, and it referred to rabbits :-)
Anyway, what's missing from the English translation of "in alter Frische" is this:
1) the contradictory tension "old vs fresh"
2) it's an oxymoron (https://en.wikipedia.org/wiki/Oxymoron)
3) it's used humorously, ironically or even mockingly
4) it's used rather by middle-aged or older people. That is, children oder teenagers probably never use this expression.
5) It tends to presuppose a certain familiarity between people who know each other.
For some background info in German see also:
https://www.dwds.de/wb/in%20alter%20Frische (btw.,, I like the ironic answer "Alt ja. Frisch eher nicht" = "Old yes. Fresh rather not" in this context :-) ).
This is more of an idiomatic structure, but such nuances apply - mutatis mutandis - to many other collocations that have not achieved the status of established idioms as well.
And to become fluent in an L2, you need to understand such semantic and pragmatic nuances...
"I think the aformentioned chatGPT can definitely help with this."
Yes, it should be able to create some example sentences for some collocations.
However, can it provide / explain the semantic and pragmatic nuances that are involved here? I don't think so.
That's usually "native speaker territory". So, the winning combo for the implementation of such SLA solutions is, IMO, rather a tech (AI, corpus linguistics, etc.") and native speaker hybridization :-)
I would, of course, also resort to specialized dictionaries for "collocations", but combine them with contemporary tech solutions (AI and Co)!
Never trust blindly a native speaker: say "no" to rabbits and "yes" to squirrels :-)
"Bright-eyed and bushy-tailed are two terms that developed independently of each other. Bright-eyed supposedly comes from the late 1500s, while bushy-tailed is said to have hailed from 1865-1870, though no direct source is clear for either of them.
They were seen together for the first time in talking about a squirrel, which did, in fact, have bright eyes and bushy tail."
Thanks Peter for the insight.
I definitely know the phrase "bright eyed and bushy tailed". My parents would use it when I was growing up. "You need to get some sleep so you'll be bright eyed and bushy tailed in the morning!" =)
I'm a little surprised deepl.com didn't get "in Alter Frische" as it often has these idioms down. I wonder if given a little more context if it would've figured it out. Then again, as great as it is, it always has room for improvement.
I usually look in dict.cc for these idioms for German. It's had just about any I've every needed to look up:
pons usually does pretty well with these too, although it didn't have this one exactly (it referred to some contextual examples).
I have been struggling with a very similar problem in Chinese. Here is a technique I use to improve my ability to understand the vernacular language:
I select a podcast intended for native speakers, that I don't understand well and have it transcribed. Previously, by enlisting the services of a commercial service (AWS transcribe), but nowadays I'm just using whisper by openAI.
The output comes in the form of a subtitle file, I can import this into LingQ, together with the audio file. This way I can use the sentence mode to listen to the audio sentence by sentence. I will repeatedly play the audio, while looking at the sentence, the translation, use the dictionary, whatever is necessary. But I will not pass to the next sentence until I am able to understand the current one without looking at the screen. This can often entail 10-20 repetitions. This exercise is rather intense, so you might want to balance it with some free-flow reading and listening. 30 minutes often feel like a good workout already.
An alternative source is YouTube, either import a video into LingQ or by utilize a browser extension like Language reactor, this allows repeating the current subtitle as well.
I have previously written a bit about this approach here: https://www.lingq.com/en/community/forum/open-forum/advice-please-slow-japanese-pr?post_id=318935
On the topic:
I don't use SRS. Not only do I find this activity inherently boring, but also unnecessary. Even in Chinese, I have never felt my rate of learning words to be a bottleneck, my inability to recognize those words in regular speech has been way more frustrating. The podcasts I import typically contain hardly any blue words. Now that I'm a bit more intermediate and deal with more rare literary words, I find monolingual dictionaries to be very helpful, because they provide an actual explanation of the word, often with example sentences, this information appears so much more meaningful than just a list of synonyms.
Yes, that's a good technique. And I use(d) something similar with all kinds of podcasts, Youtube videos or Netflix in various languages (English, French, Spanish or Portuguese). But I'm not happy with this approach because the "nuances" (mentioned above) are often missing.
For example, it took me about 15 min, just to find a satisfactory answer (Deepl, Google-Fu, discussion with my colleague) for the "in alter Frische - bright-eyed and bushy-tailed" problem.
IMO, we can do better than this nowadays when it comes to (tens of) thousands of collocations (idioms included)!
"I don't use SRS. Not only do I find this activity inherently boring,"
Well, I find the solution LingQ-to-Anki quite handy (for testing).
However, ff I had to create (tens of) thousands of Anki cards manually myself,
I probably wouldn't use it.
But the advantages of the LingQ-Anki-combo are:
1) I know the context from reading / listening on LingQ
2) LingQ provides often complete sentences
3) I can translate from my L1 in various L2s, which is much harder and, therefore, more effective than the other way around.
4) It's faster than re-reading texts on LingQ.
Re the "boring" part:
I often do the same things every morning: working out, running, taking a cold shower, doing Anki drills, and using LingQ.
It's just habit - and I'm never bored. It's the same for brushing my teeth every day... :-)
BTW, that reading is inherently more "interesting" than flashcards is a myth in my opinion, because there are so many "bad" texts out there (from didactically graded readers and depressing Assimil and Co beginner texts to AI-based text generators - "yes, OpenAI, I'm talking about your GPT3 models, too" :-) - that I prefer short and crisp flashcard exercises.
But, of course, I love reading and listening as well, but this refers usually to more advanced stuff (history, social sciences, linguistics, great novels by professional authors and not just well-meaning didacticians). :-)
- Reading is great (and more exciting than flashcards) when the content is not only comprehensible, but compelling.
- Reading is not great when the content is comprehensible, but produced by second- or third-rate (human or AI) storytellers.
SRS is an auxiliary and secondary part of committing vocabulary to long-term memory by deliberately fighting the forgetting curve. Acquiring vast amounts of words more efficiently depends more on our ability to make associations by encoding them into our brains when interacting with the language.
I can memorize and retent way better when I am at the intermediate level compared to the beginner level. The key things lie in comparable skills we have developed in the course of learning a language and applying such skills consciously or subconsciously in all activities that we engage in the language. These essential skills include mastery of phonology, familiarity with the writing system, ability to sound out words, use contextual clues, follow the storyline, connect with personal experience, make cultural, social, and historical references, etc. I also find that monolingual dictionaries are extremely helpful in providing precise definitions and sample sentences in context. However, sometimes I need help understanding the sentence, even with the translation in sentence mode on lingq.com.
How does everyone deal with more complicated sentences loaded with figurative speech, metaphors, cultural and historical references, or any other idiosyncrasy in the language?
I agree it's important to consider the forgetting curve and commiting words and phrases to long-term memory. The question is, Why have you chosen to use the strategy of using an SRS to do it? It comes with the opportunity cost of less reading and listening. There are other strategies (listed below), which seek to also "deliberately fight the forgetting curve" and come with the additional benefits of more reading and listening. Namely, exposure to the word/phrase in different contexts compared to the single sentence on the flashcard, and exposure to other words and phrases in the process, further strengthening your intuitive knowledge of them.
15 minutes of Anki per day is almost 100 hours per year. It adds up. Imagine 100 more hours of reading and/or listening. Even half of that Anki time is still equivalent to several extra books in a year.
Hi Florian, sorry if this is a stupid Q, but how do you use Whisper? On https://openai.com/blog/whisper/ I cannot see any button to get it running or to uload an audio.
Forum threads with lots of replies are always interesting, no clue where my response might end up...
@JanFinster Re: Whisper:
The whisper repository is located here: https://github.com/openai/whisper Instructions are in the readme. But note that you essentially need a modern Nvidia GPU with lots of memory (8GB+). Since I don't have such a system, I use a re-implementation of the original called 'whisper.cpp' https://github.com/ggerganov/whisper.cpp It is optimized for use on CPUs and handily outperforms the original on my system (Apple Mac mini M1). The medium model converts here in about 3x real time and the large one in about 1x.
As for installation, if you're regularly compiling things, this should be trivially easy, if not then you might have a bit of a learning curve ahead...
A less involved way to try it out was introduced here:
The original thread that introduced whisper to the forum is here: https://www.lingq.com/en/community/forum/open-forum/best-way-to-generate-subtitles
I'm happy to share some transcripts, Here are some example transcripts from a podcasts called 忽左忽右 https://0x0.st/ohRr.zip
The ones named wav.srt are by whisper, the others are by AWS transcribe. Please note that whisper.cpp has recently improved noticeably, so these samples are not representative of its current quality.
Also, I probably should have added to my other post, that whisper is really good in terms of word error rate (WER) but pretty bad at timestamp accuracy. So, the exercise I proposed doesn't work well with whisper's transcripts. Commercial services typically have excellent timestamp accuracy, as do YouTube subtitles.
@PeterBormann Sorry, I wasn't very clear.
Let's consider the following scenarios using both an 'artificial' and a 'natural' SRS. We will use units of days with 0 referring to the day 0, the day of exposure to a new word. 7 is day 7 and would be one week later.
'Artificial' SRS eg. Anki
Increasing intervals by doubling: 0 1 2 4 8 16 32 64 128
Fixed intervals of 5 days: 0 5 10 15 20 25 30 35 40 45 50 55
These intervals are very regular and in certain algorithms, such as the Anki one, if you get it wrong, you will see the word more frequently.
'Natural SRS'/reading and listening and following your interests
A 'natural' SRS is semi-irregular in it's intervals, or at least has more randomness involved, but may look something like this:
High-frequency word: 0 1 3 4 5 6 8 9 12 13 14 15 16 17
Mid-frequency word: 0 4 8 11 17 29 31 36 44 48 53 60
Low-frequency word: 0 40 112 190 319
As you can see here, there are intervals between every exposure to the word. The paper you referred to talks about intervals vs no intervals. A 'natural SRS' has intervals.
Unfortunately, I can't read the paper and the method, so I can't refer to the details, but I'm very sceptical if we can generalise this "relative schedule of repeated tests had no discernible impact" to "has no discernible impact." Maybe it didn't have a discernible impact in this, one experiment, but it doesn't necessarily mean it's true in the general sense. There's chance involved, right. I mean, if you believed it to be true, then why are you using Anki in the first place? My guess is because you believe the frequency of the intervals for low-frequency words is too low, right? Especially to learn the words at a decent speed? I agree.
There are several ways to alter the 'follow-your-interests' approach of the 'natural' SRS to increase the frequency of the intervals of words:
- Read and listen more. Simply increase the number of hours you spend each day reading and listening. It's not always possible for everyone, but for some it's an option. You can also achieve this by increasing your reading and listening/playback speed. If you do this, you can still follow your interests and learn many new words.
- As you mentioned, be more selective in your materials and less ad hoc. Certain words appear much more frequently in certain contexts, in certain domains, and by certain authors. For instance, economic words appear more in books on economics or financial columns of newspapers. To increase the intervals of low-frequency economic words, and hence learn them, read more economics material. On LingQ, you can see the number of yellow words in each lesson, so don't let that get too low.
- Reread and/or relisten to material. If someone is supportive of an 'increasing interval' idea, they can follow this strategy with their rereads/relistens too.
- Using LingQ, skip large portions of text, where you already know the words and focus on the yellow and blue words. On the browser, you press the right arrow key to do this. Steve mentioned he was doing this on his Persian podcasts.
With such strategies available, the question is: Why do you use an 'artificial' SRS? There are clear benefits to more reading and listening (such as learning collocations, as you mentioned, and exposure to the word in various contexts, not just the single one on the flashcard). By using an 'artificial' SRS, this comes with the opportunity cost of less reading and listening.
Good points, nfera!
But it's getting late and I'm hungry :-)
I'll answer tomorrow...
"clear benefits to more reading and listening"
Definitely - esp. using "ultrareading while listening".
1) "Read and listen more."
Well, there's a major downside to just "reading (while) / listening more and more", esp. from a B2-C1 level upwards:
In general, the mental effort involved in simply recognizing form-meaning combinations embedded in contexts is way too low compared to actually producing and using such form-meaning combos appropriately in co- and contexts.
Therefore, it's a common complaint among CI practioners (SergeyFM was one of them in this forum a few months ago, and I'm experiencing exactly the same thing in Br. Portuguese right now) that even after reading / listening for more than 1000-2000 h in the respective L2s, we still struggle a lot in speaking (and writing).
When it comes to writing in our L1s, that's even true for native speakers!
In other words, if native speakers aren't professional authors or have at least an academic background, they tend to be "bad writers" (esp. if they're are also bad readers).
And I'm not talking about "functional analphabets" in this context, i.e., almost 10 percent of the German population:
"Low-literate adults who attended school but either did not complete their education or, for a variety of reasons, did complete it without attaining the expected level of reading skill are referred to as functional lliterates. In Germany, approximately 7.5 Million adults are considered functionally illiterate." (highlighting by me)
3) Using "LingQ-to-Anki" is a kind of "intermediary" solution for this problem (the "real" solution is, of course, producing / using the vocabulary in oral and written communication, esp. with native speakers, but also with more powerful chatbots à la ChatGPT, - a lot):
- First, reading-only / (ultra-)reading while listening (URL) / listening-only based on LingQ
- Second, testing myself in L1 -> L2 translations for the context-embedded URL-vocabulary using Anki.
That is, if the Anki drills consisted only of L2 -> L1 translations, then this would be another and far too easy recognition exercise. And in that case, I might as well just resort to the reading strategies you mentioned above.
4) Apart from that (i.e., tougher "active L1->L2 recalls instead of pure L2 recognition), there are other use cases for artificial SRS. For example:
- Learning specific grammar structures (verb conjugations, L2 cases - see the current discussion with xxdb reg. Russian, etc.)
- Low frequency collocations that you mentioned above.
5) Can artificial SRS be further improved?
Definitely - see the "interaction" of AI, corpus linguistics, specialized dictionaries / SEs for collocations, and native speaker knowledge indicated above so that collocations are enriched with linguistic and socio-cultural background knowledge.
Remember the "bright eyed... - alte Frische"-problem mentioned above?
If the assignment for learners of German (let's say, Asad, Eric, Bembe, etc. in this forum) was to write a short dialogue between contemporary German teenagers using options such as "putzmunter", "quietschfidel" and "alte Frische" to describe someone who is full of energy early in the morning... which options would you use?
- Why and why not?
- Are there other options not mentioned in this context? (yes, there are!)
- And what do you think the sociocultural connotations of these options are?
Okay. I misunderstood. I thought you were referring to using an SRS to learn new words and new collocations. In particular, commiting them to long-term memory. I think this is what the majority of SRS users use it for. This is what I was referring to with the modifications to the 'follow-your-interests' approach to perhaps make it more effective than an SRS.
But you are referring to using the SRS as a stepping stone to speaking/writing by only doing L1 -> L2 production. You mentioned Comprehensible Input learners having the problem at B2/C1 of struggling to speak. You also mentioned yourself that the most effective solution would be speaking with and writing to native speakers or AI. This leads to the next question, Why don't you do that to improve your speaking/writing instead of using L1 -> L2 production with the SRS?
I, too, don't believe that you can become a fluent speaker by only input. Even Stephen Krasher I think doesn't believe this. One of Steve's phrases that he loves to say is: "If you want to speak well, you have to speak a lot."
"Why don't you do that to improve your speaking/writing instead of using L1 -> L2 production with the SRS?"
There are two reasons:
1) Such Anki drills are less time-consuming than speaking / writing in Br. Port. at the moment.
2) It's an experiment: to what extent do these Anki drills make speaking/writing easier (and more meaningful) in the L2?
Another interesting approach may be just to use ChatGPT or the combo " L1-L2 drills (based on Ultra-Reading while Listening vocabulary) plus ChatGPT".
I'm not a fan of ChatGPT "stories", but using AI-generated dialogues based on URL-vocabulary could be an interesting option (see the current thread reg. ChatGPT).
For the experiment, fair enough.
Regarding the time-consumption, they may be less time-consuming, but as you mentioned, they aren't as effective as speaking/writing to people (referring to your comment of the 'real' solution to the problem). I think it should be thought of as how effective something is per minute/hour invested.
That said, if you only have 15 minutes, it's reasonably fast to log on to ChatGPT for 15 minutes and have a texting chat. It's also relatively fast to open up Google Maps and write a review of the restaurant you just visited in your L2. Regarding tutors and what not, yeah, you need a bit bigger time block and a little organisation, generally.
Regarding your comment about "at the moment," my experience with Italian is that my first 5 or so hours of speaking (so maybe 8 or 10 hours of conversation) was particularly challenging. It's still not easy (I also have a limited passive vocabulary at 11k Known Words), but it became significantly easier after the first few hours. I really saw a massive jump. Maybe it might be similar for you.
I want to clarify one thing I do flashcards but not so much combining with the SRS technique. I use flashcards for note-taking and identify the areas I must work on to overcome a hurdle or advance my learning to the next level. Fundamental language skills and higher cognitive power to process the massive volume of information efficiently should be deliberately practiced and enhanced by consuming or producing tons of context in relevance.
I, too, don't believe that you can become a fluent speaker by only input. Even Stephen Krasher I think doesn't believe this. One of Steve's phrases that he loves to say is: "If you want to speak well, you have to speak a lot."
I couldn't have agreed more that every core skills in the language need to be developed over time with tons of effort. A couple of things I am aware of that hinder my efficiency with the comprehension of text are pronunciation and grammatical structures. Flashcards with multiple sample sentences and authentic audio provide more concentrated and detailed information on the grammatical structure or specific knowledge about a colocation in the language. These practices are stepping stones to opening the gateway to further studying for a specific subject or at a particular language level, enabling me to consume or produce related content with much less difficulty.
I have lingqed terms tagged with "grammar," "idiom," "phrase," "culture and custom," and "metaphor," etc. It would be a little too daunting to dive right into a book such as the one for etymology or specific colocation like Chengyu in Chinese with historical anecdotes, especially if someone's not at that level or has no basics in the subject.
A good question is when would SRS, whether natural or artificial, become less relevant as we continue developing our core language skills with engagement in the language corresponding to the framework of Bloom's Taxonomy?
I get you don't like anki and if you have developed your own method good for you.
That said, I'm going to point you to some hard-coded logic that is unassailable:
An SRS is focused on YOU. It repeats the words that YOU do not remember, specifically giving you focus on them, while allowing you to relax on those you have already learned. That is to say, it is optimized for your particular learning curve.
On the contrary, "natural SRS" is fake. It is *not* SRS. Encountering words in video or books is averaged out based on frequency. It has nothing to do with the forgetting curve and is just based on how often it comes up. And you can't change it based on words that are easier or more difficult for you. It's fixed.
Anyhow. That said. Both of them complement each other but for different reasons.
Unless you are memorizing specific phrases (ugh) you are only getting exposed to words that are out-of-context, and as (maybe you?) and definitely Peter have said, collocations and set phrases are definitely a thing you will not get from individual word bulk memorization. All the different techniques have a place. For me SRS is epically helpful. You don't seem to like it, that's great. If you have a method that works for you without SRS that's great but I whole-heartedly disagree with the concept that it's useless or "beaten out" by "natural SRS". They are complementary instead.
Anyhow, love these kinds of chats.
Anki requires SO MUCH TIME... I think there's better ways to put your hours into.
In spite of this, Anki is a great tool and it depends exclusively on you.
Honestly, I also used Anki for a long time, but I'm way better on LingQ. I'm loving investing my time here and I don't intend to use Anki again...
The thing about SRS is it's supposed to *optimize* your time. You're spending the minimum amount of time required to commit the item to memory instead of just hitting it based on frequency from reading or listening to a tremendous number of words.
I only spend 10 - 20 minutes a day on Anki now that I am at lower intermediate level. It is useful at beginner level to get a set amount of vocab under your belt and practice grammar. I used to add words/sentences every day and was spending an hour reviewing every day which felt really gruelling. If you want to spend less time on Anki, stop adding words so there is less to review.
I now spend 10-20 minutes on it every day which is enough to refresh what I've learnt but not overwhelming. Now i rely on reading and listening to youtube podcasts every day in Linq to pick up new vocab and learn about grammar in context. Much more fun! Don't worry about forgetting words that come up in Linq, they will come around again and they will fix in your brain the 2nd, 3rd, 4th or 5th time around.
The question I'd love the answer to is this: what is the *optimal* way to burn in the words?
I'm not convinced those on the side of "reading or listening" is better.
If it was, anki and tools like it would not exist.
The problem with anki is simple: it's painful. LingQ is fun.
"The problem with anki is simple: it's painful. LingQ is fun." (xxdb)
I wouldn't say that LingQ is "fun in itself".
It's rather the case that advanced texts (from a B1 level and above) that learners can choose for themselves are usually more fun than doing Anki, Glossika, etc. drills.
However, nce the use of Anki becomes a habit, it's no longer painful. You just do it :-)
Absolutely not. I would say it is a potentially useful tool, but it should be used sparingly. Currently some language learners spend the majority of their study time with Anki. This is really not the best way to acquire a language.
Of course it isn't. You can't learn a language just by memorizing individual words. You *can* however, get to beginner level just by memorizing words.
To get to intermediate you NEED to either read or listen to language.
I use flashcards with sample sentences in different contexts mainly for pronunciation/speaking practice and to familiarize myself with the grammatical structures in the language. I focus more on the essential abilities to be developed and acquired than the number of words to be memorized. Once I become more comfortable deciphering more complicated text, intensive and extensive reading would be perfect for committing vocabulary words into long-term memory.
I practice "lazy SRS" in LingQ, which I find very effective:
I have it set to never have to type in the words as I loathe feeling tested in learning a language.
I pass every card no matter what.
I set it to 25-30 cards per session.
I set each session for the 1-4 rating of knowing the words/phrases
I set it to the "important" option, not creation date or status. As status biases cards with a rating of 1 and creation date has me only going over new words.
For each rep I listen to the word with the text to speech. Of which I never use for sentences but I don't find it a problem (artificial robotic speech) for individual words and small phrases.
90% of all of these words come from a show that I've already watched, so each sentence chunk has a subtle memory to it.
So essentially I'm sentence mining with LingQ.
I've tried exporting these cards to Anki, but I'd rather just power through them with LingQ being able to hear each word.
The reader is fantastic for intensive reading but I find some mindless repetition to really accelerate things.
Anki is not necessary and I believe at the late beginner stages and beyond it's not the most efficient way...in my opinion.
I started my German "journey" with Memrise (their old A1 course that you can't find anymore). I really think it was great for these beginning stages and I think Anki could certainly be used at this stage too. I think this is the best time for these SRS programs. A lot of the words are nouns and verbs so they're easier to associate the meanings in these type of programs. These beginning words also typically have a simple straightforward meaning. As you get higher in levels, the types of words and the meanings behind them start to get a lot more complicated and definitely words can take on an entirely different meaning in phrases.
After my Memrise A1, I found LingQ and started importing easy news articles...my vocabulary skyrocketed. The problem with Memrise and SRS systems is that you spend all your time reviewing and never learning anything new (at least with my limited time). I didn't see a future with this as I would simply not have time...LingQ was a saving grace.
As I'm getting into the advanced levels, I can perhaps see a place for Anki. *Maybe*. I doubt I will try though. I think I have better ideas...
If you really want to "review", go through a lesson you've done. If you don't want to re-read the whole thing again, just skip to the yellow words. Read the sentence and the word(s) in context. Try to guess the meaning. If you need more context, read the surrounding sentence. If you still can't guess the meaning look it up. Move to the next yellow word and do the same. At least with this way you can see the word in context and you will likely get a better meaning of the word from that, as well as start to associate with other words which may help you learn the word better next time around. As someone has mentioned already, Anki or SRS with single words doesn't provide this and it really gets hard to associate or remember the multiple meanings a word may have, just from looking at that word alone.
If you like doing Anki and SRS, by all means do it. There's a lot of people here that do. There's a lot of successful learners that do use it. I'll admit I find it super boring so I will likely never do it for any stretch. One thing...I would limit its usage outside of the beginning stages to maybe at most 10% of your time with the language. I think input...reading and listening will better serve you. Of course, everyone is different. You may hate reading and listening. In that respect Anki may be the better fit...but I think it will be less efficient and at some point you're going to have to read/listen anyway.
I'm kind of reaching towards the idea that at the beginning it's indispensable.
In the middle stage you don't really need it because the mid-frequency words probably come up enough times while doing something *enjoyable* i.e. engaging with content by listening or reading.
At the advanced stage I think it becomes useful again: you just don't get enough exposure to the advanced-but-necessary words in any given time period for them to be above your memorization threshold in the forgetting curve.
But I don't know what the answer to that is, because who can say what a *useful* low frequency word is. By definition it's low frequency so you might never hear it again on average.
The compromise idea I have come up with as a hypothesis is target a particular genre or subject area (say narnia books, a particular scifi show or cooking books or whatnot). In those areas there might be domain specific "jargon" that is worth learning in order to be able to easily understand those specific domains.
Any case. Right now I'm stuck: I'm intermediate with no clear roadmap to move to "advanced" other than slogging it out so instead I'm pinpointing focus areas and going for them.
A particular telenovela. One crime show. One Scifi show. The Narnia series.
Once I have completed those I'm hoping I will be "done" enough. LOL. We'll see in July.
"I'm kind of reaching towards the idea that at the beginning it's indispensable.
In the middle stage you don't really need it because the mid-frequency words probably come up enough times while doing something *enjoyable* i.e. engaging with content by listening or reading.
At the advanced stage I think it becomes useful again: you just don't get enough exposure to the advanced-but-necessary words in any given time period for them to be above your memorization threshold in the forgetting curve." (xxdb).
Yes, that's also my experience with several L2s.
My current issue is to optimize accumulating enough advanced vocabulary to be able to understand the more advanced content.
Logically low frequency words across the entire corpus of the language are far, far, far too low frequency to help out with more *specific* topics such as e.g. Friends episodes or gardening books.
That is typically handled by just watching tons of friends episodes (if friends is your thing) or reading tons of gardening books (if gardening is your thing).
Following the logic chain the question then becomes:
within those specific domains of language is there then a *new set* of higher frequency words that come up more often within those domains?
I think the answer is likely to be YES. We even have a word for it (in English). It is JARGON. There is gardening jargon and there is likely friends jargon.
In which case the option opens up for identifying the higher frequency words (jargon) within those domains and just memorizing THOSE.
So how to do that? Just watch everything over and over and read over and over. Seems much more time consuming than it has to be.
But how to identify the higher frequency words (jargon) that are only within those domains and are not otherwise high frequency?
I don't have a good answer to that. Maybe going through the friends episodes and spitting out the "new" words would be half a solution, but there's no easy way to tell if it's repeated e.g. 9 times in the series (maybe worth learning, maybe possibly it's a jargon word) or just said once in the entire series (i.e. not jargon but lingQ has identified it as a new word).
I don't know how to solve that.
Edit: I found this scientific paper for "de-jargonizing" TED talks.
Likely the tool they used has some kind of jargon detector in order to be able to de-jargonize. Therefore, being able to identify jargon would IMO potentially allow us to spit out the jargon for the particular domain (i.e. twilight books or how-i-met-your-mother episodes).
I recommend Refold's anki decks as they're used to help a beginner onramp into a new language while immersing. Worth the money but even they state that anki is optional in Refold and not a great way to acquire a language
"Do you think it is necessary to use Anki in addition to all these for a good vocabulary?"
People have been learning language long before Anki existed. In fact, you learnt your mother tongue without Anki, if I may be so bold to guess. No, it is not "necessary" to use Anki to obtain a good vocabulary. Grandmaster Steve does not use Anki nor the LingQ SRS.
There are certain uses of Anki or an SRS, which can be very powerful, while there are other uses, which are less so. The devil's in the details.
Personally, I do not use Anki. I have in the past, but reading/listening everyday is a 'natural SRS' for me these days. Reading/listening everyday gives you large amounts of exposure to high- and mid-frequency words. You both learn these words and review them constantly through content. It's the lower-frequency words, which you may have more trouble to review through the 'natural SRS' of consuming content. In that case, at an advanced level, it may be worth considering using Anki. But at a beginner and an intermediate level, it's just not necessary.
But if you really like using Anki, enjoy life and rep away. ;)
I will probably be controversial here, but I've tried different review methods - Anki, Goldlist, manual SRS, LingQs own - and i have to say that they bore me silly, and I hate doing them...so I don't. I don't think that you NEED to do SRS, but if you LIKE doing them and theyt work for you then most likely any of the platforms will work for you. I've realised that the thing that matters is that you enjoy the experience...which makes you do it more...which gives you exposure...which then works.
In my opinion, the words that you need, and the phrases, will continually crop up when reading, and the only reviews of words I do are to click on the words in LingQ to advance them up my "known" level. But even then I forget them, and remember them, and forget them again, and them at some stage when my brain is ready, they stick.
So, to summarise...I don't think it makes the slightest difference which one you use.
I'm no expert so take this with a grain of salt. I don't think you want to memorise a language but attain it. I use Anki (for Japanese) to learn kanji and be exposed to conjugations. I think its better to learn vocabulary in context so that it's tied in to a conceptual framework.
I'm also doing Spanish and tried Anki with it to learn the most common words. It didn't help. I was divorced from every word and its relationships with other words or concepts etc.
I guess it depends on how you use Anki. If you can use it to help with intergrating concepts rather than storing all that information to memory, it can be useful. Generally though, people use Anki to straight up memorise, I don't recommend that. I could be completely wrong though. I'm not confident in langauge learning.
Just my two cents.
And conversely I learned Spanish by doing exactly that: I memorized the first 6,000 words of Spanish then started watching TV shows. Last week I gave a presentation to a C-level board in Chile. So I don't suck. Anki definitely has its place if you can tolerate the pain and boredom.
Anki overwhelms me every time I try using it. Its algorithm doesn't work that great on me, I just drown after a week of studying.
I think flashcards are good to drill down a limited set of vocabulary or useful phrases, fast. But they are not a necessity.
You can get overwhelmed very easily, that is true.
In my path I found Anki to be helpful at beginner and lower intermediate stages.
The more advanced I get, the less I find Anki helpful.
You're trying to burn a very specific "clip" of the language, which is great when attempting to get a foothold in the language, but as one becomes more advanced, the less needs to be repeated deliberately.
Although the SRS in LingQ has been criticized, I prefer it, just 30 cards a day. Just for some repetition powering through but not caring about some complex Anki algorithm.
I could also continuously read the sentences of the vocab words I've looked up and written down in the novels I read. That would be like a random SRS.
seems like Memrise would be pretty similar to Anki-- it's just which program you prefer.
I use it but it's mostly when I don't feel like reading I still feel like I touched the language. Not sure how much it helped me at the beginner level. Maybe a little bit!
Maybe it is good at intermediate levels to review words you don't see very often.
There's a lot to unpack here.
I would say that the more you do, to a point, the better. Anki and similar apps will help close gaps that may take you longer to find in LingQ or a similar program. However, when you speak a language, you never do it by imagining a series of flashcards and then putting them in order. You just use it. Slowly and poorly at first, sure, but you get better at it.
Also, flashcards are almost always out of context. It's just a word on a card. If you can read authentic material, you will learn words in a variety of contexts.
So, I would say that Anki is useful early on when you're still learning key vocab, but it's really, really hard to learn more than maybe two thousand words using a flashcard app. At some point, I would either leave Anki behind entirely, or use it to focus on a particular set of vocabulary I need to know about a particular topic.
Just my two cents as someone who used Anki in combination with other apps and LingQ to learn Russian. Also, I basically never use LingQ's flashcard review thing, or if I do, it's specifically to focus on the "4" value, really important words. I mostly just keep reading.
DU MÖCHTEST EINE NEUE SPRACHE LERNEN?
Lerne von Inhalten, die dir gefallenKostenlose Registrierung