[bug] generate timestamps overwrites created timestamps
gbonnema

[2022-06-11 saturday] Is a feature still broken if we don't use it anymore?
It shouldn't be, but it is. Today I decided to skip the "generate time-stamps" entirely and just enter my own time-stamps. Of course, that was not what happened.
After changing the first two entries, the third entry, upon using "copy previous end time" copied a seemingly random value, but one that look suspiciously like a generated time-stamp. The previous time-stamp was nine, but upon copy I got 13.2.
So, I exited editing (using view lesson) and re-entered the edit-session. Now, I saw that all time-stamps had been generated. I re-iterate: I did not press generate time-stamps!!
Apparantly the "generate time-stamps" is not just broken, it is a dominant terror even if you do not use it.
Can you please tell me, this helps you find out what is wrong? At least tell me something. If there is no way for me edit time-stamps without the "generate time-stamps" interfering, then .... I don't know how to proceed.
gbonnema

[2022-06-09 thursday] Again, a partial overwrite. This time, I had to join 2 lines into the first of the two, deleting the second one (start of text). Secondly, I added 2 lines in the last part of the text. Then I started editing the time-stamps and after the 3rd or 4th time-stamp, suddenly the audio was one line off again.
The program had decided to leave the joined line as is, but re-insert the deleted line. The lines at the end were still inserted. This is an image of the result; The time-stamp on the second line, was with the line following the un-deleted line. So, it undeletes, but leaves the changed time-stamps, causing all following time-stamps to shift one line.

EDIT1: a few hours later, I edit the file again, and it had changed again. As I cannot add another screenprint, I have to describe it. The joined line had be replaced by the original first half of the split line. The second line I had deleted (as it was in the joint line) was still deleted. The time-stamp had been replaced by old values.
Additionally, the 2 new lines at the last part of the file, were gone, undeleted.
I don't know what to say. I am exhausted from worrying about this error every day: will it overwrite the time-stamps? Will it leave it alone? WIll it delete lines? Or will it just ignore parts of my edits?
How long does it take before it is safe to edit a lesson? Why is the save not immediate, following a generate time-stamps? Why is it delayed? The saving-mechanism is seriously broken! Pfffffffffffff.
gbonnema

Yesterday the time-stamps were overwritten again. When I did not get the pop "successfully imported" straight away, I knew this was going to happen, question was "when?". So I updated the time-stamps partially and then left it. Today, time-stamps were reset. It is easy for me to see, because the generate time-stamps always over estimates the first few time-stamps by a lot, so I usually reset to 9.0 seconds or less.
I will update and publish this lesson today. The name is https://www.lingq.com/en/learn/fi/web/reader/13428265 the news of wednesday.
I am getting to be really fed up with this aspect of the system. Is there any update of progress? I even forgo mentioning any other errors I find, because it feels hopeless.
gbonnema

Today, 2022-06-01, the set of time-stamps and text was overwritten again, after doing generate time-stamps and successively an add of a sentence plus modifying time-stamps. It happened while being about 4/5th of the lesson. I noticed because the time-stamps were suddenly 1 sentence off.
So I reloaded the lesson through the lesson overview and the time-stamps had been reset, plus the sentence I added was gone.
I noticed, as before, that the popup "imported successfully" did not appear at the start. And as it shows only for a short period, I might have missed when it showed later on.
EDIT: during the repair, I noticed that the sentence was gone, but also the time-stamps were only gone up to point. Time wise this makes sense. Somewhere along the line the procedure decided to reset timings. Also, from the point where the sentence was deleted, the timings were shifted by one sentence, which also makes sense.
So: only timings until the unfortunate save were altered (about 1/3rd) and the added sentence was deleted, but the new timing (from 1/3rd to 2/3rd) were not: they just shifted a sentence.
Can you please give me an update on what is going on here? Is there any progress? I am really feeling pretty depressed about this error as it causes a lot of extra work. And there does not seem to be a lot of hope of repair.
gbonnema

Today, another weird occurrence. Lesson is https://www.lingq.com/en/learn/fi/web/reader/13269907. I had imported the lesson, and got an "imported successfully" message immediately. Then, I added a new sentence and added the audio, did the "generate time stamps" and started editing time stamps to my liking.
After that, I analyzed the words (view lesson) and listened to the lesson in sync. Everything was in order.
When the listening in sync was done I got the terrifying message popup "imported successfully" as soon as I pressed the sync button to get back to normal text.
I suspect the timings are overwritten again and the sentence I added is gone.
However, I can no longer see the lesson. I can open it, and I get the template that you usually see while the opening is in progress (all words are grey bars).
EDIT: 8 hours later I can still not open the lesson. I get the icon with the appropriate picture, but the text is all gray bars. I *can* open other lessons, no problem.
I really hope this is part of the problem being fixed ....
gbonnema

Today, again, I had a total overwrite of timings. The lesson is "https://www.lingq.com/en/learn/fi/web/reader/13117096" with the origin being from "https://yle.fi/uutiset/osasto/selkouutiset/keskiviikko_185_radio/12451032".
After importing the text I added the sound and did a generate time-stamps. Then I immediately started changing time-stamps in one go. After the last change I did a view lesson and checked the time-stamps through another edit lesson, and everything was in order. The listen with sync however, did not sync at all.
After a few hours I wanted to do the analysis in view lesson and, just to be sure, I checked the timings and noticed that the first timings were back to the old value. From the first timing to the last timing, everything was overwritten.
It shouldn't really matter, but I did the first phase (import + generate + changes) in firefox. I closed firefox and did the view lesson (hours later) in chrome.
I really hope you guys can figure out what is going on. It is extremely demotivating to have to redo changes hours later. That is additional to the idiotic click twice philosophy. The editor just has to save every change, but waits until I clicked something to do it, then blocks and then forces me to do the click again. So I already did the change twice, now I have to do it twice again!
EDIT: I am going to do the changes again, because the current timings really don't work for me.
zoran

Hello again. Just to let you know that we have identified some issues and our developers are looking into it. We will have some improvements soon.
gbonnema

Today, for lesson https://www.lingq.com/en/learn/fi/web/reader/13049357 from article https://yle.fi/uutiset/osasto/selkouutiset/sunnuntai_1552022_radio/12446503. After changing the timings and starting to the "view lesson" i.e. my analysis of words, at some point I started doing sentence mode to see the translation and hear the sound of the sentence, and again like before I got tty instead of the news reader. So I checked the "edit lesson" for the timings and the recording of the suspect sentence, and everything was in order. I thought: Ok, cache. So I ended the browser: no difference, still tty voice. I changed browser, twice, same result. Whether on Chrome or on Firefox or on the free version of chrome, they all have the same result: in sentence mode I get the tty voice when pressing <a>.
Hope this helps?
gbonnema

One other observation of this mornings exercise. Firstly, it seems that I now get the popup "imported successfully" immediately after importing. I only connect the sound after this point. As far as I can see at the moment, the timings seem to be left alone once I do the sequence "generate time-stamps" followed by adjusting the time-stamps.
Observation. When I was done adapting timings, I went to view lesson and listened to the content (with sync-ed text). I immediately noticed that the sync was on basis of the generated timings. So I returned to the "edit lesson" and found the timings the way I left them: in order. So I went back to "view lesson" and the timings were off again. Suspecting a cache problem, I ended the browser and started it again. Listening to the text with sync was now fully adapted to the new timings.
Hope this helps.
gbonnema

I am preparing lesson https://www.lingq.com/en/learn/fi/web/editor/12972728. The webpage is https://yle.fi/uutiset/osasto/selkouutiset/keskiviikko_1152022_radio/12441772. This lesson is currently private, but will be public as soon as I am done with it.
As usual, if the weather report does have an underwriting (grey bar directly below the image), then the importer skips the first paragraph. This is a known bug, at least to me.
After importing the lesson and adding the sound file, I checked the weather report (last section in the text) and inserted the 2 lines missing, making sure to do it before pressing "generate timestamps".
I briefly considered adding the lines first, and then adding the sound, but I wanted to know whether the issues were resolved, so I added sound first and then the 2 lines.
EDIT: after adding the 2 lines, I went to "view lesson" first to check whether the lines had been added Ok, and they had been, so I returned to "edit lesson".
After this, I did a "generate time-stamps" and lo and behold, the 2 lines were gone!
Next time, I will only add sound after the text lines are complete! Apparently, there is an after image after connecting the sound to the lesson that "generate time-stamps" uses to overwrite anything I added manually.
I really hope you guys will find the problem.
gbonnema

Today I did the exact procedure that I described in the previous post, but the lines I added were not deleted. The only difference I remember in the sequence of events is that I got the popup "successfully imported ... " before I connected to the sound file, and thus also before generate time-stamps.
I will stop reporting these small things as I don't think they really add to understanding the problem. My gut says this is a timing problem that typically occurs with asynchronous communication when the network is busy enough to re-order sequence of events and does not happen when it can easily cope. For that reason you guys (testers) may never see this problem as you are on top of the site being tested, while I am working from Europe and busy network times here are unpredictable.
gbonnema

Lesson: https://www.lingq.com/en/learn/fi/web/reader/12950681. The timings seem to be all there, but the listen-synced does not sync the text like it should. It does not emphasize the sentence that is timed. Could this be a different problem? Or is it related?
gbonnema

A weird timing difference. I am looking at lesson https://www.lingq.com/en/learn/fi/web/reader/12928912 where I access the sentence "Venäjän hyökkäys Ukrainaan helmikuussa on nostanut Naton kannatusta Suomessa." in sentence mode. When I press <a> I get the sound, but the timing of the sentence is not the same as the lesson editor or the sentence editor show. Those 2 are clean, while the <a> hardly pronounces the first character and the end overlaps with the next sentence (which is music).
EDIT: I also found out that the same lesson in sentence mode plays tty in stead of the recorded voice when pressing <a>, although in "edit lesson" it plays the original voice. Something messes up the sync and sentence mode voice vs the time stamps and original recording.
It could be a related but different error.
gbonnema

I feel like I shouldn't add to this unending series of posts, but I also continuously come across this error in different situations. I also feel like this one could help.
I forgot initially to add 2 missing sentences. So I did a generate time-stamps first, then added sentences that I had forgotten, then I generated time-stamps again.
Then I started modify time-stamps from the start. When almost done, I checked the listening sync and it seemed in order, until I got to the new sentences: they were gone. However, none of the modified time-stamps were gone.
The lesson was https://www.lingq.com/en/learn/fi/web/reader/12894153. The original page was https://yle.fi/uutiset/osasto/selkouutiset/lauantai_752022_radio/12435950.
Hopefully this helps a bit.
zoran

Thanks, your comments are helpful. I asked our testers to check your updates and provided URLs and try to find a way to identify the issue.
gbonnema

The import part of edit lesson is really broken. Today, I decided to import a lesson, and add 2 sentences that the importer skipped (reason unknown). After adding the sentence and doing a view lesson to check the result (they were still there), I quit the browser and switched to a different browser, both independent of each other (so separate cache).
In the new browser (chrome) I edited all the time stamps and went to view lesson to review the result. I also did a listen with sync to check that the time-stamps were ok. To my surprise I got a message in the middle of the listening session that the import was successful, which was surprising because the import I had done with a different browser.
The result is even more surprising. The 2 new sentences were gone. In stead the complete original file had been restored but leaving the time stamps in. As the time stamps were generated and changed including the new sentences, what happened was that the timings of the 2 sentence were imposed on the next 2 sentences and the last 2 timings were gone.
This utility is really broken: can you please repair it so that the import part does not get restored?! And also, please respond. Silently ignoring my reports will not make the error go away.
zoran

Our testers are looking into this. I'll let you know as soon as I have more information. Thanks for your patience.
gbonnema

Thank you, I appreciate that!
gbonnema

Do you have an eta? Today I had to redo everything again!
zoran

Our QA testers were testing this and they got back to me saying they were unable to reproduce the issue you reported here.
I am trying to figure it out on my end now. Can you please post a link to that lesson, where you said the timestamp for the first sentence switch back from 8 to 12.6? I want to try and test it in that lesson.
I tried testing in some of my imports but it worked fine, timestamps remained saved as they should.
gbonnema

I can tell you when the occurrence happens, and also state that it is not dependable. There are currently 2 courses where the overwriting happens and both concern changing time stamps and adding or changing sentences.
News
I use Finnish in simplified terms (uutiset selkosuomeksi) daily. It has audio that I can download and text without timestamps. Mark that I use radio only. The uri is https://yle.fi/uutiset/osasto/selkouutiset/. This is a generic url that gives the news of yesterday until local Finnish time of somewhere around 6 o'clock) and then publishes the new radio bulletin. I do the procedure every day in the same way:
1. I look up the page.
2. The audio bar contains an "i" button which I press to get to the link.
3. I open the link with description "Kuuntele Yle Areenassa" in a new tab.
4. I use the button "Laata tiedosto" to download the audio. Once succeeded, I close the tab. I get back to the news page (automatically).
5. On the rightmost is a column headed with "Edelliset lähetykset" where I click the relevant radio broadcast, in order for the import to get the right title.
6. For example today I clicked the link called "Keskiviikko 4.5.2022 (radio)".
7. The result should be the same page containing the radio text, but not the television video.
8. Now I import the lession using the extension (after filling in the tags "finnish", "uutiset" and "selkouomeksi". Also, I make sure it imports into the right course (which are gathered by month of the year).
9. After the import, I use the button "open lesson" on the import. Now I have the text, but no audio, so I go in "edit lesson".
10. I add the level "intermediate 1" and add the audio I downloaded. After a short while the phrase "Generate timestamps" appears.
11. Now I press "generate timestamps" and affirm the are-you-sure query.
12. After this, I open de first line (lesson title) to see the result and adapt the timing to my liking (which is mainly the music plus announcement of yle).
13. Then I open the second for the intro of the newsreader (title of the first set of paragraphs) and adapt the timing to start and end appropriately.
14. After this I adapt the timing of every next line. The procedure is always the same:
14.1 copy end time of previous line
14.2 truncate the end time to a whole number
14.3 listen to the result
14.4 Adapt the time if necessary.
15. I do this until the last line, then I share the lesson.
If you were following along for today's lesson, you should have noticed that the importer did not import the first line of the first paragraph. I added it manually, making sure there is a new paragraphj between the title and the new line. This error often happens when Yle make the first word of the first line blue i.e. an internal link.
When I get the green popup after editing for some while, or when done, then I know the timings were overwritten and my changes were rolled back to the point where only the generated time stamps exist. This time it happened while I was editing time-stamps. The sentence I added was gone. The time stamps were back to their generated time stamps.
P.S. I deleted this lesson today, because the title was wrong. I accidentally imported it from the television page. As all timings and additions were reversed, I might as well import again.
PS-edit: after importing the second time around (using the radio only page) the first line was imported correctly, very weird.
Politics
I often import lessons from the channel "yle rec". Here I have to join sentences to undo the damage caused by splitting up a sentence into subtitles. Also, I check the times by listen to the audio of each sentence under edit lesson. It is the same here: after receiving the popup, my added sentences are gone, and the original time-stamps are back. If I added a sentence here, the time-stamps will often be connected to the wrong sentences. I know the overwrite took place when the green popup comes with "successfully imported .... ".
I hope this helps in reproducing the errors.
EDIT: one more remark. If I get the popup "successfully imported .. " straight away, I normally don't get the overwrite. This however, I am not 100% sure of.
gbonnema

Today with lesson https://www.lingq.com/en/learn/fi/web/listen/12859102 which is the simple Finnish news page of this morning, I had the same problem. The original URL is https://yle.fi/uutiset/osasto/selkouutiset/torstai_552022_radio/12433335.
I had added 3 new sentences directly under sää (weather, last chapter). I knew this would happen, because the importer skips the first paragraph after an image, if the image does not have a gray bar below. I wrote about this before, but I guess it got buried after the beta started running.
I fully changed the time-stamps and started analyzing words in view lesson. I checked through synced listening and everything was fine. I had analyzed about half way through and went to do groceries. When I returned the generated timings were back and the 3 new sentences were gone.
I hope this helps in determining the problem.
I won't touch this lesson until you give the green light i.e. you are done analyzing.
gbonnema

The lesson today (i.e. news yesterday) had the popup for successful import way after the import and after the regenerate time-stamps and modifying the first few, but, the time stamps were not overwritten. I don't know whether the problem was found and fixed (was it?) or this was a fluke. In other words, the conditions for overwrite happening are not as simple as I would like it to be.
What I did notice was that the popup did not appear until I had ended the lesson to go to the overview, at which point I suddenly got the popup "import successfully ... ". It was hours after importing. But, no overwrite.
gbonnema

Again, today the same problem occurred in a smaller example, maybe easier to trace. I imported an article like always, added the audio and did a generate time stamps. I went to view lesson and played the audio with sync, noticing that the sync did not work. So I went to check the time-stamps: they were generated all right. Still, just to be sure, I modified the first ending time stamp from 12.6 to 8.0. Then through view lesson played the audio with sync and suddenly the sync worked! So I quit the browser (see my previous post in this thread - step 5) and restarted the browser. Again I played audio with sync and I was back at the original time stamp: 12.6 in stead of the one I modified (8.0).
There really is something wrong in "edit lesson". It looks like it saves the generate time stamps results, and overwrites all or some of the consecutive changes at free will.
Can you please find out what is wrong and correct the error? To be complete, the errors are:
1. The module saves and blocks every click following an update.
2. The module saves an after images of "generate time stamps" and uses it to overwrite some or all of the following time stamp changes.
3. The module does not recognize the generated time stamps in synchronizing the text when listening to audio. only after a change does it sync.
You also might want to tell us if and when you have solved one or more of these issues. The silence around changes is deafening.
gbonnema

Today, the problem reappeared in a different sequence. I describe the steps below. The lesson id is : https://www.lingq.com/en/learn/fi/web/listen/12541047.
I intend to share it, but for now (with the error) I am holding it back.
1. I imported the lesson (as normal and described above).
2. I did not do step 5 (quit the browser). So I stayed in the same session.
3. After generating time stamps, I realized the importer had skipped one line (which is a systematic error too, something for another bug-report)
4. I inserted the line where it was in the original article and did the "generate time stamps" right after. I did not change any of the generated time stamps.
5. I listened to the result checking whether all sentences were there and they were. I checked using the ability to scroll the text while listening.
5. I did a view lesson and started analyzing the words.
6. When done analyzing I listened to the result once more (scrolling the text automatically again). Everything was in order, the new sentence was still there.
7. I wanted to share the lesson, so I went into "edit lesson" mode and immediately noticed that the newly inserted sentence was gone. It was back to the situation after the first generate time stamps.
8. I checked the listen lesson with automatic scrolling and the sentence was really gone. Time stamps were as if the sentence had never been there.
Can you please find the cause of the problem and fix it?
P.S. Tell me again why these automatic saves are a good thing? I keep forgetting and I only see disadvantages! Even changing time stamps is too much of a bother with these cruel "let's save after every other click and force the user to re-cliok, again, and again, and again ad infinitum" . Yes, this annoys me, to no end, every day!
gbonnema

What I do is as follows:
1. I import a lesson. It has text, but no time stamps, adding tags and the right course from the import add-on.
2. I open the lesson from the import add-on "open lesson".
3. I execute "edit lesson" and add audio and level.
4. I wait for the "generate time stamps" to appear and activate it.
<now it generates the time stamps and eventually the button "generate time stamps" reappears>
5. At this point, in order to prevent problems, I leave the browser (chrome) in order to clean the cache.
6. I re-open the browser, open the lesson and start editing time stamps.
7. After analyzing the text, I play the audio and the time stamps are in tact, no problem.
Yesterday I forgot step 5 (I did not quit the browser). What happened was that the time stamps up to a point were back at the values they had before I had changed them. Of course I had not pressed the "generate time stamps" again, that would have been weird, because I edited them.
I also noticed that the popup claiming that the lesson was successfully imported only appeared when I was half way through editing the time stamps, which was just about the amount of time stamps that were overwritten. The time stamps after that point were unaltered.
I discovered by accident that quitting the browser remedied the problem.
EDIT: today I consciously omitted step 5 and weirdly enough the time stamps remained in order (i.e. after generate-modify they stayed as I modified them). Also, the popup "successfully imported" appeared immediately after importing and opening the lesson (contrary to the example I gave above). It looks like some process has a stale before-image saved and uses that when giving the popup thereby overwriting changes. Just my perception of what is happening.
zoran

That's strange. After you make edits to timestamps, are you saying that auto timestamps generate by itself and re-write your edits, even without you actually click on the "Generate Timestamps" option on the top left?
What if you edit timestamps and then click on the "View Lesson" button after you edit them all?