I can read the script and understand as I listen

I can read the script and understand as I listen but I cannot understand at normal conversational speed - should I go down to Intermediate 1?

I have this problem too in French. I guess the solution is a mixture of material. Choose easier material for listening or listen repetitively to material that you have read and understand well.