Thus the paradox of pastels...
You didn't do anything wrong. In fact, I would guess that nearly every modeler tries shooting dull coat after they've applied pastels and discovers what you did.
You really have two options with pastels. You can go for a subtle effect and apply light coats of various pastels to your model. This is usually done over a dull coat because the pastels adhere a little better. Then, you leave it alone...don't touch it, don't spray anything on it, just let it sit and admire it. Eventually, the pastels might start to fade or be blown away or rubbed off. If needed, you just apply a little more.
Your other option is a more severe effect. You can dampen your brush with a little (or a lot of) alcohol as you apply the pastels and they will adhere to the model almost like paint. Unforetunately, it is very easy to over do it and end up with way more pastel coloring than you were looking for (I've done it). When you apply the pastels in this way, over spraying the model with dull coat won't have nearly the same effect as dry applications do, but you will still see a slight moderation of the pastels nonetheless.
Practice with both of these techniques and you will eventually discover the right combination for you. Nothing is funnier than watching a bunch of modelers at a serious competition as they hover over their models with a make-up brush in one hand and a bottle of pastel dust in the other. Funny, but also very easy to understand. Nothing will bring an armor model to life quite as much as that final light pastel dusting.
Hope this helps.
Enjoy your modeling...