I've never had a problem with Vallejo's grey primer, so my first question would be what color are you using? I've heard white is a nightmare for some reason. Second, what are you using to clean? I've invested in their airbrush cleaner, I thin that half and half with water for an 'inbetween' shots clean and strait for a 'finished for the day' clean. It will tip dry, so you want a bit of a cleaner (whether their brand or not) and a q tip to clean the nozzle tip as you go. I thin the primer all the time, about fifty fifty with their brand of thinners (I find water makes it too wet) and have had no ill effects. The only drawback to their primer I've found is that it does not like sanding or remedial work.
At the end of all of my priming work, I put the brush through an ultrasonic cleaner, a bit of an investment (about $80) but well worth it.
Also, to add to the notes above, alcohol and Vallejo paints are a huge mess, I've never had a problem using windex to clean my tools. I find it's not as effective as an ultrasonic or their brand of cleaner, but it does work for me.
Hope that all helps, and if you're using a different color of primer, give the light grey a try, it's always worked like a dream for me.