My issues with not priming are two:
1) If I don't, it seems like I always miss some bump or dip or other flaw in the surface, which I then have to fix and paint over again anyway. Nothing seems to scream "fix this!" like a coat of light gray primer.
2) While superglue generally takes paint without that part of the model looking different or funny under the final finish, different kinds of putty and filler can give you uneven results which can range from "Nobody's going to notice this" to "My God, what was he thinking?"
Bottom line, while priming may be an extra step, I think that most of the time, you end up saving time in the long run.