That depends on the particular brand of white spirit. As a rule, pure white spirit theoretically does not damage acrylic paint at all, but of all white spirit brands I've used, none of them were pure; they do not damage acrylic paint by themselves, but if you start to wipe excessive wash out, you have only limited amount of rubs at one place you can do untill you'll start to rub off acrylic paint. Very annoying. You have to try it for yourself, on some scrap piece, before starting on a model.
Currently I use two things for oil washes - a mix of white spirit and lineseed oil, which does not damage acrilycs at all; however, it cures forever and then a week more. On a plus side, you can experiment with it as much as you want without fear. On the minus side, after you are satisfied, you'll have to leave painted piece somewhere where it wouldn't be disturbed by anyone for at least a week, or else.
Second mix is something which is called "pinen" - as with the first mix, I've bought it in the artist's shop. It cures fast, and allows a lot more rubbing than all the brands of white spirit I was able to locate here. A bit too fast for my tastes, though. :)