I say it is a big factor, but not as important as the airbrush itself.
I have a $140 unknownbrand compressor, it works just fine... oh and its diaphram. no inconsist finishes here!
I have used a $250+ iwata power/sprint/smart jet, and its not much different
A good thing to invest in is a Moisture trap and a Pressure Regulator. I dont have either... Moisture trap, MikeV can you use Automotive types? Pressure regulator, i use the welfare style: pinching the hose with a clothspin held on with a ribber band. its aztek.