Nice sized booth! Spraying on the patio should need nothing more than a particle mask for you.
4" is a bit small when you move inside, though. That will move less than 40 CFM.
For the size of your booth, I suggest a minimum of a six inch outlet.......(many, many fan options) which will move over 100CFM.
Many people are particular about particulates! Listen to advice and wear a mask if you are worried. But a six inch (round pipe) exhaust will move all of the air you need moved. Buy a fan that moves at least 500 CFM and you should be fine.
Unless you're painting a full-sized A/C....MY opinion is that you will have no fumes.
I prefer venting from the top of the booth for ease of installation.....but down draft is great if you can do the work!
HTH