Michael's and AC Moore arts & craft stores have supplies that we use--brushes, paints, putties, plus lots more things that I can't think of, but you need to go there are just browse, and you get ideas.
Check your local phone listings for hobby shops. There may still be some good, old-time hobby shops in your area, and it's worth it to check them out.
Hardware stores, too, have some supplies that we use. Most carry Dremel or their own version of a rotary tool, with accessories, and you can get things like mineral spirits and acetone, and good primers, at better prices than you'll pay to a hobby store for much smaller quantities.
Rite-Aid, Walgreen's and other drug store chains, for isopropyl, unless your grocery store sells it (I couldn't find it at the Giant).
Try model shows, and also, tool or woodworking shows. Most shows have vendor areas, and you're bound to find someone selling supplies there, too, and you can often find things at bargain prices. Flea markets are good, too. I've picked up stainless steel dental tools, useful for sculpting/scribing, at a flea market, I got three or four picks for $5, and a hobby knife set, like X-Acto, that MicroMark sells for something like $20, for $5, included a couple of handles, a variety of blades, all in a little blue plastic box.
Online, MicroMark is a good source of tools, but you might find the same things elsewhere, cheaper, and you're paying shipping & handling, too. I also buy some paints directly from the manufacturer, rather than through a vendor. I buy Tamiya's aircraft paints in the rattle cans direct from TamiyaUSA, because my local HobbyTown doesn't stock the colors, and by the time they could order them for me and have them delivered, I can finish the model (weeks vs the couple days it takes Tamiya to deliver direct to me).
If you can, I'd also seek out and join the nearest modeling club, if there's one close by. The Internet is great, but live, face-to-face meeting with other modelers can't be beat. And just as with shows, most clubs usually have raffles, or swap meets, or generally allow their members to bring things in and sell or trade.
Hope that helps!
Best regards,
Brad