The modern walls are just breeze (cinder) blocks covered with cement. Older ones are hand made bricks, again covered with cement. That is usually then whitewashed, but it doesn't last long in the heat, and peels off, usually from the bottom up. The lower half gets dirty quickly, so there's an excuse for wall weathering!
In model form, whatever you can find for the basic shape, then cover it in filler & paint it.
But if you want to show patches where plaster has fallen off you'll have to replicate the bricks or blocks somehow. What about some scribed particle board?
Pete