One idea for rubble: every couple of weeks, I thoroughly sweep my work bench into a dust pan. Pull out the dog hair, and what I'm left with is a nice pile of random scrap. Bits of PE, plastic, resin, wood shavings, twisted material from pin vise drilling, etc. You can add this to the material, mentioned by Hans, and end up with some nice, nonspecific, rubble detail.
For a church, adding lots of scrap wood to simulate destroyed pews and alter, colored transparent PVC (I think) for shattered stained glass, sawdust...
If you're really ambitious, you could try for a destroyed pipe organ.
Steve