My partner got me into it, now I'm hooked.

Hi! 
I'm Star and I'm a disabled nonbinary artist, creating art with my partner for our Etsy shop. 
One of my main items that I enjoy making is these little rock buddies I create from polymer clay. I like to give them details like rock textues and little crystals embedded in them and little critter friends. Some sit on your desk or on a shelf, some sit on little bases with stumps or such. 

My partner is really into model kits, miniatures, etc, and has gotten me into all that plus scratch building and kit bashing. She's awakened the wannabe hoarder in me. Oh no!

But now I'm playing with flocking and grass and other terrain materials to add to the little rock guys I make. For my current project, I made a base out of polymer clay spread over a wood circle. The clay is baked and I am trying to add flocking and grass. I have tried Elmer's Glue All and Aleena's Tacky Glue and I find the flocking comes right off after it's dried. 
I tried super glue and that works, but it is expensive for covering all the base. 

I'm wondering what I'm doing wrong or how to do it right. 
I don't know if it's because my base is polymer clay?
I see others say use the glue all or tacky glue and it's fine, but it's not working for me.

Looking for advice and to learn more about this hobby.

Star