Does the sanding drum mandrel have a screw head on the end of it?
Normally with the Dremel drum sander mandrel, there's a little slotted screw on the end of it, and when you put the sanding drum on the mandrel, tighten the screw, and the rubber mandrel expands, trapping the drum and making it impossible to get off without undoing the screw. Without tightening up the screw, theres no friction at all between the rubber and the drum, so it'll never be able to be used correctly.
Cheers,
LeeTree