You may have used too much glue, which would leech out into the water, poisoning the fish, and would simultaneously have weakened to plastic to the point that, instead of creating a hard bond, you created a soft one, and buoyancy just took over after a few weeks, tearing the pieces off and sending them to the top.
My suggestion would be to place in your fish tank only items designed to decorate the tank, and even those should be carefully cleaned per instructions. Fish get their oxygen from the water. Toxins leeching into in the water will poison them just as surely as large open vats of toluene-type glue in your home will eventually overcome you. And goldfish are hardier than most.