RazoBlade;
It may be as simple as you have a leak in the seam of your vacuum chamber.
You could try and re-seal all the seams inside and out with a caulking gun or similar item. Once this has set run a small bead of liquid dish detergent along the outside edges and seal the top of the box with a plastic wrap or cling film depending on the overall size. Use Duct tape to secure all the edges tight to the box. Turn on the vacuum and see if any bubbles form within the centre of the box. This will give you and idea of where the leak(s) still remain.
It may very well be that you have built too big of a box. Remember it comes down to how many Cubic Feet per Minute you are trying to remove from the box. If re-sealing does not work and you are going to re-build your box shorten the height of your existing box by half and keep the same outside dimensions. This should increase the draw level of the shop-vac as there is less air to remove.
Hopefully this will help you out and get you on your way.
Cheers;
Gregory