I believe most of the times I have seen this type of fan used, the intake side/hole was affixed to a "connector/seal" made out of metal, wood or foam to bridge and seal the gap between the cabinet and the fans input hole.
This would assist in keeping the fan in a straight orientation to the cabinet. Then, the output flange would be attached to the duct work.
Your cabinet sounds like a winner, already