Depend upon the ship and era.
The modern smaller combatant vessels--FFs, DDs, CGs--use a modified DC/MD-10 gas turbine fueled by Navy Distillate #2. Which has a passing resemblance to kerosene--for far too much info, see here: en.wikipedia.org/.../Fuel_oil) That fuel can also be used in the various diesels used, too.
Support vessels and large combatant vessels run on steam generated either by bunker-fired boilers, or fission powered heat exchangers.
We refer to the storage spaces as bunkers as a legacy of the time when coal was the power at sea, and it was bunkered in spaces formed between the ship's frames. The tankage is still formed by closing up the voids between frames, and plumbing them to the engineering spaces. (some of the frames are doubled or trippled as needed.
There are some very good stability reasons to use skinny tanks wrapped around the ship--much too complicated to go into here.
After much experimentation, USN has gotten out of the combined (hybrid before it was cool) engineering strategies. While a good concept, it has a flaw in needing a much more complicated set of reduction gears to connect the various powerplants to the propeller shaft(s).
Turns out that CODAG is very handy in patrol craft, though--you set up a diesl on the centerline, and run a pair of gas turbine plants on both outboard sides. Any drag from the turrbine running gear is offset by the cruise efficiency from the diesel plant. (Or, you rig up a set of disconnectors at the reduction gears and let the idle shafts freewheel.
Marine diesels have improved in amazing amounts in the last few years. Most will will now run on raw crude (which is a tad easier when the cylinders on a in-line marine six-cylinder have a diameter near 1 meter, with a 2 or 3 meter stroke supplying plenty of compression).