When boarding a vessel at sea what is the best weapon regarding ricochet?
9mm MP5 or M4 14.5" ?

Have you ever been to sea?  In any normal seas?  Smaller vessels?  Stainless firearms, or teflon coated (Birdsong).  A pistol grip extended mag shotgun is your best bet.

In fact, a pistol is better than a long gun, will let you have one hand available to grab handrails, etc., or just hang on.

Calibre is the least of your worries.

having been on a few boardings in my day, I think the optimum is a mix of .45s and shotguns with  buckshot.  MSC used to teach counter boarding tactics with Ithaca and Remington pump guns.

Small craft (yachts, etc) are made with fiberglass and wood penetration with just about anything is a given.  Ships are made of steel and sometimes aluminum, ricochets and not penetration are the order of the day.

Challenge #1 is getting on, #2 is getting in the skin of the ship.

The MSC chartered freighters that shuttled between Guam, Philippines, Singapore, Diego Garcia used to have gratings that were locked down over the outboard ladders to prevent getting off the main deck and the water tight doors were locked from the inside.  Anybody that successfully was able to catch up, come alongside and some how get up a ladder or rope couldn't get in or get to anybody.

Other ships used similar methods.  Boarding a ship that uses more active countermeasures will really get tough.  Boarding a cooperative ship underway in anything more than a dead calm is damn dangerous. Boarding one that doesn't want you to come aboard is close to impossible without using force to make them cooperative.

I was gonna say something in 12guage.

MP5

of your choices, I'd pick the MP5.

BUT, in so doing, I'd assume we're talking about trained professionals.

What? No Weekend Warrior boardings for you?

No, I was watching the movie Navy Seals and they were using MP5SDs. I got to looking at the walls of the ship- thick welded steel- and I just started thinking about the ricochet factor.
In that case I think the sub-sonic 9mm rounds would be the best choice of a few bad ones.
I wouldnt want to miss either.

"Challenge #1 is getting on, #2 is getting in the skin of the ship."

EXACTLY!

In any but the smoothest seas, getting on takes utmost cooperation between BOTH vessels.