For audio restoration work, Audacity won't cut it with its built in tools. It might get you decent results....but that depends upon how good you want it. You will want additional plugins.
Cool Edit Pro 2.0 (13-15 years old I know but has a very good noise reduction plug built in) has been one of my go to editors for many years just for that purpose (plus it's cuts and fade algorithms are excellent even by today's standards).
To do what you want, I'd recommend a good editor with plugins that allow you to readjust where the "center" is so you can adjust where in the audio field the main source audio is (in your case, the speaker). Then, use a side adjust tool to balance and reduce the crowd noise so that you can clear up the intelligibility of the main speaker. From there, export to two track (or stereo interleaved) and expand the center so that it doesn't disappear in certain listening environments. EQ, compress, and balance after that to make it perceptively louder and more balanced. And this is the basic stuff.
Sorry for the wonkiness.....post production audio is something I do to support my BRD......