It's called a fish market but they sell all manner of seafood.
We went there near closing time so unfortunately most of the shops were closed.
However, we did manage to try some oysters (fresh ones and ones with cheese in them!) and some other stuff.
The sign says it all - Sydney Fish Market.
Bad lighting in the picture but the bridge in the background is kinda cool.