I spearfish off of Jacksonville and also bottom fish and enjoy both. I find it more difficult to spear than to bottom fish. There are a lot of spots that i see diving where the grouper and snapper are very weary of divers and swim out into the sand well before you get close enough to take a shot. If I were to fish these places I don't have any doubt I would be more successful.
I think a lot of the divide between hook and line and divers is due to disrespectful people on both sides of the fence. I have had fishermen pull up while divers are down and drop right on top of them just as I have heard a lot of stories about disrespectful divers. Bottom line is that there are a few bad apples in every bunch.