I would say the most physically competitive sport would probably be Rugby League. The hits those boys take are literally bone-shaking.
Nothing compared to American football. A study a few years ago found that in an average NFL game, there are something like 34 collisions that have the same impact on the body (i.e. net of the pads they are wearing) as being hit by a car at 30 mph. The next highest sport (this excluded boxing) was just under 10 (rugby union, from recollection).
For physical toughness, narrow margins of error, profile, pressure on the athlete, etc, I'd go with the NFL personally.