I assumed someone would bring up the weight differences so I went out and looked. For an '03 SE with automatic they are:

4X2 3857 lbs
4X4 4156 lbs

So the difference is right at 300 lbs, or about 7.5%. And the difference of 5% gas mileage (20mpg vs 19mpg) could easily just be due to the weight difference. The numbers seem to show it now that we look at it.