I had a Cherokee prior to my X.

Most my off-highway driving is on forest service roads and some deactivated logging roads. I enjoy exploring the old roads and tackling some tougher stuff, but this constitutes less than a quarter of my off-road time.

There were times driving my Cherokee on the rutted, pot-holed forest service roads when I thought the bloody truck was going to fall apart, with all the banging and rattling. I find that I can drive my X on those same roads at nearly twice the speed with less than half the rattling. Two things contribute to this. One, the general quality of construction of the vehicle. Two, IFS. This is one of the few conditions off-road in which IFS is better than solid axle, however these conditions constitute the majority for me.

Leaving aside reliability (which definitely favours the X), my thoughts as to the better vehicle:

General off-highway driving: Xterra
More gnarly rock-crawling, etc.: Cherokee (altough you would want to mod it up a little...)

Once again, it all depends on what you're using it for...