The temperature sensor for the mirror is mounted in front of the radiator.
http://www.xterraparts.com/Merchant2/graphics/00000001/inst_ecmirror_04.html
Should give a fair reading if the vehicle is moving, but when you stop it will pick up heat from the engine or pavement.

dave and xtoy - better I don't know