Long shot here, but if you lifted the back more, you just threw your caster off in your front end. You'll have marginally more wind resistance on the roofrack, and particularly the basket, but I wouldn't think it would make that much difference.

Another dumb question - are all your tires up to road PSI?