Did you ever swap the front and back tires?
Take the wheel spacers off your front tires and see if it goes away. If it does then keep on reading.
I was asking around the office about this and one of they guys thinks it's because your wheel spacers aren't hubcentric. Here's how it was explained to me:
A stock Jeep has a lip on the hub that the wheel sits on. The lug nuts don't keep the wheel centered, they only provide tension - the lip centers the wheel and supports the vehicle weight.
Your new spacers don't have that lip (they are called lugcentric). The lugs not only center the wheel, but also have more sheer force applied to them, since they now support the weight of the vehicle. There is probably a small amount of play, so every time you remount your tires there is a chance they could be in a very slightly different position.
You may want to check out some hub centric spacers, assuming that your rims are hubcentric, and can make use of this lip. They should keep everything lined up a bit more.
In an effort for full disclosure, it appears that there is debate if Jeep rims are truly hubcentric or not - the jury appears out on this.