The Browning X-Bolt looks (and feels) like a sweet rifle. Haven't owned one, though, which is the real acid test. The 26-inch barrel they are putting on the .300 and .338 magnums is just a little too long for my taste.
The .338 is probably a better choice, as suggested (unless you just want to collect a variety of .30 caliber rifles)! The .338 will do the job nicely, and expand your capabilities to include brown bear as well as elk. You can load it to different velocities with lots of different bullet choices and weights, depending upon the game you are hunting.
If you really like your .30-06, you might think about trying Hornady's new Superformance ammunition for the relatively few rounds you will be sending downrange at elk.