For a rifle, I'd go with the 54. For several reasons.
The 54 is large enough to carry well and buck wind without being an overly heavy gun to carry in the woods. Once you get up to a 58, according to the stats in the Lyman Black Powder handbook, once you get to a 58 you lose more than you gain in muzzle velocity and energy. In addition, a 58 caliber requires sufficiently more barrel wall and heft to start getting too heavy for some folks to carry all day. I agree with alot of folks who say that velocity and energy isn't everything. A 58 caliber hole has a big advantage in shock and blood loss. But even with modern steel, you can only put so much powder behind a prb and stay within the safe range.
How much more will the finished gun weigh? If there is no difference, go with the 58. Even with the lower ballistics for the 58, most long hunters have open sights and are hunting under 125 yards. At that distance trajectory can be adjusted. Beyond that distance, open sights have difficulty even drawing a bead on a white tail's boiler room.
So, if there is no weight differential, go 58, if 58 requires stepping up to a heavier barrel, stick with the 54.