Variables for improving group size (in no particular order of importance):
-Round ball diameter
-Powder granulation (2F vs 3F, primarily)
-Powder charge volume (70gr, 80gr, etc)
These can all bear what the gun produces for group sizes. Once you have that determined, then you can work on the sights.
You'll often find that the right combo for improving group size also starts to correct the point of impact for you, as well.
I shoot 120gr of 2F in my 62-caliber rifle. It's not because I want the thing to be a cannon...it's because that's where I get good groups and sights hitting as I want. The groups aren't noticeably better at 120 than they are at 90, but at 90 it's not quite at point of aim, where it is nicely hitting point of aim with the charge at 120. Since we're in the 21st Century and not the 18th Century, I don't have to be terribly concerned with conservation of powder. So...120 it is.
That particular rifle is a long, beefy rifle that was made specifically for me. Result is that charge produces very manageable recoil for me with that rifle.
If you're shooting full bore projectiles, such as maxiballs or PA Conicals or similar, you have little to work with besides powder charge/granulation.
As a GENERAL rule of thumb, shorter barrels will run 3F better than 2F. Smaller bore sizes will run 3F better than 2F. Longer barrels/larger bores....2F over 3F.
GENERAL rule of thumb. And there are always exceptions.