What rifle(s) do you currently use for groundhogs?
There is quite a difference between the .17HMR and the 22-250 but each can be excellent groundhog killers depending on the field conditions. If you hunt around dwellings and your shots are under 150yds then a .17 would serve you well, especially if you already have a larger centerfire rifle. If you don't have a bigger rifle and you can shoot at extended ranges, and/or this purchase will be your primary GH rig, then the 22-250 would be a good choice.
I couldn,t have put it better myself.Plus if noise is a issue for you and you know your shot distance will be in the prescribed distance then the .17 will shine cause you don,t even need ear protection and their is no recoil whatsoever.
Honestly, I'd opt for a nice varmint weight .17HMR in your situation. As long as you are able to close the distance to target to under 150yds the kills will come quick and quietly. The .17 is one of my favorite woodchuck rigs and has allowed me to take several woodchucks from one position, back to back, without the neighboring chucks being any the wiser. The neighboring people will be happier with the lack of noise as well.