I agree with others that loud noise is a big factor. Get good hearing protection and let him listen while you shoot. The Limbsaver recoil pads do wonders as well. Winchester was making Low Recoil/Low Noise AA loads in both .410 and 20 ga. If you can find lite loads and, again, let him watch and hear you shoot them first, that may help. If he's still not ready, be happy to let him continue with the .22 and maybe introduce him to .22 shot shells for added fun. At that age the priority is to have him enjoy shooting whatever he wants and is ready to shoot.
The older I get, the stronger gravity becomes.