I have seen it done . You may guy your fish streamside but I dont throw the guts in the water. Rather I either put them back away from the creek or under a rock on the shore. That way the coons or other critters can find them.
I could be wrong, but I don't believe that it's legal to do till you are home/camp. I did it when I was younger, and see it done every year, but too many people just throw them anywhere, and that looks and smells really bad, especially if you are a guest on someones property.
FIELD DRESSING AND DISPOSAL OF FISH – It is unlawful to possess a fish in any form or condition other than in the whole or having the entrails removed while on shore, along the waters of the Commonwealth, onboard a boat or on a dock, pier, launch area or parking lot adjacent thereto. Fish may be processed fully if they are being prepared for immediate consumption on site. This does not apply to fish processed at a fish cleaning station officially recognized by the Commission or by a permitted charter boat/fishing guide operation.
It is unlawful to discard any fish carcass or parts thereof into the waters of the Commonwealth or upon any public or private lands contiguous to the waters unless disposal is on lands with permission from the landowner or it is where fish are properly disposed into suitable garbage or refuse collection systems or at an officially recognized fish cleaning station.