The length of the tang and the position of the front trigger guard screw is different. The old hole would need to be plugged and a new hole and inletting made. The internal inletting will need to be opened up too. Carve it out and glass bedding the receiver might be one way to go. Also the receiver is slightly different profile.
Barrel inletting will need to be opened up as the No.4 barrel is fatter and has a slightly different nocks form profile and receiver ring.
It can be done, but much whittling is involved for little if any gain over a No.4 stock.