The bolt catch is spring loaded and will be pushing up, causing the flanges on your retaining pin to catch on the edges of its hole. Press down slightly on your bolt catch overtop of where the pin is located while you attempt to remove it. I had a bit of a ##### of a time with this too initially.
Tip: If you're using your factory Swiss spring, the bore for it on the NEA lower isn't deep enough. This will cause your bolt catch to have excessive spring pressure (which by default keeps your catch in the down position) which will prevent it from catching your bolt when using certain magazines (in my experience, all magazines except LAR-15 mags do not exert enough pressure on the catch to get it to engage with the factory spring on the NEA lower). So your choices are:
- Put up with it.
- Trim your factory spring.
- Use a different spring.
- Ask NEA to send you an appropriately sized spring
I took the spring out of a disposable lighter and cut it to size. Functions perfectly now.
When installing the parts on your lower, the bolt catch along with the mag release will be the first parts to get installed. Do a function check before you install the hammer, trigger and safety selector. Does the mag follower push the bolt catch up? Or is the spring pressure so great that it doesn't move and in fact prevents the follower from rising? Better to fix this now before you get all the other components installed.
Edit: I just read that you've already installed the other parts into the lower. You may find that your hammer spring gets im the way while installing the bolt catch retaining pin.