I've got 3 brothers. I wouldn't lend out anything a but a beater to 2 of them.
But in the OP's situation, I'd simply say that I want to be the one who pops this rifle's cherry, but I'd be more than happy to lend out a different gun.
I'd try to make it clear that I trust the guy asking, that I like him, and I'm willing to give as well as receive, but there are some exceptional circumstances. So I'd invite him over to take a look at my gun cabinet and pick out the rifle he'd like to use, but I'd remove the ones that were off limits before he got there.