I've always done it, but I think it's one of those things where if you THINK it matters, it does.
Mike Venturino wrote an article about shooting with Clint Smith at his school with .308's on steel targets at ranges out to 700 yds. Clint loads unsorted range pick ups with a run of the mill powder measure, occasionally checking it's accuracy to plus or minus a couple of 10ths of a grain.
Dave Scovil of "Handloader" magazine occasionally reports on a friend's accuracy shooting one hole groups using .45 ACP range pick ups from a ransom machine rest. I stopped worrying about it after that.
My wife, on the other hand, insists that I sort out her Star-Line and WW brass from my Norinco, GI military (yes, I remove the crimp), Federal, Sellier-Beloit, etc. Why? Because she once had a failure to go into battery with some of my mixed brass which turned out to be an error in seating depth, nothing to do with the brass. Lately she has deigned to shoot my mongrel brass with good results in her Ruger SR1911.
Another pet peeve of mine is the belief that you need to trim your .45 ACP brass to uniform length as the cases MUST head space on the case mouth. In "Hatcher's Notebook", he talks about shooting .380's in 9mm pistols and other such combos, getting acceptable accuracy if not function. The extractor holds the cases in position sufficient for a primer strike and the round fires.