Hmm... I haven't encountered anything like that yet.. I have rock band unplugged also on my PSP downloaded and i always hit the home button to quit the game most of the time but didn't have any problems with it except i can't download songs from the music store.. Every credit card i enter it keeps saying invalid information, some crap like that..

Why won't you try deleting the rockband.cso file when you hook your PSP up on the computer then just re-copy it, see if it'll work again..

Goodluck!