I thought if you already knew it that it couldn't be removed from you. If you just heard it in a spell of something it vanished instantly.