There's really no reason to "leave" a non-group conversation, sans being PA.

You ought to have two sections, and only have "leave" enabled in the group tab if possible. And in there you can have "Leave", "Clear History", and "Leave+Clear History."