Aren't the Greymanes the pro-stormcloaks? The Battle-Borns would likely be tolerant of Altmer visiting Whiterun, unlike the Greymanes, unless im mixing them up.