I think anyone who claims to be part of any ideological group, especially those advocating social / cultural change, should clarify that the movement doesn’t have a centralized membership roster that they control. You can’t excommunicate someone who claims to be a feminist, even when they represents feminism in a distorted and/or immature manner due to ignorance of the true points or any other reason.
It’s very understandable that men doubt feminists, because sometimes what they mean isn’t totally clear in their expressions. I seriously think feminists need to be impeccable with what they say, or else the people who hear them may misinterpret what they mean. This is why the “not all men” argument loop comes up so often, because women assume that people know they’re not talking about all men, but seriously… I did not. When talking about subjects such as subtle sexism and/or racism, you really need to be exact about what you’re talking about.
You also need to be patient and willing to answer questions, even if they seem a bit hostile, since you are choosing to represent an ideology and people may be threatened by. People with this impression just need you to represent that it’s not militant or hostile. You can clarify that of course at times you’re going to be angry, if you are a person that is being marginalized. When it becomes clear that you’re not talking to an open minded person, you can choose to walk away, but give some people a chance if they seem to be asking questions neutrally and with sincere curiosity.
Telling people that it’s not your responsibility to educate them on subtle cultural issues and their effects, right after you claim that they exist and are very important, is how you get people to distrust and disrespect you.