Imagine if every French speaker suddenly agreed that nouns were one of 'animate' and 'inanimate', or 'chocolate' and 'strawberry', or 'A' and 'B' instead of 'masculine' and 'feminine'. The language could go on being used identically to how it was before.
Given this it's not obvious why their grammatical genders are 'masculine' and 'feminine' in the first place. Are these just arbitrary labels applied by ancient linguists to the different noun forms after they came into use, or were the words we use for different grammatical genders generated naturally (and perhaps obviously, to them) by people who spoke the language?