Most often I hear that only birth languages, those languages first learned, are native languages to the speaker. Seems like a pretty lazy way to define if a speaker is a native speaker, since to speak a language you've got to speak it, and just speaking a language has no direct relationship to how well you speak the language.
Case in point, say two non-native speakers have a child, and decide to only speak the native language of the area they're in with the child, but are not native speakers of that language. The child, if only talking to the parents and other non-native speakers, would never become a native speaker. While extreme, it does make the point that birth languages are a poor measure of if a speaker is expressing themselves in a native tongue.