I saw this and i am a native speaker of Swedish and i had to answer. I have no specific grammar training more than any other Swede and i speak what is referred to as "common" Swedish with no specific dialect (i am from the Stockholm region).
When speaking casually with friends i would respond
"Nej det är dom inte"
and when we say it fast it actually is more pronounced "Nä deärom inte". "Deärom" is not a correct word, but to speak faster we drop som letters and that is more of how we pronounce it.
Swedish people in general have a hard time differentiating between "de" and "dem" thats why we commonly just use "dom" to safeguard us against both in spoken language. A lot of people write "spoken languge" thats when "dom" sneaks in, in written language, but this is considered bad practice.
I was taught the following in school when i was young:
If you don't know when to use "de" or "dem" try replacing it with "jag" or "mig".
Is "save a piece of cake for them" called:
"Spara en tårtbit åt de?"
"Spara en tårtbit åt dem?"
If i want to save a pice of cake for myself i say "Spara en tårtbit åt mig" and "mig" is the same form as "dem". So it is "Spara en tårtbit åt dem?" that is the correct version.
Placing this context in your example it will become:
"Nej det är jag inte" and "jag" is the same as "de" so correct Swedish grammar would be
"Nej det är de inte"
But this was what i was taught in school as a 10 year old kid.