I commonly calculate the precision and recall for information pulled from text but I'm not sure how to calculate the margin of error for those precision and recall values.

So for example, if I have a sample 1,000 given names out of an unknown amount of data. My goal is to indentify the gender of of each name. My system does its magic and I am able to determine that I am assigning gender with 90% precision and 90% recall.

How do I calculate the margin of error for the precision and recall given that the values I've calculated are over a sample of data?

I've found a formula for the Maximum Margin of Error on http://www.had2know.com/business/compute-margin-of-error.html

but this doesn't seem like it would really apply in this kind of situation.

  • 3
    It's an interesting question but I feel like even though your experiment is based on linguistics, this particular question might be better served on Cross Validated, the SE for statistics.
    – acattle
    Apr 16 '13 at 1:27
  • 2
    If you want to have a margin of error on those precision figures you would need to run the test on multiple samples. I agree with @acattle that you'll get a decent answer on Cross Validated, but don't feel shy asking here.
    – user483
    Apr 16 '13 at 3:07
  • I've posted this question under Cross Validated now as well here
    – mmoosman
    Apr 16 '13 at 14:32
  • @mmoosman: Precision and recall are well-defined terms used often in corpus linguistics. These terms may not have that meaning to other kinds of statisticians. You'll have to elaborate on how these terms relate to True Positive, False Positive, etc. I mean on the CV.SE, not here.
    – prash
    Apr 19 '13 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.