Well, there are several considerations:
1 What is your research question?
You always make a corpus with a purpose. So you design a corpus that is useful with respect to your research question. When you want to analyse the language differences according to social class, make sure that you have balanced amounts of material assigned to speakers or writers of different social classes, when you want to analyse diachronic development of language, be sure to have some diachronic depth of the material.
2 Think of FAIR principles from the beginning
The FAIR principles revolve around the re-usability of the corpus. It needs a good description with metadata to be findable, and it needs a suitable licence to be re-usable. It may be necessary to choose texts according to the licence because altering licences of texts is often difficult, time-consuming, and not guaranteed to work at all.
3 Think of the format
The most important thing is that the format is sustainable over a long time. Plain text obviously works, the vertical file format (vrt) of the Corpus Query processor (cqp) is a very simple XML format, TEI is a sophisticated XML format that can do at lot of detail. csv is a viable alternative to that, and there are some more formats outside, you may chose whatever fits to your tools chain. Refrain from the file formats of commercial word processors (they may change or become obsolete without prior announce).
4 Think of a repository
It is probably a good idea to plan for depositing the completed corpus with a specialised repository. There are some criteria that you may want to apply in the selection of the repository, like
- Has it undergone some certification like Core Trust Seal
- Does it belong to larger community, like CLARIN
- Is it based on a research institution or is it commercial
- When it is commercial, what is their buisyness model