It came about like this. (Details about Greek here)
When Greeks adopted the Punic abjad into an alphabet, they changed a lot of the letters, and added some new ones. The Punes hadn't needed vowel letters, for instance, due to the nature of the language, but the Greeks did. On the other hand, the Greeks didn't need the post-velar and emphatic consonants of Semitic, so they used those letters for vowels.
Alpha, for instance, represented a post-velar consonant -- glottal stop [ʔalif] -- in Punic (and still does in Hebrew and Arabic), but it represents the vowel /a/ in the Greek alphabet, and has since then in the Latin.
One of the clever things the Greeks did was invent special letters for combinations of stops and the sibilant /s/. There were three of them, one in each of the three Greek places of articulation -- labial, dental, and velar.
The Labial + S letter is Psi (Ψ ψ /ps/), which was the letter you used when any of the three labial stops -- voiceless Pi (Π π /p/), aspirated Phi (Φ φ /pʰ/), or voiced Beta (Β β /b/) -- occurred before Sigma (Σ σ ς /s/).
The Dental + S letter is Zeta (Ζ ζ /ts/), which got voiced to /dz/, metathesized to /zd/ and thence to /z/ in historic times.
The Velar + S letter is Xi (Ξ ξ /ks/), which was the letter you used when velar stops occurred before Sigma. (Note, by the way, that this letter is not "X" but "Ξ".)
So why did they need special letters for sounds before /s/? Because
- Greek was heavily inflected, and many of the suffixes began with /s/.
- Many of the roots ended in stop consonants, right before adding suffixes.
- Before /s/, the three-way voiced/voiceless/aspirated stop distinction neutralized;
i.e, there was no difference and they all came out the same.
So it was sensible to use a special stop for that common combination. When the Greek alphabet was later adapted for Latin, Ψ (psi) didn't make it into Latin at all, but Ξ (ks) did, and was used for the /ks/ cluster. However, they changed the shape of the letter. Having no need for a letter for aspirated /kʰ/ in Latin, they used Χ instead of Ξ, possibly because it was easier to write.
So it's an accident. You can find out all about the history of each letter in any good encyclopedia, btw. Especially David Crystal's Cambridge Encyclopedias, of Language and of the English Language.