These suffixes do not change the part of speech, so are they inflectional endings?
Introductory textbooks sometimes tout "changes the part of speech" as being the key feature of a derivational affix. But this definition has a lot of flaws, and you've run into one of them.
One, there are some affixes that do change the part of speech, but don't seem to be derivational. "Run" is a verb, but "running" is a noun or an adjective depending how it's used. And that certainly doesn't seem like a derivational change.
And two, there are some affixes that don't change the part of speech, but do seem to be derivational. "Un" is a classic example of this: "do" and "undo" are both verbs, but would you really say they're forms of the same word? In the comments, Yellow Sky also mentions violin → violinist, both nouns.
(Some textbooks also say it's derivational if the meaning changes, but then you have to define "meaning" and explain why "run" and "running" have the same meaning when clearly there's a reason to use one instead of the other.)
So the rule of thumb I tell my students is that derivational affixes make a new word that deserves its own dictionary entry. Native speakers' intuition tends to work well with this rule. More formally, you can look at how regular this process is; I can't think of a single verb in English that can't take "-ing", while very few can take "un-" (it's more of an adjective thing). If you can only use an affix on a particular category of meanings (rather than a part of speech), that's a good sign it's derivational. But the categories of "inflection" and "derivation" aren't actually that strict and clear in the real world, and linguists can (and do!) disagree about what category something can fall into.
For your particular example, I would say "-th" is derivational, because "four" and "fourth" feel like distinct words, and it can't be applied to all adjectives, only the ones that mean numbers (mathematicians say "n-th" all the time, and I've heard "k-th" in computer science, but you can't say *"blue-th" or *"cold-th").
The other ones, like fir-st, seco-nd, thi-rd, are not affixes at all in my opinion. What is a *seco on its own? These are just exceptions to the pattern. The formal name for "first" and "second" is suppletion, when the regular, expected form gets replaced with a form taken from a totally different word, while "third" is fossilized: long ago it was completely regular, but it was commonly-used enough that it didn't change as the language around it changed, and now it's an exception.