It's generally assumed that you can't stick material of arbitrary length and near-abitrary content into the middle of a word. As mentioned in Yellow Sky's comment, we can say, in addition to "an apple", "a yellow apple", "a lovely, delicious yellow apple", and so on.
"Of" is not analyzed as a head-marked possessive suffix for a various reasons. There are many contexts where "of" phrases are used where "of" is not directly after the head of a possessive construction. "Of..." can be used predicatively or used in coordination ("the well-being of the people and of the government").
There are some real cases where it's unclear whether to analyze something as one word, but I don't think any of the examples you gave are of this kind.
Some examples I know of that have actually been considered to cause some difficulty for analysis:French verbs and the prefixed pronouns (especially the non-subject pronouns, but to an extent even the subject pronouns); French prepositions before proper nouns (like country names); English compound nouns of the form adjective + noun that are written closed (like "blackbird") vs English adjective + noun sequences that are written open, but stressed on the first element (I can't think of an example right now--I'll add one when I add the citation).