Cases like this don't even require borrowing.
Consider English "newt". This is a native English word, but it's got half an article stuck on it. Old English "efte" became Middle English "ewte", but later, "an ewte" was reanalyzed a "a newte".
The same thing also happens in the opposite direction: Old English "naedre" became Middle English "naddere", but later "a naddere" was rebracketed as "an addere", hence modern "adder".1
Obviously, Middle English speakers weren't unfamiliar with Middle English.
So, does that demolish your guess, or Draconis's answer? Not at all. It's pretty clear that this kind of rebracketing is much more common in Arabic->Spanish borrowings than in language-internal evolution. There are a handful of cases like "newt" and "adder", but many dozens of cases like "alcohol" and "algorithm".
Sometimes people hear unfamiliar words even in their own language.2 This is especially true for words with dialectal variation.3 It isn't that common that a rebracketing or similar error spreads through the language, but it happens occasionally. Now consider a Spanish trader with a little Arabic4—it's a lot easier for him to mishear something, and there's a lot less pressure from other Spaniards correcting the error, so it's much more likely that his reanalysis will spread.
For the reasons you suggest, English-Polish borrowings don't have most of the same factors as Arabic-Spanish borrowings that make this kind of reanalysis more likely. And Adam Bittlingmayer's answer suggests further differences.
Also, consider that many modern borrowings from English are triggered by exposure through TV, movies, music, games, etc. If you mishear something from a trader who comes to your village, you're probably the only one who heard it; if you mishear something from a TV show broadcast at prime time, half your friends saw the exact same TV show. And, even if you all mishear it, a good chunk of the larger Polish-speaking community also saw it, so it's less likely to propagate from your group.
Also, many Poles who speak English are also literate in English, and in fact read quite a bit of English (internet forums, game on-screen prompts, etc., even if they don't sit around reading Dickens all day). It's a lot harder to mix up "an ewte" and "a newte" in writing than in speech. That's even more true given English's orthographic convention of clearly separating words. Especially since Polish has the exact same convention. And even more so when 99% of what you're reading is digital or typeset, where the spacing is perfectly regular. Compare all of that to a medieval Spanish monk reading Arabic calligraphy, and it seems obvious that your literacy should counter rebracketing a lot more than his did.
For extra fun, look at "orange".5 Going from Dravidian to Sanskrit to Persian to Arabic "naranj", there was surprisingly little change, so where did English lose the "n"? It looks like a case of Arabic-Spanish article rebracketing, and it also looks just like English "adder", but it's actually neither. While Italian and Occitan both rebracketed it, losing the "n", Spanish borrowed it as "naranje", and French borrowed it from Spanish, not its nearer neighbors. But then, French calqued Italian "mela arancia" as "pome narenge", and then reanalyzed the "n" away and ended up with "pome orenge". And "orenge" is what got borrowed into English.
1. Dutch did the exact same thing with the same word, but Frisian, even closer to English, did not, and ended up with "njirre".
2. There are other ways it can happen. For example, one theory on "naunt" was that people jokingly pretended to misunderstand "mine aunt" as "my naunt", it spread as slang, and eventually it became a perfectly normal colloquial word (although a few centuries later it was lost, at least in standard English).
3. See Caxton's famous case with "egg" for a good illustration.
4. Again, this isn't the only possibility. Consider a Spanish monk reading Arabic and Greek treatises in parallel, who's rarely heard Arabic spoken aloud.
5. Some steps of this are disputed, and I might be pushing a now-discredited theory at some point along the way… but whatever happened is interesting.