Yes, Icelandic is quite conservative compared to many other Indo-European languages - it could be compared to Latin in terms of the complex morphology it retains.
There are a number of explanations for this. The geographical isolation of Iceland (being an island) means that speakers of Icelandic had less direct contact with speakers of other languages, and therefore there were fewer competing influences from other languages. Prolonged contact with languages that have very different features is generally considered to have a role in the stripping-down of inflectional systems - English is a good example of this.
(Though there was, apparently, a Basque-Icelandic pidgin spoken in the 17th century.)
Similarly, the population of Iceland is mostly made up of people who are Icelandic, and the number of Icelandic-speaking people living outside of Iceland is relatively small. Iceland is also a relatively small geographical area, with the majority of the population concentrated around the Reykjavik area (200,000 out of 320,000). The dialectal differences are minimal, because the population is not dispersed in the same way as in other countries, and homogeneity in a language can be self-reinforcing of its characteristics.
Other factors may also contribute to the conservative nature of the language - there is a recent dissertation on language change and stability in Icelandic which suggests that "strong linguistic nationalism and a stability-oriented language policy, are instrumental in creating the sociolinguistic conditions in Iceland which support language stability". This is interesting, because while 'language academies' such as the Académie française and similar policy-creating bodies generally hold little sway against the tide of language change, the attitudes of speakers themselves can be very influential, and this may be of particular relevance in Iceland.
Lastly, while Icelandic is still remarkably similar to Old Icelandic (which is essentially Old Norse), the ability of speakers to easily read the old sagas is usually exaggerated (so i've been told by Icelandic people). It's a bit like English speakers reading Shakespeare - you usually have to read a modern version with lots of footnotes and explanations and modernized spelling (because the pronunciation has changed quite a bit over the centuries). Even so, this is pretty impressive - Shakespeare is only four hundred years worth of change for English speakers, while the Icelandic sagas are about twice as old.