Finnish for Sweden is (I believe) Ruotsi, and it's quite likely that
the word Rus' derives from this. However, this has been a topic in scholarly
debate for many decades, and there no answer can be taken as simply "correct."
In any case, we are not talking here about Old Norse, but about Old Swedish.
Rurik is not the source for Rus'. The name Rurik has a Scandinavian derivation,
something like Hroerekr. Other "Russian" names come from Scandinavian
too: Igor (Ingvar), Olga (Helga) and so on.
Now about Old Norse in English. If you are new to the topic, watch out
for your dictionary. Is it giving the Old Norse word as the origin of
the English, or listing it for comparison? Almost every Old English word
can be compared to a word in Old Norse. But in some cases, we can clearly
see that the source of Modern English is not Old English but Norse. This
is the case with "egg," and "law." I believe "yard" is Old English, while
"garden" is Old Norse. They both come from the same Germanic root.