Fríðr Dofradórrir is described thus in Kjalnesinga saga: 'Hún var
mikil á allan vöxt'; and she describes the hero Búi to her father as a
'skeggbarn' "beared child" (i.e. a human man). And in Bárðar saga,
Dofri's daughter Flaumgerðr called 'kvenna stærst'. I guess that
might be an understatement, and the point of mentioning the detail
must be to indicate that they aren't exactly human, but it doesn't
mean that they had to have been collosally huge themselves, does it?
Just abnormally big or well-built by non-monstrous standards.

I like the detail in Kjalnesinga saga about the food in Dofri's hall:
Borð stóðu um allt herbergið og vist fram sett og sú ein er mönnum
sómdi þann tíma að neyta. -- Just in case we were worried that the
hero might have had to eat anything untoward (such as people) during
his stay with these beings under the mountain.

Of course these legandary sagas are late medieval texts and don't
necessarily tell us about how risar, jötnar and þursar were perceived
in pre-Christian times.