Difficulty curve needs to be adjusted. The difficulty settings increase linearly with HP and damage while your character's power level increase exponentially due to weapon/armor material/quality/enchantment strength all stacking multiplicatively (This is also the reason playing a mage in vanilla Skyrim feels bad, because of the lack of exponential scaling as spells are mostly flat damage)
Bigger towns with more verticality would also be nice, Whiterun was like 12 buildings in total, but that's more of a technical issue.