You're right, if you haven't watched all of the Trek shows you're definitely not going to get nearly as much enjoyment from Lower Decks! There are even references to things that weren't previously canon, like the Star Trek helmet, so even then you won't get all the jokes.
I really enjoy SNW, it's always pretty high on my list. It's been especially great since this is the first time I've had a Trek show I really like that I'm watching while it's still in production! I have some annoyances with it here and there, mostly due to the fact that it's a prequel, but I can more easily overlook that because most of the rest of it is good.
I really think Enterprise and Voyager both were missed opportunities. Enterprise for a lot of the reasons you mention, and Voyager I felt could have had a lot more continuity. It was the perfect place to have a lot of recurring characters and DS9 style arcs, but they never really committed.
There are a lot of good episodes in both of those!