I do it as well. If I'm not actively speaking and the person speaking isn't presenting something that I need to look at, I usually end up bouncing back and forth between the speaker and my image.

I don't want to necessarily apply logic to it because I don't think it's a conscious, logic decision I'm making. But if I had to try, I'd say that the reasons are A) I cannot "look them in the eye" as I would in person without looking directly at my camera, which is both weird and means I'm literally looking away from them. That is the paradox of video calling. B) Looking at them, versus looking literally anywhere else on your screen makes no tangible difference to anyone else looking at your video feed. C) I want to make sure I am not looking ridiculous while blasting my image to a dozen people. No double chin, no resting bitch face, no glazed look, no boogers, etc. D) Staring at anyone else would feel weird, invasive, and distracting to me, including the speaker, if they are not actively speaking to me. It feels like I'm eavesdropping when I'm not actively being addressed. E) Gotta take advantage of having eyes on the back of your head. Never turn your back on your enemy. Stay vigilant. The cat will not pounce me and claw my back mid-meeting again.