Android

I definitely think Apple and Samsung can do a smaller premium phone if they want to as Apple makes the lion's share of profits in the industry and Samsung is the only other manufacturer that makes a significant amount of profit. Heck Samsung can probably do a smaller midrange phone as well. I do not think it makes sense for other manufacturers as they make 3% profit and targeting a very vocal but demanding minority isn't going to help them improve that.

Small phone users should also temper their expectation, I saw a lot of comments from people saying they didn't get the iPhone mini because they expected the Pro Max's specs in the mini form factor. They have to understand that the cameras on the biggest flagships occupy a lot of space and it isn't feasible to bring it to a smaller form factor. Increasing the thickness might help with the battery but that's about the only component that benefits with an increase in one dimension.

I hate small phones. I switched back to an iphone SE years ago and realized it was a pita to use anymore. Everything is.. too small.

It started as a hardware problem and doesn't seem to be slowing down. LTE needed more and larger antennae for lower frequencies than older tech. Four cellular antennae are now pretty standard. Then you have wifi, Bluetooth (which can share if they can TDM), wireless charging, NFC, ultra wideband, GNSS. Then the chips are so powerful they need heat dissipation systems installed (or just lame thermal throttling like what Apple does.)

The modems require more power, (especially at the beginning of LTE) which means bigger batteries. LTE and NR have reduced range compared to the older narrowband technologies, so the phone needs to use more power to transmit, especially when carriers like Verizon didn't backfill cell sites to compensate for the reduced coverage.

Then, cameras, one wasn't enough, 4 or 5 are very common now (usually 3 primary and depth or low res sensors for aiming.)

When tablets became popular, many people decided to just have a large phone screen rather than a tablet, further entrenching the size.

The tech is more mature now, a 2-antenna MIMO antenna for cellular would suffice, albeit at the expense of network performance. Likewise one camera with a depth sensor would work, although mobile photography would be more limited. Dropping some limited-use items like wireless charging and ultra wideband could further shrink space.

So it would be possible now, but as others here have mentioned, the supply side focuses on larger hardware.

Ironically, at this point I'd almost prefer a smart watch with LTE and stop carrying a phone altogether. However, the aforementioned antenna issue makes it so watches generally have poor to unusable signal, poor battery life in cellular mode, no camera, and the 5G NR low power spec/chips aren't fully done yet, so it's LTE only on them, which, with carriers transitioning to 5G will make it so watches can only access a handful of congested bands.

Also, that device manufacturers tend to design smart watches to be companion devices to a smartphone rather than primary makes that concept's execution problematic.

Another idea I had that was anti small phone but huge battery boost was to just bring a backpack or a satchel or whatever. Carry a full sized tablet around, and use a Bluetooth headset for calls. However, tablets are also often crippled by carriers/manufacturers so they can't do common things like SMS or voice calls, and Apple has basically monopolized that market.

Bigger screens make it easier to use as my main computer, thank you.

I actually think the opposite is true as well. Phones these days mostly exist between 6.2 and 6.8". We don't actually have any huge phones as like the Galaxy Mega 6.3 (which with modern 19:9 aspect ratio screens and smaller bezels would be more like 7.5") don't exist either