I would suggest:
- Use any debian 12 iso
- When you get to the stage where you get to choose what to install (it's called tasksel and looks like : untick whatever desktop environment is selected. You'll get a bare minimal install with no graphics server
- Reboot into the installed system, and install whatever you want from there.