His secret 5th phase is brutal.
Jokes aside, he can actually damage you if you let him. I tried once after missing the aerial deathblow (but still aggroing him so I couldn't even do the normal one either) and figured I'd just let him kill me for that humiliation. I waited, just standing still, and he has that one slow punch move that the other red nobles do. It does so little damage that even on minimum physical attributes he would have taken a good while to whittle me down. I got bored and quit out instead, but it goes to show that even the weakest enemy in the game can be a threat!
Not much, but a threat still.