As Parenti so eloquently put it
Class gets its significance from the process of surplus extraction. The relationship between worker and owner is essentially an exploitative one, involving the constant transfer of wealth from those who labor (but do not own) to those who own (but do not labor). This is how some people get richer and richer without working, or with doing only a fraction of the work that enriches them, while others toil hard for an entire lifetime only to end up with little or nothing.
Those who occupy the higher circles of wealth and power are keenly aware of their own interests. While they sometimes seriously differ among themselves on specific issues, they exhibit an impressive cohesion when it comes to protecting the existing class system of corporate power, property, privilege, and profit. At the same time, they are careful to discourage public awareness of the class power they wield. They avoid the C-word, especially when used in reference to themselves as in "owning class;' "upper class;' or "moneyed class." And they like it least when the politically active elements of the owning class are called the "ruling class." The ruling class in this country has labored long to leave the impression that it does not exist, does not own the lion's share of just about everything, and does not exercise a vastly disproportionate influence over the affairs of the nation. Such precautions are themselves symptomatic of an acute awareness of class interests.
Yet ruling class members are far from invisible. Their command positions in the corporate world, their control of international finance and industry, their ownership of the major media, and their influence over state power and the political process are all matters of public record- to some limited degree. While it would seem a simple matter to apply the C-word to those who occupy the highest reaches of the C-world, the dominant class ideology dismisses any such application as a lapse into "conspiracy theory." The C-word is also taboo when applied to the millions who do the work of society for what are usually niggardly wages, the "working class," a term that is dismissed as Marxist jargon. And it is verboten to refer to the "exploiting and exploited classes;' for then one is talking about the very essence of the capitalist system, the accumulation of corporate wealth at the expense of labor.
The C-word is an acceptable term when prefaced with the soothing adjective "middle." Every politician, publicist, and pundit will rhapsodize about the middle class, the object of their heartfelt concern. The much admired and much pitied middle class is supposedly inhabited by virtuously self-sufficient people, free from the presumed profligacy of those who inhabit the lower rungs of society. By including almost everyone, "middle class" serves as a conveniently amorphous concept that masks the exploitation and inequality of social relations. It is a class label that denies the actuality of class power.
The C-word is allowable when applied to one other group, the desperate lot who live on the lowest rung of society, who get the least of everything while being regularly blamed for their own victimization: the "underclass." References to the presumed deficiencies of underclass people are acceptable because they reinforce the existing social hierarchy and justify the unjust treatment accorded society's most vulnerable elements.
Seizing upon anything but class, leftists today have developed an array of identity groups centering around ethnic, gender, cultural, and life-style issues. These groups treat their respective grievances as something apart from class struggle, and have almost nothing to say about the increasingly harsh politico-economic class injustices perpetrated against us all. Identity groups tend to emphasize their distinctiveness and their separateness from each other, thus fractionalizing the protest movement. To be sure, they have important contributions to make around issues that are particularly salient to them, issues often overlooked by others. But they also should not downplay their common interests, nor overlook the common class enemy they face. The forces that impose class injustice and economic exploitation are the same ones that propagate racism, sexism, militarism, ecological devastation, homophobia, xenophobia, and the like.
https://archive.org/details/michael-parenti-blackshirts-and-reds
To attempt to add to what you're saying, I would venture to say a lot of it comes down to who is doing the talking. A USian white person saying "identity politics doesn't matter, stop centering race" to a black person is sus at best and likely to be coming from (in the best case scenario) a place of underestimating how deeply embedded white supremacy is in the makeup of the US and its history. OTOH, if a black person growing up in the US were to tell a well-meaning but overthinking and self-castigating white person that they don't need to wallow in guilt about racism 24/7, that's a different story. Even then, though, there is a real risk that a white person who has been raised to think of black people as a monolithic oversimplified entity, will view this kind of commentary as some kind of "pass" to act a certain way, when it's just the viewpoint of one black person. So there is a real need to combat racism, for example, in a context like this because it is a real part of the makeup of things; it's not only fabricated at this stage of things. And overcoming it to reach for class solidarity means dismantling racism (creating an organized process to help people view the "othered" people as human like them and working to dismantle the institutions that keep white supremacy going), not ignoring it (statements like "I don't see color, just focus on working class solidarity"). Or to put it another way, in some contexts, the obstacle is there and real through the enforcement of powerful institutions, and so it must to some extent be dismantled in order to reach reliable class solidarity; otherwise, we run into the kind of issue you mentioned, of groups who turn their backs. The tricky part seems to be in how to dismantle those institutions, while also organizing in a trustworthy enough fashion in spite of them still existing, and some of that may come down to the kind of dynamic Mao talks about in On Contradiction with the Kuomintang (I think that's the one) where some alliances are going to have to be tenuous in order to tackle the most pressing issues first.
But how to do this in the context of a place like the US, I don't pretend to know what works best. I tend to think imperialism is that current biggest issue and may have the clarity to bypass at least some of the issue with deceptive reactionaries who focus only on local class struggle rhetoric, as it gets into colonialism and the development of these splintering dynamics like racism and sexism, and reactionaries I don't think usually want to get into that too deeply, even when they're willing to pretend to agree with the political goals of communism.
TL;DR: Centering decolonization and anti-imperialism (and making sure it is clear on what those terms mean, not just what they sound like) may be the current most effective way to get at the most pressing concern without as much distraction in the weeds of it, where people are disagreeing in the margins on issues of racism or sexism or the like, isolated out from everything else, and losing effectiveness in the overall struggle. After which these things can be dealt with and dismantled more effectively, without the institutional shadow of the western empire cast over them (but not to say they cannot still be fought against along the way). Hope that makes sense.