this post was submitted on 11 Jul 2023
17 points (100.0% liked)

/kbin meta

12 readers
1 users here now

Magazine dedicated to discussions about the kbin itself. Provide feedback, ask questions, suggest improvements, and engage in conversations related to the platform organization, policies, features, and community dynamics. ---- * Roadmap 2023 * m/kbinDevlog * m/kbinDesign

founded 1 year ago
17
lemmy.ml is no longer shadowbanning kbin. (lemmy.ml)
submitted 1 year ago by [email protected] to c/[email protected]
45 comments fedilink hide all child comments
 

Apparently one of the lemmy.ml admins was overzealous in banning all User-Agent strings that contained the word "bot". Bans were entered for all of the individual strings containing that word which were observed in their webserver logs, which impacted kbin's reported agent of "kbinBot".

The issue has been fixed, and I observed that one of my kbin posts to a lemmy.ml community was successfully pushed to the original instance.

Edit:

Here are all the links that I've found with the lemmy.ml admins discussing the issue:

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

There are plenty of high volume, non-malicious bots that do. robots.txt is a thing for a reason, and we can see here that lemmy.ml has implemented it. Not all bots that ignore robots.txt are malicious though, just poorly designed. You can basically lump them into three categories:

  • Well-behaved bots that announce that they're bots in the User-Agent header and obey robots.txt (note that they may still slam the server even if they obey it)
  • Mediocre bots that announce that they're bots in the User-Agent header but ignore robots.txt (or vice versa)
  • Bad bots (malicious or otherwise) that announce their User-Agent as other things, often pretending to be other software.

Their logs told them they had a lot of traffic from stuff identifying itself as bots. Throwing that traffic out wouldn't break lemmy but would help them deal with the capacity problems that all of the mainstream lemmy/kbin instances had to deal with shortly after the Reddit exodus began. They fucked up and tagged kbin in the process, which definitely would have been one of the highest volume ActivityPub consumers matching their criteria.