this post was submitted on 02 Jul 2023
7 points (100.0% liked)

Lemmy.world Support

3228 readers
10 users here now

Lemmy.world Support

Welcome to the official Lemmy.world Support community! Post your issues or questions about Lemmy.world here.

This community is for issues related to the Lemmy World instance only. For Lemmy software requests or bug reports, please go to the Lemmy github page.

This community is subject to the rules defined here for lemmy.world.

To open a support ticket Static Badge


You can also DM https://lemmy.world/u/lwreport or email [email protected] (PGP Supported) if you need to reach our directly to the admin team.


Follow us for server news ๐Ÿ˜

Outages ๐Ÿ”ฅ

https://status.lemmy.world/



founded 1 year ago
MODERATORS
 

Most communities on other instances (especially lemmy.ml, but most other instances seem affected) seem still desperately empty and dead when viewed from here, despite vibrant activity as seen directly from there.

Are there still big issues getting new and/or post lost during the version desynchronisation?

all 7 comments
sorted by: hot top controversial new old
[โ€“] [email protected] 7 points 1 year ago (1 children)

FYI, this is due to a confluence of issues.

  • We are the largest instance with the highest active user count - and by a good margin.
  • We are dealing with a premature software base that was not designed to handle this load. For example, the way the ActivityPub Federation queues are handled are not conducive to high volume requests. Failed messages stay in queue for 60 seconds before they retry once, and if that attempt fails it sits in queue for one hour before attempting to retry. These queued messages sit in memory the whole time. It's not great, and there isn't much we can currently do to change this, other than to manually defederate from 'dead' servers in order to drop the number of items stuck in queue that are never going to get a response. Not an elegant solution by any means, and one we will go back and address when future tools are in place, but we have seen significant improvement because of this.
  • We have attempted contacting Lemmy devs for some insight/assistance with this, but have not heard back yet, at this time. Much of this is in their hands.
  • We were able to confirm receipt of our federation messages (from lemmy.world) to other instance admins instances at lemm.ee and discuss.as200950.com. As such we do know that federation is working at least to some degree, but it is obviously still in need of some work. As mentioned above, we have reached out to the Lemmy devs, who are instance owners of Lemmy.ml, to collaborate. I cannot confirm if they are getting our federation at this time. Hopefully in coming Lemmy releases this becomes easier to analyze without needing direct server access to both instances servers.

As you can see, we are trying to juggle several different parameters here to try and provide the best experience we can, with the tools we have at our disposal. You may consider raising an issue on their GitHub about this to try to get more visibility to them from affected users.

[โ€“] [email protected] 3 points 1 year ago (1 children)

I feel sorry for the admins. This must be stressful to deal with.

[โ€“] [email protected] 2 points 1 year ago

I appreciate the concern, but we are doing this because we enjoy doing it and our skill sets allow us to contribute. These are exciting times, even with the issues we are working through :) No worries.

[โ€“] [email protected] 1 points 1 year ago

The same thing has been happening with kbin. Anything crossed with a kbin instance is dramatically late. Like hours or more.

[โ€“] [email protected] 0 points 1 year ago (1 children)

Dunnow if it is related, but everything is just worse this the upgrade than it was before. Now everything takes forever to load, responses take longer, pictures don't load the correct way. So for me this was a downgrade in my experience here or what the Germans call a "Verschlimmbesserung" (making it worse by trying to improve).

[โ€“] [email protected] 1 points 1 year ago

I bet it's just the dramatic uptick in traffic. It takes quite a bit more money and effort to make a highly available and performant service, which most of these are side projects or paid for by a normal person's income which can't compete with a Corp.