Meta

About to reboot the server, hold onto your hats.

I've been seeing various 5xx error codes (usually 504s, but occasionally 502) sporadically most of the day. It doesn't seem to be a constant thing, just every 5-10 or so loads/clicks, it errors out. I've also noticed it's sometimes taking a while to load. Seems 50/50 whether it takes 1-2 seconds, or 6-10 seconds to load.

Is this something on my end, or is today a lazy Sunday for the server, too?

(I took a screenshot, but I'm getting errors when I try to upload that, too. A classic "ffprobe failed with exit code 1: success")

I'm the developer of Fediverser Project, which is a set of services to make it easy for people on Reddit to migrate to the Fediverse. It lets people use their Reddit credentials (OAuth) to sign up and create an account on a Lemmy server.

It also offers a cool onboarding feature: during signup, we can fetch the user's subscribed subreddits, and we use this information to automatically subscribe them to the corresponding Lemmy (or Kbin/Mbin) community. This "subreddit -> fediverse group" map is crowdsourced and people can sign up if they want to contribute. The "main" site also provides a "Find an instance" feature: it can track all the servers that use the Fediverse software and redirect users to their closest instance.

To enable this service, the Lemmy admin needs to add a couple of docker services to their setup and needs to get their own Reddit API key (which is used only for authentication, so well within the rate limits and certainly not incurring any prices).

I'd really like to see aussie.zone becoming part of the network. I believe this would make it faster and simpler to get more people in the fediverse, and I'm willing to provide all the support and help needed to get the "country-based" services getting started with it.

Any questions, don't hesitate to ask.

This is a simple software bug. You see, the way Lemmy federates is that updates to a community only federate to instances that have a user subscribed to that community. If you take a community like [email protected], I'm the only person on all of Aussie Zone who subscribed to it. I know it because when I subscribed, we started getting updates from Blahaj.Zone about that community.

A few days ago, I was mistakenly banned from that community. And this caused Blahaj.Zone to stop sending our instance updates on the community. I spoke with the admin of the community and got unbanned, but the unban didn't federate. The instance will only let us know I'm unbanned if someone here subscribes over there. And I can't subscribe, because this server thinks I'm banned. It's a deadlock. If it wasn't for the ban, I could subscribe and the server would realise there isn't a ban anymore. But I can't, because this server thinks there is still a ban.

When I try and upload an image for my avatar, I receive the message:

{"data":{"files":null,"msg":"ffprobe Failed with exit status: 1"},"state":"success"}

This happened with both a WebP and a PNG file, so I suspect it's a bug.

I can upload images fine here, and when I export my account details it shows a valid image URL for my avatar.

So, as per images below, when you search for an Australian community associated with lemmy, lemmy.world is more likely to come up than Aussie Zone in all i've tried, bar Melbourne our most active community.

My question: Is this a problem we should consider intentional action to correct? And if so what could we do?

Sorry if this has been asked/answered previously, but it just struck me that I hadn't really noticed any impacts of CloudStrike on aussie.zone.

Was wondering if it was a non-event, or maybe there are some war stories with heroes deserving medals!

So, I've just done something that I said I'd be reluctant to do in future without community consultation and blocked an instance.

I don't think anyone will have any issues with it, but I figured I'd let everyone know since this executive decision affects all aussie.zone users.

Hate content

Would people be interested in a community dedicated to discussing lemmy's in general but, specifically aussie zone's growth?

I'm thinking a pretty wide ranging set of ideas including,

• exposure and promotion of fediverse-lemmy-aussie zone,
• discussions on server structures and dispersal of user-bases,
• philosophical discussions on growth as an objective in the case of lemmy,
• discussions on different platforms aussie zone users might want to gather round as the potential use cases of the federated social web start to be explored.

I'm thinking like [email protected] , but a sister community that has regard to aussie zone user's corner of the fediverse.

To be used by aussie zone as a general compass setting and project brainstorming community for how we might like to develop, if at all.

Or is [email protected] the better place for these sorts of discussions?

There's no communities I think need better moderation, or any that have no active mods (that I know of), but if there is one day, can somebody make a request to have ownership or moderation powers transferred over to them?

Hey all, following the work over the weekend we're now running Lemmy 0.19.4. please post any comments, questions, feedback or issues in this thread.

One of the major features added has been the ability to proxy third party images, which I've enabled. I'll be keeping a closer eye on our server utilisation to see how this goes...

This weekend I'll be working to upgrade AZ to lemmy 0.19.4, which requires changes to some other back end supporting systems.

Expect occasional errors/slowdowns, broken images etc.

Once complete, I'll be making further changes to enable/tweak some of the new features.

UPDATE: one of the back end component upgrades requires dumping and reimporting the entire lemmy database. This will require ~1 hour of total downtime for the site. I expect this to kick off tonight ~9pm Perth time.

UPDATE2: DB dump/re-import going to happen ~6pm Perth time, ie about 10 minutes from this edit.

UPDATGE3: we're back after the postgres upgrade. Next will be a brief outage for the lemmy upgrade itself... after I've had dinner 🙂

UPDATE34: We're on lemmy 0.19.4 now. I'll be looking at new features/settings and playing around with them.

I know there's been a few posts about this before, but it's been a month since the last one and it's still ongoing. It doesn't seem that any of the LW admins responded to Zag's post on their help community, and the last response from lodion/Nath I'm aware of was from 4 months ago when there were outright federation failures as opposed to just lengthy delays.

@[email protected] posted a comment on the post from last month about the delays stating that it's an issue on our end as our server isn't keeping up. I'm not sure whether this is the case or not, and I'm not sure how to interpret the Grafana dashboard they linked to, but as it's a new reply on an old post, I wanted to note it.

Current federation delays seem to be around 7 days. It doesn't seem to be affecting posts themselves on Lemmy.world communities, but does affect all replies to them (even from users on other instances), and all upvotes on the posts. [Edit: on further investigation, this isn't the case. The current delays are at least 13 days, and this does actually affect posts too]

I don't want to sound too pushy, since the LW admins and Lodion/Nath are all volunteers, but I was hoping we might be able to get an update on what the cause is, and if it's an issue in Lemmy itself, if anybody's opened an issue on GitHub and the developers are aware.

(NB: I don't interact that much with LW, so all of my testing has been on the Boost for Lemmy community.)

Hey all, can anyone tell me the date we're gona land on for the first anniverssary?

I guess it'll be when the instance was spun up? Or i don't know, maybe when it was a twinkle in Lodion's eye?

Its been a little while since I posted stuff :)

CPU:

Memory:

Network:

Storage:

Cloudflare caching:

Comments:
Not much has changed in quite a while. I still have a cron-job running to restart Lemmy every day due to memory leaks, hopefully this improves with future updates. Outside of that, CPU, memory and network usage are fine.
Object storage usage is growing steadily, but we're a long way from paying more than the monthly minimum Wasabi fee.

I know when things first kicked off we would get server updates semi-regularly such as finances and server load. I can’t say I’ve seen one for a while, just wondering how things are going?

Posted a query in the LW support channel to see if anyone there has any idea.

Just thought I'd give people a heads-up. I've just noticed some upvotes and posts haven't federated from lemmy.world to aussie.zone .

Hopefully it's just delays again, and they'll work their way through the queue reasonably soon.

Comments, votes don't appear to be federating for one of the last posts I made over at [email protected].

https://aussie.zone/post/7871931

https://lemmy.world/post/13089616

I'm still seeing cases where comments don't seem to be federating correctly. For example:

I'm not sure what the cause is, but the last few days it seems more of my comments are failing to federate than are succeeding.

We’ve already defederated one tanky instance, and frankly I think I’ve rarely had a positive interaction with Hexbear users. Their defence of Russia’s invasion of Ukraine, or of China’s genocide against the Uighur people, is hard enough to stomach. But what prompted me to post this is their commentary in this thread where even without any of the propaganda-based reasons to justify their beliefs, they seem to delight in being rude arseholes to people.

Personally, I don’t think they provide us any real value, no we’d be better off without them, just as we are better off without lemmygrad and exploding-heads.

~~I'll be restarting AZ today for an update to lemmy 0.19.~~

Upgrade complete.

This is a major upgrade, so I expect there to be some issues. Strap in, enjoy the ride.

Expect:

• further restarts
• bugs
• slowdowns
• logouts
• 2FA being disabled
• possibly issues with images, upgrading pictrs to 0.4 at the same time

Meta threads will open federation to the fediverse soon, and while this is mainly to mastodon it will still affect lemmy. They are acting like they won't be evil, but let's be real this is Facebook when have they ever done that.

This article which has been trending lately explains some of the issues. https://ploum.net/2023-06-23-how-to-kill-decentralised-networks.html

This comment here is a simple analogy if you can't be bothered reading the article. https://lemmy.ca/comment/5702922

@[email protected]