Home

467 readers
1 users here now

Lemmy.zip instance discussion.

For all things relating to Lemmy.zip.

Main instance rules apply, with the additional rules below:

founded 1 year ago
MODERATORS
51
54
submitted 9 months ago* (last edited 9 months ago) by Demigodrick to c/home
 
 

Hi all, we've finally made it through January after what feels like 6 years, and we're into the second month of 2024!

Here's a nice little update on what has happened and is happening with Lemmy.zip at the moment.

New Communities

We've had a few new communities pop up this month, please check them out and subscribe if you like them!

We've also had MaddieTian kindly take over the Playstation community - please check it out and subscribe if you haven't already!


Server Updates

Since the January update, we've applied Lemmy updates 0.19.2 and 0.19.3 which fixed the previous federation bug. All servers on 0.19.3 upwards should have much more reliable federation now.

We've somewhat stabilized the site now too, with thanks to locuester - we've further tuned the database performance, and in the last week we've made some configuration changes to Lemmy's database (such as a memory leak fix), plus some further configuration changes to the operation of the server which has resulted in no more daily database restarts or the image processing software breaking every day. This is likely a bit of a stop gap until the Lemmy Devs formalize the move to Postgres 16 and the testing can prove that this improves performance. But it is one less thing to worry about for now!

On the topic of images, we've now doubled the image upload size from 1mb to 2mb quietly to see if this would have impact in day to day performance or storage. With the improvements to Pictrs (the software the serves and stores images) we've actually freed up lots of space in the image storage bucket and so have room to play. Given that we've had no issues with image upload sizes at 2mb, I am considering doubling again to 4mb to see what (if any) impact that has on performance. We'd love to allow high quality image uploads so we'll continue to work towards that.

We've also updated our Code of Conduct just a tiny little bit to cover more around downvote manipulation. Downvoting bad content is fine, downvoting all content in a community that you've never interacted with before and is in-line with the rules of that community is not fine. Remember, you can always report things you don't like and mods/admins can take action.

We'll also be tightening some of the rules up around interactions in remote communities - if we're receiving mass reports regarding a Lemmy.zip user's conduct in another community, we'll consider if that user is breaking the rules of that community and take action ourselves. We'll post more about that when those changes are finalized.

Remember, you can see all of our legal stuff at legal.lemmy.zip if you're every really really bored.

Finally, we'll be releasing a new survey soon so we gather feedback from you all as to how you find your Lemmy experience and we can do better.

We did this back when the instance was 3 month's old, so I feel now would be a good time to see what has changed and how we can continue to improve.


Interaction reminder (help support the instance!)

If you’re new to Lemmy.zip - WELCOME! I hope you’re enjoying your time here :)

I just want to let everyone know that the easiest way you can support Lemmy.zip is to actively engage with content here: upvoting content you enjoy, sharing your thoughts through posts and comments, sparking meaningful discussions, or even creating new communities that resonate with your interests.

It’s natural to see fluctuations in user activity over time, and we’ve seen this over the wider lemmy-verse for some time now. However, if you’ve found a home here and love this space, now is the perfect opportunity to help us thrive.

If you want to support us in a different way than financially, then actively interacting with the instance helps us out loads.


Donations

I hate banging on about money and I'm not very good at it, but here is the general reminder that Lemmy.zip only exists because of the generous donations of its users.

We keep all the details around donations on our OpenCollective page, with full transparency around income and expenditure.

Therefore, my ask is that if you’ve been enjoying your time here, please consider a donation - it will help keep Lemmy.zip online and all donations go to paying for the server and for the surrounding services like image hosting and email. We have options for one-off donations of any amount, or recurring donations at set amounts.

We're starting to look at how we reward users with "perks" for donating, although I want to be very clear that this will never detract from the Lemmy experience.

So far we have the name in the thank you thread perk, and we're also introducing me.lemmy.zip - a place to keep links to various online identities (here's mine!) or links to anywhere really (as long they abide by our code of conduct obviously!). This is open to any past or present donator of any amount, just drop me a message and I'll set you up.

We're on the look out for good perks we can offer, so if you've seen any on other websites you think would be a good fit for here, let me know!


Graph time

Here's some exciting graphs on performance. This is slightly different this time, I'm now switching to grafana for all performance monitoring so graphs will come from there now.

CPU:

Memory:

This one requires some explanation - on the left you can see the spikes where the DB would max out RAM usage and restart itself, causing the site to break/slow down for a couple of minutes at a time.

On the right, you can see the implementation of all our changes, and how memory usage has steadied out.

Disk:

(The big jump on the right was due to swap being implemented)

Images:

The newest version of Pictrs has done some cleanup on images that haven't been accessed in a while, so the used space has dropped quite a bit.

Cloudflare

Overall stats:

Requests over 30 days:

Bandwidth over 30 days:

Unique visitors over 30 days:

And finally traffic per country over 30 days:

Thats all for this month - as always, if there are any other stats you'd like to see let me know! Otherwise, hope everyone has a great February.

Cheers,

Demigodrick

52
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

53
 
 

cross-posted from: https://lemmy.ml/post/11023519

Go to the original post if you have a question for the devs 🙂

This will take place ~24 hours from now. Feel free to post and upvote questions beforehand in this post, as it will turn into the AMA tomorrow.

TimeZone Converter Link

This is a chance for any users, admins, or developers to ask anything they'd like to myself, @[email protected] , SleeplessOne , or @[email protected] about Lemmy, its future, and wider issues about the social media landscape today.

54
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

55
6
submitted 10 months ago by treadful to c/home
 
 

Anyone else seeing most images not loading today?

56
57
2
Daily log out? (self.home)
submitted 10 months ago by Blaze to c/home
 
 

Hello everyone,

New account on this instance, thank you for having me.

I just noticed that I have to log in every day (I would just start my day and notice I'm logged out, while I was connected the day before), is this a known issue?

58
 
 

cross-posted from: https://lemmy.ml/post/10449815

Here is our regular update that explains what we have been working on for the past two weeks. This should allow average users to keep up with development, without reading Github comments or knowing how to program.

Yesterday we released Lemmy 0.19.2, which included various federation fixes, as well as a way for admins to view votes.

@phiresky fixed a federation loop bug, and worked on lemmy-federation-state, a tool to help visualize federation status.

@sleepless fixed a lemmy-ui theming issue, as well as an issue with lemmy-ui's error pages.

@dessalines fixed an issue with resolving reports, added the ability for admins to view votes to prevent downvote trolling, upgraded our woodpecker-ci to 2.1, and helped fix various CI issues, and is adding comment and post removing to jerboa.

@nutomic is fixing an issue with mastodon follows, working on local-only communities, fixed an issue with the lemmy-stats-crawler, fixed an issue with cache-control headers, better handling of federated reports from mastodon and kbin, and much more.

@dullbananas is fixing up some Lemmy DB triggers, adding a better-organized cursor-based pagination library, and a query plan viewer.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

59
 
 

I still don't really understand how the fediverse interacts very well, so this might be something basic. When I access this post through the original lemmy.world, there are (currently) 44 comments showing, but when I access it through lemmy.zip, only 9 comments show up. This entire comment thread, for example, doesn't appear through .zip. When I go to that thread's OP's profile through .zip, the comment is missing there as well.

As far as I know we're not defederated from lemmy.world, but that's the only reason I was aware of that a user's comments might not appear. Does it have to do with the fact that it's a new instance or something?

60
-7
New Mascot? (self.home)
submitted 10 months ago by possiblylinux127 to c/home
 
 

Now that Steamboat willy is public domain someone should edit it to make it out new Mascot.

sb willy

61
 
 

Welcome to 2024! I hope everyone had a nice and enjoyable festive period, and I'm excited to see what the next 12 months holds for us all!

Here is the monthly update on everything happening with Lemmy.zip.


New Communities


Server Updates

So quite a lot has happened since the December Update including the release of Lemmy 0.19 and 0.19.1, troubles with Lemmy federation across the network, a change to our hardware, and some moderator-friendly additions to Zippybot.

- Lemmy 0.19 Release

On the 16th December we took the server briefly offline in order to apply the 0.19 update, which took about 20 minutes to do some database migrations and then all came back online quite smoothly. We also took the opportunity to update the server OS and all of the supporting services to their latest versions too, which all in all took about 2 hours, most of which was waiting for various backups to complete. We have updated Pictrs (the software which shows you the images!) to its latest non-beta version, but there is a new update coming for Pictrs which requires some downtime to allow for prep work for this update to take place (although core Lemmy will still work). We'll post more on this when it happens.

It was quickly discovered we had quite a big issue (well, technically two but we worked through the second one which was caused by docker-specific settings which I'm happy to expand on if anyone is interested) which was that suddenly our CPU was maxing out, our RAM was maxing out, and our site was grinding to a halt. Something has changed, most likely due to the new persistent federation queue, which is causing the database to continuously eat server resources. Here's a lovely screenshot showing how the server was maxing out during my "wtf is happening" phase that I posted to our matrix chat:

Due to this, I've finally had to admit defeat and move us off a VPS to a Dedicated Server (technically dedicated vCPUs) which has kind of sorted our performance issues, by delaying the restart of the database from every hour and half to once a day.

This graph shows the before and after of our RAM usage. The left side shows the RAM being maxed out by the database and then the system killing and restarting the database in a vicious cycle (with our measly 8gb of RAM and the right hand side shows our new RAM usage - its still going up, but with 16GBs the cycle only occurs once a day, and for a second or two at most. Unfortunately this is a bug I can't fix, and isn't replicated by all Instances apparently (although I have seen a fair few other admins mention it) so hopefully it can be fixed at some point.

There is also a bug in relation to federation which due to all the DB restarts we weren't actually facing until we had a bit more power, when suddenly the server stopped federating with lots of other instances. There is a really handy public tool where you can see all this info live - It's important to note that the failing instances numbers are fine, if you try to visit them you'll probably get an error. The dead instances are instances our server hasn't been able to communicate with in 3 days, and so has stopped trying. Feel free to check that list and if you think any shouldn't be on there, let me know and I can manually reset them.

The issue lies with the "lagging" instances - we suddenly saw a thousand instances on that list, including some of the biggest instances, which indicated an issue. Its normal to see one or two on there, but not a thousand. Thankfully, a database restart does fix this issue so I am manually checking this website once or twice a day and will restart it if necessary, but if you do notice any issues ping me a message and I can take a look. In the future I pan to automate this but have not yet fully recovered from the festivities!

Lemmy 0.19.1 was also released, fixing some bugs such as the "show read posts" option not working, which was quietly applied.

- ZippyBot Moderator updates

Work continues to build some form of serviceable "automod" for moderators, so Zippy now includes the functionality for community mods to schedule posts to their communities, either on a one-off or ongoing basis.

The command for this is #autopost followed by some qualifiers - you can get the full info by sending Zippy a message with the command #autoposthelp, which will send you back something like this:

Updates will continue to happen to this for added functionality and I'm always open to ideas.


Donations

This is probably a more important topic than it previously has been - due to the extreme generosity of some of our users, we've been in a financial position that has allowed us to not need to worry about funding too much. However, with the move from a VPS to a dedicated server, our costs have almost doubled from £16.36ish a month to £30 a month. This means, assuming nobody donated another penny, we'd run out of funds in less than 8 months. Our current level of donations is just short of this new monthly figure too, which does mean we will eventually run out of funds at some point too even if nothing changed with donations.

Therefore, my ask is that if you’ve been enjoying your time here, please consider a donation - it will help keep Lemmy.zip online and all donations go to paying for the server and for the surrounding services like image hosting and email.


Interaction reminder (help support the instance!)

If you’re new to Lemmy.zip - WELCOME! I hope you’re enjoying your time here :)

I wanted to take a moment to encourage everyone to actively engage with our platform. Your participation can take many forms: upvoting content you enjoy, sharing your thoughts through posts and comments, sparking meaningful discussions, or even creating new communities that resonate with your interests.

It’s natural to see fluctuations in user activity over time, and we’ve seen this over the wider lemmy-verse for some time now. However, if you’ve found a home here and love this space, now is the perfect opportunity to help us thrive.

If you want to support us in a different way than financially, then actively interacting with the instance helps us out loads.


Server Performance

Graph time!

CPU usage over last 30 days:

Disk usage over 30 days:

Network usage:

Image hosting stats: (over 200gb now!)

Cloudflare stats:

Deeper dive into those metrics:

And finally traffic stats from around the world:

As always, if there are any other metrics you'd like to see let me know!

Otherwise thats it for this month, thank you all for being here :)

Demigodrick

62
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

63
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

64
 
 

As we approach the end of the year, I wanted to take a moment to extend a heartfelt wish to each of you for a joyful, peaceful, and relaxing holiday season. This year has been a remarkable journey for us, and we grew to almost 2000 users in our first 6 months (a figure that widely smashed my expectations!). Thank you for being a part of Lemmy.zip and helping to build our corner of the fediverse, and I look forward to what next year will bring for Lemmy and the wider community.

65
 
 

cross-posted from: https://lemmy.ml/post/9631322

Here is our regular update that explains what we have been working on for the past two weeks. This should allow average users to keep up with development, without reading Github comments or knowing how to program.

Last Friday we finally released Lemmy 0.19.0, after a long development time and extensive bug fixing. Read the announcement to find out about the major changes. A few days later on Wednesday we had to publish 0.19.1 to fix a few more bugs that slipped through.

@phiresky fixed the critical bug with outgoing federation in 0.19. Previously he fixed an authentication bug in lemmy-ui which was blocking the 0.19 release.

@dessalines fixed the broken logic for "hide read posts". He also fixed a problem with email login being case sensitive

@nutomic reenabled pushing to crates.io so Rust developers can easily interact with Lemmy. He also made performance optimizations for /api/v3/site and the optimized the Activitypub context sent by Lemmy, reducing the database size and the amount of data sent between instances. He fixed various tests to prevent random failures in continuous integration 1 2

@dullbananas has long been busy improving the database queries for Lemmy, such as fixing a bug in the way different posts sorts are combined, and improving the test cases.

This is our last update for 2023. It was a very busy year for Lemmy, and it looks like 2024 might have even more changes in store. So lets enjoy these holidays, have a Merry Christmas and Happy New Year!

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. Recurring donations are ideal because they allow for long-term planning. But also one-time donations of any amount help us.

66
5
submitted 11 months ago by code to c/home
 
 

Are you looking at federating with threads or not? Since they are rolling out Im interested as Ill need to move servers if so thanks.

67
48
0.19 Issues (self.home)
submitted 11 months ago* (last edited 11 months ago) by Demigodrick to c/home
 
 

Hey all,

Just wanted to loop everyone in to the bugs/issues that have been discovered so far with 0.19. I'll try to keep this updated, however if you notice any others please put them below.

The link to the Lemmy github issues page is here.

We're experiencing most of these (except the outgoing federation one AFAIK, I've not had it reported).

We are especially suffering from the high server load one, and you may see downtime where Lemmy.zip is not loading for short periods (or the message "lemmy is starting") while the DB restarts itself. This should only last a couple of seconds.

To mitigate this, I've temporarily upgraded the performance of the server so this should happen less frequently (once a day instead of once an hour!) but I do sincerely apologise. With any luck this will be fixed soon.

Thanks

Demigodrick

68
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

69
-3
submitted 11 months ago* (last edited 11 months ago) by sarmale to c/home
 
 

Don't think it was ever useful, sometimes the opposite

70
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

71
 
 

Hi All,

Hope everyone had a good November.

Here's the monthly update on what is happening at the moment on Lemmy.zip!


Here's some communities you might be interested in!

New Communities


Server Updates

At the moment we're in a bit of holding pattern pending the release the of Lemmy 0.19 - The latest dev post indicates that the release of 0.19 should be quite soon once they're able to fix the remaining bugs. Following this, we'll schedule some downtime so we can update the server.

Some of the main points of 0.19 are:

  • per user Instance Blocking, i.e. the ability for a user to defederate a whole instance themselves.
  • 2FA fix and reset. This will stop you being able to lock yourself out of your account when setting up 2FA, but if you do have it enabled currently it will reset and you'll need to redo the setup.
  • user settings import/export between servers

You can see more information in this dev post on the changes coming.

In case you've missed it, we also shared the public "Federation Stats" dashboard which is linked in the side bar, or you can click here to go straight there - this is a live(ish) view of Lemmy.zip's federation status.

We've also added some small functions to ZippyBot around reporting posts/comments (working on messages) to acknowledge that the report has been reviewed. We're looking at expanding this (after 0.19 is released) to provide some feedback on the action taken/not taken etc, with the ethos of trying to be as transparent as possible.

We're always on the lookout for new ideas or things we can do better - let us know if there is anything you can think of!

Finally, next week Lemmy.zip turns 6 months old!


Interaction reminder (help support the instance!)

If you’re new to Lemmy.zip - WELCOME! I hope you’re enjoying your time here :)

I wanted to take a moment to encourage everyone to actively engage with our platform. Your participation can take many forms: upvoting content you enjoy, sharing your thoughts through posts and comments, sparking meaningful discussions, or even creating new communities that resonate with your interests.

Your involvement is vital in shaping and nurturing our Lemmy.zip instance. It’s natural to see fluctuations in user activity over time, and we’ve seen this over the wider lemmy-verse for some time now. However, if you’ve found a home here and love this space, now is the perfect opportunity to help us thrive.

Together, we can make Lemmy.zip an even more vibrant and welcoming space for everyone. Thank you for being part of our community!


Donations

As always, we're extremely thankful to people who are kind enough and in a position to donate. You can always see the instance's financial position on OpenCollective, and what we're aiming towards. If you do donate, you'll get your name in the Thank You Thread.

If you've been enjoying your time here, please consider a donation - it will help keep Lemmy.zip online and all donations go to paying for the server and for the surrounding services like image hosting and email.

Want to help but not in a position to do so financially? Then please remember to interact with the instance! The more users who interact = the better Lemmy.zip becomes!


Server Performance

Time for some graphs, numbers, and stats!

CPU usage over the last 30 days:

Network stats over the last 30 days:

Some disk stats over the last 30 days:

Here's our current image hosting storage:

And here is the headline graph from Cloudflare (please note, all these graphs are still affected from Cloudflare's API downtime last month, so traffic is slightly higher in reality)

This is a more detailed look at the requests through Cloudflare:

This is a more detailed look at the bandwidth:

And this is a more detailed look at the unique visitors:

Finally, this is the traffic breakdown for the top countries over the last 30 days:


Finally, I'd just like to thank everyone for being here - without you, this project wouldn't be as amazing as it is!

As we head in to December I'm mindful that my next update will in 2024! So I would like to take this opportunity to wish everyone a joyful and peaceful festive period, and I hope you get to spend it with your family, friends, and the people you love.

Thank you,

Demigodrick

72
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

73
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

74
 
 

Hey there, this is the weekly thread for all general topics and any questions. You can also join us on Matrix for a chat!

75
19
Public Federation Dashboard (grafana.lemmy.zip)
submitted 1 year ago by Demigodrick to c/home
 
 

Hi all,

This is a soft launch of the public federation dashboard - i'll keep it up for a couple of hours and see what impact the performance of having this up has on the server, with a wider view to rolling it out full time if the performance is acceptable.

I will update this post if/when i take the page down.

Thanks,

Demigodrick

view more: ‹ prev next ›