Hello all,

That time of the week again! In case you haven't already read them:
Previous Parts

• Part 1
• Part 2
• Part 3
• Part 4

TL;DR

• Lemmy version update - completed to 0.18.2!
• Mobile site - tap away!
• Bot updates - cool bot does even more stuff!
• Email - we have more emails now!
• Lemmy.zip interaction - please interact!
• Funding - thanks!
• Server update - a few fun graphs!

Lemmy update

At the beginning of the week we attempted to upgrade to 0.18.2 a couple of times, each time resulting in the UI not talking to the backend server software and leaving a blank Lemmy page with no interaction. It appears that mobile apps that queried the backend directly continued to work.

Thankfully we backed up before the upgrade (and have daily backups anyway) and did a restore, but this took upwards of 20 minutes to rebuild. After a couple of attempts of this, we gave up testing in production and spun up a temp dev site so we could run further tests without impacting user exeperience.

Thankfully on the dev site we were able to work out that the issue lay with the extra security measures introduced in 0.18.2 and so we worked backwards (with some external help) to figure out it was an issue being caused by Cloudflare's Rocket Loader functionality, which was causing a violation with the new security measures. I toggled the switch in Cloudflare and everything immediately started working, and so we applied this to live and everything upgraded straight away.

Not a fun period of time but I look at it as a learning experience!

PS - we have a chat on Matrix where you can keep up to date on whats going on, get the latest updates etc. Anyone is free to join!

Mobile Site

For those not aware, we now have a mobile site hosted on the server. This is the Voyager software (previously known as Wefwef) - It is a webapp that you can install to your homescreen.

You can see the site by going to (https://m.lemmy.zip). While anyone from any instance can use it, its tailored towards Lemmy.zip users.

Bot Updates

Zippybot continues to get smarter. Zippybot will now PM new users to Lemmy.zip upon signup, so they can get all the information they need straight away.

This is a copy of the PM new users will get:

This has allowed us to unpin the "Read Me" post from the Local page so you should have a cleaner browsing experience, and we'll reserve stickied posts in Local to announcements or short term posts like this one.

The bot is being worked on to help provide advice, suggestions and guidance when users interact with certain parts of the instance, i.e. signing up. We'll look to expand this to creating new communities shortly, and then we'll continue to look for ways to help out.

Email

We've changed email providers from Brevo (who allowed 300 emails a day) to another provider that allows 300 emails per hour. It also now allows us to send and receive email from other accounts now, and so our new contact email is [email protected]. If you have any questions you're more than welcome to email us at that address.

Interaction

As always we'd like to welcome new users to Lemmy.zip, and you can find some really helpful information here and here.

What I'd really to encourage people to do is interact. There are lots of memes popping up about it and lots of posts explaining the importance. We don't have a userbase the size of Reddit, so what we need is people to be that core userbase Reddit had and become posters not lurkers.

Lemmy will live and die based on your interactions with both this instance and the wider Lemmyverse so please, if you create an account, subscribe to what interests you and then create posts, upvote, and comment on everything you can!

We have an open community create policy so feel free to create communities on subjects that interest you.

Funding

I'd once again like to thank everyone who has donated towards the upkeep of Lemmy.zip, it has allowed us to expand and improve the experience in many ways.

We're pretty much sorted, as things currently stand, for the next 10 to 12 months. However we'd like to upgrade the server for better performance, improve page loading times and look to expand the functionality of Lemmy.zip. So if you've been enjoying Lemmy.zip and would like to see it grow, please consider donating.

Server update/graphs

Now for the fun part!

This is CPU usage of the server over the last 24 hours. Due to restoring the server within the last 7 days I can't pull usage graphs for it, but will be able to show these next week again.

Here you can see the image storage stats:

Almost 40 gigs of images!

Here are the Cloudflare stats over the last 7 days:

And here are the visitor numbers: USA has pulled in front of Finland again!

Thats all for now, but if there are any other stats you'd like to see, let us know!

Thanks Demigodrick

We have successfully upgraded to 0.18.2. It only took the best part of 5 hours, 4 restored backups and a whole temporary dev site spun up to figure out it was one tiny setting in Cloudflare that was causing all the issues. The joy.

I'm off to bed with an incredible headache. I sincerely apologise for all the downtime this evening/day, it is not something I want to make a habit of.

If you're on a mobile device and want an Apollo-like Lemmy experience, you can head over to m.lemmy.zip and see Voyager, a web app built for Lemmy.

You can log in with your Lemmy.zip account, and browse and post like you would with the browser, but in an easy-to-use interface.

Edit: RC version of lemmy-ui has been released to fix this issue for emojis, which has been applied to lemmy.zip.

Hello all,

Around 5 hours ago Lemmy.world and lemmy.blahaj.zone experienced a "hack" targeting admin accounts, which then altered the sites and spread spam etc.

Thankfully the attack vector was figured out quite quickly and mitigations were put in place. Sami was very quick to act and defederated us from those instances to prevent their exploit spilling over into our site.

The attack vector is custom emojis, which allowed attackers to exploit weaknesses via cross site scripting. More info is available here for those interested: https://github.com/LemmyNet/lemmy-ui/issues/1895

This attack gives attackers access to your "session". They won't know your password as they did not have access to the database or the server. Attackers would have had access to your user settings page, so they could potentially see your email address.

Lemmy.zip had a custom emoji in place from testing a few weeks ago, so as an extra precaution we've reset the secrets table in the database which should have logged everyone out (sorry!). This would prevent attackers still having access to any accounts on our site.

Its important to add that at this stage I don't believe any of our users have been compromised, due to Sami's quick action to defederate and remove the custom emoji once this was known as the attack vector. No accounts on this instance were involved in the posting of spam and none of our admin accounts were compromised either.

If we find out any more information we'll add it here. We'll continue to implement all security fixes as they become available.

Hi All,

That time of the week again!

I've decided to combine these topics together as they're somewhat related and I don't want to spam people's home pages with more than 2 pinned posts at any one time.

Previous Parts

• Part 1
• Part 2
• Part 3

TL:DR

• Threads Vote - We'll defederate for now
• Lemmy.zip interaction - please interact!
• Bot updates - cool bot does stuff!
• Funding - thanks!
• Server update - bigger server faster website!

Threads (Threxit)

Following the poll for defederation with Threads, the community has voted and has decided that this instance should defederate from Threads.

Therefore within the next few hours I'll be adding Threads.net to the defederation list, and we'll keep an eye out in case there is a different URL they use for federation. However, this is not a final say - we'll monitor how Meta connects to the fediverse and what impact this has on Lemmy, if any.

I am all for user choice/independence, so if the Lemmy devs add the option for users to set their own federation stances then we will look to use that instead of having the instance be defederated in totality.

As a community you get a say in what happens, so if there is strong enough feeling that we want to federate with them in the future, it will go back out as a vote.

It is also important to note that Threads is not yet using ActivityPub and we don't know if/when that move will happen. This is a preemptive decision.

Interaction

Just a reminder that platforms like this one live or die based on interaction with the content. I've seen stats that suggest over 90% of users on social media are lurkers, which ties up closely with our stats too. We have just broken the 1000 registered uses mark (Woo! 🥳) and so we're looking at ways to encourage participation on the instance. We will also look at how we introduce new users and what tools we can implement to encourage them to introduce themselves and contribute to communities.

Please, if you're a lurker and you're reading this, pop into a community and join in with the discussion. Lemmy needs you!

Bot updates

I've been working on a "friendly" bot that will allow users to interact with it in various ways, including using it to vote on polls, providing information to new users and anything else I can manage to get working. Thanks to those who helped test out the #score functionality, the bot passed with flying colours.

If you know anything about Python and databases, please hit me up on Matrix.

For anyone interested the git repo is here

Funding

We continue to have some very generous support from Lemmy.zip users. Thank you all who have donated or are thinking about it. As I'll go into in a minute, we've grown to a point where the server needed to be upgraded which produces additional costs.

Therefore if you’ve been enjoying Lemmy.zip and want to see it stick around, please consider donating. The first option is a "one time donation" option and the other three are recurring options.

Anything you're comfortable sparing will go 100% towards keeping Lemmy.zip running.

I've also started to add expenses to OpenCollective, so as the invoices start to roll in, you'll be able to see what we're spending on the server.

Server Update

There are two big server updates this week. Firstly, we've increased the performance of the server, and secondly we've upgraded to Lemmy 0.18.1.

Both of these have brought performance improvements with them, which is great. Since upgrading, we've had no 502 errors (where Lemmy.zip goes offline for 5 or so minutes) and we've not had to force-restart the server either.

Our server is now "CPX21" with Hetzner for those interested - 3 vcpus, 4gb RAM and we've doubled the storage space to 80GB for the Lemmy database etc.

This is the CPU usage over the last 7 days:

This is the storage space on the server now:

Not much interesting has happened with emails this last week, these are the stats though:

Our images storage continues to grow quite nicely too:

And our Cloudflare stats show some lovely figures:

We also appear to be more popular in Finland now than the USA!

1 Month Old

Finally, Lemmy.zip turns 1 month old tomorrow. I'd just like to thank everyone that has been here and contributed to this lovely community, everyone that has donated, anyone that has made a suggestion or provided advice to help grow this instance and special thanks to Sami who has spent lots of his time helping out on the backend and keeping things running smoothly.

Thanks all,

Demigodrick

Edit: Poll now closed, results shortly.

Hello all,

I made the decision from the start that Lemmy.zip was an instance for the users and not one that I would push any personal political/social beliefs on to. Therefore in situations like this, I feel it is only right the community makes the decision.

There are many posts detailing the threat that Meta poses to the fediverse, how their strategy is likely to have a negative impact and how user data will be scraped and sold. However, on the flip side Threads is not yet using the ActivityPub protocol so we don't yet know the impact (if any) that Threads will have on Lemmy instances, given it is more of a Mastadon competitor. Being defederated will mean that users of this instance will not be able to interact with or receive activities from Threads.

So far we've not defederated from any instances, instead allowing users to choose what content they want to see and interact with. Unfortunately Lemmy doesnt yet allow users to defederate at user level, which would be the ideal outcome here. So until that feature is implemented, we'll be looking at an instance wide approach.

Wider discussions (worded far better than anything I can put together) can be found here:

• https://lemmy.dbzer0.com/post/433248
• https://sopuli.xyz/comment/879382
• https://fedipact.online (warning - very pink)
• https://lemmy.zip/post/266908
• https://lemmy.zip/post/266620
• https://lemmy.zip/comment/524509

Based on this, below is a link to a poll where you can vote for what you would like to see happen. We will leave the poll open for 24 hours after posting this.

I can only stress that this is not a win-win situation and not everybody will be happy with the result. While I'm happy for debate to take place in the comments, personal attacks are never OK. Please don't make me ban anyone in this instance for the first time.

The poll is as follows:

Should Lemmy.zip defederate from Threads?

• Yes
• No
• Wait and see the impact Threads has on the Fediverse

Please do vote. Before any action is taken, we'll look at and share the results of the poll and if there is any ambiguity to the outcome we'll come back to community.

Finally, this vote is not a permanent decision. If we decide to change the outcome of Threxit, we can hold another vote and review what has changed.

Thanks
Demigodrick and Sami.

Appears to have been quite smooth - please let me know of any issues. We'll be monitoring performance for the next hour.

Should be down for a short time while the update is applied.

You may need to do a hard refresh on your browser to refresh your cache - Shift + F5 on chromium browsers, Ctrl + F5 on Firefox.

As always you can check status.lemmy.zip for updates if you can't access the site.

• Space Engineers
• [email protected]
• https://lemmy.zip/c/spaceengineers

Hello all - small request if you have a couple of minutes. Please could you send ZippyBot a message with just #score

Im testing the bots capacity to do multiple things at once, so the more it gets spammed the better! I'll leave this message up for 10/15 minutes or until the bot breaks.

If you get no reply, RIP ZippyBot

Edit: going to leave it running for a while, see how it gets on.

I suppose your score would be closest to reddit "karma"

Hey all, its that time of the week again with another look at the back end services keeping Lemmy.zip running and what changes we're looking at implmenting.

If you've not read them yet, the previous parts are here:

• Part 1
• Part 2

TL;DR

• Matrix Chat
• Automod Bot
• Please remember to interact with Lemmy.zip
• Funding update

Matrix Chat

We've decided to put together a chat room for Lemmy.zip users to come and interact with each other (if they want!). There are various channels there, including a support channel. If there are any issues with Lemmy.zip, a bot over there will automatically notify the channel.

Come and check it out!

Automod Bot

Sami and I have spent the last week trying very hard to learn Python and put some form of automod bot in place. Currently, we've managed to get a bot together that will post various weekly chat threads (which should appear tomorrow for the first time!). The bot is @[email protected]. For those interested, he's using the Pythorhead python library to interact with Lemmy.zip.

Are you a Python dev? Fancy helping out with the bot? Please send me a PM or join the Matrix chat and let us know!

We're currently nowhere near having it be in place for moderation purposes, however my philosophy around the bot will be that it will never take direct action against a user. Rather, we're looking for ways for the moderation team to be able to be notified of anything that may break the instance rules, new accounts that begin post spam, that sort of thing. Then a human will have to read and action that report.

New Users & Interaction

Sami has put together a fantastic post for new users. This is just a general reminder for everyone to please interact with Lemmy.zip. Even a comment or an upvote really helps the instance out. Lemmy counts these interactions for each user and this is used to rank Lemmy instances. The more interactions = the more active this instance = better community.

Funding update

We've had some very generous support from Lemmy.zip users. Thank you all who have donated or are thinking about it. We've now covered the rest of the year in server costs for both the core server and the image server (as things stand!).

We'd like to be able look at upgrading to a dedicated server for more performance power, but with great power comes great ~~responsibility~~ cost. If you've been enjoying Lemmy.zip, please consider donating.

Server stats

I'll start by covering the infrastructure in a bit more detail now we've settled down a bit.

We have the "Core" lemmy server, hosted on Hetzner Cloud. More detail on the Server specifics is in Part 1.

We also have a separate "Status" server that is hosting the status page. This is a really cheap, pretty weak VPS running Uptime-Kuma via docker.

All emails are done via Brevo (formerly Sendinblue for those that have used them before). We're still within the free tier plan (300 emails a day) but it has come close a few times 😅.

Images are hosted on Backblaze via their B2 Cloud Storage Buckets.

Cloudflare is used as a CDN.

The domain is with Google registrar but I'll be looking at moving this ASAP to Porkbun given that the domain service has been sold to Squarespace.

Pretty graph time

Here is a graph showing CPU usage over the last 7 days. You can kind of see how usage has increased over the last 3 days in line with people fleeing from Reddit.

This shows CPU usage over the last 24 hours:

These are email stats over the last 7 days:

And this is the current usage of the Image storage:

This is the overview of Cloudflare usage over the last 7 days:

This is traffic across countries over the last 24 hours Interestingly, Finland has overtaken USA over the last 24 hours, however over the last week USA has smashed it with over 900k requests, and France in second place with almost 500k requests!

Hope everyone has found this interesting, I will look to try and do these weekly where possible.

Finally, welcome to all the new users! Any questions or anything extra you'd like info on, please let me know.

Thanks Demigodrick