this post was submitted on 29 Jul 2023
42 points (100.0% liked)

Announcements

433 readers
2 users here now

lemmy.zip annoucements

The same rules as the main instance apply here.

founded 1 year ago
MODERATORS
 

Due to a corrupted database following the upgrade we've needed to roll back to a point before the upgrade happened.

I'm really sorry about this, but we've had to take this step so we can continue to properly run the site.

Anything added to the site in the last 12 hours will have been wiped, i.e. posts, communities. We're looking at ways to make sure this doesn't happen in future.

Happy to answer any questions below.

all 19 comments
sorted by: hot top controversial new old
[–] incogtino 9 points 1 year ago (1 children)

All good, thanks for your efforts

FYI the discussion under the upgrade announcement included comments about a minor bug affecting Voyager web app and Memmy users where we could comment using logged in accounts but trying to access our account details resulted in a 'user doesn't exist' type error

No errors again after the rollback

[–] Demigodrick 9 points 1 year ago (1 children)

Thank you - appreciate the patience too.

I think that was probably related to the db getting corrupted. Something went seriously wrong with the migration but it seems unique to this instance, and replicating it without using live data is almost impossible unfortunately.

[–] stratoscaster 7 points 1 year ago

Shout-out to backups

[–] JooberDoober 2 points 1 year ago (1 children)

No worries. This kind of stuff is bound to happen while building the foundations for something this intensive. Take your time and get things right. Anything that can be done to prevent loss of data like this should be the biggest priority, imo.

[–] Demigodrick 3 points 1 year ago

Yeah I agree, we're having discussions on changing infrastructure so we can spin up clones of the site to do testing on before pushing to live, but this would be too big/intensive to host on the VPS we're on so looking at alternative dedicated servers which would probably be better value than spinning up two VPS instances, which is what we have to do at the moment.

[–] Tag365 2 points 1 year ago (1 children)

The federation propagation seems bugged now, posts and comments from here aren't properly propagating onto other instances! I can't see my two recent polls about favorite Pokémon on the other forums. How long is it until this is fixed?

[–] Demigodrick 2 points 1 year ago (1 children)

I can see a post from you if i go to https://lemmy.ml/c/pokemon from about an hour ago - is this the one?

[–] Tag365 2 points 1 year ago (1 children)

Is it the one about the Pokémon genes? I've noticed it spread from Lemmy.zip to Lemmy.ml, but not to Lemmy.world as well.

[–] Demigodrick 1 points 1 year ago* (last edited 1 year ago) (1 children)
[–] Tag365 1 points 1 year ago (1 children)

The posts seem to be propagating now...

[–] Demigodrick 1 points 1 year ago (1 children)

There is definitely an issue, we can see it in the logs, but I think its slowing down federation.

[–] Tag365 2 points 1 year ago

It seems to be catching up... the missing Pokémon posts I made are starting to appear on Lemmy.world.

[–] Ultrawide1654 2 points 1 year ago

No worries!

[–] JohnnyLX91 2 points 1 year ago (1 children)

Btw, is it intentional that you are using the lemmy icon with the ears and whiskers in front instead of the fixed one? :D

[–] Demigodrick 3 points 1 year ago

Haha I just thought it looked cooler to me, makes it stand out more.