Fediverse

28380 readers

908 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to [email protected]!

Rules

Posts must be on topic.
Be respectful of others.
Cite the sources used for graphs and other statistics.
Follow the general Lemmy.world rules.

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration), Search Lemmy

founded 1 year ago

MODERATORS

[email protected]

405

The Fediverse Desperately Needs Sustainable File Hosting (evergreenfiles.com)

submitted 1 week ago* (last edited 1 week ago) by [email protected] to c/[email protected]

89 comments fedilink hide all child comments

Disclaimer: I wrote this article and made this website.

There was some talk of this issue in the recent fediverse inefficiencies thread. I'm hopeful that in the future we'll have a decentralized solution for file hosting but for now I deeply believe that users should pay for their own file hosting.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 3 points 1 week ago (1 children)

I think the major advantage is the deduplication - when an image goes viral across Mastodon (or Lemmy) it's currently stored hundreds or thousands of times, each with its own cost. Do you dedupe (for either your customers' benefit or your own)?

[–] [email protected] 2 points 1 week ago* (last edited 1 week ago) (1 children)

Are the images duplicated when shared? My understanding is that only a link to the file is replicated across servers and duplication comes from users manually uploading the same file to another server.

My website does not do any deduplication at this time.

[–] [email protected] 4 points 1 week ago* (last edited 1 week ago) (1 children)

Yes, for example go to https://infosec.exchange/explore

I see the top post as https://infosec.exchange/@[email protected]/113433063621462027 and the image is https://media.infosec.exchange/infosec.exchange/cache/media_attachments/files/113/433/063/582/671/258/original/71da3801e4e4f08c.png

The link is to the original on https://files.mastodon.social/media_attachments/files/113/433/062/676/773/993/original/f828afef5cc7ed1c.png but when you click image the javascript loads a modal with the local cached version (same image as the thumbnail that infosec.exchange loads.

There's lots of different codebases across the fediverse so perhaps some hotlink, but local copies is the default.

[–] [email protected] 5 points 1 week ago* (last edited 1 week ago) (1 children)

The Lemmy server config indicates that is an optional setting to improve user privacy so requests don't ever hit the original server from the client. Those cached files are only temporary and will be deleted after some time. So it's not really full blown duplication.

The default setting is to only generate the thumbnails and store those locally (indefinitely?) but even that can be turned off. I checked and it appears that lemmy.world has the thumbnail generation disabled so all images from other instances just link to the original on that instance.

[–] [email protected] 2 points 1 week ago (1 children)

Ok, so Lemmy doesn't cause the same amount of duplication, but I'd still argue that dedupe is valuable: it saves on hosting costs (your costs, in this case) and users will get a small advantage in having slightly higher cache hits.

[–] [email protected] 3 points 1 week ago

For sure, I'll add it to the list. :)