84
submitted 1 month ago* (last edited 1 month ago) by [email protected] to c/[email protected]

This is a question that comes to mind every time I spend a few days focusing on the fediverse. Normally I'm on the microblogging side, but now I have a Lemmy account it might start a proper discussion.

So, to the point, pretty much every fedi platform has similar problems with small servers taking a beating whenever a post goes viral. This ends up costing the server owner a bunch of money trying to keep their server alive while thousands of instances attempt to pull large static files from the original host's post. This recently instigated this call to action on this forum.

I've never seen the question of torrents answered and it feels like a lot of effort and a bit self entitled to get the ear of fedi software devs to implement torrents as a solution, so I'm putting this here.

If media files were made into torrents when a post was being created, an extra object could be added to post objects like

'torrentcdn': {
  'https://imagePathAsKey.jpg': {
    'infohash': 'ba618eab...',
    'torrentLocation': 'https://directlinkto.torrent',
    'webseed': 'https://imagePathAsKey.jpg',
    ...
  }
}

This would not break compatibility as it would just be ignored by anything not looking for a 'torrentcdn' object, yet up to date instances could use this instead of directly pulling the static files.

This would benefit instances as when a post goes viral, the load would be distributed amongst all instances attempting to download the file.

This could also benefit clients and instances as larger files like short videos could be distributed using webtorrent, massively reducing the load on server when many people are watching the same video.

Thoughts?

you are viewing a single comment's thread
view the rest of the comments
[-] [email protected] 5 points 1 month ago

Additional suggestion - using magnet links wouldn't even require torrent files

https://github.com/webtorrent/magnet-uri

[-] [email protected] 2 points 1 month ago

I've just been reading up on that. Apparently a magnet link won't work without at least one proper seed, as it still needs to download the torrent file from somewhere. https://github.com/webtorrent/webtorrent/issues/1393#issuecomment-389805621

[-] [email protected] 7 points 1 month ago

A torrent link won't either? In either situation, the site needs to seed their own data, at a minimum.

[-] [email protected] 3 points 1 month ago

A torrent file and a webseed is enough. The client uses the torrent file to validate the download from a standard http source.

The webseed can be the same source as the file your browser would normally download.

So yeah the site needs to seed the file, but not necessarily using a torrent client.

this post was submitted on 05 Jun 2024
84 points (94.7% liked)

Programming

16311 readers
149 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]



founded 1 year ago
MODERATORS