this post was submitted on 29 Jul 2023
29 points (93.9% liked)

Selfhosted

40152 readers
433 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
29
Is it detrimental to the Fediverse network to self host, for only oneself, a Federated service? (lemmy.world)
submitted 1 year ago by [email protected] to c/[email protected]
21 comments fedilink hide all child comments
 

cross-posted from: https://lemmy.world/post/2357075

It seems that self hosting, for oneself, a federated service, like Lemmy, would only serve to increase the traffic in the network, and not actually serve the purpose of load balancing between servers.

As far as I understand it, the way federation is supposed to work is that the servers cache all the content locally to then serve to the people that are registered to that server. In doing so, the servers only have to transmit a minimal amount of data between themselves which lowers the overhead for small servers -- this then means that a small server doesn't get overwhelmed by a ton of people requesting from it. Now, if, instead, you have everyone self hosting their own server, you go right back to having everyone sending a ton of requests to small servers, thereby overwhelming them. It seems that it's really only beneficial to the network if you have, say, hundreds of medium sized servers instead of, say, thousands, of very small servers. While there is the resilience factor, the overhead of the network would be rather overwhelming.

Perhaps one possibility of fixing this is to use some form of load balancer like IPFS to distribute the requests more evenly, but I am no where even remotely close to being knowledgeable enough in that to say anything definitively.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 12 points 1 year ago (1 children)

If it worked like torrenting where you have seeds, etc, it'd scale almost infinitely. I don't think we should change to fit the algorithm. We should change the algorithm to make it scale.

[–] [email protected] 0 points 1 year ago (2 children)

If it worked like torrenting

That would require at least one instance having 100% of the fediverse, would it not?

[–] [email protected] 2 points 1 year ago (1 children)

Why is that?

I’m not super familiar with torrenting protocols, but would have naively assumed that the very fact that subs have a single source of truth (e.g. [email protected] is hosted on lemmy.world in its entirety, and then only cached on other lemmy instances) would be enough?

I guess we’d need to federate the sub list, we wouldn’t want a central source of truth for that, but that bit isn’t any different to what we have currently AFAIK

[–] [email protected] 4 points 1 year ago (1 children)

When you initially start a torrent, you define what "100%" is - all of the files. When you update a torrent, you need all of the updates. The beauty of a federated network is that the network can persist without all of it being available.

I run my own instance. If every other server on the planet crapped out overnight, my instance would still be operable (with whatever content from the federation that I've consumed).

The Fediverse is currently decentralized not distributed, and it should most definitely stay that way, for the sake of my disk space.

[–] [email protected] 2 points 1 year ago

Torrents are both decentralised and distributed.

When you start a torrent, you don't define a 100%, you define only your torrent and nothing else.

To follow your example, if you run your own torrent instance and the network goes down, then of all torrents out there you will have whatever your instance managed to download. It works the exact same way in this regard.

The main issue with decentralised P2P systems is that they're very slow when user count is low.

[–] [email protected] 2 points 1 year ago

Every instance just needs to store the communities they use, just like now. But once cached, any other instance could grab those messages from any of those instances. It'd be a peer to peer sort of organization.

I can think of lots of caveats regarding freshness of content and trust and ensuring the tree of instances is auto organized to minimize depth. Maybe for trust you could have signatures for all content signed using keys that every instance could pull from the original instance just once every now and then.

But, I think it's similar to other things that already exist. These problems seem solvable.