this post was submitted on 20 Aug 2023
39 points (95.3% liked)

lemmy.ml meta

1406 readers
1 users here now

Anything about the lemmy.ml instance and its moderation.

For discussion about the Lemmy software project, go to [email protected].

founded 3 years ago
MODERATORS
[email protected]
39
Should lemmy.ml block chatgpt scraping in robots.txt? (lemmy.ml)
submitted 1 year ago by [email protected] to c/[email protected]
14 comments fedilink hide all child comments
 

Some context about this here: https://arstechnica.com/information-technology/2023/08/openai-details-how-to-keep-chatgpt-from-gobbling-up-website-data/

the robots.txt would be updated with this entry

User-agent: GPTBot
Disallow: /

Obviously this is meaningless against non-openai scrapers or anyone who just doesn't give a shit.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 1 points 1 year ago

I think this is a general question and problem for the whole fediverse, and can easily lead to the question of whether, or even when the fediverse is going to embrace having closed or private spaces or even invite only spaces, in order to try to secure some "human interaction only" social media.