this post was submitted on 09 Oct 2023
239 points (87.5% liked)

Technology

59174 readers
1934 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

A nightmare scenario previously only imagined by AI researchers, where AI image generators accidentally spit out non-consensual pornography of real people, is now reality.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 3 points 1 year ago (1 children)

“If the models are trained on images of specific individuals, the models can reproduce images that resemble those people. In the worst case, the model may even directly output verbatim copies of images from the training set,”

[–] [email protected] 3 points 1 year ago

Oh, it's a reference to that paper.

Firstly, that paper was written in January and examined a Stable Diffusion model that was already obsolete due to its poor training even back then. Secondly, even with that poor model they had to move heaven and earth to find a handful of examples out of hundreds of millions of training examples where they could get a blurry replica out.

Here's a Reddit thread from back in the day discussing how, really, this sort of thing just proves how difficult it is to do this.

Secondly, as mentioned, that model is long obsolete due to issues exactly like this. Modern models work better in part because they have better curated training sets that eliminate this sort of "overfitting." There's no indication in this article that the website in question is using one of those old models, it's just presented as a hypothetical concern.