this post was submitted on 18 Sep 2023
7 points (65.2% liked)

Privacy

32120 readers
384 users here now

A place to discuss privacy and freedom in the digital world.

Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.

In this community everyone is welcome to post links and discuss topics related to privacy.

Some Rules

Related communities

much thanks to @gary_host_laptop for the logo design :)

founded 5 years ago
MODERATORS
 

I know that scanning images for Scam is kind if a dystopian and scary. However, that doesn't mean that we need to open ourselves for abusive materials to be sent to us.

What I think we need is some publicly available ML models that can be run on each device voluntary to block SCAM from being shown or stored.

Publicly available models would help but implementing them could be a slippery sloap. If popular encrypted messaging apps start having this feature built in its possible it will become illegal to turn it off or use versions of the app with scanner removed. This would mean that we would effectively stuck with a bad egg in our code.

Maybe the best answer is to not give individuals with questionable history the ability to message you.

Does anyone else have a thought?

top 11 comments
sorted by: hot top controversial new old
[–] [email protected] 24 points 1 year ago (1 children)

The point of CSAM scanners is not to protect children, but to circumvent due process by expanding warrantless surveillance. That is antithetical to FOSS.

So, in a word, no.

[–] [email protected] 15 points 1 year ago (1 children)

There Is a tool that someone built directly to scan images uploaded to lemmy for CSAM.

It is really quite clever. The image is put through a ML/AI model, which describes it (Imange to text), then the text is reviewed against a set of rules to see if it has the hallmarks of CSAM. If it does, it is deleted.

This is fully self hosted.

What I like is that it avoids the trauma of a person having to see those sort of things

[–] [email protected] 2 points 1 year ago (1 children)

Poor guy who had to define the rules.

[–] [email protected] 8 points 1 year ago

you mean the ML model?

I dont think it is too bad, as it is more like look for a description that has children and a sexual context. This can be trained without CSAM as the model generalises situations it has seen before - a pornographic picture (sexual context) and kids playing at a platground (children in the scene).

[–] [email protected] 8 points 1 year ago (1 children)
[–] [email protected] 8 points 1 year ago

Im assuming the latter.

[–] [email protected] 4 points 1 year ago (1 children)

The justifications for closed source scanners are slim, even knowing how a scanner works it would be difficult for CSAM to be altered to completely avoid detection and those gaps could quickly be closed.

We need an open source scanner that can be integrated safely and with trust into FOSS.

This will only happen with government permission as anyone developing this without permission obviously opens themselves up to legal action.

The FOSS community needs to get Governments on side with this but I don’t know where lobbying would be best started. Potentially the EU would be most receptive to this approach?

[–] possiblylinux127 1 points 1 year ago

I figured they could just release a ML model that was trained on CSAM internally

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

Don't see why not. You can download a database of hashes and compare that locally. Granted, those hashes aren't "free", but that's due to the legal status of such material. The principle itself - comparing hashes - can be foss.

Yea people can look into the algorithms to see how they work and circumvent etc., but that's no different than with... Anything else. If someone is motivated enough to distribute the material, they'll make their own network. Foss doesn't make any difference here.