Child safety org flags new CSAM with AI trained on real child sex abuse images
arstechnica.com
AI will make it harder to spread CSAM online, child safety org says.

Today, a prominent child safety organization, Thorn, in partnership with a leading cloud-based AI solutions provider, Hive, announced the release of an AI model designed to flag unknown CSAM at upload. It’s the earliest AI technology striving to expose unreported CSAM at scale.

  • hendrik@palaver.p3x.deEnglish
    211·
    16 hours ago

    And will we get that technology to keep the Fediverse and free platforms safe? Probably not. All the predecessors have been kept away for sole use of the big players, despite populism always claiming we need to introduce total surveillance to keep the children safe…

    • BetaDoggo_@lemmy.worldEnglish
      21·
      12 hours ago

      If everyone has access to the model it becomes much easier to find obfuscation methods and validate them. It becomes an uphill battle. It’s unfortunate but it’s an inherent limitation of most safeguards.

      • hendrik@palaver.p3x.deEnglish
        2·
        9 hours ago

        You’re probably right. I’m not sure if it’s a good idea to walk close to the edge with things like this, though. Every update to the detection model could change things and get them in jail… So I certainly wouldn’t play a cat and mouse game with something that has several years of jailtime attached… But then I don’t really know the thought process of the average pedo. And AI image detection comes with problems anyways. In the article they say it detected 6 million pictures already. While keeping quiet about the rate of false positives. We know people have gotten in serious trouble for (false) claims. And I also wouldn’t want to be the Fediverse admin who has to go through thousands of flagged pictures and look at them and decide which is which. With consequences attached… Maybe a database of hashes would be the only option. That doesn’t detect new pictures, but at the same time it comes without flase positives and you can’t draw conclusions from hash values.

    • Riskable@programming.devEnglish
      152·
      15 hours ago

      I was going to say… Sure would be nice to have this feature in all the open source AI image generator tools but you’re absolutely right 😩

      • hendrik@palaver.p3x.deEnglish
        9·
        14 hours ago

        Yeah, unless someone publishes even a set of hashes of known bad content for the general public… I kind of doubt the true intentions are preventing CSAM to the benefit of everyone.