> I'm really surprised that they tried it in the first place.
Compare the approach Apple floated in their CSAM white paper to what Google is already doing today.
Google:
Is scanning for images of any naked child (including images of your own children that Android automatically backed up) and reporting parents to the police for a single false positive.
>Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the database of known CSAM hashes.
The device creates a cryptographic safety voucher that encodes the match result. It also encrypts the image’s NeuralHash and a visual derivative. This voucher is uploaded to iCloud Photos along with the image.
Using another technology called threshold secret sharing, the system ensures that the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content.
Only when the threshold is exceeded does the cryptographic technology allow
Apple to interpret the contents of the safety vouchers associated with the matching CSAM images.
Apple designed a system where they don't know anything about the data you upload to their server, and where they don't do anything at all about positive matches, until you cross the threshold (later revealed to be 99 images) that match against the database of known kiddie porn.
Even then they have a human review the images you shared before taking further action to protect against the possibility that there might be 99 false positives on a single account.
vs.
Google's system where taking a picture of your own child at the request of your doctor can result in Google reporting you to the police for a single false positive and deleting your account with no human in the loop.
The problem is that Google’s system “works better” from the point of view of law enforcement. By that I mean it’s much less restrictive, and will find “novel” CSAM.
The problem with Apple’s privacy-conscious approach is that it conceded the fundamental principle and agreed that scanning private (unshared) photo repositories was reasonable. Having conceded that point, everything is just a confusing technical argument about effectiveness-versus-restrictions.
Apple would have been under continuous law enforcement pressure to enhance their limited technology so that it was “as effective” as Google’s ML-based scanners. They might have resisted the pressure for a while, but once you’ve conceded the easily-understood principle then all you have to respond with is dull technical arguments that society at large won’t understand. Law enforcement would have (correctly) argued that Apple had agreed that CSAM scanning of private files was justified, so why are they using a creaky old technology that can’t find novel CSAM and is letting bad guys get away with child abuse? Eventually law enforcement would have won that battle and Apple would have been forced to deploy an ML-based scanner, which would have undermined the thoughtful privacy protections deployed in their first version.
> Law enforcement does not have unlimited resources that can be wasted every single time Google's algorithm screws up.
Are you arguing from LE perspective? Or from taxpayer perspective? LE is happy to enforce anything that gives them more power (vide civil forfeiture laws). And signal-to-noise problem is not really a problem if real signal is political expediency. Selective enforcement is the way.
All you're saying is that Google needs to do a more careful human review before reporting to law enforcement. This is not a strong argument for privacy-preserving tech.
No. I'm saying that Google's system of using machine learning to look for images of naked children and reporting parents to the police when there is a single false positive (instead of only looking for known examples of kiddie porn) is exceptionally problematic.
The fact that Google refuses to put humans in the loop when they know the decisions made by their algorithms on this and other subjects are highly unreliable simply adds insult to injury.
Loosing access to your account because Google's algorithm screwed up is bad enough. Being accused of child abuse because you took a picture of your child's first bath is a bridge too far.
Apple's proposed system did scanning on your device, whereas google's system does scanning in the cloud. That is the distinction. A person's own hardware shouldn't be used to report them to the police. You cannot interpret technology like this in terms of what it is applied to today, because its use will be broadened in the future.
> Apple's proposed system did scanning on your device, whereas google's system does scanning in the cloud. That is the distinction.
No, the distinction is that Apple's proposed system only scanned photos you uploaded to publicly accessible iCloud albums and did so in a way that not even they had access to the results until you crossed the "99 images that match known kiddie porn" threshold. Even then, they had an actual human being review the situation to make sure there weren't 99 false positives on a single account before calling the police.
There was absolutely no possibility of calling the police for a single false positive, which is what Google is already doing today.
False positives are inherent to the justice system. While unfortunate, they do not cross any lines or send us down any slippery slopes. They are priced in. Thus, "what about google" is a mere "what-about-ism". People are ocassionally falsely accused, charged, and even convicted of crimes. That is not what is under discussion here.
> False positives are inherent to the justice system
This is in no way an excuse for Google calling the police to report child abuse because their algorithm made a bad call, and Google doesn't want to hire human beings who can intervene when their algorithms very frequently screw up.
Compare the approach Apple floated in their CSAM white paper to what Google is already doing today.
Google:
Is scanning for images of any naked child (including images of your own children that Android automatically backed up) and reporting parents to the police for a single false positive.
https://www.nytimes.com/2022/08/21/technology/google-surveil...
Apple's proposed system:
>Before an image is stored in iCloud Photos, an on-device matching process is performed for that image against the database of known CSAM hashes.
The device creates a cryptographic safety voucher that encodes the match result. It also encrypts the image’s NeuralHash and a visual derivative. This voucher is uploaded to iCloud Photos along with the image.
Using another technology called threshold secret sharing, the system ensures that the contents of the safety vouchers cannot be interpreted by Apple unless the iCloud Photos account crosses a threshold of known CSAM content.
Only when the threshold is exceeded does the cryptographic technology allow Apple to interpret the contents of the safety vouchers associated with the matching CSAM images.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
Apple designed a system where they don't know anything about the data you upload to their server, and where they don't do anything at all about positive matches, until you cross the threshold (later revealed to be 99 images) that match against the database of known kiddie porn.
Even then they have a human review the images you shared before taking further action to protect against the possibility that there might be 99 false positives on a single account.
vs.
Google's system where taking a picture of your own child at the request of your doctor can result in Google reporting you to the police for a single false positive and deleting your account with no human in the loop.