AI Content Moderation and Deepfake NCII
Platform AI moderation catches a fraction of deepfake intimate content. User reports often delay the review process compared to more direct pathways.
Major platforms use AI content moderation to detect and track prohibited intimate content including deepfakes. However, current automated moderation systems catch only an estimated 40-60% of deepfake intimate content, sophisticated deepfakes, properly labeled content, and content in closed communities routinely evade automated detection. Standard user reporting feeds into the same content moderation queue as automated flags, which can slow down the review of flagged content.
How platform AI moderation works
Major platforms use a combination of hash-matching (PhotoDNA and similar), AI classifier models trained to detect explicit content, and human review queues for borderline cases. Hash-matching is effective for exact copies of previously reported content but fails for new deepfakes. AI classifiers flag explicit content but struggle with deepfakes that appear realistic. Human reviewers handle the final determination for flagged content.
Why deepfakes evade automated moderation
Deepfakes evade moderation through: high realism that fools AI classifiers; use of multiple distribution platforms to stay ahead of hash databases; strategic timing of uploads; and distribution through closed communities where moderation is less active. The most sophisticated deepfake operations specifically target moderation system weaknesses.
User reports and content tracking
User reports enter the moderation process and can be impacted by the overall volume of content being processed. Bypassing standard reporting methods can help in expediting the tracking and discovery process.
Frequently asked questions
If I report deepfake content through a platform's standard reporting tool, why might it not be addressed quickly?
Standard reports enter the moderation queue, which processes thousands of items. Priority is determined by automated systems and available reviewer capacity. Exploring alternative reporting options may lead to quicker resolutions.
Can I request that a platform add deepfake content to its hash database?
Yes. ScanErase includes hash-blocking requests in its reporting mechanisms. When a platform adds content to its hash database, it prevents re-uploads of the same content by any user.
Find and remove your deepfakes now
Upload a face photo. We scan 2.4 billion face embeddings for both authentic and AI-generated content featuring your likeness and send legal removal notices in 48 hours.
Scan my face