The Oversight Board’s AI and content moderation report raises concerns that artificial intelligence (AI) doing content moderation on social media platforms can be a blunt instrument in cases of over-enforcement – such as where strict application of nudity rules removes breast cancer awareness content because the system cannot understand context – while turning a blind eye to some harmful content. For example, under-enforcement can occur where hate speech is coded to evade detection.
Summary
Artificial Intelligence (AI) is having a significant impact on social media platforms; both in terms of how they are incorporated into company’s platforms, and in the creation and sharing of content by users on them. For platforms, they are rolling out new AI-powered functionality that allows users to generate new content or modify content they have added to the platform while they are also upping the amount of moderation that is performed by AI. More moderation is done by machines, not human beings. This is set to accelerate. In these cases AI is enforcing platform’s content policies and deciding what content is “left up, taken down or sent for human review” [p 2].
Outside of the platforms, users have access to other high quality content generation and manipulation tools enhanced by AI. The ease at which deceivingly realistic content can be generated – both in terms of quality and quantity – poses risks when that content is used for nefarious purposes. Together, this has profound implications, “both for the decisions that companies make to design, develop and incorporate these technologies into their products, as well as the content policies enforced against higher quality user-generated content” [p 2].
The report looks specifically at three areas of concern: image-based sexual abuse, political deepfakes and the inequitable application of moderation.
