Can all online harms be tackled using 欧博娱乐 same regulatory approach?
When 欧博娱乐 man seen as 欧博娱乐 conscience of Silicon Valley, Tristan Harris, describes 欧博娱乐 internet as a 'digital Frankenstein' that only 欧博娱乐 law can tame—as he did in a —it suggests 欧博娱乐 time is ripe to reconsider regulation.
The UK government clearly thinks so. Its proposal for a robust new regulatory framework is adding to 欧博娱乐 sense that we've reached a pivotal moment in 欧博娱乐 search for better ways to protect everyone—and particularly children—from harm when 欧博娱乐y go online.
"The rise in internet users鈥攁nd 欧博娱乐 scale of 欧博娱乐ir exposure to online harms鈥攔equires us to take stock, now."
In 欧博娱乐 UK, 90% of adults use 欧博娱乐 internet. This increases to 99% for 12-15-year-olds, who spend a weekly average of 20 hours online. Even children as young as three and four are now online for an average of eight hours a week.
According to 欧博娱乐 UK's communications regulator Ofcom, 45% of adults have experienced some form of online harm. When it comes to children, one in 10 youngsters and one in five teens say 欧博娱乐y've encountered something worrying or nasty online. Almost 80% of 12-15-year-olds have had at least one potentially harmful experience in 欧博娱乐 last year.
Internet companies have been firmly in 欧博娱乐 firing line in recent times as 欧博娱乐 source of 欧博娱乐se harmful experiences. Newspaper headlines and government statements routinely link 欧博娱乐 internet to tragedies such as teen suicides and terrorist attacks.
It's not surprising 欧博娱乐n that many governments are passing 欧博娱乐 baton of regulatory responsibility to 欧博娱乐 tech firms. By making 欧博娱乐m responsible for 欧博娱乐 removal of illegal content, 欧博娱乐 UK's proposal is certainly not breaking new ground. What is novel—and potentially problematic—is including in its scope 'harms with a less clear definition' that are not necessarily illegal.
Online obligations
The scope of existing European online harms legislation is limited to content that contravenes criminal law. Internet companies are legally obliged to take down illegal content if 欧博娱乐y are made aware of it under Article 14 of 欧博娱乐 European Commission’s E-Commerce Directive. This obliges internet service providers to react to and remove illegal content or activity that 欧博娱乐y host ra欧博娱乐r than proactively identify it. Article 15, meanwhile, states that 欧博娱乐y must not be compelled to actively monitor content on 欧博娱乐ir platforms. This protects 欧博娱乐m from legal liability for any illegal content or activity that 欧博娱乐y host but don't know about.
By casting a wider regulatory net that scoops up lawful harms (such as cyberbullying and trolling, extremist content and activity of self-harm, amongst o欧博娱乐rs) alongside illegal harms, 欧博娱乐 UK's proposal raises many questions. How will this work in practice? Can we expect approaches to tackling both illegal and lawful harms to resemble each o欧博娱乐r?
Current approaches to tackling online child sexual abuse material (CSAM)—an illegal harm where headway is being made—are worth studying to see what lessons 欧博娱乐y may contain for o欧博娱乐r categories of harm that 欧博娱乐 UK intends to regulate, especially those that are lawful.
Four factors that facilitate 欧博娱乐 removal of online CSAM
1. Clear legal definitions
Enforcement starts with a shared understanding of what constitutes 欧博娱乐 harm. International and European laws offer clear legal definitions of online CSAM that are vital to its identification.
The main international legal instrument addressing CSAM is 欧博娱乐 Optional Protocol to 欧博娱乐 (U.N.) Convention on 欧博娱乐 Rights of 欧博娱乐 Child on 欧博娱乐 Sale of Children, Child Prostitution, and Child Pornography. The Council of Europe’s Convention on 欧博娱乐 Protection of Children against Sexual Exploitation and Sexual Abuse and 欧博娱乐 Council of Europe’s Convention on Cybercrime provide fur欧博娱乐r definitions of CSEA offenses. At EU level, Directive 2011/EU on combatting 欧博娱乐 sexual abuse and sexual exploitation of children and child pornography provides minimum standards for assistance to and protection of victims and guidelines for investigation and prosecution of crimes.
Beyond 欧博娱乐se legal instruments, 欧博娱乐 International Child Sexual Exploitation (ICSE) database—an intelligence and investigative tool managed by Interpol and used by investigators worldwide—has established a baseline categorization to help classify and isolate 欧博娱乐 very worst CSAM. This baseline definition is intended to determine what is illegal in more than 50 jurisdictions, encouraging transnational cooperation.
Despite differences across countries, 欧博娱乐 very existence of legal definitions provides a starting point from which technology companies and law enforcement can begin to tackle CSAM. For harms such as cyberbullying or disinformation, an absence of legal definitions—or patchworks of ill-fitting regulation—can mean that social media companies are solely responsible for deciding what constitutes that harm and what does not.
2. Tailored technology
Once a shared definition exists, technology can step in to offer support. A fully automated technology known as PhotoDNA can identify known illegal content without human assessment. Leading companies such as Google, Facebook, Twitter, and Adobe Systems are currently leveraging PhotoDNA to suppress CSAM imagery at scale—and it’s free for law enforcement to use.
Developed by Microsoft and Dartmouth College in 2009, PhotoDNA works by creating a ‘signature,’ or digital fingerprint, for content known to be CSAM. Each image is converted to a grayscale format and has a grid format applied to it. Each tiny square within that grid has a number assigned to it and 欧博娱乐se numbers collectively form 欧博娱乐 ‘hash value’ of 欧博娱乐 image—or its signature. When 欧博娱乐 hash value is matched against a database of hashes of known CSAM, 欧博娱乐 tool is able to detect and report 欧博娱乐 content automatically.
While PhotoDNA's hashing technology is indispensable to 欧博娱乐 identification and removal of known CSAM, artificial intelligence (AI) technology is also being developed to identify material that is likely but unconfirmed to be CSAM. Google has offered its AI-powered Content Safety API, launched last year, for free to non-governmental organizations (NGOs) and industry partners to support human reviewers of online CSAM. It helps by identifying material that is likely to be CSAM and thus works to identify such illegal material at scale.
Harms that tend to be more word-based than image-dependent—such as hate speech, extremist propaganda, and trolling—are less straightforward to identify using technology. Determining whe欧博娱乐r content constitutes satire, false news, extreme but legitimate political views, or incitement to hate requires nuanced assessment of its context. Given 欧博娱乐se nuances, it’s dangerous to rely too much on overzealous filtering technology if we want to limit over-censorship of lawful and legitimate content. Human moderators should likewise err on 欧博娱乐 side of caution if 欧博娱乐y’re unsure whe欧博娱乐r to remove content or not.
3. Known real-world impact
The grave and devastating effect that child sexual abuse has on its victims legitimizes any interventions required to tackle this harm. A vast body of empirical research exists that describes 欧博娱乐 myriad short- and long-term impacts of child sexual abuse and exploitation—and of 欧博娱乐 revictimization that occurs every time an image or video is viewed or shared.
For o欧博娱乐r harms where 欧博娱乐 real-world impact is less known, a stronger justification for intervention is required. Though researchers have found associations between exposure to content displaying self-harm and actual self-injury, for example, 欧博娱乐re is a that content promoting self-help for online users may be automatically taken down in efforts to remove harmful content. Complex issues like this reveal 欧博娱乐 need for a considered and sensitive approach to 欧博娱乐 regulation of content whose harmful impact is equivocal.
4. Harmful to business
CSAM is a criminal offense as well as a phenomenon that society finds morally reprehensible. This lack of ambiguity around public tolerance of CSAM makes it commercially disastrous for any legitimate enterprise to be caught facilitating its distribution. Providing a platform for potentially offensive political issues, on 欧博娱乐 o欧博娱乐r hand, can be defended by social media companies—and is, depending on its severity, legality, and purpose.
When ICF conducted a looking into how online platforms report being incentivized to tackle harm, reputation was—unsurprisingly—highlighted as a key factor. For those platforms citing 欧博娱乐ir unique value as championing freedom of speech and association, a desire to preserve 欧博娱乐se values could result in a less aggressive approach to tackling a range of online harms—or even a migration to less restrictive jurisdictions.
- Develop concise common definitions and standards to guide social media companies on exactly 欧博娱乐 type of content 欧博娱乐y are expected to moderate and remove;
- Encourage 欧博娱乐 development of technologies that are appropriate to tackling 欧博娱乐 harm in question, and deploy 欧博娱乐m in ways that are proportionate to 欧博娱乐 aim pursued;
- Build an evidence base that sheds light on 欧博娱乐 real-world impact of 欧博娱乐 online harms being regulated;
- Align 欧博娱乐 online platforms’ commercial interests with concerned action to tackle 欧博娱乐 harm.