US government report reveals how Discord, Roblox, Reddit and others co-operate with the FBI and Homeland Security over content like 'user-generated re-creations of mass shootings'
The Biden administration describes domestic terrorism and "the most urgent terrorism threat facing the United States."
A newly published US Government Accountability Report (GAO) looks at the role of the Federal Bureau of Investigation and the Department of Homeland Security in combating domestic violent extremism online, and in the process reveals what we surely all knew anyway. Discord? Reddit? Roblox. All your base are belong to the Feds!
The GAO report (first picked up by The Intercept) is more wide-ranging than just gaming, but is essentially focused on online communication platforms of all sorts, how US government and law enforcement agencies interact with them, and what the companies themselves are and should be doing about extreme content. The report is based on interviews with five companies that have been coordinating with the FBI and DHS: Roblox, Discord, Reddit, plus one gaming publisher and one social media company that asked to remain anonymous. Insert your guesses here.
The nature of this cooperation when it comes to the gaming companies is essentially that the DHS holds meetings at which they share information "about online activities promoting domestic violent extremism" or even stuff that violates their terms of service. The companies provide leads on anything that may potentially be illegal and the FBI follows up. In addition to this Discord and Roblox have a "trusted flagger" program through which "approved subject matter experts in extremism can use a streamlined channel to report potentially violating content."
We'll get into the weeds of what the government thinks such content may include, but the report offers some examples. "Researchers have identified user-generated re-creations of mass shootings on the gaming platform Roblox, such as the 2019 shooting in Christchurch, New Zealand, and the 2022 shooting in Buffalo, New York. Roblox officials told us their moderation team reviewed their content moderation tools and adjusted them to block these games and prevent content glorifying violent incidents from appearing on the platform."
Recreations of real world mass shootings (and on a platform with a huge young audience) are one thing, but what may worry some is how vague and potentially over-reaching the US government is about what may constitute domestic violent extremism. Since 2019 it has defined five types of domestic terrorism threat: racially motivated extremism, anti-government and anti-authority extremism, animal rights and environmental extremism, abortion-related extremism, then the catch-all of any other domestic terror threats.
These categories are so broad that people may rightly wonder where the line is between responsible intervention (such as with the Roblox examples) and the potential for Big Brother overreach when someone is having a normal one online about how much they hate the government or the repeal of Roe vs. Wade. To give the report its due, it does acknowledge the big unanswered questions about the friction between the above and Americans' free speech rights, but it's not like the FBI is known for its nuanced approach towards potential extremists: in recent decades it's been behind some shocking lapses and botched schemes, and been accused of entrapment of suspects multiple times.
Nevertheless, this is not going away: the GAO was asked to investigate domestic violent extremists' use of games, gaming-related platforms and social media by the House Homeland Security Committee, and it follows Congressional pressure being applied to various major gaming publishers early last year.
This all arguably stems from the storming of the US Capitol on January 6, 2021, by supporters of former President Donald Trump, an event which saw the then-incoming Biden administration swiftly designate domestic terrorism as "the most urgent terrorism threat facing the United States" once in office and create the first nation-wide strategy for combating it. In this strategy the US government specifically cites "online gaming platforms" as the kind of places where "recruiting and mobilizing individuals to domestic terrorism occurs."
This report was commissioned in order to make recommendations and, generally speaking, it spells-out what US authorities are currently doing and what they should be doing in the future. It ends by saying both the FBI and the DHS's Intelligence and Analysis division should develop their own "strategy and goals for sharing information related to domestic violent extremism with social media and gaming companies."
In civilian speak, that means by this summer both agencies will be issuing their own forward-looking plans for coordinating with gaming and gaming related platforms, and basically telling the publishers what they want to see. Meantime? Enjoy Roblox. Just maybe save the venting about Climate Change for Disc… oh.