OSA phase 3 : User Empowerment Tools

In the third in our series of explainers on the draft OSA Phase 3 proposals published by Ofcom for consultation, Prof Lorna Woods looks at Volume 2 (user empowerment tools).

Background

Duties are not just about protecting people from exposure to specific content but about trying to ensure greater choice through transparency and user control, with Category 1 services being subject to the most wide-ranging duties. The provisions on user empowerment are found in sections 14-16 requiring an assessment of specified types of content (that is not illegal content) available (see s 16 OSA) and the provision of tools to adult users enabling those users to reduce the likelihood of encountering this content or to be alerted to its presence. They must also offer adults the ability to filter out non-verified users. This latter duty is linked to duties requiring providers to give all adult users the option of verifying their identity (ss 64 and 65). Ofcom describes these as being “about giving adult users of Category 1 services greater choice and control over the content they see and the people they interact with”, which is apparently “one of the key online safety objectives set out in the Act” – referring to paragraph 4(a)(v) to Schedule 4. Ofcom also refers to its own research on (adult) user experience noting that some adults do not want to see some or all of these types of content, or want to avoid interacting with some users – and that women and minoritised groups, as well as young adults and people with mental health conditions, are more likely to be affected by the content.

The cross-cutting complaints obligations (s 21) applies in this context too, as well as the obligations relating to records and review. Note that Cat 1 service providers are under extra duties in this regard – they must supply Ofcom with the record of assessments made in relation to section 14 (other assessments must be recorded by all providers but need not be sent to Ofcom). Under section 23(3)-(4) of the Act, providers must keep written records of measures taken (or alternative measures taken) to comply with the duties in section 15 (user empowerment), section 17 (content of democratic importance), section 19 (journalistic content) and section 21 (complaints procedures) (See Ofcom Record-keeping and Review Guidance).

Ofcom’s Documents

Ofcom is required to provide a number of documents to help services subject to these obligations to carry out the duties (and Ofcom has noted that all these Cat 1 duties will be dealt with in one code as this is administratively efficient). It has aimed at ensuring flexibility while maintaining consistency with the Illegal Content Codes of Practice and the Protection of Children Codes of Practice. Ofcom, as required by the Act, has “paid particular attention to the needs of vulnerable users and users with certain characteristics”.

In addition it is required to provide guidance on the types of content caught by these rules (instructions for suicide or deliberate self-injury; instructions about/promotion of eating disorder; abusive content based on protected characteristics; content inciting hatred in relation to protected characteristics), as well as guidance on doing an assessment of the content.

Guidance on Content

Given the similarity between the relevant content here and that relating to types of content harmful to children, the guidance follows the guidance in relation to that content, describing the content in almost exactly the same way. This allows providers to use the same systems for identifying these sorts of content for children as for adults. The Guidance gives different examples to reflect the adult context, however. Given the similarity of the content, we do not intend to comment further on the description of content.

Note: Ofcom also states that providers should:

“be aware that code words, substitute terms and phrases, hashtags, sounds, and comments could amount to relevant content. Providers will need to be aware of how language will evolve over time and be deployed by users as a method to avoid detection when posting relevant content.” [4.31]

Guidance on Assessment

Ofcom proposes a 4-stage process: Understand the relevant content; assess likelihood of adult users encountering such content – distinguishing between content types and assigning a risk level to each; decide on appropriate measures; report, review and update. As Ofcom notes, this is basically the same four-step approach as for its risk assessments guidance – which presumably allows the services to use the same process for all three sets of duties. It has also used the idea of core and enhanced inputs to describe the sorts of evidence services should be looking at to do the assessment (Table 5.2).

Core Inputs

• User complaints and reports.
• User data.
• Available evidence relevant to understanding the likelihood of adults with a certain characteristic or who are members of a certain group encountering relevant content which particularly affects them.
• Retrospective analysis of incidents that lead to an increase in relevant content.
• [draft] Guidance on Relevant Content and relevant sections of the Illegal Harms Register and Children’s Register.
• Relevant findings of the provider’s illegal content and, where in scope, children’s risk assessments.
• Results of content moderation systems.
• Evidence drawn from existing controls.
• Other relevant information may also be considered.

Enhanced Inputs

• Results of product testing
• Consultation with internal experts
• Views of independent experts
• Internal and external commissioned research
• Consultation with users
• Result of engagement with relevant representative groups.

Note the language of the assessment is different here from the risk assessments – Ofcom states that this means providers do not need to assess the impact of the content on the user but just the likelihood of adults encountering it – and Ofcom has provided a “Likelihood Level Table” (Table 5, draft Guidance) identifying various factors that are relevant for making an assessment as to likeliness. It seems the threshold for action is a more than negligible likelihood. Ofcom has also suggested that providers “should also consider the likelihood of adult users encountering relevant content that particularly affects them as part of this assessment.” [5.29]

Although Ofcom has added to the list of “core inputs” which providers should take into account (including the results of content moderation systems), the concerns about the sorts of evidence required as a minimum is subject to the same criticism as for the risk assessments for the Illegal Content Codes and Protection of Children Codes – specifically as regards the lack of product testing requirements. As we have said before (see, for example, p22-23 in our response to the protection of children consultation), this approach means that product testing is not something that companies should automatically be doing. This is the more noteworthy because Ofcom has recognised [5.39] that because Category 1 services are defined by size, and that they would therefore expect them to have insights from the content moderation systems, which have been included as a core input. Conversely, Ofcom has removed from the core inputs matters which relate only to children – as these duties relate only to adults [5.42].

The Act requires a suitable and sufficient risk assessment which must include (at least) all the matters listed in the Act (see s 14(5)). Ofcom also suggests that following the guidance will mean that an assessment meets the qualitative threshold (but does not following it mean that an assessment is not suitable and sufficient??).

As for the other risk assessments, Ofcom has proposed an annual review of assessment to catch incremental changes, as well as the reviews required when there has been a significant change. See [5.60] on significant change.

User Empowerment Duties

Ofcom has identified 4 categories of measure:

• default control features and accessibility
• content specific measures (dependent on provider identifying relevant content)
• content-agnostic measures
• tool to filter out non-verified users.

Only the last one is mandatory for all Cat 1 services; the relevance of the others will depend on likelihood. Note also link with duty to enforce ToS (s 72). This note does not discuss the issues with regard non-verified users.

Control features defaults and accessibility

The Code contains a requirement – reflecting the terms of the Act – that users must be given the opportunity to change defaults and that the settings to control the features are easy to access (ADU A1). Note the Act does not specify whether features should be default on or default off and the requirement to present users with a choice applies only in the first instance. The measure as drafted by Ofcom seems devoid of content in the sense that there is no indication what the required measure would look like – though Ofcom has proposed illustrative examples. Ofcom also notes that the nature of the system providing this initial choice will “vary depending on the specific control feature in question, the way that feature is designed, and the nature of the service on which it is offered” [7.11].

Content Specific Measures

There are four elements to this heading:

• content determination measure (ie a system to identify relevant content) (ADU A2)
• measure to give those who switch it on/leave it on the choice not to encounter 1 or more categories of relevant content (ADU A3)
• measure to give those who switch it on/leave it on the choice to be alerted to 1 or more categories of relevant content (ADU A4)
• measure to allow reporting of suspected relevant content (ADU A 5).

Again, Ofcom has taken a flexible approach to what the measure might look like. As regards, ADU A2, Ofcom emphasises it will work with content moderation systems providers already have in place for other duties – and note how interconnected some types of content are for the three different sets of duties. Rather than one category of content, Ofcom sees the rules as applying to three distinct categories, blending suicide and self-harm into one category (rather than 2 as set out in the Act as they have done in relation to the Illegal Content Codes), all the different types of hate into another, and eating disorder into the other category.

On the one hand this arguably gives the user more choice (a point Ofcom makes [8.17]) but doesn’t recognise that people may be sensitive to one sort of abusive speech but not others, so users there may be forced to compromise – and it may be that the hate speech doesn’t arrive in the same way across the categories of affected group (women for example may suffer from deepfakes which men might not, even from minoritised groups). Ofcom notes that more granular tools engage people more [8.17] and it notes the differences between groups and risk of different groups encountering different sorts of content but does not apply it below the level of these three categories. Also the tools might be more effective at one type of speech than others in the same group.

Interestingly Ofcom notes that content determination might not be effective all the time [8.112] which is why it has also introduced content-agnostic measures as well.

Ofcom identifies a range of mechanisms by which providers could satisfy ADU A3 (option not to encounter content):

• filtering out or excluding relevant content from content recommender feeds of relevant users;
• reducing the prominence of content within display mechanisms, for example, related‑content recommendations;
• not displaying this content to users in search results; and
• where providers cannot take content-level action, giving users the option of whether to proceed to an area of a service that contains relevant content.

In its discussion Ofcom notes a range of ways in which users can encounter content: when scrolling through their feed or ‘For You’ page; when they see comments and replies under other content; as a result of the way content recommender systems have interpreted their engagement with related content; autoplay; livestreaming; messaging; community spaces; users personally targeted by other users. Ofcom concludes that users may encounter relevant content at any point on a service so that the obligation to provide measures applies across the whole service – subject to technical feasibility (though what does that mean). Where a service cannot take content level action (eg encrypted direct messaging), that functionality should have measures at access level [8.60].

NB the wording here is “not encounter” rather than reduce the likelihood (as in the Act) [8.61]

For ADU A4, Ofcom gives the following illustrative examples:

• blurs (content is still somewhat viewable as a blur) and overlays (content is still somewhat viewable as it is partially covered behind an opaque element),
• interstitials (content is not visible at all due to being fully covered behind an opaque element), and
• a warning that a part of the service contains relevant content.

Again, users should be able to choose categories of content (but presumably this is just the three categories Ofcom has mentioned??). Ofcom refers to POC where there is more detail on interstitials. It clarifies “it would be most effective to give providers discretion over how to alert users to content, rather than setting out the form an alert should take.” [8.108] But Ofcom notes that taking different approaches to different categories of content may be more effective [8.109] and refers to its Media Literacy Best Practice Principles [8.111]. Ofcom further clarifies:

we considered prescribing the specific technical actions providers could apply that would allow them to discharge their duties. Our provisional view is that this could stifle technical innovation of new ways of alerting users to relevant content. [8.110]

Reporting seems to be considered to be parasitic on the already required reporting functions. Ofcom sees it as a way of empowering users when systems don’t function properly – and user reports will have a role in understanding the categorisation of content.

Content-agnostic measures

The "content agnostic” measures give all registered adult users the option to:

• block individual user accounts on the service,
• block all unconnected user accounts (global blocking) on the service, and
• mute individual user accounts on the service (ADU A6)
• give all registered adult users the option to disable comments on their own posts (ADU A7).

ADU A6 mirrors measures in Illegal Content Codes and Protection of Children Codes. Ofcom describes the required functionalities at 9.9-9.13. It seems Ofcom envisages that the services do not have a choice about this but must offer them (though this may be default off).

Ofcom notes in its rights analysis that:

“rights to freedom of expression and association can include the right of users to choose whose information they wish to consider and with whom they want to associate. If users are blocked under our proposed measure, they would only be prevented from sharing information with the adult users who blocked them, but not more widely.” [9.41]

They do not make express the fact that nobody has a right to force people to listen to them!

Disabling comments, whether at the time of posting content or after, aligns with ICU Code and PCU Code. Again minimal impact on would be commenters’ rights of expression (assuming they have rights against private actors) but they can comment elsewhere and nobody has the right to be invited in to somebody else’s space to talk at them – a point not made by Ofcom!

Comment

In general terms, we welcome Ofcom’s approach to user empowerment tools and their recognition that it is for the providers themselves to understand their services and to provide appropriate tools -this is potentially both flexible and future proof. However, this will mean that the standards Ofcom sets for enforcement will be particularly important in determining appropriate standards for ease of use, visibility and – importantly – effectiveness. We note the attempt to limit constraints of “technical feasibility” by introducing content agnostic measures as well as those that depend on identification of particular content. We also commend the fact that Ofcom is joining the dots between different aspects of its work – here engaging with the media literacy work it has done. In general, Ofcom here is at pains to demonstrate that there are parallels here between systems required to be set up for the other duties and those anticipated as relevant here.

The framing of the user empowerment duties is interesting. The emphasis on giving people control as one of the core objectives of the Act does not map on to the fact that this point is mentioned only in schedule 4. Of course, this is of real substantive significance, save to argue that such a framing should indicate a wide approach to the scope given to the areas in which people can exercise this choice. While concerns about limitations on freedom of expression might suggest that illegal content and content harmful to children should be understood narrowly as those terms trigger restrictions on users’ speech, the reverse is true here. The obligations are speech enhancing (if we see control over what and who users engage with as part of freedom of expression). Services should be encouraged to view the contexts in which these tools are applicable broadly.

It is somewhat unclear how the group of the suicide and the self-harm material together impacts the assessment of likelihood and the need to take action. Moreover, it is unclear whether this group of the areas of harm together undermines the need for tools to be effective for all groups of content, and that users should have a granular choice as to whether to toggle the tool on or off in relation to each type of harm separately.

Although user empowerment tools are not naturally part of safety by design, save to the extent that those tools need to be built in to or allowed to interact with a service, it remains disappointing that Ofcom does not envisage that product testing is a key part of a services development and deployment process – as indicated by Ofcom’s approach to core inputs.

Perhaps the area we have the greatest concern about is Ofcom’s comments in relation to Terms of Service and user empowerment. Ofcom excludes content prohibited by the providers terms of service from the obligations. It seems to think that that type of content would not be available. Yet it is unlikely that any system for dealing with content would be 100% accurate and so this is removing users’ ability to protect themselves. Moreover, this suggests that the obligation to enforce terms of service and the provision of user empowerment tools are alternatives- this is not reflected in the wording of the Act.