Beyond technology: Protecting the mental health of content moderators

The growth of User Generated Content (UGC) is staggering — the World Economic Forum estimates that, by 2025, we will be creating 463 exabytes of data each day, globally. Not only is data increasing exponentially, it is also becoming an extremely powerful tool to shape opinions and elicit responses. Needless to say, there are serious consequences to both business and society where UGC is fake, hateful, or disturbing. Social platforms and businesses are under immense pressure to moderate the swelling load of content to ensure online safety for all users. However, this extracts an emotional toll on UGC moderation experts who are exposed to it in the line of duty.

Content moderation has become a deep science with the use of Artificial Intelligence, Machine Learning and Natural Language Processing. These are automating moderation, helping manage the vast amount of UGC originating from the billions of active social media users. This is besides UGC on websites, blogs, news sites, e-commerce platforms and travel, food and other review sites in the form of text, video and audio. While technology does help, human moderators must still attend to exceptions. They must use their expertise around local laws, business guidelines, social sensibilities and contextual inferences to cull out what is acceptable and what is not.

Therefore, it is imperative to develop a holistic wellbeing and resilience framework informed by evidence-based research and practice knowledge. It must put team wellness at the center of recruitment, training, operations and exit.

An evidence-based approach to the mental wellbeing challenge

To begin with, the framework must use a variety of tests to assess a candidate’s behavioral competencies. Psychometric testing tools, like the Connor-Davidson resilience scales, could help determine the resilience levels of candidates.

The onboarding process must put candidates through complete training on the concepts of wellbeing and introduce a team of wellness leads. Once on the job, round-the-clock onsite counselling services must be made available with a dedicated HR team and Mental Wellbeing Coaches conducting monthly 1X1 meetings, counselling, yoga and meditation classes, gamification of wellness activities, and workshops that create awareness of the risk to mental health.

On the other side of operations, leads and managers should be trained to spot risk patterns and behavioral changes of employees that help red flag issues. Given the professional hazards involved, moderators must not be exposed to long work hours. Companies must consciously limit the shifts of moderators involved in sensitive workflows, depending on the type of content and the resilience of the moderator.

Perhaps the most important intervention is the introduction of AI/ML driven wellness bots that provides wellness information along with the opportunity to have conversations and interactions in complete privacy. The bot:

• Acts as the first level of screening by proactively identifying patterns in mood/ behavior over time
• Provides behavioral change nudges in small steps to moderators
• Provides solutions based on the mental and physiological profiles of moderators
• Helps create typified nutritional plans that reduce stress levels and improve overall wellness
• Provides a constant feedback loop that customizes wellbeing programs and delivers insights from data points to assist teams/ counsellors
• Streamlines scheduling of wellbeing programs

The heightened use of intelligent technology will reduce the burden on moderators and keep them mentally fit. The technology constantly knows the risks involved and is ready to act, when necessary, without delay. A resilient framework builds confidence in the team to deliver the demands of their roles without faltering.

Anees Khan, Vice President and Global Head of Consumer Business for Wipro-iCORE

An industry veteran, Anees has more than 22 years of IT/ITES experience spanning sales, solutions, and services for global markets, and leading strategic planning and operations for various businesses. In his 19 years with Wipro, Anees has held various leadership roles across industry verticals. He is an accomplished sales and delivery professional, known for leveraging quality management methodologies to streamline processes, drive productivity, and increase competitive positions. He is also a strategic problem-solver with an aptitude for enhancing value propositions and increasing team performance through effective training and management. He is based in the UK.