As AI technology rapidly evolves, schools are facing new safeguarding challenges around the misuse of online imagery. In this article, Alex Dave, Safeguarding Lead at edtech charity LGfL-The National Grid for Learning, explores the growing risks of AI image manipulation and synthetic imagery in education settings. She outlines the real-world impact this can have on pupils, staff and school communities, while offering practical guidance to help school leaders reduce risks, strengthen policies and protect children online.
Navigating the risk of AI image manipulation in schools
As school leaders, our duty to safeguard children and young people seamlessly extends into online spaces. While publishing images on our websites and social media channels helps to share and celebrate achievements and show the vibrancy of school life, advancements in technology have introduced an alarming risk: the unauthorised harvesting and manipulation of school imagery.
A growing concern is where photos of pupils or staff are taken from websites, social media channels and marketing materials. These images are then processed using artificial intelligence tools – such as freely available ‘nudification’ apps – to create non-consensual, synthetic sexual imagery, including AI-generated child sexual abuse material (AI CSAM). The AI tools used to create such abhorrent and illegal images (photos and videos) are neither difficult to find nor use. The reality of this threat requires our attention and proactive leadership.
The Internet Watch Foundation (IWF) has also published recent data about the significant increase in AI Generated CSAM being found by their analysts, suggesting that this technology is already in the wrong hands.
This has led to new guidance being published this month by the UK Online Harms Early Warning Working Group.
Understanding the harm and risks
Is this just fear-mongering? Whilst, of course, we want to continue celebrating the successes of our school and children, the risk landscape has undoubtedly changed. This is no longer a hypothetical scenario, and neither is it victimless. If most parents and carers knew that their child’s image on a website or social media channel could be used to make very real looking, sometimes violent abuse material, would they still consent to their images being published?
The creation of synthetic intimate imagery carries severe real-world consequences
- Vulnerability to blackmail:
Perpetrators can use manipulated images to blackmail individuals or the school itself, demanding money under the threat of public exposure.
- Significant risk to already vulnerable children:
The non-consensual sharing and modification of student photos directly compromises the safety of vulnerable children, such as Looked After Children or those on child protection plans, by exposing personal data or location routines.
- Profound emotional distress:
The emotional toll on the victims, whether they are pupils or staff members, and their families, is immense. Dealing with these breaches is also heavily distressing for the safeguarding teams managing the fallout. And if deepfakes are shared on the web, they are difficult to totally remove, and so the consequences can be long term.
Practical Actions
Senior leadership teams should arrange for a review of what imagery of staff, children, and leaders is publicly available, and then consider the actions required to prevent harm. Such as:
-
- Re-evaluate image necessity – Ask the fundamental question: Are face-on photos actually required? Why not replace direct portraits with alternative angles, group shots, or photos taken from a distance or over the shoulder? We have seen some schools now using avatars or drawings of school staff, leaders and governors instead of photographs.
- Utilise protective technology – Deploying blur or manipulation technology on student faces or background features makes imagery harder to misuse*. Additionally, you can strip out all embedded EXIF metadata (which can reveal device details, times, and routines) before uploading files. But this is not a quick process and requires some technical expertise.
- Implement quality controls – Publish lower-resolution images online to hinder the effectiveness of AI manipulation tools.
- Assess your social media channels – Closed groups are always preferable when posting photos, but any member can still save, edit and share any images posted in the group. If you continue posting images in closed social media groups, regularly review the security settings of the groups and their memberships.
- Remind staff about reporting – Staff should already be familiar with how to report a safeguarding concern and to avoid looking at, saving or sending child sexual abuse imagery (even AI-generated content) – see UKCIS guidance for a reminder. But it could be helpful to communicate to staff about how to report concerns relating to the synthetic imagery of teachers/adults being created.
- Update your policies and AUPS – Make sure this issue of image sharing is covered in your policies, e.g. Online Safety Policy and Acceptable Use Policies. We will be updating our template policies and AUPs shortly.
- Re-issue image consent forms – Ensure that parents and carers (and children over the age of 16 years) are aware of the risks, so can make an informed decision about using their child’s photos. Opt in systems, rather than opt out, would seem more responsible in this new landscape.
* A warning on schools altering images and Data Protection
While navigating these technological threats, schools must exercise extreme caution regarding their own use of AI tools to alter images. Please see edtech charity LGfL’s AI Policy Toolkit for further support with adopting a safeguarding-first approach to using AI in school.
Manipulating student images via unapproved and unchecked AI can lead to privacy breaches and violate UK GDPR obligations. Completing a Data Protection Impact Assessment on any AI tools used is paramount to ensure schools are compliant with legislation.
