Research Scientist, Open Source Technical Safeguards 1 day ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. About the AI Security Institute About the AI Security InstituteThe AI Security Institute is the world's largest and best-funded team dedicated to understanding advanced AI risks and translating that knowledge into action. We are in the heart of the UK government with direct lines to No. 10 (the Prime Minister’s office), and we work with frontier developers and governments globally. About the Role This is a research scientist position focused on developing technical safeguards against tampering with open weight models. The role will focus on mitigating AI-generated CSAM and NCII by targeting the real‑world supply chain driving harm: open‑weight models, adaptation artifacts (LoRAs, guides), and downstream distribution infrastructure (hosting platforms, app stores, operating systems). Our approach prioritises downstream mitigations and actors beyond frontier model developers. This role will build technical tools, protocols, and evidence that platforms and OS/app ecosystems can adopt. Example Projects

Publish a Problem Book framing the technical challenges and research directions for preventing CSAM/NCII misuse across model and hosting layers. Develop threat models for how AI generated CSAM and NCII are created and shared. Design and pilot scalable, automated screening methodologies platforms can run pre‑publication on uploads (topic‑general prototypes that avoid exposure to illegal content). Develop approaches for identifying and tracking known or novel CSAM LoRAs to enable platform blocking at upload. Co‑develop best‑practice protocols with NGOs (e.g., Thorn/IWF) for hosting, app store, and OS enforcement. This is an individual contributor role with no line management responsibilities. You will report into a senior Research Scientist overseeing our team's misuse workstream.

Impact Your work will raise safety standards across hosting and distribution layers, reduce the availability of CSAM/NCII‑generating artifacts (e.g., LoRAs) on major platforms, inform industry protocols and possibly standards, and provide actionable evidence for government decisions. Crucially, we do not expect this role to handle NCII or CSAM material. Role Requirements Essential

At least 3+ years of relevant experience in applied ML, trust & safety tooling, content moderation, security engineering, or adjacent technical fields; we also welcome strong earlier‑career applicants (2‑3 years) with demonstrated impact in open‑source technical work. Deep familiarity with open‑weight image/video models (diffusion, LoRA), model hosting ecosystems (e.g., Hugging Face, GitHub), and the limitations of pre‑deployment safeguards. Strong methodological rigor and creativity; able to design automated, scalable evaluations and detection methods that generalise and avoid reliance on illegal content. Strong Python and ML stack (PyTorch/JAX), data engineering, and systems skills; experience building pipelines and tooling that run at platform scale. Knowledge of fingerprinting and detection approaches (e.g., perceptual hashing, embedding‑based similarity, behavioural signatures), and their privacy and robustness trade‑offs. Excellent writing and communication for technical and policy audiences; ability to translate evidence into practical governance guidance. High agency, ethical judgment, and safe‑working practices for sensitive topics. Commit to work from our London office in Whitehall for parts of the week, with flexibility for remote work. We’re looking for full‑time commitment but are open to part‑time arrangements.

Preferred

Experience collaborating with hosting platforms, app stores, OS vendors, or regulators (e.g., Ofcom) on safety‑by‑design initiatives. Familiarity with Online Safety Act requirements and platform trust & safety operations; prior work with NGOs such as IWF, Thorn, or STOPNCII.org. Expertise in diffusion models and adaptation techniques (LoRA), model evaluation, and secure tooling for sensitive domains. Experience with privacy‑preserving computation, metadata‑poor detection, and standardization efforts (RFCs, protocols). Open‑source contributions (tools, libraries) and evidence of leading cross‑sector technical projects.

Example backgrounds

Senior trust & safety engineer who built automated content integrity pipelines for a large platform; strong OSS track record; experience with model hosting ecosystems. Applied ML researcher with a PhD/postdoc in computer vision or ML safety; hands‑on with diffusion/LoRA; led evaluations and published tooling used by industry. Security/data engineer with 3+ years building scalable detection systems; experience in fingerprinting, hashing, and privacy‑preserving methods; collaborated with regulators/NGOs.

What We Offer Impact you couldn't have anywhere else

Incredibly talented, mission‑driven and supportive colleagues Direct influence on how frontier AI is governed and deployed globally Work with the Prime Minister's AI Advisor and leading AI companies Opportunity to shape the first & best‑resourced public‑interest research team focused on AI security

Resources & access

Pre‑release access to multiple frontier models and ample compute Extensive operational support so you can focus on research and ship quickly Work with experts across national security, policy, AI research, and adjacent sciences

Growth & autonomy

If you're talented and driven, you'll own important problems early. 5 development days per year, an annual L&D budget, and travel support for conferences and external collaborations. Freedom to pursue research bets without product pressure Opportunities to publish and collaborate externally

Life & family

Modern central London office (cafes, food court, gym) or option to work in similar government offices in Birmingham, Cardiff, Darlington, Edinburgh, Salford, or Bristol Hybrid working with opportunities for occasional remote work abroad At least 25 days' annual leave, 8 public holidays, and extra team‑wide breaks Generous paid parental leave (36 weeks of UK statutory leave shared between parents + 3 extra paid weeks + option for additional unpaid time) Plus: 27% government‑funded pension contribution on top of salary, work from home equipment and dental insurance

Salary

Level 3 - Total Package £65,000 - £75,000 inclusive of a base salary £35,720 plus additional technical talent allowance of between £29,280 - £39,280 Level 4 - Total Package £85,000 - £95,000 inclusive of a base salary £42,495 plus additional technical talent allowance of between £42,505 - £52,505 Level 5 - Total Package £105,000 - £115,000 inclusive of a base salary £55,805 plus additional technical talent allowance of between £49,195 - £59,195 Level 6 - Total Package £125,000 - £135,000 inclusive of a base salary £68,770 plus additional technical talent allowance of between £56,230 - £66,230 Level 7 - Total Package £145,000 inclusive of a base salary £68,770 plus additional technical talent allowance of £76,230

Interview Process

Initial interview Technical take home test Second interview and review of take home test Third interview Final interview with members of the senior team

#J-18808-Ljbffr