We have been awarded a grant from the Advanced Research + Invention Agency (ARIA) within their Safeguarded AI programme funding projects to develop the safety standards needed for transformational AI. Our project Aggregating Safety Preferences for AI Systems: A Social Choice Approach (ASPAI)aims to develop formal methods for eliciting and aggregating safety specifications and risk thresholds for AI systems, with the goal of contributing to a “gatekeeper AI” that mitigates risks posed by AI agents.
There is a 12-month postdoc position available at HPI as part of this project. Apply by June 15 here.
Team members include Markus Brill (University of Warwick), Paul Goldberg (University of Oxford), Davide Grossi (University of Groningen), Jobst Heitzig (Potsdam Institute for Climate Impact Research and Zuse Institute Berlin), and Wes Holliday (UC Berkeley).
See also the announcement by ARIA.