Keeping Humans in the Loop: Co-Creating Design Principles at HPI d-school

Rather than starting with technology, the workshop focused on design principles as a practical tool. Design principles act as a shared “north star”: they capture what a product should stand for and help teams make tough decisions when many stakeholders and opinions are involved. Participants were encouraged to go beyond classical principles like “simplicity” and explore AI-specific principles such as transparency mechanisms, opt-out and override options, explainability on demand, human feedback loops and ethical design heuristics. These principles make abstract debates about AI governance tangible and actionable in concrete product decisions.

In four domain-focused teams, participants translated their principles into mini use cases:

• Transformative education: A tutoring system where learners and parents stay “in the loop” when setting learning goals. Principles like transparent ethical principles and inconvenience ensure that AI shows its value compass, challenges learners instead of only pleasing them, and actively fosters real-world collaboration.
• Safety-critical everyday decisions: An app for identifying poisonous mushrooms highlighted the need for intuition and strong safety defaults. The system should explain options, show confidence levels, and, when in doubt, clearly recommend: “Don’t eat it.”
• Automation of work and the state: For decisions like housing benefits, participants emphasized trust in the process of iterative improvement: systems should expose risks, collect feedback, and offer an “I don’t think so” button that triggers human review instead of leaving people alone with opaque outcomes.
• AI in healthcare: In an “AI doctor” scenario, principles around trust and transparency focused on making data sources, probabilities and reasoning steps visible, so that patients, doctors and insurers can understand and challenge recommendations over time. 

Across all groups, one message was clear: responsible AI needs built-in ways for humans to understand, question and reshape decisions – before, within and after the automated process. 

Based on the workshop results, the HPI d-school has developed a compact Design for Trust & Human-in-the-Loop mini-playbook, which will soon be available for download on this website. It will offer key principles, templates, and exercises for teams who want to keep human agency at the centre of their AI projects. If you would like to be notified when the playbook goes online, please subscribe to our newsletter.

Thanks to our participants, as well as to our moderators, speakers, and facilitators, Flavia Bleuel, Monika Frech, Dr. Julia Kleeberger, Brian Tepfenhart, Mads Pankow, Christian Bleuel, Maria-José Juarez, Lukasz Lata, Robin Mehra.