Post

🔮 How can we empower online communities to design AI agents tailored to their unique needs and norms? In our #CHI2026 paper, we introduce #Botender, a system that enables collaborative design of AI agents through 🔥case-based provocation🔥

2/ AI agents, or bots, play diverse and important roles in community governance across online platforms. However, many communities rely on third-party bots developed by outsiders that misalign with their specific needs.

3/ This misalignment occurs because third-party bots typically offer limited customization options. Meanwhile, developers lack the community-specific knowledge and capacity to address tailored requests over time.

4/ Recently, LLMs have lowered the technical barriers to designing AI agents, allowing communities to shape the behavior of LLM-powered bots using natural language, without relying on external developers. However, several challenges exist:

5/ First, non-AI experts often focus narrowly on editing LLM prompts for a single interaction scenario, failing to account for unintended bot behavior in other relevant scenarios.

7/ To address these challenges, we present Botender, a system that supports communities in collaboratively designing LLM-powered bots that reflect their specific needs. With Botender, community members can directly propose, iterate on, and deploy custom bot behaviors on their community platform.

8/ To support this collaborative design process, we introduce 🔥case-based provocations🔥: Concrete interaction scenarios that are algorithmically generated to spark collective reflection and discussion about desirable bot behavior, rather than simply validating expected outcomes.

9/ Botender generates these provocations to surface opportunities for prompt improvement, based on common issues in novice-authored prompts. In turn, these cases are also useful in surfacing differing viewpoints about desirable bot behavior within a community.