Human Agent Teaming

A browser-based testbed that measures how an embedded AI changes analyst trust, attention, and coordination.

An AI assistant can summarize accurately and still change what a team trusts and what it stops reading, an effect field deployment hides. With the University of Maryland and Duquesne University, Fraunhofer USA built HATIT, a browser-based testbed that puts participants inside a realistic analyst shift (427 pages across 60 documents, timed data drops, scripted teammates) and instruments trust, workload, attention, coordination, information search, and accuracy. Across 151 participants we varied only the summary type an embedded AI produced: indicative, describing each document at ~5% of its length, or informative and entity-based, replacing it at ~31%. Over a 48-minute shift, trust rose measurably for the informative summarizer and not the indicative one, and participants reached newly arrived teammate information faster, with no difference in accuracy. One design choice changed how analysts worked before it changed how well they worked, precisely what a testbed is built to discover.

Publications

Paletz, S. B. F., Kane, A. A., Diep, M., Nelson, T. M., Porter, A., & Vahlkamp, S. H. (2025). Human-Agent Teaming on Intelligence Tasks (HATIT): A Testbed for Evaluating AI in Intelligence Analysis. Proceedings of the Association for Information Science and Technology, 88th ASIS&T Annual Meeting, 483–494.

Kane, A. A., Paletz, S. B. F., Diep, M., Hajkowski, A., & Porter, A. (2025). Virtual Collaborative Analysis: Effects of Two AI Summarizers. Small Group Research, 56(5), 821–863.