Top

HUBEDA

Workshop on Human Behavior Data Acquisition for Human-Robot Interaction

Co-Located with IEEE RO-MAN 2024
Workshop Date: August 30, 2024
Workshop Venue: Westin Pasadena, 191 N Los Robles Ave, Pasadena, CA 91101

A vast number of advancements have been made in making robots engage in more natural human-robot interaction (HRI). To close the gap on enabling safe seamless human-aware HRI, collection of large data on human behavior, specially diverse multi-person interactions spanning large participant counts, is still an open challenge. Data on human behavior spans visual information on human movements, gestures, and interactions, spoken content, textual information, and physiological data such as heart rate, electromyography, and electrodermal activity.

Through invited talks, demos, papers, and posters, HUBEDA addresses fundamental questions on the acquisition, analysis, and use of large-scale data on human behavior and interactions to enable data-informed HRI. Following topics are of interest.

  • Datasets on single-person behavior and multi-person interaction for HRI
  • Multimodal setups integrating visual, audio, inertial, tactile, and physiological sensing for full-range capture
  • Augmented reality / virtual reality (AR/VR) approaches to scale up human subject data collection for HRI
  • AI, machine learning (ML), vision, and natural language processing (NLP) approaches to estimate parameters of interest for HRI from datasets on human behavior
  • Leveraging large vision and language models to decipher human activity, text, and spoken content
  • Robotic implementations driven using human behavior datasets
  • Robot learning from human behavior datasets and multi-person interactions
  • Applications involving multiple agents such as collaborative assembly, handover, repair
  • Ethical practices in data collection for single/multi-person behavior
  • Addressing challenges in reaching out to diverse population groups, e.g., children, older adults, and individuals with disabilities, to scale up data collection

The workshop is supported through a generous donation from the De Luca Foundation.

PROGRAM SCHEDULE

  • 9:00 AM - 9:10 AM: Opening Remarks (HUBEDA organizers: Natasha Banerjee, Maria Kyrarini, Sean Banerjee)
  • 9:15 AM - 10:15 AM: Talk by Arash Ajoudani
  • 10:15 AM - 10:55 AM: Paper Presentations and Robotics Demo from TARS
    • Analysis of Power Consumption Data Sets for Human-Robot Interaction Using Knowledge Graph and eXtended Reality by Ru-Guan Wang, Chu Hsien Tsai, Mei Cheng Tseng, Chi-Yun Ke, Yi-Fan Chien, Mei-Ling Chuang, and Chien-Cheng Chou
    • VR to AR 3D Display: Enhancing Remote Tele-operation in Healthcare with User-Friendly Interface by Himanshu Vishwakarma and Pradipta Biswas
    • A preliminary assessment of the detection of ERPs with visual stimuli in HRI task by Alessandra Fava, Valeria Villani and Lorenzo Sabattini
    • Toward an Ethical Framework for Using AI in the Assessment of Emotions and Other Mental States by Max Parks and Mark Allison
    • Systematic Enhancement of Neural Sensory Equipment (SENSE) of Tactile Feedback for Assistive Robots by Mashrur Wasek, Nathan Park, Jungkwon Kim and Chung Hyuk Park
    • Robotics Demo from TARS by Ava Megyeri
  • 11:00 AM - 12:00 noon: Robotics Demo from Universal Robots by Rafael Mancilla and Pat Uetz
  • 12:00 noon - 1:00 PM: Lunch and Farewell (organized by RO-MAN)
  • 1:00 PM - 2:00 PM: Talk by Elaine Short
  • 2:00 PM - 3:00 PM: Talk by Vesna Novak
  • 3:00 PM - 3:10 PM: Coffee Break
  • 3:10 PM - 4:10 PM: Talk by Paul Robinette
  • 4:10 PM - 5:10 PM: Talk by Pradipta Biswas
  • 5:10 PM - 5:15 PM: Closing Remarks (HUBEDA organizers: Natasha Banerjee, Maria Kyrarini, Sean Banerjee)

INVITED SPEAKERS

ARASH AJOUDANI
Director, HRII Laboratory
Italian Institute of Technology
VESNA NOVAK
Associate Professor, Department of Electrical Engineering and Computer Science
University of Cincinnati
ELAINE SHORT
Assistant Professor, Department of Computer Science
Tufts University
PAUL ROBINETTE
Assistant Professor, Department of Electrical and Computer Engineering
University of Massachusetts Lowell
PRADIPTA BISWAS
Associate Professor, Center for Product Design and Manufacturing
Indian Institute of Science

DEMOS

At HUBEDA, we will have demos from Universal Robots on current advances and from TARS on integrating physiological sensing with robotic to enable HRI applications. Attendees will receive the opportunity to interact with robots from UR and TARS, and ask the presenters questions about the technology.

DEMO PRESENTERS

RAFAEL MANCILLA
Business Development Manager
Universal Robots
PAT UETZ
Business Development Manager
Advanced Technology Consultants
AVA MEGYERI
Graduate Student, Department of Computer Science and Engineering
Wright State University

SPEAKER INFO


Title: TBA

Abstract: TBA

Bio: Arash Ajoudani is the director of the Human-Robot Interfaces and Interaction (HRI²) laboratory at IIT. He also coordinates the Robotics for Manufacturing (R4M) lab of the Leonardo labs, and is a principal investigator of the IIT-Intellimech JOiiNT lab. He is a recipient of the European Research Council (ERC) proof-of-concept grant 2023 Real-Move and the ERC starting grant 2019 (Ergo-Lean), the coordinator of the Horizon-2020 project SOPHIA, the co-coordinator of the Horizon-2020 project CONCERT, and a principal investigator of the HORIZON-MSCA project RAICAM, and the national projects LABORIUS, COROMAN, and ReFinger. He is a recipient of the IEEE Robotics and Automation Society (RAS) Early Career Award 2021, and winner of the SmartCup Liguria award 2023, Amazon Research Awards 2019, of the Solution Award 2019 (MECSPE2019), of the KUKA Innovation Award 2018, of the WeRob best poster award 2018, and of the best student paper award at ROBIO 2013. His PhD thesis was a finalist for the Georges Giralt PhD award 2015 - best European PhD thesis in robotics. He was also a finalist for the best paper award on human-robot interaction at ICRA2024, the best paper award mobile manipulation at IROS 2022, the best paper award at Humanoids 2022 (oral category), the Solution Award 2020 (MECSPE2020), the best conference paper award at Humanoids 2018, the best interactive paper award at Humanoids 2016, the best oral presentation award at Automatica (SIDRA) 2014, and for the best manipulation paper award at ICRA 2012. His main research interests are in physical human-robot interaction, mobile manipulation, robust and adaptive control, assistive robotics, and tele-robotics.



Title: Measuring human behavior in robotic rehabilitation gyms: from simulation to real-world studies

Abstract: As the cost of rehabilitation robots decreases, it is increasingly common for rehabilitation clinics to own multiple robots. Patients can use these robots to exercise together in a group setting, tentatively dubbed a robotic rehabilitation gym. Such gyms have significant potential as a new rehabilitation delivery model, as they could both increase patient motivation through social interaction and increase cost-effectiveness by reducing the number of therapists needed per patient. However, such gyms may benefit from additional software support (e.g., group performance monitoring, patient-robot assignment suggestions) to avoid overloading the supervising therapist and allow them to focus on the most critical aspects of therapy. However, very little data exists on how humans behave in such gyms, limiting further development. In this presentation, I will thus discuss our ongoing work on simplified simulation models of robotic rehabilitation gyms, which allowed us to develop and evaluate software such as intelligent dynamic patient-robot assignment via stochastic optimization or learning from a human expert. I will then discuss possible next steps for actual human subjects studies of such robotic gyms.

Bio: Vesna Novak is an Associate Professor in the Department of Electrical and Computer Engineering at the University of Cincinnati, with secondary appointments in biomedical engineering and computer science. She received her diploma and PhD in electrical engineering from the University of Ljubljana in 2008 and 2011, respectively. She was a postdoctoral fellow at ETH Zurich, Switzerland, from 2012 to 2014, and then an Assistant and Associate Professor at the University of Wyoming from 2014 to 2021. She is currently Principal Investigator for two NSF grants and one NIH grant, and has authored 49 peer-reviewed journal papers as well as over 50 peer-reviewed conference papers, with over 3500 total citations per Google Scholar. Her research interests include rehabilitation robotics, wearable robotics, serious games, affective computing, and human activity recognition.



Title: Human-Centered AI for Accessible and Assistive Robotics: Towards a Disability-Centered HRI

Abstract: Powered by advances in AI, especially machine learning, robots are becoming smarter and more widely used. Robots can provide critical assistance to people in a variety of contexts, from improving the efficiency of workers to helping people with disabilities in their day-to-day lives. However, inadequate attention to the needs of users in developing these intelligent robots results in systems that are both less effective at their core tasks and more likely to do unintended harm. The Assistive Agent Behavior and Learning (AABL) Lab at Tufts University seeks to apply human-centered design thinking, especially disability ethics, to the design of state-of-the-art robot learning algorithms and interaction frameworks. This talk will explore how disability-community-centered thinking can be used to inspire new directions for intelligent interactive robotics and review recent work from the AABL lab at the intersection of assistive robotics, robot learning, and human-robot interaction.

Bio: Elaine Schaertl Short is the Clare Boothe Luce Assistant Professor of Computer Science at Tufts University. She holds a PhD and MS in Computer Science at the University of Southern California (USC) and a BS in Computer Science from Yale University. Her research applies human-centered design and disability community values to the development, deployment, and evaluation of AI and machine learning for robotics, including: human-centered human-in-the-loop machine learning; disability-friendly assistive robotics; autonomous HRI in groups, public spaces, and other human-human contexts; and accessibility and disability inclusion in robotics education and the computing research community. She is as committed to human-centered research practices as she is to algorithm and robot design: her work spans from designing a low-cost open-source open-hardware robot platform, to understanding family group interactions with socially assistive robots, to designing new neural network architectures for improving human-in-the-loop robot learning. As a disabled faculty member, Elaine is particularly passionate about disability rights in her service work. She is a co-PI of AccessComputing and co-Chair of AccessSIGCHI, an advocacy group that works to increase the accessibility of the 24 SIGCHI conferences.



Title: Collecting Data to Build Trustworthy, Fieldable Robotics

Abstract: As robots enter everyday life, it is important to understand how and why people trust them. Too much trust leads to over-reliance and the expectation that a system performs flawlessly. Too little trust causes people to under-utilize robots, reducing the effectiveness of a deployed system. This talk will discuss data collection experiences, including some cautionary tales, over the last decade of human-robot trust experiments in multiple domains.

Bio: Paul Robinette is an assistant professor in the Department of Electrical and Computer Engineering at the University of Massachusetts Lowell (UML). He has performed extensive experiments on human-robot trust in time-critical situations in virtual simulations, the lab, and the field. In recent years, he focused on field robotics: in-situ human-robot teaming experiments in the marine domain and field experiments for river navigation of autonomous surface vehicles. These projects have yielded datasets that have been released for the human-robot interaction community and the marine robotics community.



Title: AR, VR, MR – How Does It Matter

Abstract: The Covid 19 pandemic and associated work-from-home culture reignites the importance of immersive media. Major industries used various terms like Metaverse (Facebook), Mesh (Microsoft), Nth Floor (Accesnture) and so on to invest and commercialize immersive media related products. Traditionally, immersive media is described through a continuum between reality and virtual reality along with intermediate systems known as Augmented and Mixed Reality systems. In the middle of the commercialization efforts and advertisements from software giants, end users are often confused about the best solution for their needs. There are not many studies on analyzing end users’ feedback across the continuum of immersive media. This talk will present analysis on ocular parameters and EEG power bands of different brain regions while users undertook pointing and selection tasks using state-of-the-art AR, MR and VR media. In parallel, novel case studies will be presented from Aerospace, Smart Manufacturing and Assistive Technology involving XR technologies. The talk will be accompanied by a plethora of video and live demonstrations of XR technology.

Bio: Pradipta Biswas is an Associate Professor at the Department of Design and Manufacturing and associate faculty at the Robert Bosch Centre for Cyber Physical Systems of Indian Institute of Science. He has been elected as a vice chairman of ITU Study Group 9 and also a Co-Chair of the IRG AVA  at International Telecommunication Union. His research focuses on user modelling and multimodal human-machine interaction for aviation and automotive environments and for assistive technology. I set up and lead the Interaction Design (I3D) Lab. His research won Microsoft’s AI 4 Accessibility Grant Award, Facebook’s Responsible AR/VR Award and research grants from Collins Aerospace, Siemens, British Telecom (on AR/VR Systems),  Faurecia Groupe Services Ltd. (on Automotive UI), Wipro Ltd (On Autonomous Vehicle), Defence Research and Development Organization (DRDO), Departments of Science & Technology (DST) and BioTechnology (DBT) and was featured in New Scientist, NDTV, All India Radio and various other media outlets. Earlier, he was a Senior Research Associate at Engineering Department, Research Fellow at Wolfson College and Research Associate at Trinity Hall of University of Cambridge. He completed PhD in Computer Science at  University of Cambridge Computer Laboratory and Trinity College in 2010 and was awarded a Gates-Cambridge Scholarship in 2006. He is a member of the UKRI International Development Peer Review College, Association of Computing Machinery (ACM) and was a professional member of the Society of Flight Test Engineers British Computer Society, Associate Fellow at the UK Higher Education Academy and Royal Society of Medicine.


RAFAEL MANCILLA


Bio: Rafael Mancilla is Business Development Manager at Universal Robots. He spent four years as an application engineer ensuring customers in the West region had successful deployments prior to moving into sales. He works daily with partners and customers to vet out and assist in cobot implementation and is a Certified Universal Robots Instructor. Prior to this role, Rafael worked as a controls engineer on power generating systems, and he obtained his Bachelor of Science degree in Computer Engineering from Cal Poly, Pomona.


PAT UETZ


Bio: Pat graduated from Western Michigan University, with a degree in Business Marketing. After spending the first two years of his career for ATC in Michigan, Pat and his family relocated to Southern California in 2012. With 14 years of experience in Career and Technical Education, Pat is passionate about providing educators with the latest equipment and curriculum related to Advanced Manufacturing and Automated Technologies.


ORGANIZERS

NATASHA BANERJEE
Associate Professor
Terascale All-sensing Research Studio (TARS)
Wright State University
MARIA KYRARINI
Assistant Professor
Human-Machine Interaction & Innovation Lab (HMI2)
Santa Clara University
SEAN BANERJEE
Associate Professor
Terascale All-sensing Research Studio (TARS)
Wright State University
For questions, contact us as hubeda2024 a gmail d com