Please ensure Javascript is enabled for purposes of website accessibility Publication | Donghoon Shin

Conference & Journal Paper

Trkic G00gle: Why and How Users Game Translation Algorithms

Soomin Kim, Changhoon Oh, Won Ik Cho, Donghoon Shin, Bongwon Suh, Joonhwan Lee
CSCW 2021.

PDF Video Abstract ▼ BibTeX ▼

Individuals interact with algorithms in various ways. Users even game and circumvent algorithms so as to achieve favorable outcomes. This study aims to come to an understanding of how various stakeholders interact with each other in tricking algorithms, with a focus towards online review communities. We employed a mixed-method approach in order to explore how and why users write machine non-translatable reviews as well as how those encrypted messages are perceived by those receiving them. We found that users are able to find tactics to trick the algorithms in order to avoid censoring, to mitigate interpersonal burden, to protect privacy, and to provide authentic information for enabling the formation of informative review communities. They apply several linguistic and social strategies in this regard. Furthermore, users perceive encrypted messages as both more trustworthy and authentic. Based on these findings, we discuss implications for online review community and content moderation algorithms.

title={Trkic G00gle: Why and How Users Game Translation Algorithms},
author={Kim, Soomin and Oh, Changhoon and Cho, Won Ik and Shin, Donghoon and Suh, Bongwon and Lee, Joonhwan},
journal={Proceedings of the ACM on Human-Computer Interaction},
publisher = {ACM},
address = {New York, NY, USA},
url = {},
doi = {10.1145/3476085},
keywords = {Human-AI Interaction, algorithmic experience, gaming, translation algorithm, online review, recommendation algorithm, peer-to-peer platform}

Design Guidelines of Computer-based Intervention for Computer Vision Syndrome: Focus Group Study and In-the-wild Deployment

Youjin Hwang, Donghoon Shin, Jinsu Eun, Bongwon Suh, Joonhwan Lee
Journal of Medical Internet Research 23(3), 2021.

PDF Abstract ▼ BibTeX ▼

Prolonged time of computer use increased the prevalence of ocular problems including eyestrain, tired eyes, irritation, redness, blurred vision, and double vision, collectively referred to as computer vision syndrome. Approximately 70 percent of computer users have vision-related problems. To design the effective screen intervention for preventing or improving computer vision syndrome, we must understand the effective interfaces of computer-based intervention (CBI).

In this study, we aim to explore the interface elements of computer-based intervention for computer vision syndrome to set design guidelines based on pros/cons of each interface element.

We conducted iterative user study to achieve our research goal. First, we conducted workshop to evaluate overall interface elements that are included in the previous systems for computer vision syndrome (N=7). Second, we designed and deployed our prototype LiquidEye with the multiple interface options to the users in the wild (N=11). Participants used LiquidEye for 14 days and during these period, we collected participants’ daily log (N=680). Also, we conducted pre and post survey and post-hoc interviews to explore how each interface element affects system acceptability.

We have collected 19 interface elements for designing intervention system for CVS from the workshop, then, deployed our first prototype LiquidEye. After deployment of LiquidEye, we conducted multiple regression analysis with the user data log to analyze significant elements affecting user participation of the LiquidEye. The significant elements include instruction page of eye rest strategy (P<.05), goal setting of resting period (P<.01), compliment page after user complete the resting (P<.0.001), middle-size popup window(P<.05), and symptom-like visual affect that alarms eye resting time (P<.0.005).

We suggest design implications to consider when designing CBI for computer vision syndrome. The sophisticated design of the customizing interface can make it possible for users to use the system more interactively which results in higher engagement and management of eye condition. There are important technical challenges still to address, but given the fact that this study has been able to sort out various factors related to computer-based intervention, it is expected to contribute greatly to the research of various CBI designs in the future.

author = {Hwang, Youjin and Shin, Donghoon and Eun, Jinsu and Suh, Bongwon and Lee, Joonhwan},
title = {Design Guidelines of Computer-based Intervention for Computer Vision Syndrome: Focus Group Study and In-the-wild Deployment},
journal = {J Med Internet Res},
year = {2021},
month = {Feb},
day = {25},
url = {},
doi = {10.2196/22099}

  TalkingBoogie: Collaborative Mobile AAC System for Non-verbal Children with Developmental Disabilities and Their Caregivers

Donghoon Shin, Jaeyoon Song, Seokwoo Song, Jisoo Park, Joonhwan Lee, Soojin Jun
CHI 2020.
Honorable Mention Award (top 5%)

PDF Slides Abstract ▼ BibTeX ▼

Augmentative and alternative communication (AAC) technologies are widely used to help non-verbal children enable communication. For AAC-aided communication to be successful, caregivers should support children with consistent intervention strategies in various settings. As such, caregivers need to continuously observe and discuss children's AAC usage to create a shared understanding of these strategies. However, caregivers often find it challenging to effectively collaborate with one another due to a lack of family involvement and the unstructured process of collaboration. To address these issues, we present TalkingBoogie, which consists of two mobile apps: TalkingBoogie-AAC for caregiver-child communication, and TalkingBoogie-coach supporting caregiver collaboration. Working together, these applications provide contextualized layouts for symbol arrangement, scaffold the process of sharing and discussing observations, and induce caregivers' balanced participation. A two-week deployment study with four groups (N=11) found that TalkingBoogie helped increase mutual understanding of strategies and encourage balanced participation between caregivers with reduced cognitive loads.

author = {Shin, Donghoon and Song, Jaeyoon and Song, Seokwoo and Park, Jisoo and Lee, Joonhwan and Jun, Soojin},
title = {TalkingBoogie: Collaborative Mobile AAC System for Non-verbal Children with Developmental Disabilities and Their Caregivers},
booktitle = {Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems},
series = {CHI '20},
year = {2020},
isbn = {978-1-4503-6708-0/20/04},
location = {Honolulu, HI, USA},
url = {},
doi = {10.1145/3313831.3376154},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {AAC, developmental disability, assistive technology, caregiver collaboration, accessibility}

Poster & Workshop Paper

Characterizing Human Explanation Strategies to Inform the Design of Explainable AI for Building Damage Assessment

Donghoon Shin, Sachin Grover, Kenneth Holstein, Adam Perer
NeurIPS 2021 Workshop. (AI for HADR)

PDF Poster Abstract ▼ BibTeX ▼

Explainable AI (XAI) is a promising means of supporting human-AI collaborations for high-stakes visual detection tasks, such as damage detection tasks from satellite imageries, as fully-automated approaches are unlikely to be perfectly safe and reliable. However, most existing XAI techniques are not informed by the understandings of task-specific needs of humans for explanations. Thus, we took a first step toward understanding what forms of XAI humans require in damage detection tasks. We conducted an online crowdsourced study to understand how people explain their own assessments, when evaluating the severity of building damage based on satellite imagery. Through the study with 60 crowdworkers, we surfaced six major strategies that humans utilize to explain their visual damage assessments. We present implications of our findings for the design of XAI methods for such visual detection contexts, and discuss opportunities for future research.

author = {Shin, Donghoon and Grover, Sachin and Holstein, Kenneth and Perer, Adam},
title = {Characterizing Human Explanation Strategies to Inform the Design of Explainable AI for Building Damage Assessment},
booktitle = {NeurIPS 2021 Workshop on Artificial Intelligence for Humanitarian Assistance and Disaster Response},
year = {2021},
location = {Sydney, Australia},
keywords = {explainable AI, satellite imagery, damage detection, disaster response}

BlahBlahBot: Facilitating Conversation between Strangers using a Chatbot with ML-infused Personalized Topic Suggestion

Donghoon Shin, Sangwon Yoon, Soomin Kim, Joonhwan Lee
CHI 2021 Extended Abstracts. (Late-Breaking Work)

PDF Poster Video Abstract ▼ BibTeX ▼

It is a prevalent behavior of having a chat with strangers in online settings where people can easily gather. Yet, people often find it difficult to initiate and maintain conversation due to the lack of information about strangers. Hence, we aimed to facilitate conversation between the strangers with the use of machine learning (ML) algorithms and present BlahBlahBot, an ML-infused chatbot that moderates conversation between strangers with personalized topics. Based on social media posts, BlahBlahBot supports the conversation by suggesting topics that are likely to be of mutual interest between users. A user study with three groups (control, random topic chatbot, and BlahBlahBot; N=18) found the feasibility of BlahBlahBot in increasing both conversation quality and closeness to the partner, along with the factors that led to such increases from the user interview. Overall, our preliminary results imply that an ML-infused conversational agent can be effective for augmenting a dyadic conversation.

author = {Shin, Donghoon and Yoon, Sangwon and Kim, Soomin and Lee, Joonhwan},
title = {BlahBlahBot: Facilitating Conversation between Strangers using a Chatbot with ML-infused Personalized Topic Suggestion based on Social Media Posts},
booktitle = {Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems},
series = {CHI EA '21},
year = {2021},
isbn = {978-1-4503-8095-9/21/05},
location = {Yokohama, Japan},
url = {},
doi = {10.1145/3411763.3451771},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {chatbot, topic suggestion, computer mediated communication, chat moderation}

Applying the Persona of User's Family Member and the Doctor to the Conversational Agents for Healthcare

Youjin Hwang, Donghoon Shin, Sion Baek, Bongwon Suh, Joonhwan Lee
CHI 2020 Workshop. (Conversational Agents for Health and Wellbeing)

PDF Abstract ▼ BibTeX ▼

Conversational agents have been showing lots of opportunities in healthcare by taking over a lot of tasks that used to be done by a human. One of the major functions of conversational healthcare agent is intervening users' daily behaviors. In this case, forming an intonate and trustful relationship with users is one of the major issues. Factors affecting human-agent relationship should be deeply explored to improve long-term acceptance of healthcare agent. Even though a bunch of ideas and researches have been suggested to increase the acceptance of conversational agents in healthcare, challenges still remain. From the preliminary work we conducted, we suggest an idea of applying the personas of users' family members and the doctor who are in the relationship with users in the real world as a solution for forming the relict relationship between humans and the chatbot.

author = {Hwang, Youjin and Shin, Donghoon and Baek, Sion and Suh, Bongwon and Lee, Joonhwan},
title = {Applying the Persona of User’s Family Member and the Doctor to the Conversational Agents for Healthcare},
booktitle = {CHI 2020 Workshop on Conversational Agents for Health and Wellbeing},
year = {2020},
location = {Honolulu, HI, USA},
keywords = {chatbot, persona, healthcare}

Linguistic Features to Consider When Applying Persona of the Real Persona to the Text-based Agent

Youjin Hwang, Seokwoo Song, Donghoon Shin, Joonhwan Lee
MobileHCI 2020 Extended Abstracts. (Late-Breaking Result)

PDF Abstract ▼ BibTeX ▼

As artificial intelligence (AI) technologies advance, the possibility of developing virtual agents capable of mimicking human beings is increasing. Furthermore, AI techniques applicable to mimicking certain features of a specific person (e.g., facial expression, voice, motion) are becoming more sophisticated. Although the HCI community has explored how to design or develop AI agents mimicking a real person, limited studies on mimicking someone's text-based behavior shown in the instant messaging exist. This study investigates the features that make users perceive text-based agents as people they know in reality. On top of the previous efforts of designing human-like virtual agents, our work suggests design guidelines for applying the persona of the real person (PRP) to text-based agents.

author = {Hwang, Youjin and Song, Seokwoo and Shin, Donghoon and Lee, Joonhwan},
title = {Linguistic Features to Consider When Applying Persona of the Real Persona to the Text-based Agent},
booktitle = {22th International Conference on Human-Computer Interaction with Mobile Devices and Services},
series = {MobileHCI '20},
year = {2020},
isbn = {978-1-4503-8052-2/20/10},
location = {Oldenburg, Germany},
url = {},
doi = {10.1145/3406324.3410723},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {chatbot, chat analysis, personality, authorship attribution}

Domestic Conference

AmslerTouch: Self-testing Amsler Grid Application for Supporting a Quantitative Report of Age-related Macular Degeneration Symptoms

Donghoon Shin
HCI Korea 2022.

PDF Slides Video Abstract ▼ BibTeX ▼

Age-related macular degeneration (AMD) is a progressive chronic disease that is led by damage in the macula. Due to its irreversible characteristics and disastrous effects on the patients, a precise diagnosis of the symptoms is extremely important. Yet, paper-based Amsler Grid, the most prevalent testing method, is highly limited in that it requires the indirect report of patients and quantitative reporting is difficult. To address this, I propose AmslerTouch, a touch-based Amsler-testing web app that supports patients to self-report AMD symptoms. Based on the heuristic evaluation for identifying limitations and gaining insights, I also discuss possible enhancements of my proposed system.

author = {Shin, Donghoon},
title = {AmslerTouch: Self-testing Amsler Grid Application for Supporting a Quantitative Report of Age-related Macular Degeneration Symptoms},
booktitle = {Proceedings of the 2022 HCI Korea},
series = {HCI Korea '22},
year = {2022},
publisher = {HCI Korea},
keywords = {Healthcare, Medical informatics}

Tech Report

An Analysis of K-MOOC Learners’ Data and an Investigation of its Future Applications

Seoyoung Kim, Soonwoo Kwon, Donghoon Shin, Juho Kim
Issue Paper of Korean National Institute for Lifelong Education, 2019-2.

PDF Abstract ▼ BibTeX ▼

2008년 Prince Edward Island 대학교의 Dave Cormier에 의해 명명된 MOOC는, 평생학습, 열린 교육, 그리고 상호유대라는 가치 아래 널리 확산되고 있다. MOOC는 2010년대 초반부터 Stanford(xMOOC), MIT(MITx), Harvard(HarvardX) 등 대학을 중심으로 제공되었고, Udacity, Coursera 등 영리 목적의 MOOC 제공업체도 생겨나 다양한 강좌를 제공하고 있다. 또한, MOOC는 비단 한 국가의 현상으로 머무르는 것이 아닌, 전세계적인 트렌드가 되고 있다. 많은 국가에서 국가, 대학, 또는 기업을 중심으로 MOOC 플랫폼을 개발하였고, 현재 중국 칭화대학교의 XuetangX, 일본의 gacco, Fisdom, 영국의 FutureLearn 등 다양한 플랫폼이 개발되어 각국에서 MOOC 강좌들을 제공하고 있다. 우리나라의 경우, 2015년 10월 국가주도로 한국형 온라인 공개강좌 ‘K-MOOC(’이 시범 개통되어 운영되고 있다. 2019년 1월 현재 69개 대학에서 인문, 사회, 공학, 자연과학 등 다양한 분야에 걸친 500여개 강좌를 개설하여 제공하고 있다. 이에, 2018년 5월을 기준으로 가입자 27만명, 수강신청 58만명을 달성하는 등 학습자들의 관심이 지속적으로 증가하는 추세이다. 이와 같은 거대하고(Massive), 수강인원의 제한이 없는(Open), 온라인을 통한(Online) MOOC의 특성상 수많은 학습자가 대규모의 로그 데이터를 남긴다. edX 기반으로 된 플랫폼에서(K-MOOC, MITx, HarvardX, XuetangX 등) 학습자는 학습자 정보, 학습자가 특정 강좌를 들을 때의 event 등의 로그를 남기는데, 이러한 방대한 로그를 이용하면 수강자의 행동을 분석할 수 있다. 이를 통해 MOOC 플랫폼과 강좌를 개선할 수 있기 때문에 MOOC에서는 양적 연구를 질적 연구와 함께 병행하는 것이 상당히 중요하다. 이미 HarvardX, MITx, XuetangX 등의 해외 유명 MOOC 기관에서는 사용자 로그 데이터를 기반으로 한 여러 양적 분석을 하고 있다 (Ho et al., 2014). 특히 ACM SIGKDD가 후원하는 데이터마이닝 콘테스트인 KDD CUP 2015에서는, 중국 칭화대의 MOOC 플랫폼인 XuetangX의 학습자 데이터를 제시하고 사용자의 수강취소 패턴을 분석 및 예측하는 것을 주제로 내건 바 있다 ("KDD Cup 2015", 2015). 이는 방대한 학습자 데이터를 이용하면 MOOC 서비스를 개선할 수 있다는 것을 암시하며, 이러한 이유로 많은 MOOC 플랫폼에서 사용자 수강자 통계(demographics), 수강자 행동 양식 등을 통해 서비스의 개선을 시도하고 있다. 이에 따라 하지만 우리나라의 이러한 연구가 부족한 상태이다. 이러한 측면에서, K-MOOC 플랫폼의 지속적이고 성공적인 운영을 위해선 사용자 로그 데이터를 기반으로 한 정량적 연구가 필수적이다. 본 연구에서는 K-MOOC의 기존 사용자 데이터들을 분석하고, 이를 통해 유의미한 결론을 도출하여 추후에 K-MOOC 플랫폼 내에서 1) 사용자, 2) 교수자, 그리고 3)기관의 측면에서 개선안을 제시하는데 도움이 되고자 한다.

author = {Kim, Seoyoung and Kwon, Sunwoo and Shin, Donghoon and Kim, Juho},
title = {An Analysis of K-MOOC Learners’ Data and an Investigation of its Future Applications},
institution = {Korean National Institute for Lifelong Education},
year = {2019},
address = {14 Cheonggyecheon-ro, Jung-gu, Seoul, Republic of Korea},
keywords = {MOOC, online learning, data analysis}

Work In Progress

Evaluating the Impact of Human Explanation Strategies on High-Stakes Human-AI Visual Decision Making

Advised by Kenneth Holstein & Adam Perer
CSCW 2022 (under review)

Exploring the Effects of AI-assisted Emotional Support Processes in Online Mental Health Community

Advised by Hwajung Hong & Jinwook Seo
CHI 2022 LBW (preliminary results under review)