Introduction

The potential of artificial intelligence (AI) to revolutionize border management has been recognized for some time (Beduschi 2020). In the European Union (EU), different AI-based technologies for border control are either already in use or are being tested for future deployment, such as biometric identification and verification, risk assessment, or emotion detection (Dumbrava 2021). This contribution focuses on a subset of the latter: The so-called automatic deception detection system (ADDS), part of the iBorderCtrl project funded by the EU, was developed to detect cases of illegal border crossing by video-interviewing travelers to analyze their facial microexpressions for indicators of deceit. The technology is intended to support border guards in their decision-making process, providing recommendations in the form of risk assessments regarding individual travelers.

Who gets to cross European borders, an already complex social issue with major implications for migrants and society at large, thus becomes embedded in discourses around AI-based decision support. These discourses pertain not only to one state or one scientific discipline. A responsible use of AI-based systems at border crossing points requires responsible policy-making based on an interdisciplinary and transnational perspective. Against this background, in this paper, we bring together epistemological, technological, and social science arguments in order to contribute to an informed assessment of the technology.

After giving a more in-depth description of ADDS, its purpose and theoretical basis, we focus on two interlinked questions:

1. What are the concerns about the use of ADDS and are they warranted? We examine epistemological and empirical arguments against the deployment of ADDS. The first group of arguments targets the system’s theoretical foundation, contending that ADDS rests on a scientifically unfounded basis. The second group attempts to show that the mechanisms underlying ADDS are not sufficiently accurate.
2. Having discussed the above concerns, we turn to the second question: Given the concerns outlined, what would be the social implications of using ADDS in terms of public trust? The concept of public trust is related to various dimensions of technology assessment (TA) such as trustworthy technology, acceptance of new technologies, as well as the promotion and maintenance of trust in technology-related policy-making (Weydner-Volkmann 2021). We address these aspects at the end of the paper, arguing in favor of more transparency in the implementation of systems such as ADDS.

The automatic deception detection system

Purpose and working principle

ADDS is a machine learning (ML) based system designed to identify deception[1] when crossing a state border (O’Shea et al. 2018; Podoletz 2023). It has been dubbed an “AI Polygraph” or “AI lie detector” (Kaminski 2019, p. 178). ADDS is part of iBorderCtrl, which is designed to facilitate and accelerate the registration and control of travelers coming to the EU, including refugees. It has already been tested in Hungary, Latvia, and Greece, with the test phase ending in 2019. Currently, it is not known whether and in what form iBorderCtrl will be deployed; however, with the EU extensively testing this kind of technology and funding several related border control and surveillance projects (iBorderCtrl 2023), critically examining these systems remains relevant.

iBorderCtrl works in the two stages of pre-registration and border crossing. It includes procedures such as biometric identification, document matching, risk analysis, and ADDS, on which this contribution focuses. During pre-registration, travelers undergo an online video interview with a police avatar. ADDS analyzes the recorded interviews, more specifically combinations of the travelers’ microexpressions, very subtle facial expressions, normally invisible to the naked eye, to quantify the probability of deceit. O’Shea et al. (2018, p. 3) point out a difference between microgestures and microexpressions: The former are “more fine-grained and require no functional psychological model of why the behaviour has taken place”. However, since the original description by Rothwell et al. (2006, p. 759) uses the term “microexpression”, we follow this publication. Based on the microexpression analysis and other components of the iBorderCtrl system, a risk estimation regarding the traveler is provided. The system is intended as a human-in-the-loop system: When travelers attempt to cross the border, a border guard makes the final decision after performing a security check against the background of the data provided by the system.

The main component of ADDS is a subsystem called “Silent Talker” (ST) (iBorderCtrl 2018, p. 15). By using several artificial neural networks, ST learns to recognize combinations of microexpressions (by means of supervised or unsupervised methods). The actual classification of these combinations as truthful or deceptive is based on a conceptual model of non-verbal behavior (NVB): “This model assumes that certain mental states associated with deceptive behavior will drive an interviewee’s NVB when deceiving. These include Stress or Anxiety (factors in psychological Arousal), Cognitive Load, Behavioral Control and Duping Delight” (O’Shea 2018, pp. 3–4). It is worth noting that a combination’s classification as deceptive cannot be regarded as proof of deception but as a probabilistic result obtained by inductive learning.

The assumption that microexpressions are capable of revealing deceptive intentions thus forms the theoretical basis of ST, which will be further discussed in more detail in the next chapter.

Theoretical foundation

Deception consists of or involves mental states, especially intentions, while microexpressions are a kind of behavior open to intersubjective investigation. ST and ADDS analyze the latter and thereby provide a basis for human actors to obtain information about the former. Against this backdrop, the question arises as to which possible combinations of microexpressions can indicate an intention to deceive on the part of the subject. In other words, the development of deception detection systems requires a psychological theory about the connections between microexpressions and mental states. In their description of the ST, the inventors explicitly state that they take some key elements from the psychological work of Paul Ekman, such as Ekman’s definition of microexpression (Rothwell et al. 2006, p. 759).

Ekman became famous for his cross-cultural studies of emotional expression. In the 1960s, for instance, he asked indigenous people in New Guinea, at that time largely isolated from the Western world, to assign terms like ‘sad’ to pictures of Europeans expressing different emotions. Experiments with various cultures as well as later studies (Elfenbein and Ambady 2002) resulted in relatively high accuracy rates, leading to the assumption of universal emotional expression. Regarding universal microexpressions, Ekman analyzed video recordings of proven liars frame by frame, finding that subjects with deceptive intentions could not consciously control all facial muscle movements while experiencing a particular emotion (Ekman 1985, p. 133).

Ekman’s theory indicates, therefore, that some unconscious and uncontrollable facial expressions can provide evidence of deception. However, this idea is controversial; in the coming chapter, we will explore criticisms of microexpression analysis and ADDS.

Criticism

Deception detection has been criticized as ‘pseudoscience’ (Whittaker et al. 2018). Against the background of such normative characterizations, we want to assess if the use of ADDS is justified, focusing on the system’s theoretical background and its accuracy; the first aspect is the subject of an epistemological criticism, the second of an empirical one.

Epistemological criticism

The psychological foundation of ST and ADDS has been criticized as flawed, which, according to the systems’ opponents, means their use cannot be justified. A major part of this criticism is an observed lack of scientific consensus on the assumption that deceptive intentions can be derived from microexpressions. In general terms, the epistemological criticism states that the use of deception detection systems is not justified unless there is widespread agreement on their theoretical foundations (Podoletz 2023, p. 1071). Given that Ekman assumes one can derive deceptive intentions from microexpressions, it is reasonable to ask whether there is disagreement among experts about his theory. By the term ‘disagreement’, we mean the incompatibility of other psychological theories with Ekman’s position.

A literature review shows that, indeed, there are disagreements on different aspects and at different levels of abstraction: First, the interpretation of microexpressions as indicators of deceit is not conclusive. Psychologists have made other reasonable assumptions on what microexpressions might indicate (Zhang and Arandjelović 2021). This disagreement touches the very core of ST’s theoretical foundation, for if microexpressions are not indicative of deceit but of suppressed emotions, the system does not measure what it is supposed to. Second, at a deeper level, we find psychological disagreement on whether facial analysis can provide a universal reading of emotions as fixed states (Whittaker et al. 2018, p. 14). Although studies report some advancement in the field (Varghese et al. 2015), it has for instance been argued that emotional expressions are contingent on cultural and social factors (Feldman Barrett et al. 2019), which fundamentally challenges the basis of Ekman’s theory. Lastly, there is no consensus regarding the appropriate classification of emotions; Ekman himself conducted a survey among experts in psychology of emotion to identify the preferred classification model in the field. His results suggest that only 16 % of experts favor the model his theory implies (Ekman 2016, p. 32). The theoretical basis of ST thus seems to be approved only by a minority.

Ekman’s theory indicates that some unconscious and uncontrollable facial expressions can provide evidence of deception.

While this line of thought appears convincing, there are some limitations. To start with, although it is unclear how much disagreement is too much, it seems uncontroversial that perfect consensus is not necessary for justifying the use of a system. On the other hand, even if there were a perfect consensus on the theory as a basis of a system, the use of this system would not be justified without a sufficiently high level of accuracy. This indicates that a lack of consensus on a system’s theoretical basis does not necessarily impact the justification of its use; it can be argued that it is irrelevant whether ST is based on a controversial theoretical basis as long as it is able to distinguish deceptive statements from truthful ones with sufficient accuracy. In the third part of this paper, we will revisit the relevance of the theoretical foundation.

Empirical criticism

The empirical criticism of ADDS considers the accuracy of such systems (Sánchez-Monedero and Dencik 2022). As part of iBorderCtrl, ADDS is intended to support the decision-making of border guards. The use of such subsidiary systems, particularly in high-risk application contexts like border control, is arguably only justified if their accuracy rates exceed those of human experts. Since human actors are expected to make the final decision, the systems would otherwise have no added value; in the worst case, they would direct human experts towards wrong decisions due to the perceived authority of automated outputs (Helm and Hagendorff 2021).[2]

The concern is that in the case of ST this condition is not met: Its accuracy rate has been reported to be 63 to 70 % (Rothwell et al. 2006) and 74.6 % (O’Shea et al. 2018). Empirical findings suggest that while human performance is below these values for untrained subjects, ST’s performance does not exceed human experts. According to a meta-analysis, subjects without special training perform only slightly better than chance (54 %) when attempting to distinguish deceptive from truthful statements (Bond and DePaulo 2006). However, the mean accuracy of trained parole officers has been found to be 76.7 % (Porter et al. 2000) and thus beyond the highest values found for ST.

A second empirical problem is that ST was trained and tested on a surprisingly small number of subjects under controlled experimental conditions. Rothwell et al. worked with 39 subjects and O’Shea et al. with 30, with most of them being male Europeans. Since the aim of this system is to control non-EU citizens, Rothwell et al. openly admit this could lead to data bias (Rothwell et al. 2006, p. 768): Underrepresenting certain groups – such as people of color or women – in AI training datasets can lead to unreliable assessments regarding individuals belonging to these groups and therefore to discriminatory outcomes (Brandner and Hirsbrunner 2023; Selbst 2017). The results of the real-life tests in Hungary, Latvia, and Greece have not been publicly disclosed, and it is thus so far unclear whether they show biases.

These concerns are independent of the epistemological criticism since they relate solely to the accuracy of the system in question. Thus, even ignoring potential flaws in the theoretical foundations, there seems to be no sufficient justification for deploying ADDS. Taking into account the addressed empirical criticism, the next chapter assesses the potential social implications of ADDS. To do so, we build on empirical studies on public attitudes toward the police and AI-based technology.

Social implications

Trust in automated deception detection

AI-based policing and border control evoke divergent public attitudes. For controversial real-time facial recognition technology, acceptance appears to depend on the general trust invested in the police, with a nearly fifty/fifty split regarding the question if the police should be able to use this technology (Bradford et al. 2020). While trust is a much debated, complex notion, someone trusting in an institution like the police can be described as “having confidence that the institution is reliable, observes rules and regulations, works well, and serves the general interest” (Devos et al. 2002, p. 484). Trust in the police varies greatly between European nations and regions, particularly between Scandinavia (high trust) and Eastern Europe (low trust) (Pfister 2020).

Public attitudes toward AI decision-making and support are far from uniform and depend on multiple interrelated factors, such as application context, geographical and cultural differences, or other (perceived) characteristics of the respective technology, such as fairness and transparency (Starke et al. 2022). It has been found that in the application context of justice, automated decisions are perceived as less risky and fairer than decisions made by human experts (Araujo et al. 2020). Others, however, suggest a preference for decisions made by human actors in a policing context (Hobson et al. 2023). Human decisions to accept or reject AI suggestions are furthermore not only contingent on the confidence vested in the system but also on personal self-confidence (Chong et al. 2022). Both over- and undertrust can thus lead to problematic outcomes in high-risk situations such as border control, where humans are meant to make reliable final decisions based on AI suggestions. Given the lack of social consensus, the coming paragraphs assess the implications of both low and high trust in a system like ADDS.

Implications of low trust

Public trust is fundamental for the societal acceptance of AI-based systems and therefore their sustainable adoption (Gillespie et al. 2023). The deployment of unreliable AI-based technology can actively lead to a loss of trust in public institutions (Starke et al. 2022). While, to our knowledge, trust in automated deception detection in a border control context is yet to be studied empirically, automated emotion recognition appears to predominantly evoke negative attitudes: Interviewees describe it as “invasive” and “scary” (Andalibi and Buss 2020, p. 6). Those who believe emotion detection to be accurate can also perceive this accuracy as concerning, i. e., as a threat to human agency (Grill and Andalibi 2022). Others question if the technology can work at all due to the complexity of human emotions. This indicates that academic criticisms of the theoretical basis and empirical accuracy of ADDS are also reflected in citizen concerns.

While ADDS is meant to control non-EU nationals and therefore poses no immediate personal risk to EU citizens, based on the qualitative studies just mentioned it seems plausible to assume that many would fundamentally oppose the use of deception detection in a high-risk setting such as border control. Particularly persons with a general “anti-surveillance” viewpoint (Ezzeddine et al. 2023, p. 869) emphasize the importance of personal freedom over security and oppose all police AI that might flag individuals as suspicious, regardless of who the system is used on. Those generally critical and distrustful of AI policing might perceive the use of ADDS at European borders as threatening to human agency rather than as a trustworthy security measure. Given the already uneven trust in the police within the EU, deploying ADDS might further erode this shaky ground, leading to increased dissonance between nations and political unrest.

The perceived transparency of systems impacts trust, with higher transparency entailing more trust (Aysolmaz et al. 2023). As has been shown in TA studies, trust-building communication cannot only consist of conveying technical aspects such as reliability (Weydner-Volkmann 2021). Meaningful transparency should also include justifying the logic, reasoning, and legality of (semi-)automated decisions (Malgieri 2020). In the case of ADDS, this would inevitably include disclosing and explaining the technology’s contentious theoretical basis and potentially underwhelming accuracy; given the described criticisms that question both if and how the system works, a sufficient justification of its use to the public would thus be challenging.

Implications of high trust

Freedom of movement of EU citizens is one of the cornerstones of the European project. Yet, with the Russian invasion of Ukraine, high immigration and growing support for far-right parties, even internal borders such as Germany-Poland undergo more rigorous checks, while the majority of EU citizens support stricter external border protection and up to a third think individual nations should control their own borders (BrusselsReport.eu 2022). As opposed to groups who fundamentally oppose AI policing, parts of the population passively trust any police action on principle (Bradford et al. 2020), which might lead them to be more accepting of technologies they would oppose in other contexts. Given that the iBorderCtrl system is meant to control non-European migrants while EU citizens maintain privacy, a "Not Me group" (Ezzeddine et al. 2023, p. 872) might also be prevalent in this case; these individuals endorse AI policing for their personal safety but not on their own data and might therefore trust in systems like ADDS.

Deploying automatic deception detection systems might further erode the shaky ground of trust in the police within the EU, leading to increased dissonance between nations and political unrest.

Comprehension is not a necessary prerequisite for trust. Instead, people often trust in things they find too complex to understand (Reinhardt 2023). Considering its justification issues, trust in lie detection technology might be misplaced since unreliable systems can lead to biased decisions; it has for instance been found that emotion detection systems can have racist bias (Rhue 2018). Not only underrepresenting populations can lead to discriminatory outcomes but also overrepresenting already marginalized groups (Bacchini and Lorusso 2019). If predominantly non-EU citizens’ data are fed into ADDS, the system might for instance learn to disproportionately classify the microexpressions of individuals of non-European descent as deceptive. The use of ADDS for border control might not only continue inequalities and discriminatory dynamics but, by automating them, embed them further into the social fabric of an already divided and crisis-ridden Europe.

The question of transparency is again relevant here, but in the context of actively fostering distrust (Ammicht Quinn 2015). Distrust can mobilize citizens to refuse using certain technologies or actively protest them (Büscher and Sumpf 2015), which can incentivize governments and industries to more carefully assess the potential harms of deploying systems such as ADDS. In the case of ADDS, EU citizens’ data would not be analyzed; for EU citizens, expressing distrust in the form of refusal is therefore not possible and those with a "Not Me" perspective might overall not be interested in protesting the technology. However, deploying similar systems at EU borders has the potential to perpetuate and aggravate harmful social inequalities and therefore affect all parts of society. EU citizens should thus be comprehensively informed about the described risks and uncertainties in order to facilitate reasonable distrust in – and therefore resistance against – potentially unjustified technology.

Conclusion

Both immigration and AI-based systems are complex and controversial societal issues. With iBorderCtrl, the EU has attempted to find solutions to the former via the latter. This paper has highlighted the importance of justification for the use of such a system, particularly considering public trust. At the current state, we observe a lack of justification on two fundamental levels: The theoretical basis of deception detection is highly contentious on an epistemological level. From an empirical perspective, varying accuracy rates achieved in small-sample studies do not seem sufficient to justify the usefulness of the technology compared to human experts. At the same time, public opinions and trust regarding AI policing are already divided. A responsible European policy towards related systems must consider the criticism outlined as well as transparently disclose it to the public to prevent both a further loss of trust in public institutions and blind trust in AI-based border control.