Branchoria AI Sense AI Sense

Clear, practical explanations of what AI does, where it helps, and where it fails.

Within Face Matches

What Happens When Face Searches Stay Hidden

Disclosure is essential because defendants cannot test an algorithmic lead they never learn was used against them.

On this page

  • Why hidden facial recognition use weakens defence review
  • What courts need to evaluate a face match
  • How disclosure fits with corroborating evidence
Preview for What Happens When Face Searches Stay Hidden

Introduction

Facial recognition systems are often described as investigative tools rather than proof of identity. That distinction matters only if defendants know the technology was used. When police rely on a facial recognition search to generate a suspect and then present later evidence without revealing the original algorithmic lead, defence lawyers may be unable to examine whether the investigation began with a flawed match. Disclosure rules are therefore a crucial safeguard. They allow courts and defendants to ask how a candidate was generated, what alternatives were considered, whether reliability warnings existed, and whether investigators became overly focused on one person because of an AI-generated suggestion. Transparency does not automatically make facial recognition reliable, but it gives the defence an opportunity to test the technology’s role and challenge weak identifications. [arXiv]arxiv.orgMarch 13, 2024…Published: March 13, 2024

Disclosure illustration 1

Why Hidden Facial Recognition Use Weakens Defence Review

A defendant cannot challenge evidence that remains invisible. If a facial recognition search is treated as a confidential investigative step rather than a discoverable part of the case, the defence may never learn that an algorithm helped direct police attention toward the accused.

This creates several problems. First, lawyers cannot investigate whether the source image was poor quality, whether the system produced multiple possible candidates, or whether the match was accompanied by warnings about uncertainty. Second, they cannot explore whether investigators ignored contradictory evidence after receiving a machine-generated lead. Researchers studying public defenders’ experiences with AI-based forensic systems have found that contesting such tools is often difficult because information about how they were used and how decisions were made is hard to obtain. [arXiv]arxiv.orgMarch 13, 2024…Published: March 13, 2024

Hidden use also makes it harder to detect automation bias—the tendency to place excessive trust in computer outputs. Wrongful arrest cases involving facial recognition have repeatedly raised concerns that investigators treated algorithmic suggestions as stronger evidence than they actually were. [ACLU of Georgia]acluga.orgACLU of GeorgiaMore than a Dozen Wrongful Arrests Due to Police Reliance on Facial Recognition Technology - ACLU of GeorgiaApril 17, 2026…Published: April 17, 2026

In practical terms, disclosure serves the same function as disclosure of other investigative techniques: it allows the defence to examine whether the path to the suspect was reliable, lawful and fair.

Face-Off: Recognizing and Challenging the Use of Facial Recognition Technology [webinar]

Channel: NACDLvideo

Open on YouTube

What Courts Need to Evaluate a Face Match

For a court to assess the significance of a facial recognition lead, it generally needs more than the final accusation. It needs information about the process.

Relevant material can include:

  • The image that was searched.
  • The database against which it was compared.
  • The candidate list returned by the system.
  • Confidence scores or ranking information, where available.
  • Examiner notes and human review procedures.
  • Internal warnings about image quality, demographic performance or limitations.
  • Records showing how investigators used the result in subsequent decisions.

Without this information, a court may see only the end product of the investigation rather than the chain of reasoning that produced it.

The importance of such disclosure has become clearer through litigation involving wrongful arrests. Discovery in the lawsuit brought by Robert Williams against Detroit police revealed broader problems in how facial recognition leads were generated and used. Subsequent settlement terms required disclosure of important information about facial recognition searches and factors affecting reliability to detectives, courts and lawyers, specifically so that exculpatory information would not remain hidden. [ACLU of Michigan]aclumich.orgACLU of Michigan Facial RecognitionACLU of MichiganFacial Recognition - ACLU of MichiganDecember 19, 2023…Published: December 19, 2023

The principle is straightforward: if a facial recognition search contributed to identifying a suspect, courts need enough information to evaluate whether that contribution was trustworthy.

Disclosure illustration 2

How Disclosure Fits with Corroborating Evidence

Disclosure rules are closely connected to the broader requirement for independent evidence.

When facial recognition is disclosed, defence lawyers can ask whether later evidence genuinely corroborated the AI-generated lead or merely repeated it. This distinction is important because an investigation can appear to contain multiple pieces of supporting evidence even when each step was influenced by the original algorithmic suggestion.

For example, if a facial recognition search identifies a candidate and investigators then show witnesses a photo array built around that candidate, the resulting identification may not be truly independent. Civil liberties organisations have argued that facial recognition-generated leads can contaminate subsequent identification procedures because the initial match shapes the investigation that follows. [ACLU of Maryland]aclu-md.orgACLU of MarylandFacial Recognition “Lineups” Not Enough for Probable Cause, Says ACLU to State Police - ACLU of MarylandAugust 22, 2024…Published: August 22, 2024

Disclosure helps reveal these relationships. Once the defence knows that facial recognition was involved, it can examine whether:

  • Witness identifications were influenced by the algorithmic lead.
  • Investigators overlooked alternative suspects.
  • Exculpatory evidence emerged during the facial recognition process.
  • The supposedly independent evidence was actually derived from the same initial search.

In this way, disclosure supports the principle that facial recognition should generate investigative leads that must be verified, not conclusions that automatically become evidence of guilt.

Recognizing and Challenging Facial Recognition Technology in Criminal Cases

Channel: NACDLvideo

Open on YouTube

The Tension Between Transparency and Secrecy

Law enforcement agencies sometimes argue that extensive disclosure could reveal sensitive investigative methods, proprietary software details or security procedures. Courts therefore face a recurring tension: defendants need enough information to challenge the evidence, while agencies seek to protect operational practices.

Criminal disclosure systems already address similar conflicts through mechanisms such as protective orders, limited-access disclosures and judicial review of sensitive material. In the United Kingdom, disclosure law generally requires prosecutors to provide material capable of assisting the defence or undermining the prosecution case, while courts can consider claims that some information should be restricted for public-interest reasons. [Policing.uk]policing.ukKey Cases on Disclosure and Evidence | Policing.ukKey Cases on Disclosure and Evidence | Policing.uk

The key governance question is not whether every line of source code must be handed over in every case. Rather, it is whether defendants receive enough information to meaningfully challenge the role facial recognition played in identifying them. If the technology’s involvement remains concealed, adversarial testing—the process through which courts evaluate evidence—becomes much harder.

Detroit woman sues Detroit Police Department after arrest over false facial recognition match

Channel: CBS Detroit

Open on YouTube

Disclosure as a Due Process Safeguard

The most important function of disclosure is procedural rather than technical. It preserves the defendant’s ability to question the evidence against them.

Facial recognition systems can make mistakes, especially when working with low-quality images, unusual lighting conditions or other challenging inputs. Researchers continue to document performance variations and fairness concerns under real-world conditions. [arXiv]arxiv.orgAccuracy and Fairness of Facial Recognition Technology in Low-Quality Police Images: An Experiment With Synthetic FacesMay 20, 2025…Published: May 20, 2025

Because those limitations exist, defendants need access to information showing how the technology was used in their case. Disclosure allows lawyers to investigate alternative explanations, challenge overconfidence in algorithmic outputs and expose weaknesses that might otherwise remain hidden. It also gives judges a clearer basis for deciding whether later evidence truly corroborates a facial recognition lead.

Within the broader principle that facial recognition matches require independent evidence, disclosure serves as the mechanism that makes scrutiny possible. A defendant cannot test an algorithmic lead that they never learn existed. Without transparency, the opportunity to challenge a potentially flawed facial recognition match may disappear before the case ever reaches trial. [arXiv]arxiv.orgMarch 13, 2024…Published: March 13, 2024

Disclosure illustration 3

Amazon book picks

Further Reading

Books and field guides related to What Happens When Face Searches Stay Hidden. Use these as the next step if you want deeper reading beyond the article.

Browse more on Amazon: Your Face Belongs to Us The End of Policing Weapons of Math Destruction

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Example marketplace items related to this page. Use the search link to explore similar finds on eBay.

Using USA
Browse more on eBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

  1. Source: arxiv.org
    Link: https://arxiv.org/abs/2403.13004
    Source snippet

    March 13, 2024...

    Published: March 13, 2024

  2. Source: policing.uk
    Title: Key Cases on Disclosure and Evidence | Policing.uk
    Link: https://policing.uk/knowledge/case-law/key-cases-disclosure

  3. Source: arxiv.org
    Link: https://arxiv.org/abs/2505.14320
    Source snippet

    Accuracy and Fairness of Facial Recognition Technology in Low-Quality Police Images: An Experiment With [Synthetic]({{ 'synthetic-media/' | relative_url }}) FacesMay 20, 2025...

    Published: May 20, 2025

  4. Source: arxiv.org
    Title: Ulixes: Facial Recognition Privacy with Adversarial [Machine Learning]({{ ‘machine-learning/’ | relative_url }})
    Link: https://arxiv.org/abs/2010.10242
    Source snippet

    October 20, 2020...

    Published: October 20, 2020

  5. Source: acluga.org
    Link: https://www.acluga.org/news/more-than-a-dozen-wrongful-arrests-due-to-police-reliance-on-facial-recognition-technology/
    Source snippet

    ACLU of GeorgiaMore than a Dozen Wrongful Arrests Due to Police Reliance on Facial Recognition Technology - ACLU of GeorgiaApril 17, 2026...

    Published: April 17, 2026

  6. Source: aclumich.org
    Title: ACLU of Michigan Facial Recognition
    Link: https://www.aclumich.org/en/cases/facial-recognition
    Source snippet

    ACLU of MichiganFacial Recognition - ACLU of MichiganDecember 19, 2023...

    Published: December 19, 2023

  7. Source: aclu-md.org
    Link: https://www.aclu-md.org/es/press-releases/facial-recognition-lineups-not-enough-probable-cause-says-aclu-state-police/
    Source snippet

    ACLU of MarylandFacial Recognition “Lineups” Not Enough for Probable Cause, Says ACLU to State Police - ACLU of MarylandAugust 22, 2024...

    Published: August 22, 2024

  8. Source: ilga.gov
    Link: https://www.ilga.gov/legislation/ILCS/details?ActID=1966&ActName=Code+of+Criminal+Procedure+of.&ChapAct=725+ILCS%2F&Chapter=&ChapterID=54&MajorTopic=&Print=True&SeqEnd=11600000&SeqStart=11100000
    Source snippet

    www.ilga.gov725 ILCS 5/ Code of Criminal Procedure of 1963...

Additional References

  1. Source: torkin.com
    Link: https://www.torkin.com/insights/publication/faceoff-uk-appellate-court-finds-police-use-of-facial-recognition-technology-contravenes-laws
    Source snippet

    www.torkin.comFaceoff! UK appellate court finds police use of facial recognition technology contravenes lawsAugust 28, 2020...

    Published: August 28, 2020

  2. Source: acluohio.org
    Link: https://www.acluohio.org/press-releases/aclu-cautions-ohio-attorney-general-fbi-access-ohios-facial-recognition-database/
    Source snippet

    Cautions Ohio Attorney General on FBI Access to Ohio’s Facial Recognition Database - ACLU of OhioAugust 31, 2016...

    Published: August 31, 2016

  3. Source: cambridge.org
    Link: https://www.cambridge.org/core/journals/legal-studies/article/automated-facial-recognition-and-policing-a-bridge-too-far/347341E2BFA2EF1E3CC896A9C5ECDAD5
    Source snippet

    facial recognition and policing: a Bridge too far? | Legal Studies | Cambridge CoreAugust 27, 2021...

    Published: August 27, 2021

  4. Source: youtube.com
    Link: http://www.youtube.com/watch?v=EVeL4ZLLY1c
    Source snippet

    Facial recognition defense court challenge lead LAWYER: How to FOOL Police AI Cameras Hampton Law...

  5. Source: anrak.legal
    Link: https://anrak.legal/feed/court-reins-the-use-of-automated-facial-recognition-in-criminal-trials
    Source snippet

    Legal | AnrakLegal - India's Legal OSNovember 16, 2025...

    Published: November 16, 2025

  6. Source: youtube.com
    Title: Recognizing and Challenging Facial Recognition Technology in Criminal Cases
    Link: http://www.youtube.com/watch?v=gxCBUwGbte0
    Source snippet

    Face-Off: Recognizing and Challenging the Use of Facial Recognition Technology [webinar]...

  7. Source: newyorker.com
    Title: The New Yorker Does A.I
    Link: https://www.newyorker.com/magazine/2023/11/20/does-a-i-lead-police-to-ignore-contradictory-evidence
    Source snippet

    Lead Police to Ignore Contradictory Evidence?On March 26, 2022, a man in Timonium, Maryland assaulted a bus driver. The ensuing investiga...

    Published: March 26, 2022

  8. Source: youtube.com
    Link: http://www.youtube.com/watch?v=s4k3BLcH6OI
    Source snippet

    Wrongfully Arrested Because of Flawed Face Recognition Technology...

  9. Source: ojp.gov
    Title: www.ojp.gov Illinois vs Gates
    Link: https://www.ojp.gov/ncjrs/virtual-library/abstracts/illinois-vs-gates-probable-cause-redefined
    Source snippet

    vs Gates - Probable Cause Redefined? | Office of Justice Programs...

  10. Source: youtube.com
    Title: AI Software Tells Cops to Arrest the Wrong Guy
    Link: http://www.youtube.com/watch?v=lPUBXN2Fd_E
    Source snippet

    Detroit woman sues Detroit Police Department after arrest over false facial recognition match...

Topic Tree

Follow this branch

Parent topic

Face Matches Why a face match is not proof

Related pages 2