Because of the recent boom in generative artificial intelligence (GenAI), businesses across many industries are beginning to recognize the potential of AI and machine learning systems. Legislators and government agencies have not let the trend go unnoticed. On April 26, 2023, the Department of Justice (DOJ), the United States Equal Employment Opportunity Commission (EEOC), and other federal agencies issued a joint statement declaring that, although automated systems may offer the promise of technological breakthroughs and increased efficiency, "…their use also has the potential to perpetuate unlawful bias, automate unlawful discrimination, and produce other harmful outcomes."
Businesses using GenAI for employment purposes are already come under increased scrutiny. In 2021, New York City passed Local Law 144 in order to regulate the use of automated employment decision tools (AEDTs), a broad class of software including artificial intelligence, machine learning, statistical modeling, or data analytics that issue a simplified output, such as a score, classification, or recommendation regarding an employee or job applicant.
GenAI Systems Starting to Face Government Regulations
About Local Law 144
Local Law 144 specifically targets the use of AEDTs for purposes of hiring. Any employer who wishes to use an AEDT for screening candidates for hiring or promotion within New York City must:
1. Conduct a bias audit on the tool prior to use
2. Provide a summary of the bias audit on its website
3. Give notice to candidates of the company's use of an AEDT ten business days before using the system
4. Allow candidates the right to request an alternative selection process
Although groundbreaking in its attempt to address emerging GenAI systems, Local Law 144 also left significant questions open for interpretation. New York City’s Department of Consumer and Workplace Protection (DCWP) clarified two key questions when it issued final rules about the ordinance on April 6, 2023.
First, the DCWP addressed the applicability of the regulations when AEDTs are used in conjunction with human decision-making. Under the new rules, AEDTs are subject to the requirements of Local Law 144 if they "substantially assist or replace discretionary decision making," which the DCWP defines as when an employer either:
- Relies solely on the simplified output of the AEDT without considering any other factors
- Uses a simplified output in a set of criteria, but weighs the AEDT's simplified output more than any other criterion
- Or uses an AEDT's simplified output to overrule conclusions derived from other factors, including human decision-making
The DCWP also clarified an employers' responsibilities with respect to bias audits conducted prior to the deployment of AEDTs:
- Yearly audits must calculate both the impact ratios and scoring rates for applicants based on their sex, race/ethnicity, and intersectional categories of sex, race, and ethnicity.
- "Impact ratios" may be calculated based on selection rates or scoring rates of a particular category, divided by the respective rates for the most favored category.
New York City's Local Law 144 will begin enforcement July 5, 2023.
Stay aware
In addition to expanding your understanding and awareness of how GenAI can benefit your HR/HCM systems, you should also make a point to be on the lookout for state and local law makers’ efforts to regulate GenAI’s growing use. At the federal level, the DOJ and EEOC are two agencies to keep on your radar, too, as they have a keen interest in enforcing systems that have the potential to engage in unfair bias and discrimination.
