Data Bias on the Daily: Is AI hindering her job search?

The gender pay gap and women’s representation in leadership roles continues to captivate headlines but what action is really being taken… It’s time to take a journey through the application process for a young adult female, let’s call her Mira, looking to land an interview in a Science, Technology, Engineering or Math (STEM) focused corporation…

A large part of Mira’s job search is online, where she will turn to various social platforms to seek out new opportunities, and depending on the channel she picks, she will be shown job advertisements that ultimately will be based on biased pay per click (PPC) and purchasing algorithms.

Advertising a STEM job across various social networks reached 20% fewer females than males, even though it is illegal to gender bias in both the US and Europe.

Management Science Study

Since advertising algorithms are designed to optimise advertising spend, and maximise reach, less potential female candidates were exposed to the ad and therefore did not get the opportunity to apply. To put it simply, the algorithms are biased against female candidates due to the higher cost of reaching them, even if ultimately the female candidate would be a better hire and take less resources to train.

Despite all of these challenges and biases facing her, Mira chugs along and finds a job posting in her field that captures her attention. Upon reading further into the job and the company, she is both consciously and unconsciously influenced by the language used to describe the role which will determine her next step. The STEM industry, particularly, is imbalanced due to a lack of women being trained and therefore is an industry that remains largely male-dominated. This transfers into the biased language used in the industries’ job listings, which in turn biases the data the job board algorithms are trained on.

A University of Waterloo and Duke University study showed that male-dominated industries (STEM industries) use masculine-themed words (like “leader”, “competitive” and “dominant”) which when interpreted by female applicants deteriorated their perception of the “job’s appeal” and their level of “belongingness” even if they felt able to perform the job. Above this, it is proven that a female will only apply for a job if she fulfills 100% of the criteria, whereas males will apply if they feel they fulfill only 60%.

Determined as ever, Mira eventually finds a job description and company she feels confident about, she submits an application. Her CV and cover letter are parsed and ranked alongside other applicants, male and female. Each success factor identified within the words Mira has used, like her programming skills, is weighted according to what has been historically successful in the past for that particular company.

In the age of Artificial Intelligence, past hiring decisions are used to train algorithms to identify the best suited candidates for the company. The issue with biased data is that even if gender is excluded from the application itself, the gender distribution of the training data may be strongly imbalanced since the industry has been historically male-dominated. This means that even if Mira gets to the point where the hiring company or job board puts her resume through their fit algorithm, she still may not receive the interview based on the inherent bias in the program.


If you’re not convinced by Mira’s journey, here’s a real life example: Amazon’s experimental hiring tool was used to screen and rate resumes based on historical hiring patterns. To date, a majority of Amazon employees were males and inevitably the system taught itself to penalise female resumes, with far greater efficiency than a human.


Still unsure if data bias is perpetuating gender bias in STEM? Check out these articles from others in the industry:

How Unconscious Bias Holds Women Back

Mitigating Gender Bias

AI Is Demonstrating Gender Bias


Processing…
Success! You're on the list.

Data Bias on the Daily: Criminal Sentencing- Not all algorithms are created equal

Imagine This…. You’ve been convicted of a non-violent crime, say petty theft. Your legal team decides the best course of action is to take a plea deal. On the day of your sentencing, the judge rejects your plea deal and doubles your sentence. Why? An algorithm says that you are at high risk for violent crime in the future…

You may be reading this thinking, that can’t possibly be real? But that is an all too real scenario because of the COMPAS algorithm.


COMPAS, an acronym for Correctional Offender Management Profiling for Alternative Sanctions, is a case management and decision support tool used by U.S. courts to assess the likelihood of a defendant becoming a repeat offender.


The problem with COMPAS, as a ProPublica report states, “Only 20 percent of the people predicted to commit violent crimes actually went on to do so.” ProPublica also concluded that the algorithm was twice as likely to falsely flag black defendants as future criminals as it was to falsely flag white defendants. And therein lies the problem, the algorithm has inherently biased training data due to years of human bias in the courtroom.

COMPAS is not only biased racially, but it also has bias against age and gender. An independent study done by researchers at Cornell University and Microsoft found that because most of the training data for COMPAS was based on male offenders the model is not as good at distinguishing between male and female as it could be. They even decided to make a separate COMPAS model aimed specifically at recidivism risk prediction for women.

But why would COMPAS separate the data based solely on gender when COMPAS has also shown to have racial bias? Why are judicial systems still turning to private, for-profit, companies whose algorithms are known to support racial, age and gender bias?

Turning to these types of algorithms have long standing implications on human life and our judicial system. Criminals receiving their sentences in the early ages of algorithmic adoption should not be test samples or guinea pigs for faulty and biased algorithms. As Artificial Intelligence becomes more main stream, understanding the data sets and training methodologies is key to understanding the results – how is data bias affecting your daily life?


For more information on COMPAS and ProPublica’s report, please click here.

Up Next for NCW: Digitization and Chemical Manufacturing

Processing…
Success! You're on the list.

Data Bias on the Daily: What’s in your Amazon Cart?

With multiple billions of packages shipped per year, and even more billions of items purchased, it’s no wonder that Amazon is a household name. In the time it takes you to read this article, an estimated 100,000 items will be purchased on Amazon. But how many of those purchases will be impacted by data bias? It’s likely every single one.

Have you ever stopped to think about the algorithms behind those convenient home page suggestions, people also purchased, and related to items you viewed? Maybe in passing or idly but likely not in detail. In this post we will be exploring just how much one quick search on Amazon can be littered with bias.

Bias can end up in your shopping cart in many different stages along the way. It’s safe to assume that Amazon wants to maximize it’s bottom line, even though they claim they want to provide the consumer with a great deal. When the engineers set to writing shopping algorithms at Amazon corporate, they have to add parameters that are computable and achieve a certain goal. With this in mind, they are likely adding the bias that they would like to increase Amazon’s bottom line (aka, profitability of your purchase). Let’s apply this to an example search.

Perhaps you’ve been searching for storage solutions for your messy closet. The moment your results appear for “closet storage” you are encountering data bias. Exhibit A, the screenshot below:

You’re looking at the first instance of bias because a majority of the page is showing you “sponsored” results. “Funding Bias” is written into many algorithms all over the internet and is the reason why sponsored items always show up first. These brands: JYYG, TYPE A, OUMYJIA and JEROAL are paying to bias your search. This is one of, if not the most common search bias encountered but we see so much “sponsored” content that, at this point, we may not even think of the bias’s effects.

Regulations have been placed on funding bias, note the “Sponsored” tag and other evidence for example on social media platforms when influencers post with “#ad.” These indicators are set up to clue in the consumer that their purchasing decisions are being biased by funded posts, something relatively new the last several years. Corporations spent years biasing search history towards paid posts without having to let the consumer know, this is a huge stride against hidden data bias. There’s nothing wrong with “pay to play!” As a consumer, this can be a helpful way to find one of your new favorite brands, products or technology solutions but people now believe it’s important for the consumer to know when funding is biasing their purchase.

Another example of bias in this search are the items marked with the “best seller” tag. Under “Help & Customer Service” Amazon.com says: “The Amazon Best Sellers calculation is based on Amazon.com sales and is updated hourly to reflect recent and historical sales of every item sold on Amazon.com.” There is a litany of data bias that could be going on here. Was the “Simple Houseware” organizer at one point a “sponsored” post which led to it becoming a “best seller?” A quick google search shows that Simple Houseware only sells on Amazon, this could be another contributing factor to the status as a “best seller.” We simply do not know the parameters the calculation is setting for “sales” as that is quite a broad term.

To play devil’s advocate, the “sponsored”  results could be amazing products that you will purchase, love and even re-order in the future. The data bias written into the “Best Sellers” calculation could absolutely be favoring great products you will know and love. However, it’s critical that we don’t turn a blind eye to the bias and we continue to scroll past the “sponsored” and “best sellers” to get a full picture for purchasing.

The objective of this article isn’t to get you to stop buying from Amazon but rather to consider the data bias right in front of you. If there is this much data bias in one simple Amazon search, how many more things in your life are impacted by data bias? How much is your business and its bottom line impacted by data bias? Algorithms, AI and their inherent bias are a part of daily life. How will you use them to create change for good?


Up Next for NCW: Digitization and Chemical Manufacturing


Processing…
Success! You're on the list.


Data Bias on the Daily

Our upcoming September Blog series is Data Bias on the Daily: How Data Bias in Artificial Intelligence is Impacting You. This series will focus on data bias in various forms encountered daily and the goal is to educate consumers on how bias can enter algorithms, knowingly and unknowingly. It is important to define bias:

Bias: The systematic favoritism that is present in the data collection process, resulting in lopsided, misleading results.

How to Identify Statistical Bias

Bias in Artificial Intelligence and algorithms is sometimes intentional, can be caused by a number of things including, but not limited to, sample selection or data collection, and can be avoidable, if that is the desired outcome. (Many corporations want to write their algorithms with bias, in order to increase their bottom line.)

Maybe you’re a woman searching for a STEM job, but the data is biased against your application? Perhaps, your Amazon search is biased towards products and brands that will only increase Amazon’s bottom line? Data bias is even entering our judicial system, is it possible that algorithmic “advancement” is simply confirming long standing racial bias?


Stay tuned to learn more about how Data Bias impacts our daily lives by checking in with us on Mondays in September (9/16, 9/23 and 9/30).



Processing…
Success! You're on the list.