Data Bias on the Daily: Is AI hindering her job search?

The gender pay gap and women’s representation in leadership roles continues to captivate headlines but what action is really being taken… It’s time to take a journey through the application process for a young adult female, let’s call her Mira, looking to land an interview in a Science, Technology, Engineering or Math (STEM) focused corporation…

A large part of Mira’s job search is online, where she will turn to various social platforms to seek out new opportunities, and depending on the channel she picks, she will be shown job advertisements that ultimately will be based on biased pay per click (PPC) and purchasing algorithms.

Advertising a STEM job across various social networks reached 20% fewer females than males, even though it is illegal to gender bias in both the US and Europe.

Management Science Study

Since advertising algorithms are designed to optimise advertising spend, and maximise reach, less potential female candidates were exposed to the ad and therefore did not get the opportunity to apply. To put it simply, the algorithms are biased against female candidates due to the higher cost of reaching them, even if ultimately the female candidate would be a better hire and take less resources to train.

Despite all of these challenges and biases facing her, Mira chugs along and finds a job posting in her field that captures her attention. Upon reading further into the job and the company, she is both consciously and unconsciously influenced by the language used to describe the role which will determine her next step. The STEM industry, particularly, is imbalanced due to a lack of women being trained and therefore is an industry that remains largely male-dominated. This transfers into the biased language used in the industries’ job listings, which in turn biases the data the job board algorithms are trained on.

A University of Waterloo and Duke University study showed that male-dominated industries (STEM industries) use masculine-themed words (like “leader”, “competitive” and “dominant”) which when interpreted by female applicants deteriorated their perception of the “job’s appeal” and their level of “belongingness” even if they felt able to perform the job. Above this, it is proven that a female will only apply for a job if she fulfills 100% of the criteria, whereas males will apply if they feel they fulfill only 60%.

Determined as ever, Mira eventually finds a job description and company she feels confident about, she submits an application. Her CV and cover letter are parsed and ranked alongside other applicants, male and female. Each success factor identified within the words Mira has used, like her programming skills, is weighted according to what has been historically successful in the past for that particular company.

In the age of Artificial Intelligence, past hiring decisions are used to train algorithms to identify the best suited candidates for the company. The issue with biased data is that even if gender is excluded from the application itself, the gender distribution of the training data may be strongly imbalanced since the industry has been historically male-dominated. This means that even if Mira gets to the point where the hiring company or job board puts her resume through their fit algorithm, she still may not receive the interview based on the inherent bias in the program.

If you’re not convinced by Mira’s journey, here’s a real life example: Amazon’s experimental hiring tool was used to screen and rate resumes based on historical hiring patterns. To date, a majority of Amazon employees were males and inevitably the system taught itself to penalise female resumes, with far greater efficiency than a human.

Still unsure if data bias is perpetuating gender bias in STEM? Check out these articles from others in the industry:

How Unconscious Bias Holds Women Back

Mitigating Gender Bias

AI Is Demonstrating Gender Bias

Success! You're on the list.

Data Bias on the Daily: Criminal Sentencing- Not all algorithms are created equal

Imagine This…. You’ve been convicted of a non-violent crime, say petty theft. Your legal team decides the best course of action is to take a plea deal. On the day of your sentencing, the judge rejects your plea deal and doubles your sentence. Why? An algorithm says that you are at high risk for violent crime in the future…

You may be reading
this thinking, that can’t possibly be real? But that is an all too real
scenario because of the COMPAS algorithm.

COMPAS, an acronym for Correctional Offender Management Profiling for Alternative Sanctions, is a case management and decision support tool used by U.S. courts to assess the likelihood of a defendant becoming a repeat offender.

The problem with COMPAS, as a ProPublica report states, “Only 20 percent of the people predicted to commit violent crimes actually went on to do so.” ProPublica also concluded that the algorithm was twice as likely to falsely flag black defendants as future criminals as it was to falsely flag white defendants. And therein lies the problem, the algorithm has inherently biased training data due to years of human bias in the courtroom.

COMPAS is not only
biased racially, but it also has bias against age and gender. An independent
study done by researchers at Cornell University and Microsoft found that
because most of the training data for COMPAS was based on male offenders the
model is not as good at distinguishing between male and female as it could be.
They even decided to make a separate COMPAS model aimed specifically at
recidivism risk prediction for women.

But why would COMPAS
separate the data based solely on gender when COMPAS has also shown to have
racial bias? Why are judicial systems still turning to private, for-profit,
companies whose algorithms are known to support racial, age and gender bias?

Turning to these
types of algorithms have long standing implications on human life and our
judicial system. Criminals receiving their sentences in the early ages of
algorithmic adoption should not be test samples or guinea pigs for faulty and
biased algorithms. As Artificial Intelligence becomes more main stream,
understanding the data sets and training methodologies is key to understanding
the results – how is data bias affecting your daily life?

For more information
on COMPAS and ProPublica’s report, please click

Up next: DNA Testing: Is knowing your heritage worth risking your privacy?

Success! You're on the list.

Data Bias on the Daily: What’s in your Amazon Cart?

With multiple billions of packages shipped per year, and even
more billions of items purchased, it’s no
wonder that Amazon is a household name. In the time it takes you to read this
article, an estimated 100,000 items will be purchased on Amazon. But how many
of those purchases will be impacted by data bias? It’s likely every single one.

Have you ever
stopped to think about the algorithms behind those convenient home page
suggestions, people also purchased, and related to items you viewed? Maybe in
passing or idly but likely not in detail. In this post we will be exploring
just how much one quick search on Amazon can be littered with bias.

Bias can end up in
your shopping cart in many different stages along the way. It’s safe to assume
that Amazon wants to maximize it’s bottom line, even though they claim they
want to provide the consumer with a great deal. When the engineers set to
writing shopping algorithms at Amazon corporate, they have to add parameters
that are computable and achieve a certain goal. With this in mind, they are
likely adding the bias that they would like to increase Amazon’s bottom line
(aka, profitability of your purchase). Let’s apply this to an example search.

Perhaps you’ve been
searching for storage solutions for your messy closet. The moment your results
appear for “closet storage” you are encountering data bias. Exhibit
A, the screenshot below:

You’re looking at the first instance of bias because a majority of the page is showing you “sponsored” results. “Funding Bias” is written into many algorithms all over the internet and is the reason why sponsored items always show up first. These brands: JYYG, TYPE A, OUMYJIA and JEROAL are paying to bias your search. This is one of, if not the most common search bias encountered but we see so much “sponsored” content that, at this point, we may not even think of the bias’s effects.

Regulations have
been placed on funding bias, note the “Sponsored” tag and other
evidence for example on social media platforms when influencers post with
“#ad.” These indicators are set up to clue in the consumer that their
purchasing decisions are being biased by funded posts, something relatively new
the last several years. Corporations spent years biasing search history towards
paid posts without having to let the consumer know, this is a huge stride
against hidden data bias. There’s nothing wrong with “pay to play!”
As a consumer, this can be a helpful way to find one of your new favorite
brands, products or technology solutions but people now believe it’s important
for the consumer to know when funding is biasing their purchase.

Another example of
bias in this search are the items marked with the “best seller” tag.
Under “Help & Customer Service” says: “The Amazon
Best Sellers calculation is based on sales and is updated hourly to
reflect recent and historical sales of every item sold on”
There is a litany of data bias that could be going on here. Was the
“Simple Houseware” organizer at one point a “sponsored”
post which led to it becoming a “best seller?” A quick google search
shows that Simple Houseware only sells on Amazon, this could be another
contributing factor to the status as a “best seller.” We simply do
not know the parameters the calculation is setting for “sales” as
that is quite a broad term.

To play devil’s
advocate, the “sponsored” 
results could be amazing products that you will purchase, love and even
re-order in the future. The data bias written into the “Best Sellers”
calculation could absolutely be favoring great products you will know and love.
However, it’s critical that we don’t turn a blind eye to the bias and we
continue to scroll past the “sponsored” and “best sellers”
to get a full picture for purchasing.

The objective of
this article isn’t to get you to stop buying from Amazon but rather to consider
the data bias right in front of you. If there is this much data bias in one
simple Amazon search, how many more things in your life are impacted by data
bias? How much is your business and its bottom line impacted by data bias?
Algorithms, AI and their inherent bias are a part of daily life. How will you
use them to create change for good?

Up next: DNA Testing: Is knowing your heritage worth risking your privacy?

Success! You're on the list.

Data Bias on the Daily

Our upcoming September Blog series is Data Bias on the Daily: How Data Bias in Artificial Intelligence is Impacting You. This series will focus on data bias in various forms encountered daily and the goal is to educate consumers on how bias can enter algorithms, knowingly and unknowingly. It is important to define bias:

Bias: The systematic favoritism that is present in the data collection process, resulting in lopsided, misleading results.

How to Identify Statistical Bias

Bias in Artificial Intelligence and algorithms is sometimes intentional, can be caused by a number of things including, but not limited to, sample selection or data collection, and can be avoidable, if that is the desired outcome. (Many corporations want to write their algorithms with bias, in order to increase their bottom line.)

Maybe you’re a woman searching for a STEM job, but the data is biased against your application? Perhaps, your Amazon search is biased towards products and brands that will only increase Amazon’s bottom line? Data bias is even entering our judicial system, is it possible that algorithmic “advancement” is simply confirming long standing racial bias?

Stay tuned to learn more about how Data Bias impacts our daily lives by checking in with us on Mondays in September (9/16, 9/23 and 9/30).

Success! You're on the list.