Busting Biases

Fair Play, found within a complex blend of elements

The term ‘bias’ can mean different things in different domains. Statistical bias is when an operation is disproportionately weighted to favour some outcome. Social bias is when such operations relate to people, which may lead to unfair decisions being made.

Reducing bias is hard, because models and data are complex, and because reasonable people disagree about whether a statistically accurate finding is still socially biased. For example, men as a group are physically stronger than women as a group. However, we typically consider gender to be a protected characteristic, which is not permitted to unduly influence decisions in hiring, etc. A system can be technically correct and still fall foul of the law.

Even data run through a high-pass filter until no human could recognise it as an x-ray may still contain signatures that machine learning can pick up.

Bias can sneak in from a number of sources, for example:

1). The reproduction of human labelling or selection biases: an algorithm trained upon human appraisals of CVs may replicate the same biased patterns.

2). Bias due to error in datasets, for example incorrect geolocation data that wrongly states that a house is inside a lake, and therefore considered not able to be insured.

3). Bias due to a lack of sampling data, for example an algorithm that is trained upon a set of examples over-representative of one ethnicity or gender, which generalises poorly to underrepresented demographics in the real world.

4). Bias due to overfitting, whereby a model is trained too strongly on training data, to the degree that it maps poorly onto real world examples.

5). Bias due to adversarial error, whereby a model may fail to recognise something accurately, or may misinterpret one thing for another. Models can be reverse engineered to uncover such exploits.

We can take several steps to reduce the risk of bias within algorithmic systems.

1). Select data which appears to be minimally influenced by human perception or prejudice. This is challenging, as data generally needs to be labelled and annotated in order to be interpreted by machine intelligence.

2). Make datasets more inclusive. Ensure that data is gathered from as broad a sampling as possible, and indeed solicit less common examples to ensure that the data is more representative of a global population and global environments.

3). Ensure the accuracy and integrity of data as far as possible. Run sanity checks, hunt for signatures of error, and locate lacunae — missing data that should be repaired or, ideally, set aside. This is a core duty of data science, and much of it is performed by legions of workers in less-developed nations for very small sums of money, with uncertain credentials.

4). Rigorously test models against real-world examples. A portion of training data is typically set aside for validation, but lab results are like a battle plan: they survive only until the first contact with reality. Systems must be tested live, across as broad a range of environments and demographics as possible.

5). Harden systems against attack and exploitation. Resources should be ring fenced in order to provide bounties for Red Teams to attempt to disrupt the algorithmic system. This can help to uncover issues long before they may occur ‘in the wild’ where real people may be affected.

Machine learning is already enmeshed with our personal and professional lives. We interact with algorithms a hundred times a day, usually without noticing. These systems must not be allowed to exclude people, or to misinterpret their behaviour or preferences in ways that harm them.

Transparency matters: we need to understand what processes are running, in what manner, for what purpose, and to whose benefit. That visibility is also one of the best tools for surfacing bias.

AI has tremendous potential, but it can also become a prejudiced petty tyrant. More governmental, academic, and business resources need to go into integrating it safely.

Natural Language processing is the science of teaching computers to make sense of the kinds of language that human beings use in everyday life. It is one of the most mature forms of machine learning, and highly integrated in daily life. NLP techniques assist speech processing, by helping to provide context for speech recognition systems. For example, the spoken word bear might mean an animal, or it might mean to carry or endure. If spoken, it could actually be intended as ‘bare’ as in nude, or a name, as in Behr. However NLP can guide the interpretation of such phonemes, to parse common phrases, and to make an inference that the word bear next to arms most likely refers to brandishing weapons, instead of furry ursine appendages.

But not all cultures or subcultures use language the same way. In the US you pay the check with a bill; in the UK you might pay the bill with a cheque. NLP needs to infer not only the surrounding linguistic context but the probable culture in which someone is communicating. People also code-shift, moving from a family dialect towards a more general register that others find intelligible.

Accents are another vector. A model trained on one form of a language may ignore or mangle the stress, tone, and pronunciation of another. When algorithms form impressions about us in so many contexts, being misunderstood — or not understood at all — can lower your ranking for opportunities and carry a direct economic cost. Training data needs to include a genuinely diverse range of accents and dialects so that people are not excluded by the sound of their own voice.

There must be a clear mechanism to report failures and anomalies, and an opportunity to provide a corrective example, so that models can improve over time. People should also be able to see what impression an algorithmic system formed of them — accuracy, appropriateness, and proportionality all depend on it.