Fairness in Machine Learning is a growing area of research and of massive policy debates. The gist of it is simple. While ML algorithms are great for out-of-sample prediction, they train on historical datasets generated by human decisions, which.. as history teaches us, are oftentimes biased in favor or against of a social group. For the sake of argument, let’s take a company that wants to implement a ML software to suggest hiring and promotion decisions inside the firm. The algorithm’s job is then to predict which candidate is more likely to succeed in a given position. To do that, the algorithm would train on historical decisions of the firm. These include the candidates’ credentials and characteristics (x variables) as well as how well they performed in a given position which could be proxied by whether they got hired or promoted (y variable). Suppose now that past hiring managers had a bias for male candidates. Thus male candidates had a higher chance of being hired and promoted for the same credentials. That would then lead the algorithm to predict that a male candidate is more likely to succeed and thus suggest the hiring or promotion to that candidate. 

The problem is that simply removing sensitive variables like race and gender from a dataset is not enough to make the bias go away since most of these variables are highly correlated with other characteristics. I have had the chance to work with some incredible people on a related project where we proposed a system in which the algorithm got penalized by the prediction of those sensitive variables. We proposed a setup in which a policymaker could choose the level of fairness desired (which would obviously tradeoff with prediction accuracy) and the algorithm would make predictions given that level.

If you’re interested for a light reading I suggest this easy-to-read book “Weapons of math destruction”. For more in-depth reading I recommend going through this archived course from UC Berkeley’s Moritz Hardt who is at the forefront of research in this area. His book (coauthored with Solon Barocas and Arvind Narayanan) is also a great resource. It is available online here.