Let’s not delude ourselves – wide swaths of mathematics have been
regulated at least in their applied incarnations. At least since cryptography,
a branch of number theory, has come to be considered an “armament” requiring an
export license.
But as “weapons
of math destruction” have become commonplace and algorithms
that rule our working lives and consumer existence are used for anything
and everything from predictive advertising to policing to a virtually unlimited
number of other uses that include the internet of things as much as smart
agreements and the internet of contracts, it became increasingly obvious that
delegation of the outcome of machine analysis, evaluation, learning and
assessment would require
regulation. Cathy O’Neil, the “mathbabe”
with a comet-tail long track record in finance, has
been arguing for considerable time that algorithmic notations
project the past onto the future (so the best we can hope for is to perpetuate
the past) and are thus rife
with bias: they serve as a means of social control through pernicious
feedback loops, such as value-added models penalizing seemingly excellent educators,
perpetuating racial, class-based and other discrimination in “predictive
policing,” political
polling, prison
sentencing, car
insurance premiums, or employment
tests.
When Lufthansa, following the bankruptcy of Air Berlin, substantially
and unjustifiably hiked its airfares (to no one’s surprise, because a serious
low-cost competitor had just vanished) and was chided by the German Federal
Cartel Office, it created the “algorithmic defense”: no human was to blame, it
was all “the algorithm’s fault.” To which federal regulators remarked that
algorithms were not
written by God in heaven. We can be sure to see algorithmic defenses spring
up all over the place, almost at the speed of light.
In this context, O’Neil postulates a “Hippocratic oath” of
modeling and data science: first, do no harm. That would require that mathematical
models be purged of characteristics that allow them to serve as proxies for race and class
and start responding to ethical responsibilities – which are tricky because
there are different stakeholders. Thus, meaningful regulation in a meta way requires
auditing algorithms – which, in reality, would mean to create and continually
improve algorithms that audit algorithms. That is because mathematics is
inherently “trusted” but, because of its undisclosed assumptions and model
correlations in most algorithms, is anything but trustworthy: its formulae are secret, and, although they almost
operate like laws in some instances, their disclosure is currently mandated by
no law, not even by the Freedom of Information Act, and has proved difficult to
enforce by civil litigants. Furthermore, the constitutionality of potential
outcomes dictated by algorithmic output is reviewed by no one. For example, we
have fair
hiring laws – just that those are not applied to Big Data algorithms, and
there are no signs of an emerging nationwide conversation about it, just as
there is not about so much of data science.
In the face of increasingly overwhelming evidence that the very
analogy-based and precedent-oriented genesis of AI does, in fact, “learn” from
models that carry prejudicial patterns on race, class and gender (surely among
others), indicating “group membership” or value allegiances of algorithms and
robots steered by them may become the next frontier of disclosure – perhaps through
brand names,
though it will likely require greater and deeper efforts. But mere disclosure
of potential or predictable biases reinforced by autonomous learning may not be
enough in the absence of proactive and affirmative eradication tools. Which
raises another bizarre specter: the infiltration of AI by algorithms to secure
political correctness. So long as algorithms are written by humans, draw value
data input from humans and perform for a human target audience, it will be
difficult to see how phenomena that have existed in human valuation, rating and
triage processes could fail to leave mirroring marks on mathematical models
ultimately traced back to them.
