For years, people like Scott McNealy, long-time CEO of the now defunct computer company SUN Microsys- tems, have said that privacy is dead. After Edward Snowden’s revelations about the prevalence of covert digital
surveillance and countless reports of data
breaches at large corporations, many may
agree. And yet, surveys find that a large
majority of people in the United States and
elsewhere treasure information privacy and
want it protected. This tension has led to
a proliferation of books arguing the importance of privacy, lamenting its demise,
and advocating for its resurrection. But the
story is getting stale.
In contrast, The Black Box Society, by
Frank Pasquale, offers a good dose of fresh
thinking and may advance our debates over
privacy. It also helps greatly that it is a good
read, not just for those who are curious
about privacy but also for those who are
already familiar with the privacy literature.
Pasquale’s main argument is that power
has become concentrated in the hands of a
few players in three key sectors: reputation,
search, and finance. Reputation and search,
he says, cover how we are perceived and
how we perceive. The aim of the financial
sector is price discovery, providing information that allows people to make economic
Actors in these sectors have amassed gi-
gantic piles of personal data about all of us.
But Pasquale, a law professor and privacy
expert, points out that information about
how these actors operate, how they use
the troves of personal data they have, and
how they design the algorithms that drive
their decision-making has ebbed to a historic low. Credit scoring, a reputational device, is shockingly opaque. The algorithm
used by Google to rank search results is a
highly protected trade secret, inaccessible
to anyone outside of the company. The recent financial crisis has revealed shockingly
complex legal constructs deliberately established by banks to obfuscate ownership and
to hide risks.
Pasquale also argues that the private sector is now sharing its troves of personal data
with government agencies, leading to even
more powerful, as well as impenetrable,
data uses. He is right when one just looks at
the past few decades, but the early information privacy debates in the late 1960s were
fueled by precisely the same concerns, as
governments and corporations began planning huge data centers to combine and share
their data holdings. Massive public backlash
stopped these projects then, but more recently, a growing preference for safety over
privacy—fueled by the trauma of 9/11—has
done much to silence public concern.
Early privacy experts highlighted the importance of informational power and how
heavy power imbalances often lead to subop-timal (and unethical) outcomes for society.
However, the classic remedy has been to require informed consent from the individual.
As long as someone is told how their data
will be used, and as long as they consent, the
idea goes, privacy is maintained.
Pasquale suggests that this approach is no
longer effective (if it ever was). “Notice and
consent,” he claims, has degenerated to a formalistic farce, in which most of us consent to
terms we have neither the time nor the expertise to decipher. Current privacy protection, as Pasquale aptly notes, at best works
for the rich, who can afford pricy lawyers
and “reputation managers” to enforce their
privacy rights. “Big Data”—the ability to gain
novel insights from the collection and analysis of a large number of data points—with
its emphasis on reuse and the combination
of diverse data sets, will only exacerbate the
current system’s shortcomings.
In contrast, Pasquale argues for a fundamental shift in privacy protection, from
a focus on notice and consent at the time
of data collection to stringent regulation of
the actual use of data by corporations and
government agencies. His reasoning is sensible, and he is not alone; a growing number of privacy advocates are in favor of data
use regulation. If such regulations were to
be put in place, it would represent an epic
change in how we aim to protect privacy in
There is a second valuable and important
idea in The Black Box Society. It is the implicit red thread throughout the book but
emerges clearly in the last chapter. Many of
the information tussles that arise in reputation, search, and finance can be framed as
privacy challenges, but they are, Pasquale
suggests, more than that. They reflect how
much of our lives is now affected by information and how this has created an urgent
need for a framework of information governance, as well as a need to define what
role the public sector ought to play. Perhaps
we need a public agency offering credit
scores, not just private corporations. Perhaps the Library of Congress ought to scan
books, much like Google does, and permit
us to search them. Perhaps, Pasquale boldly
suggests, there is even a public role in finance—for instance, to provide mortgages
and loans to underserved communities. He
is right. As data-driven decision-making becomes commonplace, our ethical and legal
frameworks need to broaden. This is precisely the debate that we need to have.
Connecting the dots
By Viktor Mayer-Schönberger
The reviewer is at the Oxford Internet Institute, University of
Oxford, Oxford OX1 3JS, UK. E-mail: firstname.lastname@example.org
BOOKS et al.
The Black Box Society
The Secret Algorithms
That Control Money and
Harvard University Press,
2015. 319 pp.
When it comes to the data that can make or break us,
who holds the power?