The Innovative State - IA de Investigación especializada en el MIT

El Estado innovador

Beth Simone Noveck

To create government that is neither bigger nor smaller but better at solving problems
more effectively and legitimately, agencies need to use big data and the associated
technologies of machine learning and predictive analytics. Such data-analytical ap-
proaches will help agencies understand the problems they are addressing more em-
pirically and devise more responsive policies and services. Such data-processing tools
can also be used to make citizen engagement more efficient, helping agencies to make
sense of large quantities of information and invite meaningful participation from
more diverse audiences who have never participated in our democracy. To take ad-
vantage of the power of new technologies for governing, sin embargo, the federal govern-
ment needs, first and foremost, to invest in training public servants to work differently
and prepare them for the future of work in a new technological age.

D uring the COVID-19 pandemic, I have had the privilege to lead a team of

engineers, designers, and policy professionals in the New Jersey Office
of Innovation, a recently created administrative unit in the state’s gov-
gobierno. When the pandemic hit, the Innovation Office team used technology
and data, and unprecedented levels of collaboration across agencies and with the
private sector, to respond to the crisis.

Working with the nonprofit Federation of American Scientists, Por ejemplo,
we built a website and accompanying (Amazonas) Alexa skill to enable the public to
pose questions about the virus to more than six hundred participating scientists
and receive rapid, well-researched responses.1

A private sector company lent us the tech and the talent to create a website,
covid19.nj.gov, in three days. In the last year, the site has been visited more than
seventy-five million times since its launch in March of 2020.

Even more challenging to create than the technology was the content. Allá-
delantero, the Innovation Office collaborated with Princeton, Rutgers, Montclair, Fila-
un, and the state’s other universities to create an editorial team to translate legal-
ese from government agencies into plain English and to knit together disparate
sources of information in a single website.

A professor of data science at New York University assembled a team to pro-
duce predictive analytics about the spread of the virus. This data enabled the gov-
ernor and other senior leaders to make better decisions about the response. Cuando

121

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

the data science team could not determine the number of deaths on the basis of
race because the testing labs were not providing that information, the Depart-
ment of Human Services and the Department of Health shared key administrative
data with one another that enabled us to answer this question faster. Such sharing
would normally be accomplished in a year (or never); we did it in a day.

In three days, the team also produced the nation’s first state jobs site to list
available positions in essential businesses and thereby mitigate the crisis of unem-
ployment. We posted over fifty thousand jobs in a broad range of businesses and
salary levels. We launched a site that was far from perfect and improved it as we
went along, knowing it was more important to risk failure than not to act quickly.
Our team also worked with the federal government’s Digital Service, a unit with-
in the Executive Office of the President, to fix the state’s process of certifying for
unemployment.2 We also worked with the nonprofit Code for America to digitize
the application process for food benefits, whose paper-based rules previously re-
quired coming into a government office to demonstrate income level.

By working more collaboratively and taking advantage of new technologies
of information collection, análisis, and visualization, we were able to demon-
strate how a bureaucracy can be nimble and effective, rather than lumbering and
unresponsive.

Changing how we work in government is imperative. The COVID-19 crisis has
revealed how ill-equipped the administrative state is at dealing with novel chal-
lentes. From delivering adequate testing and personal protective equipment
(PPE) to expanding online education equitably, in too many areas the state has
struggled to respond.

Perhaps it is telling that, in the face of the unprecedented COVID crisis, muchos
public leaders chose to hire the management consultancy McKinsey and out-
source critical state responses despite the high costs.3 In the first four months of
the pandemic alone, public institutions in the United States contracted with Mc-
Kinsey to the tune of $100 millón, reflecting, a lo mejor, a perceived lack of confi- dence in the skills of bureaucracies and, at worst, a hollowing out of competence in the administrative state.4 Either way, there is an urgent need for new approach- es to how government operates in response to the crises hiding in plain sight, from the public health emergency to an unprecedented economic depression. In the United States in 2020, joblessness reached numbers not seen since the Great De- presion. The International Monetary Fund (IMF) has estimated that the global economy shrunk by 3.5 por ciento en 2020, pushing many of those who could least afford it deeper into poverty.5 While the economy is showing signs of bouncing back and vaccines are help- ing to alleviate the public health emergency, the crisis of confidence in govern- ment is chronic, not acute, because the challenges we face are not going away. En- equality persists. Pre-COVID, the average worker had not seen her wages increase 122 l D o w n o a d e desde h t t p : / / directo . mi t . / e d u d a e d a r t i c e – pd / l f / / / / 1 5 0 3 1 2 1 2 0 6 0 4 8 2 d a e d _ a _ 0 1 8 6 3 pd . / f por invitado 0 8 septiembre 2 0 2 3 Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State since the 1970s, while the average pretax income of the top 10 percent of American earners has doubled since 1980, and that of the top 0.001 percent rose sevenfold.6 Whereas life expectancy in the United States continuously increased for most of the past sixty years, it has been decreasing since 2014.7 For the poor, life expectan- cy is dramatically lower.8 Rich American men now live fifteen years longer than their poorer compatriots.9 Life expectancy for Black men is far below every other demographic.10 On top of these and countless other challenges, there is the loom- ing and existential threat of climate change. It is no wonder that most Americans today have lost confidence in govern- mento, especially the federal government. According to Pew Research Center, solo 2 percent of Americans today say they can trust the government in Washington to do what is right “just about always,” while 18 percent trust the federal govern- ment “most of the time.”11 Political scientist Paul Light has asserted that “federal failures have become so common that they are less of a shock to the public than an expectation. The question is no longer if government will fail every few months, pero donde. And the answer is ‘anywhere at all.’”12 I f embraced, the right technologies can create new opportunities for improv- ing the efficacy and agility–and, when used well, the legitimacy–of the ad- ministrative state. The technologies of big data as well as those engagement tools that enable individual and group communication and collaboration across a distance–what we might call the technologies of collective intelligence–could enable government agencies to understand problems with greater precision and in conversation with those most affected. Thanks to the ubiquitous presence of data-gathering sensors in our lives, the technologies of big data make it possible for bureaucrats to gather more: more real- time and more granular information. Instead of speculating about the cause of accidents, Por ejemplo, a city now has exact information generated by the sen- sors on traffic lights, road cameras, and even sensors built into the pavement re- vealing exactly what kind of accidents are happening, when they occur, and which vehicles they involve. Data-analytical tools like machine learning make it possible for machines to ingest and make sense of large quantities of data. They can help the administrative state analyze the new glut of information. Agencies have the opportunity to get smarter from people–their experiences and expertise–as well as from sensors and to obtain more diverse and equitable perspectives and insights. These combinations of quantitative and qualitative ap- proaches tell agency officials more about why a problem is occurring and offer a broader audience to provide solutions. The administrative agencies of government at every level have always had far greater access to information than other branches of government.13 This is why legal scholar Adrian Vermeule refers to the administrative state as the “sensory 123 l D o w n o a d e desde h t t p : / / directo . mi t . / e d u d a e d a r t i c e – pd / l f / / / / 1 5 0 3 1 2 1 2 0 6 0 4 8 2 d a e d _ a _ 0 1 8 6 3 pd / . f por invitado 0 8 septiembre 2 0 2 3 150 (3) Summer 2021Beth Simone Noveck organ” of government. Its agencies and large staffs are designed to “gather, exam- ine and cull information” and make greater sense of on-the-ground conditions.14 Technology in every era has enabled administrative agencies to engage in “seeing like a state,” in the famous phrase of political scientist James Scott (and his epony- mous book). Whereas Scott was concerned about the tendency of those who gov- ern toward reductive simplification due, in large part, to simplistic measurement tools, entrepreneurial bureaucrats today have the opportunity to use big data and human insight to understand a problem as ordinary people experience it, and to design collaboratively more-effective solutions tailored to achieving the public’s desired outcomes. If we embrace these diverse sources of external knowledge, the epistemic ca- pacity of the state has the potential to increase dramatically. De hecho, en 2018, Estafa- gress passed the Foundations for Evidence-Based Policymaking Act, requiring agencies to make better use of their data to measure and improve their perfor- mance and policy-making.15 But, on the whole, too many administrative agen- cies are still falling behind in their use of new technologies and innovative ways of working. There is an ongoing information asymmetry in that regulators lack ac- cess to the data, información, and insight they need to safeguard the public inter- est, deliver services, identify violations, and enforce the law efficiently, especially vis-à-vis those seeking to evade liability. They also lack the practices for solving problems collaboratively. For agencies to engage in transformative policy-making, they need to exploit the tools available for creating a “smarter” and more equita- ble state. B ig data refers to extremely large data sets that are too big to be stored or processed using traditional means. Hoy, new collection, almacenamiento, trans- mission, visualization, and analytic techniques have triggered a massive proliferation of data sets collected by public and private entities about everything from health and wellness to phone and purchase records. Such data are powerful raw materials for problem-solving. Take a recent example from New Orleans, which has one of the highest murder rates of any city in the nation. Determined to change this dismal fact, then May- or Mitch Landrieu in 2012 created a unit in city government called the Innovation Team, or i-Team. Using more than fifty years of data grouped by neighborhood and by rates of murder, crime, educational attainment, unemployment, and recid- ivism, the team uncovered a significant correlation between unemployment and violent crime (and thus recidivism). The data showed that a small and identifiable set of people in a few neighborhoods committed a majority of murders, usually as the result of petty disputes.16 That knowledge produced significant change. Municipal agencies instituted programs to train and hire ex-offenders in an effort to reduce the likelihood of re- 124 l D o w n o a d e desde h t t p : / / directo . mi t . / e d u d a e d a r t i c e – pd / l f / / / / 1 5 0 3 1 2 1 2 0 6 0 4 8 2 d a e d _ a _ 0 1 8 6 3 pd / . f por invitado 0 8 septiembre 2 0 2 3 Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State offending among those who had been incarcerated.17 Strategies in the NOLA for Life program included social services and job opportunities as well as threats of prosecution, using data to determine which approach was appropriate for which individual. In the i-Teams’ first year, the New Orleans’ murder rate dropped 19 por ciento. Two years in, the rate had dropped over 25 percent from the 2012 alto. New Orleans’ murder rates in 2018 y 2019, though still among the highest in the country, were at their lowest level in almost fifty years.18 There has been a significant push in recent years to increase the amount of data that administrative agencies collect from the entities they regulate to enable more targeted regulatory enforcement. En 2010, Por ejemplo, the Occupational Safety and Health Administration (OSHA) required certain employers to submit death and injury data electronically to Washington and, como resultado, OSHA was able to build a dashboard showing where injuries were occurring. (This data collection rule was scrapped by the Trump administration in 2019, though on day one of his administration, President Biden reversed course again.)19 In July 2010, Congress passed and President Obama signed the Dodd-Frank Wall Street Reform and Con- sumer Protection Act, which among other things created the Consumer Financial Protection Bureau (CFPB). The CFPB created a public complaint database in an ef- fort to pressure businesses to treat customers better. Like OSHA, this agency also collected more data in machine-readable format to be able to create the Student Debt Repayment Assistant, an online tool to help borrowers navigate student loan repayment options.20 Similarly, en 2015, the CFPB issued a rule to expand data col- lection requirements under the Home Mortgage Disclosure Act to help protect borrowers. (This rule, también, was effectively gutted by the Trump administration, which eliminated penalties for noncompliance. Joe Biden campaigned on a com- mitment to undo Trump’s actions.)21 Many describe what makes big data big as the “3Vs”: volumen, velocity, and variety. Primero, the term reflects a huge rise in data volume. En 2015, 12 zettabytes–that’s 12 x1021 bytes of data–were created worldwide. Por 2025, that number is forecast to reach 163 zettabytes. For comparison, the entire Library of Congress is only 15 tera- bytes: 1 zettabyte is 1 billion terabytes. Segundo, data velocity–the speed at which data are generated, analyzed, and used–is increasing. Hoy, data are generated in near real-time, created by humans through myriad everyday activities like making a purchase with a credit card, logging onto social media, or adjusting a thermo- stat, and by machines through radio-frequency identification (RFID) and sensor data. Much of these data are “designed data,” collected for statistical and analytical purposes. But large quantities of data are also “found data” (also known as “data exhaust”), collected for something other than research but still susceptible to anal- ysis.22 For example, the JPMorgan Chase Institute uses financial services data, en- cluding credit card purchase records, to analyze and comment on the economic future of online platforms such as Uber and Lyft.23 Third, big data reflects accumu- 125 l D o w n o a d e desde h t t p : / / directo . mi t . / e d u d a e d a r t i c e – pd / l f / / / / 1 5 0 3 1 2 1 2 0 6 0 4 8 2 d a e d _ a _ 0 1 8 6 3 pd / . f por invitado 0 8 septiembre 2 0 2 3 150 (3) Summer 2021Beth Simone Noveck lating data variety. Data come in many formats, including numbers, texto, images, voice, and video. Some data are organized in traditional databases with predefined fields such as phone numbers, zip codes, and credit card numbers. Sin embargo, more and more data are unstructured: they do not come preorganized in traditional spreadsheet-style formats but helter-skelter as Twitter postings, videos, coordi- nates, Etcétera. Sin embargo, contemporary analytical methods make it possi- ble to search, sort, and spot patterns even in unstructured data. The value of all this data collection for the administrative state is in the ability to understand past, present, and future actions.24 With the right data-analytical skills–namely, an understanding of how to for- mulate a hypothesis, identify and collect the right data, and use that data to con- firm the hypothesis–policy-makers can understand past performance of public policies and services, evaluating both their efficiency and impact on different pop- ulaciones. Economists Raj Chetty, Nathaniel Hendren, and Lawrence Katz stud- ied twenty years of income records from families that moved to new neighbor- hoods using the Housing Choice Voucher Program. They discovered that these families earned significantly higher incomes, completed more education, and were less likely to become single parents than peers who stayed in their neighbor- capuchas. Citing this research, the Department of Housing and Urban Development overhauled the formula that it had used for four decades to calculate rental assis- tance, and increased opportunities for families to move from high-poverty areas to low-poverty areas.25 Larger quantities of data also enable the delivery of more-tailored interven- tions in the present by helping governments match people to benefits to which they are entitled or to assistance they need. Por ejemplo, Louisiana’s Department of Health uses Supplemental Nutrition Assistance Program (SNAP) enrollment data to sign people up for health benefits. Of nearly 900,000 SNAP recipients, Louisi- ana has enrolled 105,000 in Medicaid without a separate application process, re- lying on a four-question, yes-or-no survey to determine eligibility. This approach has helped some of the state’s poorest residents get access to benefits, while sav- ing the state about $1.5 million in administrative costs.26

Better access to data even helps with forecasting future outcomes, such as who
is likely to be a frequent visitor to the emergency room, thereby enabling more
targeted interventions and treatment. During the COVID-19 pandemic, muchos
jurisdictions started using “symptom trackers,” simple software tools to enable
people to report their symptoms to public health officials. (In New Jersey, we cre-
ated our own, and half a million participants used it to report data and obtain in-
formation.) Especially in the absence of testing data, symptom trackers provided
an early warning mechanism, signaling where people were complaining of coughs
and fevers. Symptom tracker data enabled emergency officials to anticipate the
need for equipment, supplies, and hospital beds in the not too distant future.

126

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

Big data also creates the opportunity for regulators to spot mistakes, outliers,
and rare events and make decisions based on evidence of on-the-ground condi-
ciones. Por ejemplo, rat infestations in large cities are difficult to tackle because
rats travel in virtually unpredictable ways. Chicago’s rat problem peaked in 2011
when it received more than twenty-five thousand rodent complaints via 311 calls
(notifications from residents about problems needing attention). This call cen-
ter information generated a novel database that offered a deeper understanding
of the day-to-day patterns of rat infestations. In search of a new strategy, the city
partnered with Carnegie Mellon University’s Event and Pattern Detection Lab
to gather twelve years of 311 citizen complaint data, including information on rat
sightings along with related factors such as overflowing trash bins, food poisoning
casos, tree debris, and building vacancies. It is important to point out that these
data are not gathered by regulators but by citizens calling the city’s hotline. El
311 system “constructs a collaborative relationship between city residents and
government operations,” writes public affairs scholar Daniel T. O'Brien. “Resi-
dents act as the ‘eyes and ears of the city,’ reporting problems that they observe in
their daily movements.”27

F rom cuneiform to card catalogs, governments have always recorded data.

But the proliferation of big data creates hopeful new opportunities for in-
novation in the administrative state. Big data makes it possible for agencies
to increase their epistemic and sensory capacity and develop a more detailed and
accurate understanding of on-the-ground conditions with the engagement of a
more diverse public.

These data-analytical techniques have made possible an expanded toolkit for
change and new kinds of solutions from regulatory agencies, such as “smart dis-
closure” tools that aim to give consumers more complete data about the cost, qual-
idad, and safety of the products and services they buy, or the health, environmen-
tal, and labor practices of manufacturers and service providers.28 For example, el
Department of Education’s College Scorecard gives students and parents informa-
tion about the real costs, financial aid options, graduation rates, and postgradua-
tion salaries and employment opportunities of universities. In New Jersey, we are
building Data for the American Dream, a similar initiative to provide transparency
about vocational training programs to job seekers, and especially unemployed job
seekers, to help them make more-informed decisions about cosmetology, welding,
and green energy training programs, Por ejemplo. Using anonymized government-
collected tax data, this “training explorer” will be able to show whether those who
took a given training course saw their income go up or down.

To be sure, as legal scholar Rory van Loo has pointed out, there can be draw-
backs in the use of smart disclosure tools like Training Explorer, College Score-
card, the Affordable Care Act’s health insurance exchange websites, or the CFPB’s

127

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

mortgage rate checker tool: when under-resourced public agencies build worse
websites than Silicon Valley, consumers suffer. Al mismo tiempo, outsourcing the
development of these tools to the private sector has its own problems. The IRS
contracted with Intuit to provide a free version of TurboTax to low-income resi-
dents for their tax preparation, but the company has allegedly made that version
as bad as possible to pressure people to buy its expensive products.29

M achine learning (a subset of artificial intelligence, or AI) describes a set of

analytical techniques for using big data to make sense of and predict fu-
ture occurrences and could radically transform the ability of agencies to
deliver services and make informed policies.30 Machine learning teaches computers
to learn using training data sets. Familiar home assistants like Siri, Alexa, and Goo-
gle Home are all powered by machine learning. They learn from earlier questions
to understand and answer new questions. En otras palabras, with machine learning,
a computer learns by example rather than through explicit programming instruc-
ciones, opening up a vast array of new possibilities for administrative interventions.
Machine learning takes many forms. The most common, “supervised machine
aprendiendo,” is akin to how a teacher trains a child in arithmetic. The conclusions
are known, and the teacher shows her how to arrive at them. Similarmente, in super-
vised machine learning, the outputs are known and used to help develop an algo-
rithm to reach that conclusion. Using large quantities of labeled data (y ahí
is an ever-expanding number of labeled data sets available on the Internet), mamá-
chine learning can uncover patterns and inductively create general rules. por ejemplo-
amplio, MIT researchers used machine learning to analyze the cough patterns of
more than five thousand people and used that data set to develop an algorithm
that can diagnose COVID-19, and researchers at Stanford looked at a training data
set of cancerous moles to devise a tool that could diagnose skin cancer.31 (To be
clear, machine learning based on large-scale raw data sets, while potentially an
improvement over human diagnostics in some cases, is still error prone.)

The learning in machine learning occurs when the machine turns the data into
a model. Models make us smarter, writes political scientist Scott Page. “Without
modelos, people suffer from a laundry list of cognitive shortcomings: we overweight
recent events, we assign probabilities based on unreasonableness, and we ignore
base rates. . . . With models, we clarify assumptions and think logically. From pow-
er laws to Markov models, such heuristics give us simple ways to test our hypoth-
eses.”32 Increasingly, there are also techniques for unsupervised machine learning
that can find patterns in large quantities of unstructured data.

Machine learning could transform the workings of the public sector. It can
make it possible to target scarce enforcement resources more effectively. por ejemplo-
amplio, Chicago has more than fifteen thousand food establishments, but only
three dozen inspectors. Working in collaboration with Carnegie Mellon Univer-

128

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

sity, Chicago’s city government used its data on restaurant inspections and a wide
variety of other data to create an algorithm to predict food-safety violations. Este
project increased the effectiveness of its inspections by 25 por ciento. Chile’s Labor
Inspectorate is applying machine learning to analyze past accidents and thereby
anticipate workplace safety violations to make inspections more efficient and tar-
geted. The Department of Education is exploring how machine learning and oth-
er technologies could be used to bring down the cost and improve the quality of
creating learning assessments by automating the process of creating questions,
scoring responses, and obtaining insights.33

By making it possible to sort the extraneous chaff from the informational
wheat, machine learning could enable agencies to deliver both new and better
services to the public. But it can also enable agencies to engage a broader public in
decision-making by helping agencies to make public engagement more efficient.
The public has long had a right to comment on any proposed agency regulatory
rulemaking thanks to the Administrative Procedure Act of 1946. Although many
of the three or four thousand rulemakings agencies publish annually receive only
a handful of comments, thanks to the ease of digital commenting, some receive
voluminous responses. En 2017, when the Federal Communications Commission
sought to repeal an earlier Obama-era “net neutrality” rule requiring Internet ser-
vice providers to transmit all content at the same speeds and not discriminate in
favor of one content provider or another, the agency received twenty-two million
comments.34 In 2007, the Fish and Wildlife Service received more than 640,000
email comments on whether to list the polar bear as a threatened species.35

Mientras, in principle, it is good for democracy when more people participate in
reglamentación, the reality is that the large volume of comments–many of which are
“written” by software algorithms or are the result of electronic mass comment
campaigns–also makes it hard for agencies to read or use the material and ren-
ders the public’s engagement mere “democracy theater.” But if agencies used
machine learning to summarize and analyze comments, they could better under-
stand public participation and increase the epistemic value of engagement. Tools
already exist for rapid de-duplication of identical comments and summarization
of unique comments.36 Journalists took advantage of such tools, Por ejemplo,
when they needed to sift rapidly through the 13.4 million documents that made
up the Paradise Papers.37 Both Google and Microsoft announced in 2019 that they
had built systems that could summarize articles.38

While not yet in widespread use in federal agencies, data-analytical techniques
have begun to be used to make sense of citizen input in some contexts. A recent
State Department project offers a simple illustration for how agencies could take
a more effective approach to making sense of rulemaking comments using a com-
bination of artificial intelligence from machines and collective intelligence (CI)
from humans. En 2016, the State Department sought to improve its passport appli-

129

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

cation and renewal process in anticipation of an increase in the number of pass-
port application and renewal forms. The Department ran an online public engage-
ment process to ask people what improvements they wanted. It received almost
one thousand comments and engaged an Israeli-American software company to
help it make rapid sense of the submissions.39

Primero, commenters were asked to highlight the key points of their answers. Para
users who declined to do so, the platform encouraged other users to highlight what
they felt to be the other users’ core ideas. Then the company applied a text-mining
algorithm that scanned the highlighted text for responses containing similar key-
words in order to create summaries, or what the company calls “highlights.” Not
surprisingly, the public was clamoring for a more convenient application process.
While machine learning can make it easier to process large quantities of com-
mentos, there are also challenges inherent in using machine learning precisely be-
cause of the way it creates generalizable rules. If a machine learning algorithm is
“fed” with bad or incomplete data, it will encode bias into the model.40 For ex-
amplio, large companies use machine learning tools (sometimes known as “auto-
mated employment decision tools” or “algorithmic hiring tools”) to conduct and
score video-based applicant interviews. This reduces the costs of screening po-
tential employees. But if machine learning is used to compare applicant responses
with interview answers provided by current employees, and if current employees
are mostly White and American-born, applicants who are Black or foreign-born
will score poorly.41 Nonetheless, if applied to foster democratic engagement, estos
tools can help agencies get “smarter,” faster, from new, more diverse audiences.

T he late nineteenth and early twentieth centuries saw the rise of the profes-

sions–medicine, law, engineering, and social sciences–and of the civil
servicio. To overcome the cronyism of the past, under the Pendleton Re-
form Act of 1883, professional civil servants had to qualify based on an examina-
ción. Rules and procedures were put in place to create a culture of independence
and the tradition of working behind closed doors emerged. Governing, especially
in expert agencies, was meant to be at arm’s length from the people.42 Institutions
and bureaucracies were designed to be hierarchical and rules-based, in order to
support the new vision of the public servant as an impartial mandarin shielded
from undue influence. This culture of isolation persists today. Mike Bracken, para-
mer head of the UK Government Digital Service, writes about the British civil ser-
vicio: “Whitehall was described to me when I started as a warring band of tribal
bureaucrats held together by a common pension scheme.”43

As we saw with public 311 data about rats, thanks to the technologies of collec-
tive intelligence–those Internet-based tools that connect networks of people to
one another for deliberation, data-gathering, collaborative work, shared decision-
haciendo, and collective action–the public is capable of playing an increasingly

130

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

collaborative role in governance. As Geoff Mulgan explains in Big Mind: How Col-
lective Intelligence Can Change Our World, “every individual, organization or group
could thrive more successfully if it tapped into . . . the brainpower of other people
and machines.”44 Humans, aided by machines, are smarter acting together than
solo. They are able to collect and share the information needed to solve problems
mejor. The technologies of collective intelligence create the opportunity to inno-
vate and improve on the traditional regulatory rulemaking commenting process
by enabling agencies to get more relevant information, especially from those who
have not traditionally participated. Collective intelligence technologies do not re-
fer to specific products but to a field of research and an ever-growing set of partic-
ipatory methods and tools.

Diversifying engagement in the administrative state is especially important
because rulemaking–like civic participation generally–does not attract diverse
perspectives. Legal scholar Cynthia Farina has explained that regulated entities
tend to be more represented in rulemakings than regulatory beneficiaries. Semental-
ies by a variety of academics have found that business groups dominate the com-
menting process.45 While there is still not enough empirical research on who par-
ticipates, it appears that individuals all too rarely submit substantive comments,
in the same way that freedom of information requests come far less often from
investigative reporters or civic groups than from businesses.46 We have no data
on race and participation in regulatory rulemakings. Surveys undertaken by Pew
Research Center in 2008 y 2012 found that civic engagement is overwhelming-
ly the province of the wealthy, Blanco, and educated.47 The design of the current
notice-and-comment process exacerbates armchair activism and amplifies some
voices at the expense of others with relevant expertise and experience to share
that could inform regulatory rule writing.

But around the world, public institutions have sought to reverse the decline in
democratic trust by using new technology to enable citizens to participate in law
and policy-making processes, or what I term crowdlaw.

Por ejemplo, in early 2020, before the pandemic, New Jersey’s Future of Work
Task Force, which I chaired, used a “wiki survey” tool called All Our Ideas to en-
gage workers in defining the challenges associated with the impact of technology
on the future of worker rights, salud, Y aprendiendo. All Our Ideas is a free, abierto-
source tool developed by Princeton sociologist Matt Salganik. The wiki survey
tool was prepopulated with dozens of possible responses to the question: qué
is your greatest concern about the impact of technology on the future of work?
Respondents were then asked to decide which, between two randomly selected
statements, is more important to them. People select the response they prefer (o
“I can’t decide” as a third answer) or they may submit their own response. People
can answer as many or as few questions as they choose and, with enough people
participating, the result is a rank-ordered list of the answer choices, yielding in-

131

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

sight into the issues of greatest concern. Over three weeks in February 2020, más
than four thousand workers used the tool to engage about the impact of technol-
ogy on the future of work and share their concerns, such as “unnecessary degree
requirements for jobs have a bigger impact on low-income populations” or “costs
of living–including medical, housing, and education costs–have risen over the
last few decades.” In April 2021, the New Jersey Department of Education used
the same technology to ask parents, estudiantes, and teachers about their priorities
for schools. More than seventeen thousand participated in three weeks, resulting
in greater understanding for policy-makers and the public of the priorities of stu-
abolladuras, profesores, and caregivers, and how they diverge.48

The wiki survey method of showing people two ideas and having them choose
between them or submit a new idea has several practical benefits. It makes it hard-
er to manipulate or game results. Respondents cannot manipulate which answer
options they will see. Además, because respondents must select one of two dis-
crete answer choices from each pair (or add their own), this reduces the impulse
to add new ideas unless there is something new to be said. New submissions can
also be reviewed prior to posting to reduce duplication. También, the need to pick one
of two submissions helps with prioritizing ideas. This feature is particularly valu-
able in policy contexts in which finite resources make it helpful for agency offi-
cials to have some assistance extracting the most unique comments.

Wiki surveys are just one example of technologically enabled engagement.
Other countries are turning to online collaborative drafting platforms to develop
políticas, normas, and laws with the public. En 2018, the German government used a
free annotation platform to “expert source” feedback on its draft artificial intel-
ligence policy.49 The German Chancellor’s Office, working in collaboration with
Harvard University’s Berkman Center for Internet and Society and the New York
University Governance Lab was able to solicit the input of global legal, technolo-
gy, and policy experts. Taiwan and Brazil are turning to technology to include cit-
izens in drafting national legislation as well.50 Using an annotation platform also
made it possible for people to see one another’s feedback and create a robust dia-
logue, instead of a series of disconnected comments.

If agencies would genuinely like to ensure diverse citizen input in the rulemak-
ing process, there are proliferating examples of participatory rulemaking–crowd-
law processes–sprouting up around the world.

T aking advantage of new technology, whether big data, machine learn-

En g, or crowdlaw tools, to regulate, deliver services more effectively, y
co-design laws, regulations and policies with the public needs to start with
training public servants to work differently, imbuing those who govern with a new
set of skills. Retraining, reskilling, and lifelong learning are crucial for thriving in
the digital age, in which technology will transform every job, no less so in the pub-

132

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

lic sector than the private. The Innovator’s DNA: Mastering the Five Skills of Disruptive
Innovators explains that the ability to innovate is not innate, but a learned set of
practices that can and must be taught if businesses are to thrive.51 Yet for all the
talk about investing in private sector training, we are not doing so nearly enough
in the public sector. By failing to invest in teaching public servants how to use data
and collective intelligence–quantitative and qualitative methods–we are failing
to build the skill set of the twenty- first-century public servant.52

To create government that is not smaller or bigger but better, the public sector
needs to nurture talent, invest in training, and foster the development of a new set
of skills. British conservative politician and Minister for the Cabinet Office Mi-
chael Gove declared in a much-publicized speech in June 2020:

The manner in which Government has rewarded its workers for many years now has,
understandably, prized cognitive skills–the analytical, evaluative and, tal vez, arriba
todo, presentational. I believe that should change. Delivery on the ground; making a dif-
ference in the community; practicable, measurable improvements in the lives of oth-
ers should matter more.53

Desafortunadamente, the skills involving data and collaboration needed to make
practicable, measurable improvements in the lives of others–defining problems,
employing data-analytical thinking, using collective intelligence and other inno-
vative ways of working–are not in widespread and consistent use in public ser-
vicios. A 2019 survey I conducted to assess the use of six innovative problem-solv-
ing skills by over four hundred local public officials in the United States shows
that only half were using new data-analytical or engagement skills in their work.54
The results were similar in Australia, where I worked with colleagues at Monash
University to run a comparable survey of almost four hundred mid- to senior-level
public servants about nine skills, from problem definition to research synthesis.
Only one-third of these Australian bureaucrats, on average, used innovative prob-
lem-solving skills.55 Tellingly, sin embargo, once people knew and used a skill, ellos
applied it regularly in their work. But the application is scattershot, and the skills
are not developed for taking a project from idea to implementation.

The public sector’s failure to use creative problem-solving methods that take
advantage of collective intelligence and data is widespread.56 And when public
servants are not getting trained to work differently, that is no wonder. The surveys
showed that respondents had been trained in innovative skills like the use of data
or collective intelligence only between 8 y 30 percent of the time.57

In the Trump administration, which was openly hostile to the civil service and
even signed an executive order (E.O. 13957) giving the president the power to hire
or fire civil servants at will, investing in public sector training and talent did not
happen.58 While the Biden administration is friendlier to the civil service and re-
scinded E.O. 13957, urgent priorities of fighting COVID, climate change, and racial

133

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

equity are drawing the most attention and resources, even though training peo-
ple to work differently could help advance these important political goals. Mientras
the United States is not focusing on training, forward-thinking countries are in-
vesting heavily in training public servants in new skills. Argentina’s Innovation
Academy offers programs on human-centered design that reach thirty-six thou-
sand public servants. Germany has launched the new Digitalakademie with gov-
ernment-wide courses in new ways of working and digital competencies. Caná-
da’s Busrides program offers podcasts about new technologies such as artificial
intelligence and their application to governing aimed at the country’s two hun-
dred and fifty thousand public servants.

U ltimately, the future of the administrative state rests in the hands of peo-

ple who must embrace new ways of working. Individuals drive the ac-
tions of institutions. Futurist and architect Buckminster Fuller likened
the power of the individual change agent to the trim tab, the small rudder that
moves a big ship.59 If we want better government capable of responding to exis-
tential crises like climate change or inequality, we must invest in and train new
líderes: passionate and innovative people who are determined to go beyond mere
compliance to solve problems in new ways.

In addition to training, sin embargo, government at every level needs to recruit
more people with digital and innovation skills. The Tech Talent Project is a non-
profit effort by more than eighty technologists and former policy-makers to con-
duct a review of agency operations and recommend ways to innovate. Ellos, también,
emphasize that “agencies need leaders with modern technical expertise from Day
One” and recommend appointing people with more tech savvy in key leadership
roles as well as training existing personnel.60 It is also key to promote the agile re-
cruitment and hiring of a modern and diverse federal workforce, including hiring
a new generation of public sector leaders (currently, solo 155,000 out of 2.1 millón
federal workers are under thirty) and more people of color, to complement better
efforts at training.61 The overhaul of the Office of Personnel Management and the
Office of Presidential Appointments to facilitate faster hiring and better training,
together with the creation of a Chief People Officer or cabinet-level human cap-
ital position to oversee these efforts, would ensure a robust twenty-first-century
federal workforce and that training becomes a priority for future governments.

We also need greater understanding of the talents already in place in the ad-
ministrative state. While we have data about the age, género, carrera, and disabil-
ity status of federal public servants and know how imbalanced the distribution
of leadership positions is, we know very little about public workers’ current skill
gaps. The federal government should conduct an in-depth diagnostic survey
about the talent and competences of the current workforce to diagnose what peo-
ple do and do not know and empirically determine whether they are using quali-

134

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

tative and quantitative techniques, new technologies, and data-driven research in
how they work. Only by learning what people can do can government facilitate a
data-driven and informed training and hiring strategy. A decade ago, Por ejemplo,
the World Bank developed SkillFinder to keep track of the skills and know-how of
its employees and consultants to foster greater knowledge sharing.62 The United
States should follow the lead of Chile, which conducted a limited skills survey in
2017; Canada, which did so in 2018; and the German Federal Government, cual
is planning in 2021 to distribute the same innovation skills survey I ran among
public officials in the United States and in Australia.

But in addition to training and talent, we need the technology itself. The Gen-
eral Services Administration (GSA) should execute blanket purchase agreements
with appropriate technology vendors to make it easy for every agency to know
which tools to use and how to access innovative new platforms, including AI, mamá-
chine learning, and collective intelligence platforms. We can use the authority
provided by the America Competes Act to host a competition on the federal gov-
ernment’s challenge platform (challenge.gov) to spur the creation of new tools
designed specifically for regulatory agencies, such as platforms for summarizing
comments or undertaking collaborative drafting. The Tech Talent project specif-
ically recommends that, en 2021, the Biden administration prioritize building a
modern data infrastructure to enable robust, secure sharing of data within agen-
cíes, between agencies, and with the American public. In addition to massive in-
vestment in technology infrastructure and funding for technology research, anuncio-
vances in new technology need to be translated into more modern government.
The GSA should not give grants to fund private sector innovation without ensur-
ing that those innovations are used by government, también.

Previously, I have written extensively about using new technology to connect
federal agencies to experts in America’s industries and universities to improve
the level of understanding of science in federal agencies. In Smart Citizens, Smarter
Estado: The Technologies of Expertise and the Future of Governing, I lay out in detail how
the federal government could expand projects like experts.gov for connecting
public servants to smart, outside help to obtain data, hechos, opinions, advice, y
insights from a much broader audience. Además, technology can help to con-
nect administrative agencies to ordinary people with lived experience and situa-
tional awareness. Appellate lawyer and public interest advocate David Arkush has
proposed that administrative agencies adopt a citizen jury system that would em-
panel one thousand randomly selected citizens to provide oversight over agency
Toma de decisiones. In a variation on Arkush’s idea, Administrative Conference of
the United States counsel Reeve Bull, building on an idea expressed earlier by the
Jefferson Center in its work on citizen juries, has proposed creating citizen adviso-
ry committees: relatively small groups of citizens who would advise but not bind
an agency. In Bull’s model, participants would receive background materials gen-

135

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

erated by deliberative polling before their discussions. This is exactly what they
do in Belgium, where random samples of ordinary citizens serve on legislative
committees. Thanks to new technology, it is becoming cheaper and easier to con-
nect with ever-larger quantities of people who can bring their expertise to bear.

F rom Toby Ord to Bill Gates to Stephen Hawking, there is no lack of dooms-

day prognosticators about the dangers of new technology, especially artifi-
cial intelligence. But the greatest risk for our democracy is not the longer-
term future of hyper-intelligent machines. Bastante, the risk right now is that ad-
ministrative agencies will fail to innovate altogether and miss this opportunity
to open the processes of governance to more data and more public engagement.
While there may be a danger from machines wresting control from humani-
ty down the line, right now we have an opportunity to put these tools to use to
strengthen participatory democracy and transform the administrative state.

Desde 2019 government shutdown, the longest in U.S. historia, to the repeat-
ed insults (think “deep state” and “fire Fauci”), to undermining the work of vital
agencies like the Food and Drug Administration, the Centers for Disease Control
and Prevention, and the National Institute of Allergy and Infectious Diseases, el
Trump administration’s approach to governing reflected an emphasis on loyalty
to Trump over expertise and delivering results for the public. But the Biden ad-
ministration will need to do more than roll back enacted regulations or rehire the
people Trump fired on his way out the door. If officials are to take advantage of
data and technology to enhance both the regulatory and service delivery func-
tions of government, Washington has to: invest in broadscale training in digital,
innovation, and public problem-solving skills across the federal enterprise; aprender
who works in government and understand their skills and performance; find the
talent hiding in plain sight and take advantage of their innovative know-how;
speed up the process of bringing in more diverse people to serve; and relax the
rules and customs that prevent federal officials from exercising common sense
and creativity. We can use technology and new ways of working to steer the ship of
state toward a future in which the public sector works openly and collaboratively,
informed by data and engagement. We can overcome our fears about becoming
slaves to new technology by putting those same tools to work for us to create a
stronger, more robust democracy and better government.

136

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

nota del autor

This essay draws from Beth Simone Noveck, Solving Public Problems: How to Fix Our
Government and Change Our World (nuevo refugio: Prensa de la Universidad de Yale, 2021).

Sobre el Autor

Beth Simone Noveck is the author of Solving Public Problems: How to Fix Our Govern-
ment and Change Our World (2021). She is Professor of Technology, Cultura, and Soci-
ety and the Director of the Governance Lab at New York University. En 2018, Gov-
ernor Phil Murphy appointed her to be the Chief Innovation Officer for the State of
New Jersey. She is also the author of Wiki Government: How Technology Can Make Gov-
ernment Better, Democracy Stronger and Citizens More Powerful (2009) and Smart Citizens,
Smarter State: The Technologies of Expertise and the Future of Governing (2015).

notas finales

1 Federation of American Scientists, “Ask a Scientist,” https://covid19.fas.org/ (accedido

Noviembre 1, 2020).

2 Given the explosion of unemployment claims during the pandemic and the fact that, y-
like Social Security, unemployment is administered separately by each state, the White
House Digital Service volunteered to help states fix their systems to respond to the
demand.

3 ProPublica reported that a single junior consultant just out of school runs clients $3.5 mil- lion annually. Ian MacDougall, “How McKinsey Is Making $100 Million (and Counting)
Advising on the Government’s Bumbling Coronavirus Response,” ProPublica, Julio 15,
2020, https://www.propublica.org/article/how-mckinsey-is-making-100-million-and
-counting-advising-on-the-governments-bumbling-coronavirus-response.

4 Ibídem.

5 International Monetary Fund, “World Economic Outlet Update, January 2021” (Lavado-
tonelada, CORRIENTE CONTINUA.: International Monetary Fund, 2021), https://www.imf.org/en/Publications
/WEO/Issues/2021/01/26/2021-world-economic-outlook-update.

6 Fernando G. De Maio, “Income Inequality Measures,” Journal of Epidemiology and Com-
munity Health 61 (10) (2007): 849–852, https://doi.org/10.1136/jech.2006.052969; y
Lawrence Mishel, Elise Gould, and Josh Bivens, “Wage Stagnation in Nine Charts,"
Economic Policy Institute, Enero 6, 2015, https://www.epi.org/publication/charting
-wage-stagnation/.

7 Steven H. Woolf and Heidi Schoomaker, “Life Expectancy and Mortality Rates in the
United States, 1959–2017,” JAMA 322 (20) (2019): 1996–2016, https://doi.org/10.1001/
jama.2019.16932.

8 National Academies of Sciences, Ingeniería, and Medicine, The Growing Gap in Life Ex-
pectancy by Income: Implications for Federal Programs and Policy Responses (Washington, CORRIENTE CONTINUA.:
The National Academies Press, 2015), https://doi.org/10.17226/19015.

9 Jacob Bor, Gregory H. cohen, and Sandro Galea, “Population Health in an Era of Rising
Income Inequality: EE.UU, 1980–2015,” The Lancet 389 (10077) (2017), https://doi.org/

137

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

10.1016/S0140-6736(17)30571-8; and Samuel L. Dickman, David U. Himmelstein, y
Steffie Woolhandler, “Inequality and the Health-Care System in the USA,” The Lancet
389 (10077) (2017), https://doi.org/10.1016/S0140-6736(17)30398-7.

10 Shervin Assari, “George Floyd and Ahmaud Arbery Deaths: Racism Causes Life-Threat-
ening Conditions for Black Men Every Day,” The Conversation, Junio 1, 2020, https://
theconversation.com/george-floyd-and-ahmaud-arbery-deaths-racism-causes-life
-threatening-conditions-for-black-men-every-day-120541.

11 “Americans’ Views of Government: Low Trust, but Some Positive Performance Rat-
ings,” Pew Research Center, Septiembre 14, 2020, https://www.pewresearch.org/
politics/2020/09/14/americans-views-of-government-low-trust-but-some-positive
-performance-ratings/.

12 Paul C. Light, “A Cascade of Failures: Why Government Fails, and How to Stop It,"
Brookings Institution, Julio 14, 2014, https://www.brookings.edu/research/a-cascade
-de- failures-why-government-fails-and-how-to-stop-it/.

13 William P. marshall, “Eleven Reasons Why Presidential Power Inevitably Expands and

Why It Matters,” Boston University Law Review 88 (2008).

14 Adrian Vermeule, “The Administrative State: Law, Democracia, and Knowledge,” in The
Oxford Handbook of the United States Constitution, ed. Mark Tushnet, Mark A. Graber, y
Sanford Levinson (Oxford: prensa de la Universidad de Oxford, 2013).

15 H.R. 4174, Foundations for Evidence-Based Policymaking Act of 2018, Public Law No:
115-435, Enero 14, 2019, https://www.congress.gov/bill/115th-congress/house-bill/4174.
16 Mitch Landrieu, “New Orleans’ Top Priority: Cut Its Murder Rate,” CNN, December
16, 2014, https://money.cnn.com/2014/12/09/news/economy/new-orleans-landrieu/
index.html.

17 “30-2-2 Programs Encourage Companies to Hire Ex-Offenders,” Jails to Jobs, Febrero
1, 2018, https://www.jailstojobs.org/30-2-2-programs-encourage-companies-hire-ex
-offenders/.

18 “New Orleans: Lowest Number of Killings in 47 Años,” U.S. Noticias, Enero 1, 2019, https
://www.usnews.com/news/best-states/louisiana/articles/2019-01-01/new-orleans
-lowest-number-of-killings-in-47-years.

19 “Tracking of Workplace Injuries and Illnesses,” Occupational Safety and Health Admin-
istración, Federal Register 84 (17) (2019): 00101, https://www.osha.gov/laws-regs/federal
register/2019-01-25; and The White House, “Executive Order on Protecting Worker
Health and Safety,” January 21, 2021, https://www.whitehouse.gov/briefing-room/
presidential-actions/2021/01/21/executive-order-protecting-worker-health-and-safety/.
20 “Repay Student Debt,” Consumer Financial Protection Bureau, https://www.consumer
finance.gov/paying-for-college/repay-student-debt/# (accessed November 2, 2020);
and Christa Gibbs, CFPB Data Point: Student Loan Repayment (Washington, CORRIENTE CONTINUA.: Estafa-
sumer Financial Protection Bureau, 2017), 7–9, https://files.consumerfinance.gov/f/
documents/201708_cfpb_data-point_student-loan-repayment.pdf.

21 “Home Mortgage Disclosure (Regulation C),” Consumer Financial Protection Bureau,
Federal Register 80 (208) (2015): 66128, https://www.govinfo.gov/content/pkg/FR-2015
-10-28/pdf/2015-26607.pdf; Consumer Financial Protection Bureau, “Home Mortgage
Disclosure Act (Regulation C),” https://www.consumerfinance.gov/policy-compliance
/rulemaking/final-rules/regulation-c-home-mortgage-disclosure-act/ (updated April 16,

138

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

2020); and “The Biden Plan for Investing in Our Communities through Housing,"
Biden/Harris: A Presidency for All Americans, https://joebiden.com/presidency-for
-all-americans/ (accessed May 5, 2021).

22 Mateo J.. Salganik, Bit by Bit: Social Research in the Digital Age (Princeton, NUEVA JERSEY.: Princeton

Prensa universitaria, 2019), 16, 82–83.

23 Diana Farrell, Fiona Greig, and Amar Hamoudi, The Online Platform Economy in 27 Metro
Areas: The Experience of Drivers and Lessons (Nueva York: JPMorgan Chase Institute, 2019),
https://institute.jpmorganchase.com/content/dam/jpmc/jpmorgan-chase-and-co/
institute/pdf/institute-ope-cities-exec-summary.pdf. “The JPMorgan Chase Institute
is harnessing the scale and scope of one of the world’s leading firms to explain the
global economy as it truly exists. Its mission is to help decision-makers–policymakers,
negocios, and nonprofit leaders–appreciate the scale, granularity, diversity, and in-
terconnectedness of the global economic system and use better facts, timely data, y
thoughtful analysis to make smarter decisions to advance global prosperity. Drawing
on JPMorgan Chase’s unique proprietary data, expertise, and market access, the In-
stitute develops analyses and insights on the inner workings of the global economy,
frames critical problems, and convenes stakeholders and leading thinkers.”

24 Some examples in this section are drawn from Beth Simone Noveck, “Rights-Based and
Tech-Driven: Open Data, Freedom of Information, and the Future of Government
Transparency,” Yale Human Rights and Development Law Journal 19 (1) (2017): 18, https://
digitalcommons.law.yale.edu/yhrdlj/vol19/iss1/1.

25 Raj Chetty, Nathaniel Hendren, and Lawrence Katz, “The Effects of Exposure to Better
Neighborhoods on Children: New Evidence from the Moving to Opportunity Project,"
Revisión económica estadounidense 106 (4) (2016).

26 Louisiana Department of Health, “Louisiana Receives Approval for Unique Strategy to
Enroll SNAP Beneficiaries in Expanded Medicaid Coverage,” June 1, 2016, http://ldh
.la.gov/index.cfm/newsroom/detail/3838.

27 Daniel T. O'Brien, The Urban Commons: How Data and Technology Can Rebuild Our Communi-

corbatas (Cambridge, Masa.: Prensa de la Universidad de Harvard, 2018).

28 Beth Simone Noveck and Joel Gurin, “Corporations and Transparency: Improving Con-
sumer Markets and Increasing Public Accountability,” in Transparency in Politics and the
Media: Accountability and Open Government, ed. Nigel Bowles, James T. hamilton, and Da-
vid A. Lev (Nueva York: I. B. Tauris, 2013).

29 Justin Elliott and Lucas Waldron, “Here’s How TurboTax Just Tricked You into Paying
to File Your Taxes: Come Along as We Try to File Our Taxes for Free on TurboTax!"
ProPública, Abril 22, 2019, https://www.propublica.org/article/turbotax-just-tricked
-you-into-paying-to-file-your-taxes.

30 Gideon Mann, “Machine Learning Applications for the Public Sector,” Solving Pub-
lic Problems with Data recorded lecture, 2017, http://sppd.thegovlab.org/lectures/
machine-learning-applications-for-the-public-sector.html.

31 Jordi Laguarta, Ferran Hueto, and Brian Subirana, “COVID-19 Artificial Intelligence Di-
agnosis Using Only Cough Recordings,” IEEE Open Journal of Engineering in Medicine and
Biología 1 (2020), https://doi.org/10.1109/OJEMB.2020.3026928. See also Andre Esteva,
Brett Kuprel, Roberto A. Novoa, et al., “Dermatologist-Level Classification of Skin
Cancer with Deep Neural Networks," Naturaleza 542 (7639) (2017): 115–118, https://doi.org
/10.1038/nature21056.

139

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

32 Scott E.. Página, The Model Thinker: What You Need to Know to Make Data Work for You (Nuevo

york: Libros Básicos, 2018), 15.

33 “Artificial Intelligence and Assessment,” Conference Proceedings, Abril 30, 2021,
Smarter Crowdsourcing, The GovLab, https://education.smartercrowdsourcing.org.
34 “Restoring Internet Freedom, A Proposed Rule by the Federal Communications Com-
mission,” Federal Register 82 (105) (2017): 25568; and “Restoring Internet Freedom, A
Rule by the Federal Communications Commission,” Federal Register 83 (36) (2018): 7852.
35 Estados Unidos. Fish and Wildlife Service, “News Release: Polar Bear Range States Meet to
Exchange Information,” June 25, 2007, https://www.fws.gov/news/ShowNews.cfm
?newsId=64E14600-FE16-C275-509DA6F8C494B903.

36 Steve Balla, Reeve Bull, Bridget Dooling, Beth Simone Noveck, et al., “Mass, Computadora-
Generated, and Fraudulent Comments,” draft report for the Administrative Conference
of the United States, Abril 2, 2021, https://www.acus.gov/report/mass-computer
-generated-and-fraudulent-comments-draft-report-4221.

37 Fabiola Torres López, “How They Did It: Methods and Tools Used to Investigate the
Paradise Papers,” Global Investigative Journalism Network, December 4, 2017, https://
gijn.org/2017/12/04/paradise-papers/.

38 Patrick Fernandes, Miltiadis Allamanis, and Marc Brockschmidt, “Structured Neural
Summarization,” conference paper presented at the International Conference on Learn-
ing Representations (ICLR 2019), Febrero 2019, https://arxiv.org/pdf/1811.01824.pdf;
and “Text Summarization Using TensorFlow,” Google AI Blog, Agosto 2016, https://
ai.googleblog.com/2016/08/text-summarization-with-tensorflow.html.

39 Andrew Zahuranec, Andrew Young, and Stefaan G. Verhulst, Identifying Citizens’ Needs by
Combining Artificial Intelligence (AI) and Collective Intelligence (Nueva York: The GovLav, 2019),
https://www.thegovlab.org/static/files/publications/CI-AI_oct2019.pdf.

40 Betsy Anne Williams, Catherine F. Arroyos, and Yotam Shmargad, “How Algorithms
Discriminate Based on Data They Lack: Challenges, Solutions, and Policy Implica-
ciones,” Journal of Information Policy 8 (2018): 78–115, www.jstor.org/stable/10.5325/
jinfopoli.8.2018.0078.

41 Solon Barocas and Andrew Selbst, “Losing Out on Employment Because of Big Data Min-
En g,” The New York Times, Agosto 6, 2014, https://www.nytimes.com/roomfordebate
/2014/08/06/is-big-data-spreading-inequality/losing-out-on-employment-because-of
-big-data-mining.

42 “The role of the professional civil servant is enshrined by the law itself, which reinforces
the profession’s control over the flow of information into and out of institutions–what
Pierre Bourdieu calls the ‘officializing strategy’ of bureaucracy–in ways designed to
dissuade citizens from engagement. There is a wealth of administrative law that limits
control over speech in the public sector to public management professionals and treats
their decisionmaking with legal deference. Por ejemplo, key information law statutes
intentionally limit information sharing and collaboration and preserve the domain of
the public servant distinct from and closed to others. The public earned a right to ac-
cess information held by government relatively late in the twentieth century, e incluso
entonces, only upon request and with significant limitations. For those of us outside the
curtain, the effect is impressive.” Beth Simone Noveck, Smart Citizens, Smarter State: El

140

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State

Technologies of Expertise and the Future of Governing (Cambridge, Masa.: Harvard University
Prensa, 2015), 49.

43 Brian Glick, “Interview: Government Digital Chief Mike Bracken–Why I Quit,” Com-
puter Weekly, Agosto 13, 2015, https://www.computerweekly.com/news/4500251662/
Interview-Government-digital-chief-Mike-Bracken-why-I-quit.

44 Geoff Mulgan, Big Mind: How Collective Intelligence Can Change Our World (Princeton, NUEVA JERSEY.:

Prensa de la Universidad de Princeton, 2017).

45 Jason Webb Yackee and Susan Webb Yackee, “A Bias towards Business? Assessing In-
terest Group Influence on the U.S. Bureaucracy,” The Journal of Politics 68 (1) (2006):
128–139.

46 David E. Pozen, “Freedom of Information beyond the Freedom of Information Act,"
University of Pennsylvania Law Review 165 (2016): 1097, https://scholarship.law.columbia
.edu/faculty_scholarship/2022.

47 Aaron Smith, “Part 1: Online and Offline Civic Engagement in America,” in Civic Engage-
ment in the Digital Age (Washington, CORRIENTE CONTINUA.: Pew Research Center, 2013), https://www
.pewresearch.org/internet/2013/04/25/part-1-online-and-offline-civic-engagement
-in-america/. “A key finding of our 2008 research was that Americans with high lev-
els of income and educational attainment are much more likely than the less educated
and less well-off to take part in groups or events organized around advancing political
or social issues. That tendency is as true today as it was four years ago, as this type of
political involvement remains heavily associated with both household income and ed-
ucational attainment.”

48 “Make Your Voice Heard!” https://nj.youreducationyourvoice.org/ (accessed April
29, 2021). See also Gopal Ratnam, “Tech Tools Help Deepen Citizen Input in Drafting
Laws Abroad and in U.S. Estados,” Roll Call, Abril 20, 2021, https://www.rollcall.com/
2021/04/20/tech-tools-help-deepen-citizen-input-in-drafting-laws-abroad-and-in-u-s
-estados.

49 Beth Simone Noveck, Rose Harvey, and Anirudh Dinesh, The Open Policymaking Playbook
(Nueva York: The GovLab, 2019), https://www.thegovlab.org/static/files/publications/
openpolicymaking-april29.pdf.

50 Ratnam, “Tech Tools Help Deepen Citizen Input.”
51 Jeff Dyer, Hal Gregersen, and Clayton Christensen, The Innovator’s DNA: Mastering the Five
Skills of Disruptive Innovators (Brighton, Masa.: Harvard Business Review Press, 2011), 22.
52 Blue Wooldridge, “Increasing the Productivity of Public-Sector Training,” Public Produc-

tivity Review 12 (2) (1988): 205–217.

53 Michael Gove, “‘The Privilege of Public Service’ Michael Gove’s Ditchley Lecture,” as
reprinted in The New Statesman, Junio 28, 2020, https://www.newstatesman.com/politics/
uk/2020/06/privilege-public-service-michael-gove-s-ditchley-lecture-full-text.

54 For details on the public entrepreneurship innovation skills surveys sponsored by the
International City and County Managers Association (ICMA), see the Public Entrepre-
neur Skills Surveys, www.publicentrepreneur.org/skills.

55 Beth Simone Noveck and Rod Glover, The Public Problem Solving Imperative (Carlton, Aus-

tralia: The Australia and New Zealand School of Government, 2019).

141

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

150 (3) Summer 2021Beth Simone Noveck

56 This report builds on earlier work by the OECD, which did the first countrywide study
of the pervasiveness of innovation skills in a survey of 150 Chilean public servants, y
subsequently elaborated on this work in a report on core governance innovation skills,
both in 2017. See Observatory of Public Sector Innovation, Core Skills for Public Sector
Innovation (París: Organisation for Economic Co-operation and Development, 2017),
https://www.oecd.org/media/oecdorg/satellitesites/opsi/contents/files/OECD_
OPSI-core_skills_for_public_sector_innovation-201704.pdf.

57 Noveck and Glover, The Public Problem Solving Imperative.
58 “Executive Order 13957 of October 21, 2020: Creating Schedule F in the Excepted Ser-

vicio,” Federal Register 85 (207) (2020).

59 The TrimTab Conspiracy was the name of the “salon” that David Johnson, Susan Craw-
ford, and I ran between 2003 y 2008 in New York and then in Washington, D.C.
It was self-consciously styled as an opportunity to discuss strategies for public prob-
lem-solving using new technologies. We met either every two weeks or once a month
for these discussions for many years. See Buckminster Fuller, “A Candid Interview
with R. Buckminster Fuller,” Playboy magazine, Febrero 1972, http://www.bfi.org/
sites/default/files/attachments/pages/CandidConversation-Playboy.pdf.

60 Tech Talent Project, “A 21st Century Administration Requires Strong, Modern Technical
Leaders,” https://techtalentproject.org/government/ (accessed November 2, 2020).
61 The White House, “Strengthening the Federal Workforce, FY 2019.” See also Dan
Durak, “Government’s Lack of Diversity in Leadership Positions,” Partnership for
Public Service, Marzo 11, 2019, https://ourpublicservice.org/blog/governments-lack-of
-diversity-in-leadership-positions//.

62 The GovLab, El Banco Mundial: Skillfinder (Nueva York: The GovLab, 2016), https://www

.thegovlab.org/static/files/smarterstate/skillfinder.pdf.

D
oh
w
norte
oh
a
d
mi
d

F
r
oh
metro
h

t
t

pag

:
/
/

d
i
r
mi
C
t
.

metro

i
t
.

mi
d
tu
d
a
mi
d
a
r
t
i
C
mi
–
pag
d

F
/

1
5
0
3
1
2
1
2
0
6
0
4
8
2
d
a
mi
d
_
a
_
0
1
8
6
3
pag
d

b
y
gramo
tu
mi
s
t

oh
norte
0
8
S
mi
pag
mi
metro
b
mi
r
2
0
2
3

142

Dédalo, la Revista de la Academia Estadounidense de las Artes & SciencesThe Innovative State
Descargar PDF