Michael Mandel. Progressive Policy Institute. For presentation at IDEA Workshop. Frank H. Kenan Institute of Private Enterprise

Similar documents
Online Job Demand Up 255,000 in December, The Conference Board Reports

Online Job Demand Down 83,200 in October, The Conference Board Reports

Online Job Demand Up 169,000 in August, The Conference Board Reports

Online Job Demand Up 106,500 in November, The Conference Board Reports

For further information: Carol Courter / Release #5990. Online Job Ads Increased 229,700 in December

For further information: Carol Courter / Release #5985. Online Job Ads Increased 137,100 in November

For further information: Carol Courter / Release #5952. Online Job Ads Increased 195,600 in May

For further information: Carol Courter / Release #5996. Online Job Ads Increased 1,200 in January

For further information: Carol Courter / Release #5967. Online Job Ads Decreased 125,900 in August

For further information: Carol Courter / Release #5963. Online Job Ads Decreased 157,700 in July

For further information: Carol Courter / Release #6029. Online Job Ads Increased 170,800 in July

For further information: Carol Courter / Release #5980. Online Job Ads Increased 81,500 in October

The Conference Board Reports Online Job Demand Drops 507,000 in December

For further information: Carol Courter / Release #5942. Online Job Ads Increased 102,000 in March

For further information: Carol Courter / Release #6016. Online Job Ads Decreased 69,300 in April

For further information: Carol Courter / Release #5931

For further information: Carol Courter / Release #5862

For further information: Frank Tortorici: / board.org Release #5458

For further information: Carol Courter / Release #5916

For further information: Carol Courter / Release #5486

Online Labor Demand up 232,000 in June

Online Labor Demand Rises 164,600 in August

For further information: Carol Courter / Release #5806. Online Labor Demand Dropped 104,500 in April

2017 Competitiveness REDBOOK. Key Indicators of North Carolina s Business Climate

The Middle Market Power Index: BEST U.S. CITIES FOR MIDDLE MARKET FIRMS

Colorado River Basin. Source: U.S. Department of the Interior, Bureau of Reclamation

REGIONAL AND STATE EMPLOYMENT AND UNEMPLOYMENT JUNE 2010

REGIONAL AND STATE EMPLOYMENT AND UNEMPLOYMENT MAY 2013

Saudi Government Scholarship Program - USA. Statistical Presentation For Student Enrollment in US Universities As of February 2007

F Quarter 201 Real Estate Market Update. The Builder Developer Lender Council of the Greater Atlanta Home Builders Association

STATE ENTREPRENEURSHIP INDEX

CONNECTICUT: ECONOMIC FUTURE WITH EDUCATIONAL REFORM

Rankings of the States 2017 and Estimates of School Statistics 2018

FBI Field Offices. Louisville Division Room Martin Luther King Jr. Place Louisville, Kentucky (502)

College Profiles - Navy/Marine ROTC

CHAPTER CHAPTER DUES CANDIDATE & NEW REGULAR RETIRED DESIGNEE DUES

GROWING THE MIDDLE: SECURING THE FUTURE LOS ANGELES

Figure 10: Total State Spending Growth, ,

Travel Impact Report

Ethnic Studies Asst 55, ,755-2, ,111 4,111

Putting Nanotechnology on the Map

Dashboard. Campaign for Action. Welcome to the Future of Nursing:

Current Medicare Advantage Enrollment Penetration: State and County-Level Tabulations

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

Unemployment Rate (%) Rank State. Unemployment

NSTC COMPETITIVE AREA DEFINITIONS. UIC Naval Service Training Command (NSTC), Great Lakes, IL

U.S. Psychology. Departments

Ethnic Studies Asst 54, ,315-3, ,229 6,229. Gen Honors/UC Asso 64, ,402-4, ,430 24,430

TABLE 3c: Congressional Districts with Number and Percent of Hispanics* Living in Hard-to-Count (HTC) Census Tracts**

TABLE 3b: Congressional Districts Ranked by Percent of Hispanics* Living in Hard-to- Count (HTC) Census Tracts**

Its Effect on Public Entities. Disaster Aid Resources for Public Entities

Georgia Power Community & Economic Development

MapInfo Routing J Server. United States Data Information

APPENDIX c WEIGHTS AND MEASURES OFFICES OF THE UNITED STATES

Interstate Pay Differential

Estimated Economic Impacts of the Small Business Jobs and Tax Relief Act National Report

ON THE GLOBAL, REGIONAL & LOCAL ECONOMIC CLIMATE

Listed below are the states in which GIFT has registered to solicit charitable donations and includes the registration number assigned by each state.


Use of Medicaid MCO Capitation by State Projections for 2016

Candidate Application

Census of State and Local Law Enforcement Agencies, 1996

Key Vocabulary Use this space to write key vocabulary words/terms for quick reference later

House Prices: A pictorial review

ON THE ECONOMIC, BUSINESS AND POLITICAL CLIMATE

Single Family Loan Sale ( SFLS )

Fiscal Research Center

VOLUME 35 ISSUE 6 MARCH 2017

PRESS RELEASE Media Contact: Joseph Stefko, Director of Public Finance, ;

The American Legion NATIONAL MEMBERSHIP RECORD

Final Award Listing

US News and World Report Rankings Graduate Economics Programs Ranked in 2001

Arts and Culture in Metro Atlanta: By the Numbers. February 21, 2018

Employment Outcomes, New York / Metro NYC Law Schools

THE STATE OF GRANTSEEKING FACT SHEET

Compensation Data Colleges & Universities Participant List

California Economic Snapshot 3 rd Quarter 2014

50 STATE COMPARISONS

Initial (one-time) Membership Fee 10,000 Renewal Fee (every 8 years) $3500

Rutgers Revenue Sources

Virginia Association of Economists

University of Maryland-Baltimore County

Name: Date: Albany: Jefferson City: Annapolis: Juneau: Atlanta: Lansing: Augusta: Lincoln: Austin: Little Rock: Baton Rouge: Madison: Bismarck:

The Search for Skills

The Colorado Registered Nurse Pool and Out-of-State Recruitment

National Bureau for Academic Accreditation And Education Quality Assurance ACTUARIAL SCIENCE

Index of religiosity, by state

CAIR Conference Anaheim, CA, Nov. 6-9, 2012

The MetLife Market Survey of Nursing Home & Home Care Costs September 2004

Application for Retired Member Status

North Carolina Central University Contact Information for Filing Student Complaints

SCORING TECH TALENT

Transcription:

DRAFT: Do Not quote or cite without permission. Tracking the Real-Time Geography of Innovative Activity Using Online Job Postings. Michael Mandel Progressive Policy Institute. For presentation at IDEA Workshop Frank H. Kenan Institute of Private Enterprise UNC Kenan-Flagler Business School May 22-23, 2017 Abstract: This paper shows how publicly available online job postings can be used to track the real-time geography and concentration of innovative activity. The methodology is illustrated with machine learning/artificial intelligence; 3D printing/additive manufacturing; genomics; Crispr, carbon fiber and composites; and ecommerce. The paper finishes by discussing two innovations that appear to be getting no traction in the labor market. We thank the Alfred Sloan Foundation for its generous support for this project.

Introduction Data sets generated by companies in the course of normal business otherwise known as organic data, big data or business practice data are increasingly important to national statistical agencies and economic researcher. Such organic data sets can potentially augment government surveys, which are increasingly expensive and have falling response rates (Groves, 2011). Organic data can be used to track important economic variables even when government surveys do not exist or are untrustworthy (Rigobon and Cavallo, 2016). The use of organic data to improve economic statistics has become common knowledge around the world. Recently a Japanese newspaper editorial called for the Japanese government to start calculating gross domestic product using new methods, such as one making use of big data owned by companies (Yomiuri Shimbun, 2016). Organic data can be used as a microscope to study smaller geographic areas and more detailed data categories than can be covered by even the best national survey, as Einav and Levin (2014) note. In practice, national surveys are limited by their sample size and by the number of questions that they can realistically ask. The needs of statistical disclosure limitation create a real issue for local data analysis based on government surveys (Abowd and Schmutte, 2015). The Bureau of Labor Statistics and the Census Bureau routinely and correctly suppress local data that could potentially reveal information about individual companies. For

example, Amazon opened up a large fulfillment center (1.2 million square feet) in Jeffersonville, Indiana in 2012. Not surprisingly, starting in 2012 the BLS suppresses QCEW data for nonstore retailers (NAICS 454) in Clark County, where Jeffersonville is located. Organic data can fill in the gaps. Lazar et al (2014) note that perhaps the most valuable use of Google Flu Trends is to understand the prevalence of flu at very local levels, which is not practical for the CDC to widely produce, but which, in principle, more finely granular measures of GFT could provide. Glassdoor has started producing local pay reports, based on data reported by users of their website (Chamberlain, 2016). Moreover, as the economy changes, it typically takes years before new data categories are introduced into official surveys. Web developer was added to the Standard Occupational Classification in 2010, but the term was in common use at least as early as 1995, when visitors to the official White House web page were invited to send emails to the White House web developers and maintainers (White House, 1995). That s 15 years in which state and local policymakers, including leaders of local institutions of higher education, had no way of systematically determining the importance of this new field. Today the same lack of data for new industries and occupations perplexes state and local policymakers trying to assess the potential labor market demand for new college graduates. There is no official data available for industries such as additive

manufacturing or occupations such as social media managers. Organic data can be useful to help policymakers fill in this knowledge gap. Policy analysts also have to contend with the possibility that familiar industry and occupation categories have evolved to the point that they no longer mean the same thing. For example, many innovative manufacturers have increasingly focused on R&D, design, and marketing, while outsourcing the actual production to other enterprises both domestic and abroad. In this way, innovative and successful manufacturing firms create different kinds of jobs than in the past. Policymakers on the national and local levels are often interested in the location of innovation. Today, policymakers might want to know which areas of their country have clusters of big data jobs, which would signal innovative activity. Local economic development officials might like to know if their area is being successful compared to other areas in terms of attracting innovative industries or jobs. Online job postings This paper is part of a larger project, funded by the Sloan Foundation, to improve the usefulness of organic data for policy, especially at the state and local level. We are particularly concerned with tracking the intensity and location of innovative activity, especially in new fields that are not well-captured by government data. We will show here how one type of organic data online job postings can be used to track innovative activity on a national and local level. The first important point is that any company which is engaged in research, product development, or

production in an innovative field, is almost certainly going to be advertising for workers with skills in that innovative area. For example, in May 2017, Nexosis, a small venture-backed startup in Columbus, Ohio, was advertising for a data scientist with 2+ years of experience with Machine Learning. The second important point is that job postings are public, and contain no personal data about individuals. Companies get to choose their level of anonymity by going through recruiting agencies if need be, so there is no need for statistical disclosure limitations. Finally, real-time data on job postings are easily available. Job search engines such as Indeed, Monster, Glassdoor, Wanted Analytics (for the Conference Board) and others routinely scan job boards and corporate career listings for job postings. Our data for this study comes from Indeed, a job posting aggregator that calls itself the world's #1 job site, with over 200 million unique visitors every month from over 60 different countries. Indeed aggregates the universe of online job postings into a database that is updated in real-time and Boolean-searchable by key words and phrases. Job postings are tagged geographically and by distance, so that it s possible to ask for a count of all the job postings within 20 miles, say, of Kansas City, containing the word genomics. Indeed provides a public API for accessing the data. A very limited amount of historical data is available. This paper will briefly describe how job posting data differs from government survey data; gives examples of how job posting data can be used to map innovation; outlines the next steps in the research; and discusses possible policy uses.

How Job Posting Data Differs from Government Survey Data In general, economists who are used to working with systematically administered and carefully benchmarked government surveys want to treat organic data the same way. In particular, it is tempting to interpret counts of online job postings as if they are a macro labor demand measure, directly corresponding to vacancy counts from government surveys. It is true that employers place want ads when they have a job vacancy. The BLS defines a position as a vacancy if: 1. A specific position exists and there is work available for that position. The position can be full-time or part-time, and it can be permanent, short-term, or seasonal, and 2. The job could start within 30 days, whether or not the establishment finds a suitable candidate during that time, and 3. There is active recruiting for workers from outside the establishment location that has the opening. In other words, the concept of a job vacancy is specifically designed and standardized to be the mirror of an unemployed worker. That enables economists to calculate the vacancy rate, which is the number of vacancies as a share of total employment. Just as the unemployment rate measures the difficulty that workers have finding jobs, the vacancy rate measures the difficulty that employers have in filling positions. Both the vacancy rate and the unemployment rate are macroeconomic indicators, in the sense that they tell us something about the overall state of the labor market and the economy. However, there is no reason for a one-to-one mapping between job postings and job openings. For one, an employer can place multiple job postings for a single opening, including through employment recruiters. That s especially true in cases where the workers are in high demand, so the employer wants to make sure to gain the

maximum visibility for the ad. Some of these duplicates may be easy to remove from the data, but some are more difficult to weed out. Job postings can remain online well after the opening was filled, or represent openings that are never actually meant to be filled. Conversely, some job postings represent more than one job, especially for large employers such as retailers, restaurants, hotels, and hospitals that are perpetually trying to fill multiple job openings. A simple calculation will illustrate the scope of the problem. Walmart employs 1.5 million people in the United States. The annual hiring rate in retail was 58% of employment in 2015. If this same ratio applies to Walmart, the company would have to run 900,000 job postings per year. Looking at Indeed suggests Walmart is running roughly 600 new job postings per week nationally, or 30,000 per year. Moreover, it is well known that raw organic data can be biased and volatile. Raghunathan (2015) notes selection bias can have a big impact on the inferences from the non-survey data and could not be even checked without having a reliable survey or some external data to check against or to calibrate. Volatility is inevitable, since the process generating the data can shift according to the needs of the organization. Lazer et al (2014) suggest that the failure of Google Flu Trends to match CDC flu projections came about in part because the algorithm producing the data (and thus user utilization) has been modified by the service provider in accordance with their business model. Similarly, counts of online job postings can be affected by algorithm changes by job aggregators (Mandel and

Scherer, 2015). Cajner and Ratner (2016) note that changes in the prices of online job postings can affect the number of job postings. The implication is that job postings are problematic as a macroeconomic indicator. Indeed, the Conference Board HWOL count of online advertised vacancies fell 15% from 5.4 million in April 2016 to 4.6 million in April 2017, compared to a much more modest 2% decline in BLS job openings over the same period (5.85 to 5.74) Job Postings and Innovation However, even if job postings are problematic as a macroeconomic indicator, they may be useful in other ways. Job postings can be used to study the STEM labor market (Rothwell, 2014) and local supply and demand (Berger-Gross, 2014; Berger- Gross 2015). By bemchmarking job postings to official statistics, they can be used to compare the number of workers engaged in innovative activity say, mobile app development across different states and countries (Mandel, 2012; Mandel and Scherer, 2015; Mandel 2016b, Mandel 2017b). Job postings can be used to explore the innovation frontier (Litan, Wyckoff, and Fealing, 2014; Mandel 2016a) and to measure the strength of the startup ecosystem startup activity in metro areas (Mandel 2017a). In this paper, we will do something simpler: Use job postings to qualitatively identify the concentration of innovations such as machine learning and artificial intelligence, 3D printing and additive manufacturing, and genomics across the country.

We note that Indeed has a public API that can be used to get a count of job postings, by location, that match a Boolean expression using operators such AND, OR, and NOT. In addition to the Boolean search expression and location, the API also allows the specification of the distance from the listed location, how long the job posting has been available, and the strictness of deduplication. For that reason, there is considerable ambiguity in the job posting count. For example the total number of job postings in the country, with no age limit, is 3.3 million. With an age limit of 30 days, that number goes down to 2.5 million. With an age limit of 7 days, the count goes down to just under 600 thousand. Strict deduplication cuts that number even more. As part of the Sloan-funded research project, we are constructing a longitudinal data set of the parallel measures, so we can examine the volatility over time and decide which measure is best. For the purposes of this paper, however, we will stick with no age limit and moderate deduplication, which tends to smooth results over time. Another important point is that the location algorithm can either follow the geographic borders of states, cities and counties, or report on job posting counts for a set geographic distance around a central point. For our metro area analysis, we use 25 miles around the core city, paying careful attention to potential overlaps. All data was drawn the first two weeks of May.

Results Summary Table A shows the searches that we did for this paper on the 100 largest metro areas. Roughly speaking, that top 5 metro areas have about 20% of the job postings, employed workers, and managers and professionals. The top 20 metro areas have about 40% of the job postings, and roughly the same share of workers and managers and professionals. The rankings for job postings and employment are not identical, but include mostly the same metro areas (see Table 1, 2, and 3) That will be our benchmark to assess the concentration of innovative jobs. We start with the search machine learning or artificial intelligence, which is much more highly concentrated, with 60% of the job postings concentrated in the top 5 metro areas. From Table 4, these are Seattle, San Jose, San Francisco, New York City, and Boston. Washington DC is number 6 on the list. The top companies include Amazon and Microsoft. Daimler is investing as well. By contrast, 3D printing and additive manufacturing is far less concentrated, with only 32% of job postings in the top 5 metro areas. From Table 5, these are Boston, San Jose, New York City, San Francisco, and Los Angeles. The top 20 includes several midwestern cities, including Detroit, Indianapolis, Dayton, and Cincinnati. But they still have a relatively small share compared to the traditional tech hubs.

Summary Table A: Concentration of Innovative Jobs by Metro Area Top 5 Top 20 All job postings* 19.5% 39.9% All workers 18.3% 39.8% Managers and 20.2% 43.5% professionals Machine learning or 60.4% 86.6% artificial intelligence 3D printing or additive 32.1% 64.0% manufacturing Genomics 39.8% 69.5% Genomics and -Eurofins 46.0% 75.2% Job postings within 25-mile radius. Moderate deduplication, no time limit on postings. *For this search only, New York and Los Angeles are surveyed at 50-mile radius. Data: Indeed API, May 2017.

Now let s turn to genomics. The case of genomics illustrates one of the key issues with using job postings as a data source sometimes a large company will use a key word in the boilerplate for its job postings. In this case Eurofins, a large global testing company, has a genomics division that it mentions in all of its job postings. So the count is higher than it should be. We ran the search both with and without Eurofins included. As Tables 6a and 6b show, removing Eurofins keeps the order more or less the same, with Boston, San Francisco, San Jose, San Diego, and New York City in the top 5. However, without Eurofins, the concentration of genomic jobs increases to 75% in the top 20 metro areas. We can map the concentration of innovative jobs on the state level as well. Summary Table B reports on the concentration of job postings for the top 5 and top 10 states. We start with the search for the term Crispr, which is an acronym for a genome editing technique that holds great promise. We find that 79% of the job postings containing the term Crispr were concentrated in the top 5 states: Massachusetts, California, New York, Maryland, Washington. Leading employers are Pfizer, Crispr Therapeutics, and the Broad Institute.

Summary Table B: Concentration of Innovative Jobs by State Top 5 Top 10 All job postings Crispr Carbon fiber Composite and material 37% 54% 79.3% 89.8% 61.5% 79.1% 40.3% 56.2% and Princeton Fulfillment center 3D printing or additive 49% 70.9% 47.3% 67.4% manufacturing 3D printing or additive 53.0% 69.3% manufacturing (December 2015) Genomics Genomics (December 58.5% 74.6% 60.0% 74.8% 2015)

On the other hand, there are 34 states that have 0 or 1 job postings for Crispr. So if Crispr turns out to be the breakthrough technology that many expect, these states will not be in the lead in generating new jobs and businesses. Carbon fiber is one type of new material, and the job postings for carbon fiber are somewhat better distributed. The top 5 states are Michigan, California, Florida, Utah, and Washington, with 61.5% of the job postings. Still, 27 states have 0 or 1 job postings for carbon fiber. Top employers include companies such as Pratt & Whitney, Hexcel, and SpaceX. A more general class of new materials are composites, so we did a search for job postings that have the words Composite and material. It turned out that this initial search also caught a large number of job postings for the Princeton Review, which was looking for instructors with high composite scores on the ACT. So to weed out those, we added another term to the search expression: composite and material and Princeton. The result was that the top 5 states California, Michigan, Texas, Ohio, Florida--had 40% of the job postings, which is just slightly above the baseline. Some top employers include Pratt & Whitney, Owens Corning, Lockheed Martin, Space X, and Honeywell. All states have at least one job posting in this category. Job postings can also be used to track new business models as well as new technologies. Ecommerce took off in the early 2000s with Amazon s realization that

online customers wanted rapid deliveries. That led to them setting up fulfillment centers all around the country, with other large retailers following suit. We can track these fulfillment centers, in part, by looking at places where warehouse employment has suddenly spiked. But it s useful to have an alternative measurement source. When we search job postings for fulfillment center, we find that the top 5 states are California, Texas, Washington, Pennsylvania, and New Jersey. The main companies currently hiring are Amazon, Walmart, Chewy, Target, Blue Apron, zulily, and MSC Industrial Supply. Comparisons over time The Indeed API does not make historical information available directly. It s possible to run the same search at two time periods and compare them. However, it s important to keep in mind that at any point a provider of organic data can change its algorithm at any time. That means when comparing time periods, it is better to use analysis methods that rely on ratios or benchmarking to account for changes in the algorithm. Figure 11 shows state level job postings for machine learning and artificial intelligence, as a share of total, for December 2015 and May 2017. We see that the distribution is slightly less concentrated, but there is a remarkable degree of stability in the national share. Figure 12 does the same for genomics, without the adjustment for Eurofins (since that was not done in 2015)

The Absence of Innovative Jobs So far we have looked at the geographic distribution of innovations that have gained commercial traction. However, job postings can also give us some insight into highprofile technologies that are not generating many jobs in actuality. First, let s start with bioplastics. Bioplastics are plastics in which all carbon is derived from renewable biomass. It s gotten plenty of press attention, including a recent CNN story about making plastic out of cassava However, a search through Indeed shows precisely 1 job listing in the United States that mentions bioplastics. There are no job listings for bioplastics in the United Kingdom either, or in Germany. That suggests for all the media attention, the actual economic activity in the field is relatively small. (To be fair, there are 12 job listings in Japan that mention. バイオプラスチック, or bio plastics, but it is hard to know if that term has the same meaning). Another area of interest is superconductors. High temperature superconductors won the 1987 Nobel Prize in Physics, and were predicted to have a wide variety of uses. But searching for the term superconductor finds only 16 job postings in the United States, 16 in UK, 1 in Germany, and 17 in Japan ( 超伝導体 ). Monitoring job postings could give a contemporaneous signal for when an innovation is starting to get commercial traction. Or conversely, job postings could show which innovations are stalled.

Conclusion This paper presents work in progress. Partly it s about exploring how a new data source can be used to study the concentration of innovation

References Abowd, J. and Schmutte, I., (2015). Economic Analysis and Statistical Disclosure Limitation, Brookings Papers on Economic Activity, Berger-Gross, A., (2015). Where the Jobs Are: Supply and Demand for Occupations in North Carolina, Labor & Economic Analysis Division. North Caroline Department of Commerce, 29 April 2015 https://www.nccommerce.com/lead/research-publications/the-leadfeed/artmid/11056/articleid/82/where-the-jobs-are-supply-and-demandfor-occupations-in-north-carolina Berger-Gross, A., (2016) The Mismatch Mystery: Searching for the Skills Gap in North Carolina Labor & Economic Analysis Division. North Caroline Department of Commerce, 2 August 2016 https://www.nccommerce.com/lead/research-publications/the-leadfeed/artmid/11056/articleid/229/the-mismatch-mystery-searching-forthe-skills-gap-in-north-carolina Bureau of Labor Statistics, (2016). Job Openings and Labor Turnover Survey July 2016 http://www.bls.gov/news.release/archives/jolts_09072016.pdf Cajner, T., and Ratner, D.,(2016). A Cautionary Note on the Help Wanted Online Data FEDS Notes, Board of Governors of the Federal Reserve System https://www.federalreserve.gov/econresdata/notes/feds-notes/2016/acautionary-note-on-the-help-wanted-online-data-20160623.html Chamberlain, A., (2016). U.S. Wages Up 2.8% Year Over Year; Glassdoor Launches Local Pay Reports, Glassdoor, November 1, 2016.

https://www.glassdoor.com/research/glassdoor-local-pay-reports-oct- 2016/ The Conference Board, (2016). The Conference Board Help Wanted OnLine (HWOL) https://www.conference-board.org/data/helpwantedonline.cfm Couper, M., (2013). Is the Sky Falling? New Technology, Changing Media, and the Future of Surveys. Survey Research Methods, 7(3), 145-156. Einav, L., & Levin, J., (2014). Economics in the age of big data. Science, 346(6210). Groves, R., (2011) Three Eras of Survey Research, Public Opinion Quarterly, Vol. 75, No. 5, 2011, pp. 861 871. Houseman, S.N. and Mandel, M., eds., (2015). Measuring Globalization: Better Trade Statistics for Better Policy, W.E. Upjohn Institute for Employment Research. Kureková, L., Miroslav B., and Thum-Thysen, A., (2015). Using online vacancies and web surveys to analyse the labour market: a methodological inquiry. IZA Journal of Labor Economics 4.1 (2015): 1. Lazer D., Kennedy, R., King, G., and Vespignani, A., (2014). The parable of Google flu: Traps in big data analysis. Science 343, 1203 1205. Litan, R., Wyckoff, A., and Fealing, K.H., eds., (2014) Capturing Change in Science, Technology, and Innovation: Improving Indicators to Inform Policy, National Research Council, (National Academies Press, 2014), http://www.nap.edu/catalog.php?record_id=18606. Mandel, M., (2012). Where the Jobs Are: The App Economy, South Mountain Economics Research Report.

Mandel, M., (2014). Connections As A Tool For Growth: Evidence From The Linkedin Economic Graph, South Mountain Economics Research Report. https://southmountaineconomics.files.wordpress.com/2014/11/mandellinkedin-connections-nov2014.pdf Mandel, M., (2016a). Where is Innovation Falling Short?: Using Labor Market Indicators to Map the Successful Innovation Frontier, in The New Entrepreneurial Growth Agenda, Kauffman Foundation. http://www.kauffman.org/neg/section- 3#whereisinnovationfallingshortusinglabormarketindicatorstomapthesucce ssfulinnovationfrontier Mandel, M. (2016b). The App Economy in Europe: Leading Countries and Cities, Progressive Policy Institute Policy Memo, http://www.progressivepolicy.org/wp-content/uploads/2016/11/2016.06- Mandel_The-App-Economy-in-Europe_Leading-Countries-and-Cities-final.pdf Mandel. M. (2017a) How the Startup Economy is Spreading Across the Country and How It Can be Accelerated, http://www.progressivepolicy.org/wpcontent/uploads/2017/05/how-the-startup-economy-is-spreading-acrossthe-country-%e2%80%94-and-how-it-can-be-accelerate-final.pdf Mandel, M. (2017b). U.S. App Economy Update, Progressive Policy Institute Policy Memo, http://www.progressivepolicy.org/wpcontent/uploads/2017/05/ppi_usappeconomy.pdf Mandel, M. and Scherer, J., (2015) A Low-Cost And Flexible Approach For Tracking Jobs And Economic Activity Related To Innovative Technologies, South

Mountain Economics, Nesta Working Paper No. 15/11, June 2015, http://www.nesta.org.uk/blog/howtrack-innovative-jobs-real-time. Raghunathan, T., (2015). Statistical Challenges in Combining Information from Big and Small Data Sources, BEA Expert Meeting on Exploiting Commercial Data for Official Economic Statistics Thursday, November 19, 2015. Rigobon, R. and Cavallo, A., (2016) The Billion Prices Project: Using Online Prices for Inflation Measurement and Research, Journal of Economic Perspectives, Vol 30(2):151-78. Rothwell, J. (2014) Still Searching: Job Vacancies and STEM Skills, Brookings Institution. https://www.brookings.edu/wp-content/uploads/2014/07/job- Vacancies-and-STEM-Skills.pdf Yomiuri Shimbun, (2016). Recognize changing social structure when gathering economic data [editorial], Oct. 18, 2016 http://the-japannews.com/news/article/0003290068 White House, (1995). Send mail to the developers, https://clinton1.nara.gov/white_house/mail/html/mail_developers.html, downloaded November 3, 2016.

Table 1. Top Metro Areas by Total Job Postings, May 2017 Number of online job postings Percentage of national total 1 New York City 146178 3.6% 2 Los Angeles 112104 2.8% 3 Washington DC 101401 2.5% 4 Chicago 88085 2.2% 5 Boston 82017 2.0% 6 Dallas 80492 2.0% 7 San Francisco 69343 1.7% 8 Philadelphia 66420 1.6% 9 Atlanta 63535 1.6% 10 Denver 63081 1.6% 11 Houston 60923 1.5% 12 Phoenix 58225 1.4% 13 Minneapolis 58149 1.4% 14 Seattle 54522 1.3% 15 San Jose 52884 1.3% 16 Detroit 44582 1.1% 17 Baltimore 43204 1.1% 18 Miami 38400 0.9% 19 Tampa 37445 0.9% 20 Charlotte 35050 0.9% Top 5 sum 13.1% Top 20 sum 33.5% * Job postings within 25-mile radius. Moderate deduplication, no time limit on postings. If New York and Los Angeles are counted at 50 miles, the top 5 and top 20 sums go up to 19.5% and 39.9% respectively,. Data: Indeed API, May 2017.

Table 2: Top MSAs by Employment, 2015 share of employed workers national total 1 New York-Newark-Jersey City, NY-NJ-PA 9795868 6.51% 2 Los Angeles-Long Beach-Anaheim, CA 6390533 4.25% 3 Chicago-Naperville-Elgin, IL-IN-WI 4672798 3.10% 4 Dallas-Fort Worth-Arlington, TX 3513490 2.33% 5 Washington-Arlington-Alexandria, DC-VA-MD- 3250230 2.16% 6 Houston-The Woodlands-Sugar Land, TX 3190721 2.12% 7 Philadelphia-Camden-Wilmington, PA-NJ-DE 2945673 1.96% 8 Miami-Fort Lauderdale-West Palm Beach, FL 2858008 1.90% 9 Atlanta-Sandy Springs-Roswell, GA 2764512 1.84% 10 Boston-Cambridge-Newton, MA-NH 2553286 1.70% 11 San Francisco-Oakland-Hayward, CA 2409944 1.60% 12 Phoenix-Mesa-Scottsdale, AZ 2070696 1.38% 13 Detroit-Warren-Dearborn, MI 1959298 1.30% 14 Minneapolis-St. Paul-Bloomington, MN-WI 1911709 1.27% 15 Seattle-Tacoma-Bellevue, WA 1903626 1.26% 16 Riverside-San Bernardino-Ontario, CA 1858130 1.23% 17 San Diego-Carlsbad, CA 1552867 1.03% 18 Denver-Aurora-Lakewood, CO 1498349 1.00% 19 St. Louis, MO-IL 1387444 0.92% 20 Baltimore-Columbia-Towson, MD 1383041 0.92% Top 5 Sum 18.3% Top 20 Sum 39.8% Data: ACS, 2015 1-year

Table 3: Top MSAs by Managerial and Professional Employment, 2015 employed managers and professionals share of national total 1 New York-Newark-Jersey City, NY-NJ-PA 4061074 7.27% 2 Los Angeles-Long Beach-Anaheim, CA 2400322 4.30% 3 Chicago-Naperville-Elgin, IL-IN-WI 1802809 3.23% 4 Washington-Arlington-Alexandria, DC-VA-MD- 1661136 2.97% 5 Dallas-Fort Worth-Arlington, TX 1340166 2.40% 6 Philadelphia-Camden-Wilmington, PA-NJ-DE- 1261350 2.26% 7 Boston-Cambridge-Newton, MA-NH 1219990 2.18% 8 Houston-The Woodlands-Sugar Land, TX 1184464 2.12% 9 San Francisco-Oakland-Hayward, CA 1159597 2.08% 10 Atlanta-Sandy Springs-Roswell, GA 1105288 1.98% 11 Miami-Fort Lauderdale-West Palm Beach, FL 972402 1.74% 12 Seattle-Tacoma-Bellevue, WA 852767 1.53% 13 Minneapolis-St. Paul-Bloomington, MN-WI 828105 1.48% 14 Detroit-Warren-Dearborn, MI 737142 1.32% 15 Phoenix-Mesa-Scottsdale, AZ 733910 1.31% 16 Denver-Aurora-Lakewood, CO 642884 1.15% 17 Baltimore-Columbia-Towson, MD 626410 1.12% 18 San Diego-Carlsbad, CA 625742 1.12% 19 St. Louis, MO-IL 536290 0.96% 20 Riverside-San Bernardino-Ontario, CA 525989 0.94% Top 5 sum 20.2% Top 20 sum 43.5% Data: ACS, 2015 1-year

Table 4: Top metro areas for machine learning or artificial intelligence Search Criteria: Machine learning or artificial intelligence Number of Percentage online job of national postings total 1 Seattle 2436 17.1% 2 San Jose 2284 16.1% 3 San Francisco 1633 11.5% 4 New York City 1325 9.3% 5 Boston 919 6.5% 6 Washington DC 830 5.8% 7 Chicago 483 3.4% 8 Los Angeles 435 3.1% 9 Austin 258 1.8% 10 Atlanta 248 1.7% 11 Philadelphia 236 1.7% 12 San Diego 190 1.3% 13 Dallas 187 1.3% 14 Baltimore 177 1.2% 15 Denver 155 1.1% 16 Portland OR 115 0.8% 17 Minneapolis 110 0.8% 18 Raleigh NC 104 0.7% 19 Charlotte 103 0.7% 20 Detroit 95 0.7% Top 5 sum 60.4% Top 20 sum 86.6% Job postings within 25-mile radius. Moderate deduplication, no time limit on postings. Data: Indeed API.

Table 5: Top Metro Areas for 3D Printing or Additive Manufacturing Search criteria: 3D printing or additive manufacturing Number of online job postings Percentage of national total 1 Boston 114 9.3% 2 San Jose 99 8.0% 3 New York City 92 7.5% 4 San Francisco 47 3.8% 5 Los Angeles 43 3.5% 6 Seattle 40 3.3% 7 Portland OR 38 3.1% 8 Detroit 32 2.6% 9 Washington DC 29 2.4% 10 San Diego 29 2.4% 11 Chicago 27 2.2% 12 Charlotte 26 2.1% 13 Indianapolis 24 2.0% 14 Dayton 23 1.9% 15 Philadelphia 22 1.8% 16 Atlanta 22 1.8% 17 Minneapolis 22 1.8% 18 Cincinnati OH 20 1.6% 19 Denver 19 1.5% 20 Raleigh NC 19 1.5% Top 5 sum 32.1% Top 20 sum 64.0% Job postings within 25-mile radius. Moderate deduplication, no time limit on postings. Data: Indeed API

Table 6a: Top Metro Areas for Genomics Search criteria: genomics Number of online job postings Percentage of national total 1 Boston 452 11.4% 2 San Francisco 358 9.1% 3 San Jose 311 7.9% 4 New York City 234 5.9% 5 San Diego 217 5.5% 6 Philadelphia 188 4.8% 7 Durham 105 2.7% 8 Washington DC 102 2.6% 9 Los Angeles 96 2.4% 10 Raleigh NC 87 2.2% 11 Hartford 85 2.1% 12 Seattle 83 2.1% 13 Chicago 79 2.0% 14 Houston 72 1.8% 15 St. Louis MO 61 1.5% 16 Denver 46 1.2% 17 Tucson 46 1.2% 18 Baltimore 43 1.1% 19 Des Moines 43 1.1% 20 Bridgeport 40 1.0% Top 5 sum 39.8% Top 20 sum 69.5% Job postings within 25-mile radius. Moderate deduplication, no time limit on postings. Data: Indeed API.

Table 6b: Top Metro Areas for Genomics Search criteria: genomics and eurfins Number of online job postings Percentage of national total 1 Boston 445 13.6% 2 San Francisco 345 10.5% 3 San Jose 283 8.7% 4 San Diego 216 6.6% 5 New York City 215 6.6% 6 Durham 105 3.2% 7 Washington DC 98 3.0% 8 Raleigh NC 88 2.7% 9 Seattle 85 2.6% 10 Hartford 83 2.5% 11 Philadelphia 75 2.3% 12 Los Angeles 74 2.3% 13 Chicago 72 2.2% 14 Houston 61 1.9% 15 Tucson 48 1.5% 16 Baltimore 41 1.3% 17 Denver 40 1.2% 18 Bridgeport 36 1.1% 19 St. Louis MO 30 0.9% 20 Phoenix 21 0.6% Top 5 sum 46.0% Top 20 sum 75.2% Job postings within 25-mile radius. Moderate deduplication, no time limit on postings. Data: Indeed API

Table 7. Top States for Crispr Search criteria: "crispr" Number of online job postings Percentage of national total 1 Massachusetts 93 37.8% 2 California 53 21.5% 3 New York 32 13.0% 4 Maryland 9 3.7% 5 Washington 8 3.3% 6 Pennsylvania 7 2.8% 7 Texas 6 2.4% 8 Illinois 5 2.0% 9 Arizona 4 1.6% 10 Missouri 4 1.6% 11 New Jersey 4 1.6% 12 Virginia 4 1.6% 13 Connecticut 3 1.2% 14 Colorado 2 0.8% 15 Iowa 2 0.8% 16 North Carolina 2 0.8% 17 Ohio 2 0.8% 18 Delaware 1 0.4% 19 Georgia 1 0.4% 20 Indiana 1 0.4% Top 5 Sum 79.3% Top 10 Sum 89.8% Data: Indeed API.

Table 8. Top States for Carbon Fiber Search criteria: "carbon fiber" Number of online job postings Percentage of national total 1 Michigan 72 30.1% 2 California 28 11.7% 3 Florida 18 7.5% 4 Utah 15 6.3% 5 Washington 14 5.9% 6 Arizona 11 4.6% 7 Alabama 9 3.8% 8 Minnesota 9 3.8% 9 Oregon 7 2.9% 10 Missouri 6 2.5% 11 Nebraska 5 2.1% 12 N. Carolina 5 2.1% 13 Ohio 5 2.1% 14 Tennessee 4 1.7% 15 Texas 4 1.7% 16 Colorado 3 1.3% 17 Massachusett 3 1.3% 18 North Dakota 3 1.3% 19 Pennsylvania 3 1.3% 20 Wisconsin 3 1.3% Top 5 Sum 61.5% Top 10 Sum 79.1% Data: Indeed API.

Table 9. Top States for composite and material Search criteria: composite and material and-princeton Number of online job postings Percentage of national total 1 California 505 16.1% 2 Michigan 239 7.6% 3 Texas 190 6.0% 4 Ohio 177 5.6% 5 Florida 155 4.9% 6 Washington 118 3.8% 7 Utah 114 3.6% 8 Massachusetts 95 3.0% 9 Illinois 92 2.9% 10 Indiana 82 2.6% 11 Georgia 79 2.5% 12 Oregon 78 2.5% 13 North Carolina 74 2.4% 14 New York 72 2.3% 15 Alabama 66 2.1% 16 Pennsylvania 66 2.1% 17 Virginia 62 2.0% 18 Kansas 60 1.9% 19 Arizona 59 1.9% 20 Maryland 58 1.8% Top 5 Sum 40.3% Top 10 Sum 56.2% Data: Indeed API.

Table 10. Top States for Fulfillment Center Search criteria: "fulfillment center" 1 California 225 11.9% 2 Texas 220 11.6% 3 Washington 186 9.8% 4 Pennsylvania 172 9.1% 5 New Jersey 126 6.6% 6 Indiana 113 6.0% 7 Nevada 99 5.2% 8 Illinois 69 3.6% 9 Kentucky 69 3.6% 10 Ohio 64 3.4% 11 Tennessee 62 3.3% 12 Florida 48 2.5% 13 New York 45 2.4% 14 Massachusetts 44 2.3% 15 Georgia 42 2.2% 16 Kansas 34 1.8% 17 North Carolina 27 1.4% 18 South Carolina 26 1.4% 19 Virginia 25 1.3% 20 Minnesota 22 1.2% Top 5 Sum 49.0% Top 10 Sum 70.9% Data: Indeed API.

Table 11. Comparison, 2015 vs 2017, "3D printing" or "additive manufacturing" May-17 May-17 Dec-15 1 California 237 20.1% 23.2% 2 New York 112 9.5% 10.9% 3 Massachusetts 82 7.0% 9.9% 4 Washington 65 5.5% 2.7% 5 Ohio 61 5.2% 3.5% 6 Pennsylvania 54 4.6% 3.6% 7 Illinois 51 4.3% 4.8% 8 Florida 45 3.8% 2.4% 9 Michigan 44 3.7% 3.5% 10 Texas 42 3.6% 3.0% 11 North Carolina 37 3.1% 4.2% 12 New Jersey 36 3.1% 1.5% 13 Georgia 31 2.6% 1.8% 14 Colorado 28 2.4% 2.3% 15 Indiana 25 2.1% 0.7% 16 Minnesota 25 2.1% 1.9% 17 Arizona 23 2.0% 2.3% 18 Maryland 23 2.0% 1.8% 19 Connecticut 19 1.6% 1.8% 20 Oregon 19 1.6% 1.1% Top 5 sum 47.3% 53.0% Top 10 sum 67.4% 69.3% Data: Indeed API.

Table 12. Comparison, 2015 vs 2017, genomics May-17 May-17 Dec-15 1 California 1022 26.1% 28.9% 2 Massachusetts 478 12.2% 10.6% 3 Pennsylvania 395 10.1% 7.0% 4 New York 235 6.0% 6.6% 5 Maryland 161 4.1% 6.9% 6 North Carolina 152 3.9% 3.9% 7 New Jersey 143 3.7% 3.4% 8 Texas 126 3.2% 2.4% 9 Connecticut 114 2.9% 1.4% 10 Illinois 98 2.5% 2.7% 11 Washington 97 2.5% 2.2% 12 Missouri 76 1.9% 2.4% 13 Michigan 71 1.8% 1.2% 14 Arizona 69 1.8% 2.1% 15 Virginia 65 1.7% 1.5% 16 Colorado 50 1.3% 0.9% 17 Iowa 50 1.3% 1.3% 18 Ohio 49 1.3% 1.5% 19 Florida 41 1.0% 2.2% 20 Indiana 41 1.0% 1.2% Top 5 sum 58.5% 60.0% Top 10 sum 74.6% 74.8% Data: Indeed API