Are economic impact and effectiveness the same thing?

This article is part of an occasional series examining state and local reports evaluating economic development incentive programs.  Here we look at the “Review of State Economic Development Incentive Grants” report to the Governor and General Assembly of Virginia, prepared by the Joint Legislative Audit and Review Commission (JLARC).

This is a very good report, and I encourage you to read the Executive Summary because there are several interesting findings.  Here I want to highlight one particular topic that this report emphasizes: how to assess the economic impact of incentive grants.

JLARC considers three economic impact indicators by funded project:

  • high employment multiplier (indicating new jobs should be generated beyond those created by the project) 
  • export-based (indicating it brings new money into the State’s economy)
  • high wages (relative to the industry average)

The report assesses 3,372 grants in 18 different programs over ten years.  However, the economic impact portion only considers about 1,400 projects that are considered “completed” and had adequate data to enable analysis.  Even among this set, wage and investment data needed to be imputed for half of the projects.  

Findings include:

  • Only 3% of projects met all three indicators, but approximately one-third met at least two indicators.
  • Economic impact as measured in this manner varied considerably by grant program, with workforce and enterprise zone projects least likely to meet these objectives.
  • Overall employment impacts are estimated to be 67,000 workers (cumulatively) after 5 years.  This includes indirect, induced and construction employment; the number of direct jobs is not clear.

The report makes an interesting assumption that incentives are only responsible for 10% of business investment decisions, so the calculations are also presented assuming only 10% of total impacts on employment, income, GDP and state revenue.

This is a useful report on several fronts.  It makes a solid effort toward addressing one of the most challenging aspects of incentive evaluation: assessing effectiveness.  It is based at least in part on reported data for completed projects (not just projections). And it breaks out the analysis by program to compare impact.

I do think there are some weaknesses, however.  One is that the number of direct jobs created is not clear in the analysis, so we don’t know how much of the impact is connected specifically to the company assisted.  The need to impute wages and investment for half the projects is another.  Third, the costs from other non-grant incentives or local incentives are not incorporated so the fiscal gains may be over-stated. (I also do not think taxes associated with indirect and induced jobs belong in a fiscal impact, but I see the reasoning.)  

Mostly, though, I am not sure that the implication that high multiplier projects and the largest projects are the most worthy of grants because they have the highest calculated economic impact is right.  For one, we still don’t know how much of the calculated impact is from the model and how much actually happened. I have also become skeptical about overly large multipliers in some industries that certainly overstate some of the induced impacts.  It is also inconsistent with the latest thinking on what drives regional economic growth, which is a whole set of assets and a diverse business base rather than a few large projects.

What do you think?