Abstract

A sustained methodological critique spanning nearly four decades has established that the aggregate economic impact study is structurally ill-suited to answer the question municipal decision-makers most need answered: does this programming, in this community, retain spending that would otherwise leave, or does it redirect spending that would have remained locally regardless? Seaman (1987), Crompton (1995, 2006), and Sterngold (2004) identify the omission of substitution effects as the central structural error. McCarthy et al. (2004) identify the conflation of correlation with causation in claims about arts benefits. National studies have not resolved these critiques across successive iterations.

This paper introduces the Cultural Investment Strategy (CIS) and its Cultural Investment Index (CII) as a proposed institutional response to this measurement gap. The CIS is a municipal activation-level instrumentation system with a standardizable data collection framework that generates behavioral records at the activation level, within defined geographic and operational boundaries, using consistent measurement procedures across all programs.

Three scope conditions govern all claims in this paper. First, the CII is a management instrument in active development; the psychometric validation roadmap is described in Section VIII. Second, Factor 3 captures self-reported counterfactual behavioral intent collected at point of activation through survey response. Third, the adjacent business lift comparison is a matched-night analysis with a limited baseline. These scope conditions define the instrument's current claims boundary. Findings from Year One activation records will inform instrumentation refinement and calibration. Causal findings require a dataset of sufficient depth; the instrument is in its first operational year.

Keywords: municipal arts instrumentation, composite indicator design, cultural policy measurement, behavioral data architecture, demand validation, activation-level evidence, methods framework

Introduction: The Municipal Measurement Gap

Municipal governments invest in arts and cultural programming through direct appropriation, facility provision, and economic development strategy. These investments are typically justified through one of two mechanisms: aggregate economic impact studies, which estimate total economic activity attributable to the nonprofit arts sector at regional or national scale; or output reporting, which documents the number of events, attendees, or programming hours delivered.

Neither mechanism answers the question most relevant to municipal policy: does this specific programming, in this specific community, generate economic activity that would not otherwise occur? Aggregate studies operate at scales too broad to isolate municipal effects. Output reporting documents activity without measuring consequence. The methodological critique of aggregate economic impact methodology, advanced over four decades by Seaman (1987, 2011), Crompton (1995, 2006), Sterngold (2004), and others, has established that the dominant approach systematically overclaims through the omission of substitution effects, the misapplication of multiplier coefficients, and the conflation of correlation with causation.

This paper is an institutional response to that measurement gap, complementary to the national evidence base that serves important purposes at the scale for which it was designed. The missing layer is a standardizable, replicable instrumentation architecture that generates activation-level behavioral records at the municipal scale, within defined geographic and operational boundaries.

The Cultural Investment Strategy and Cultural Investment Index described here are proposed as a contribution to that lower layer. The paper describes the instrument's design principles, measurement components, validity boundaries, governance translation protocols, and development roadmap. Year One records will inform instrumentation refinement and calibration. Causal findings require a dataset of sufficient depth; the instrument is in its first operational year.

The Existing Evidence Base: National Breadth, Local Absence

The dominant tradition of arts impact measurement in the United States is the economic impact study. Beginning with the Johns Hopkins University model developed by Cwi and Lyall (1977), arts organizations and their advocacy partners have used input-output analysis and audience expenditure surveys to estimate the broader economic contributions of cultural programming. Americans for the Arts has institutionalized this approach through its Arts and Economic Prosperity series, which in its sixth iteration surveyed 373 communities across all 50 states and Puerto Rico, documenting $151.7 billion in economic activity by nonprofit arts and culture organizations and their audiences in 2022 (Americans for the Arts, 2023). This figure measures reported economic activity associated with the sector, as documented through audience expenditure surveys.

Parallel to the economic impact tradition, RAND Corporation research has developed a framework centered on the intrinsic and instrumental benefits of arts engagement. McCarthy et al. (2004) proposed a two-axis framework distinguishing private from public benefits and intrinsic from instrumental benefits, designed to reorient arts policy advocacy toward a more comprehensive account of how the arts create value. The Urban Institute has contributed a community indicators approach incorporating arts and culture measures into broader quality-of-life frameworks (Urban Institute, 2025). The Wallace Foundation has funded audience development research on the conditions under which sustained participation develops (Wallace Foundation, 2010).

Despite the breadth of national research, a pronounced gap exists at the municipal scale. Bloomberg Associates documented this gap in a 2021 analysis of arts data practices in fifteen cities: there is a recognized lack of standardized, locally specific data, and both the creative sector and municipal governments frequently express skepticism about whether such data can be reliably generated (Bloomberg Associates, 2021). Grantmakers in the Arts found that local arts agencies operate without a standard data collection process nationally (Grantmakers in the Arts, 2020).

Walters and Chandler (2019), in a framework paper published in Local Government Studies, identify the methodological dimension of this gap: prevailing municipal measurement frameworks take an outputs-based approach, evaluating the performance of one-off events through simple metrics, without measuring behavioral outcomes over the life of a strategy. This distinction between output measurement and outcome measurement is foundational to the instrument described in this paper.

Duncanville, Texas illustrates the local data gap. The municipality's approximately 40,000 residents generate a synthetic annual household entertainment expenditure estimate of approximately $31 million, derived from Bureau of Labor Statistics Consumer Expenditure Survey averages applied to American Community Survey household counts (BLS, 2024; U.S. Census Bureau, 2023). The figure is methodologically bounded: BLS entertainment category definitions are broader than arts and cultural spending, and the calculation applies national averages to local household counts rather than measuring observed local flows. Municipal entertainment capacity and cultural spending capacity are distinct concepts. The figure serves to contextualize the scale of the instrumentation opportunity, with those boundaries explicitly acknowledged.

The Methodological Critique: Substitution, Correlation, and the Limits of Aggregate Analysis

The critique of arts and events economic impact studies has a substantial lineage. Seaman (1987) characterized such studies as a "fashionable excess," identifying three fundamental demand-based analytical errors: failure to subtract local sources of spending and non-local uses of spending, erroneous attribution of all ancillary spending as causally related to the subject organization, and failure to adapt multipliers to specific regions. Seaman (2011) identifies these errors as still prevalent in contemporary economic impact methodology.

Crompton (1995) formalized eleven sources of misapplication in economic impact analyses for events and facilities. The most consequential for arts measurement are: the inclusion of local spectators whose spending represents reallocation rather than injection; the failure to exclude time-switchers and casuals whose spending would have occurred regardless; the omission of opportunity costs; and the confusion of total with marginal benefits. Crompton (2006) characterized the incentive structure producing these errors: most economic impact studies are commissioned to legitimize a predetermined position rather than to search for economic truth.

Sterngold (2004) applied this critique specifically to arts economic impact methodology in the Journal of Arts Management, Law, and Society. His central argument is that conventional analyses treat all arts-related spending as additive without accounting for substitution: spending that would have occurred regardless of the arts programming, directed toward other leisure or recreational activities.

Economic impact analyses that use only gross measures of impact fail to provide any evidence to support their claims because the studies overlook the substitution effects of spending by nonprofit arts and cultural organizations.

Sterngold, 2004, p. 169

The Americans for the Arts AEP series has not substantively modified its methodology in response to these critiques across six iterations.

McCarthy et al. (2004) in Gifts of the Muse review the literature on instrumental benefits and find that many studies demonstrate correlation rather than causality. Observed associations between arts participation and educational, economic, or social outcomes frequently fail to rule out alternative explanations: higher household income, greater parental involvement, better-funded school districts. Relying exclusively on instrumental benefit arguments carries a further risk. If a programmatic investment is justified on the promise of improving urban economic outcomes, but non-arts interventions prove more cost-effective at achieving those same objectives, the arts face defunding pressure on the terms of their own advocacy.

Together, these critiques define what a credible municipal-scale instrument must accomplish. It must collect behavioral data on what attendees would have done in the absence of programming, directly addressing the substitution question. It must document geographic origin of participants to establish whether any spending retention claim is plausible. It must generate activation-level records that accumulate into a longitudinal dataset suitable for trajectory analysis. And it must operate through a standardized protocol that makes records from different activations methodologically comparable. These four requirements are the design criteria for the CIS.

Instrument Design Criteria

Evidence must be grounded in what people actually did or reported intending. The instrument collects two categories of behavioral data: pre-commitment records reflecting financial acts before activation (ticket purchases, sponsor deposits), and post-activation survey responses capturing self-reported counterfactual behavioral intent. Pre-commitment is observed behavior. Survey responses are self-report.

Spending retention is a municipal concept. Spending retention evidence requires a resident-majority audience. Geographic origin data establishes whether any retention claim is plausible. For Duncanville, the target geography is defined by ZIP codes 75116, 75137, and 75138.

Aggregate studies operate at scales that obscure municipal-level variation across programs, geographies, and conditions. The CIS generates records at the level of individual program activations, with consistent data fields across all activations, enabling trajectory analysis and cross-program comparison. This is the outcomes-based approach Walters and Chandler (2019) identify as the necessary alternative to output measurement.

For activation-level records to be comparable, data collection must be consistent. If measurement quality varies by individual data collector, or if consistent procedures across activations are the prerequisite for placing records in a shared dataset. Standardization of timing, method, and procedure across all activations is a prerequisite for any subsequent comparative analysis. This is an instrumentation problem in the quasi-experimental sense used by Campbell and Stanley (1963).

The Cultural Investment Strategy as a Municipal Measurement Instrument

The Cultural Investment Strategy is a data collection architecture. Its organizing logic is demand validation: programs do not proceed to public activation without demonstrated prior commitment from the intended audience. This logic serves two functions simultaneously. As operational risk management, it prevents investment in programming without evidence of demand. As an instrumentation choice, it generates pre-activation behavioral data that few arts measurement frameworks collect.

The three-stage model — Validate, Activate, Determine — is a decision architecture. Each stage generates structured data. The Validate stage records demand characteristics before any programming expenditure is committed. The Activate stage records attendance, participation patterns, and adjacent economic activity during programming. The Determine stage produces a composite score and a governance determination based on accumulated data. These stages are data phases generating structured records.

No program proceeds to Activate without achieving 100% pre-commitment of projected costs through ticket sales, sponsor commitments, or vendor deposits. This threshold functions as a revealed preference gate: participants who purchase tickets before an event exists are demonstrating demand through a financially consequential act grounded in revealed preference prior to activation.

Year One programming operates exclusively at Arts Junction at Old Rail Station in Duncanville. Holding the venue constant across activations reduces environmental confounds: variation in CII scores across activations is more attributable to program-specific factors than to differences in venue accessibility, parking, or neighborhood conditions. The adjacent business lift comparison, which requires a stable baseline business environment, is only feasible with consistent venue operation.

Composite Indicator Design: The Cultural Investment Index

The Cultural Investment Index is a weighted additive composite scoring instrument that produces a single numeric score for each activation on a scale of 0 to 100. It is constructed from five factors, each scored on a four-tier rubric with defined criteria. The OECD/JRC Handbook on Constructing Composite Indicators (Nardo et al., 2008) provides the methodological standard against which the CII's design choices and required development work are assessed throughout this paper.

The five weights are governance parameters established through the Foundation's judgment about the relative evidential strength of each factor. Factor 1 carries the greatest weight because pre-commitment is an observed behavioral act, the strongest evidence type in this instrument. Factor 3 carries a reduced weight relative to Factor 1 because survey-reported counterfactual intent is weaker evidence than observed financial commitment. Factor 5 carries the lowest weight because the matched-night comparison is a contextual signal with restricted inferential scope.

The weights are subject to a pre-specified sensitivity protocol described in Section VIII. Pre-specification of this protocol, before Year One data is analyzed, protects against post-hoc weight adjustment in response to data. If sensitivity analysis reveals that program rank-orders are unstable under plausible alternative weight sets, determinations from that period will be reported as provisional pending weight revision.

Additive weighted aggregation allows strong performance on one factor to offset weak performance on another. This is a deliberate design choice for an operational management instrument: a program with unusually high pre-commitment but modest repeat attendance is different from a program with low pre-commitment and high repeat attendance, but both may achieve the same composite score. The CII produces a single decision-relevant number for governance purposes. For research purposes, factor-level scores must be reported alongside the composite to preserve analytical granularity. Compensatory aggregation is a known limitation of additive composite indices; Nardo et al. (2008) discuss this explicitly. The choice is accepted here on operational grounds and flagged as a philosophical assumption requiring defense.

Measurement Components

Pre-commitment measures revealed advance demand within bounded venue conditions. A resident who purchases a ticket before an event exists is performing a financially consequential act grounded in revealed preference prior to activation. F1 carries the highest weight in the CII because it is the strongest evidence type available. Two scope qualifications govern its interpretation. First, it measures demand for the bounded class of demand-validated activations specifically. Second, advance purchase rates vary by genre, price point, and marketing reach, factors the CII tracks without current normalization across activations. Cross-program comparison of F1 scores requires attention to these structural differences.

F2 addresses geographic specificity through ZIP code data collected via survey Q1 and ticketing records. Spending retention evidence requires a resident-majority audience; F2 identifies whether that condition is met. The target ZIP codes (75116, 75137, 75138) are an administrative proxy for municipal residency. ZIP code boundaries approximate municipal boundaries and diverge at the margins. Any research application of F2 data should document this approximation.

F3 asks attendees what they would have done with their time and money if this activation had not been available. Survey questions Q2 and Q3 present response options including staying home in Duncanville, spending locally at other businesses, traveling to a regional venue outside Duncanville, and not spending entertainment funds at all. Responses indicating out-of-city alternatives constitute self-reported substitution intent.

F4 serves as a habit formation proxy. Single-activation attendance may reflect novelty-seeking; repeat attendance indicates the formation of sustained local arts consumption patterns, which McCarthy et al. (2004) identify as a prerequisite for both instrumental and intrinsic benefit realization. A default score of 50 is applied to first activations where no prior participation data exists. This default is a governance parameter that compresses variance in early-activation comparisons. Any longitudinal analysis of CII trajectories that includes first activations must flag the default as an artifact. The six-activation maximum creates administrative censoring of program trajectories; programs that sunset before reaching the maximum generate truncated records that must be treated analytically as left-censored observations.

F5 uses point-of-sale data from Arts Junction tenants compared against a matched-night baseline from non-activation nights at the same venue. Matching criteria: same day of week, same seasonality window (within four weeks), excluding holidays and documented competing events. F5 is a contextual operational indicator providing corroborating signal. Its inferential scope is corroboration only.

The matched-night comparison controls for day of week and seasonality. Concurrent events, weather extremes, promotional activity by tenants, and pay cycle effects are tracked in a confound log rather than controlled statistically. Its evidential role is corroboration: when F5 and F3 point in consistent directions, the combined signal warrants greater confidence than either alone. When they diverge, the discrepancy should be investigated before any retention claim is made. The weight of 0.10 reflects this restricted epistemic role.

Threats to Validity, Reliability Protocol, and Required Development Work

A measurement instrument is evaluated by the threats its design creates and the mitigations it provides. This section follows the framework of Campbell and Stanley (1963) and Shadish, Cook, and Campbell (2002) in analyzing threats systematically, and references Nardo et al. (2008) on composite indicator validation standards.

The CII requires program managers to apply rubric criteria to observed data and assign factor scores. Before CII records from different activations are placed in a shared dataset for comparative analysis, a reliability study is required. Protocol: two independent scorers assess the same activation records and calculate inter-rater agreement using Cohen's kappa. Target threshold: kappa above 0.70. A kappa below 0.60 triggers rubric revision. Training materials and anchor examples will be developed from the first cohort of activations.

The pre-specified sensitivity protocol to be executed before Year One data is published includes three alternative weight configurations representing plausible governance priorities, rank-order stability assessed using Kendall's tau-b, and a reporting rule that any activation receiving a different determination under an alternative weight set is flagged as a borderline case requiring additional evidence before the determination is applied.

As multiple activations accumulate, factor scores will be examined for convergent and discriminant validity using standard correlation analysis. Factors proposed to measure the same underlying construct should show positive correlation; factors measuring distinct constructs should show lower correlation. If factor scores correlate near perfectly, the composite adds no information over any single factor. If they are uncorrelated, the aggregation rationale requires reexamination.

The graduation threshold of 70 will be calibrated against post-graduation commercial performance data once three or more programs have graduated and operated commercially for at least two activation cycles. The threshold is currently a governance parameter; calibration will determine whether it predicts commercial sustainability at an acceptable rate.

Governance Translation: Thresholds and Determinations

The following thresholds and determinations are governance interpretations of structured indicator data. They are governance interpretations translating composite scores into operational decisions, using boundaries set by the Foundation as an exercise of institutional judgment.

Two Potential Research Outcomes

The CIS is designed to generate structured evidence regardless of the direction that evidence takes. This section describes the two potential research outcomes and the policy inferences each supports. Both outcomes are framed as findings the instrument can generate.

Either outcome constitutes a research contribution. Path A generates evidence of the conditions under which arts programming becomes self-sustaining in a municipality with no prior infrastructure. Path B, disaggregated through competing explanations analysis, generates evidence of the conditions under which demonstrated community demand encounters commercial pricing barriers. Neither finding exists for a municipality of Duncanville's profile in the current literature because the activation-level behavioral data to support it has not been collected.

Conclusion

The measurement problem in municipal arts investment is a framework mismatch: existing tools were designed for national-scale advocacy and generate claims at a scope the underlying data supports only at that scale. The CIS is a complementary instrumentation layer, operating at the activation level, within a single municipality, under defined validity boundaries.

The Cultural Investment Index is a management instrument pursuing the status of a research instrument. The gap between those two descriptions is closed by the development work described in Section VIII: inter-rater reliability testing, weight sensitivity analysis under pre-specified alternative configurations, factor-level validity assessment as activation data accumulates, and external calibration of graduation thresholds against post-graduation commercial performance. None of this work requires the instrument to be redesigned. It requires the instrument to be operated with discipline and the data it produces to be reported at the level of confidence it actually supports.

This paper's contribution is the instrument design, the validity framework, and the explicit separation of governance decisions from research claims. The empirical contribution comes later, when the dataset exists.

The Foundation welcomes methodological review, inter-rater reliability partnership proposals, and positioning within national arts research infrastructure. Correspondence regarding the instrument design or the research roadmap may be directed to the Foundation at 202 W. Center Street, Suite 101, Duncanville, Texas 75116 or [email protected].

Appendices

CIS Evidence Scope and Boundaries

This appendix defines the evidence boundaries of the Cultural Investment Strategy and Cultural Investment Index for Year One data. Each boundary reflects the instrument's current design scope and is reassessed as the validation roadmap in Section VIII progresses.

Crompton's Eleven Sources of Misapplication: CIS Response Mapping

The table below maps Crompton's (1995, 2006) eleven sources of misapplication against the CIS design. Where the CIS addresses a source, the response is described. Where it does not, this is stated.

OECD/JRC Composite Indicator Construction: Ten-Step Compliance Assessment

The OECD/JRC Handbook (Nardo et al., 2008) prescribes a ten-step methodology for composite indicator construction. The table below maps CII development status against each step.

References

• Americans for the Arts. (2017). Arts and economic prosperity 5: The economic impact of nonprofit arts and culture organizations and their audiences. Washington, DC: Americans for the Arts.
• Americans for the Arts. (2023). Arts and economic prosperity 6: The economic impact of nonprofit arts and culture organizations and their audiences. Washington, DC: Americans for the Arts.
• Bloomberg Associates. (2021). Arts data in the public sector: Strategies for local arts agencies. New York: Bloomberg Associates. https://associates.bloomberg.org/arts-data-in-the-public-sector/
• Bureau of Labor Statistics. (2024). Consumer expenditure survey: Entertainment category components. Washington, DC: U.S. Department of Labor.
• Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
• Crompton, J. L. (1995). Economic impact analysis of sports facilities and events: Eleven sources of misapplication. Journal of Sport Management, 9(1), 14–35.
• Crompton, J. L. (2006). Economic impact studies: Instruments for political shenanigans? Journal of Travel Research, 45(1), 67–82.
• Cwi, D., & Lyall, K. (1977). Economic impacts of arts and cultural institutions: A model for assessment and a case study in Baltimore. Baltimore, MD: Center for Metropolitan Planning and Research, Johns Hopkins University.
• Duncanville Arts Foundation. (2026). Cultural investment strategy, version 2.0. Duncanville, TX: Duncanville Arts Foundation.
• Grantmakers in the Arts. (2020). Public funding for arts and culture in 2020. Retrieved from https://www.giarts.org/public-funding-arts-and-culture-2020
• McCarthy, K. F., Ondaatje, E. H., Zakaras, L., & Brooks, A. (2004). Gifts of the muse: Reframing the debate about the benefits of the arts. Santa Monica, CA: RAND Corporation.
• Moss, I. D. (2009, September). Arts policy library: Arts and economic prosperity III. Createquity. Retrieved from https://createquity.com/2009/09/arts-policy-library-arts-economic-prosperity-iii/
• Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffmann, A., & Giovannini, E. (2008). Handbook on constructing composite indicators: Methodology and user guide. OECD Publishing, Paris. https://doi.org/10.1787/9789264043466-en
• Seaman, B. A. (1987). Arts impact studies: A fashionable excess. In A. J. Radich (Ed.), Economic impact of the arts: A sourcebook (pp. 43–75). Denver, CO: National Conference of State Legislatures.
• Seaman, B. A. (2011). Economic impact of the arts. In R. Towse (Ed.), A handbook of cultural economics (2nd ed., Chapter 28). Cheltenham: Edward Elgar Publishing.
• Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
• Sterngold, A. H. (2004). Do economic impact studies misrepresent the benefits of arts and cultural organizations? Journal of Arts Management, Law, and Society, 34(3), 166–187.
• Urban Institute. (2025, October). Understanding the impact of arts and culture on communities through local data. Washington, DC: National Neighborhood Indicators Partnership.
• U.S. Census Bureau. (2023). American community survey 5-year estimates, 2019–2023. Washington, DC: U.S. Department of Commerce.
• Wallace Foundation. (2010). The road to results: Effective practices for building arts audiences. New York: Wallace Foundation.
• Walters, T., & Chandler, L. (2019). Towards a framework for measuring local government return on investment in arts and cultural development. Local Government Studies, 45(2), 165–185.