Program Evaluator

Program Evaluators answer one fundamental question on behalf of taxpayers and policymakers: is this program actually working? That sounds simple until you try to do it rigorously — defining what "working" means for a workforce development program serving 40,000 participants across 15 states, or determining whether a juvenile justice intervention reduces recidivism when you can't randomly assign kids to treatment and control conditions.

The job starts well before any data is collected. A good evaluation begins with a clear articulation of the program's theory of change — what inputs are supposed to produce what activities, what activities are supposed to produce what outcomes, and over what time horizon. Getting that logic model right requires working closely with program managers who often haven't had to make their assumptions explicit before. That negotiation, done well, is some of the most valuable work evaluators do.

Once the evaluation design is set, the work becomes methodological. Evaluators specify data needs, navigate the bureaucratic process of getting access to administrative records (which can take months at federal agencies), design surveys that will actually get responses from grantees who are already drowning in reporting requirements, and make defensible decisions about comparison groups when random assignment isn't feasible.

Analysis follows data collection, and the outputs range from descriptive statistics and simple cross-tabulations in formative work to difference-in-differences models and instrumental variable approaches in impact evaluations where causal inference is at stake. The analysis only matters if it gets communicated effectively — and translating regression output into a finding a congressional staffer will act on is a distinct skill that takes years to develop.

The institutional context matters too. A GAO evaluator operates under different constraints than an evaluator at a state health department or a management consulting firm doing OMB PART work. Federal evaluators navigate agency politics, data access bureaucracy, and formal review processes. State government evaluators often work with smaller teams and less data infrastructure but closer proximity to programs. Consulting evaluators work on faster timelines with more varied methodological demands.

Education:

• Master's degree in public policy (MPA/MPP), public health (MPH), social work research, applied economics, or a related field — the standard credential for mid-career federal and state positions
• PhD preferred for research director and senior fellow roles at federal agencies, think tanks, and evaluation research centers
• Bachelor's degree plus 5–7 years of directly relevant evaluation experience accepted at some state agencies and nonprofits

Methods competency:

• Quantitative: regression analysis, difference-in-differences, propensity score matching, interrupted time series — at minimum one causal inference method at publication-ready proficiency
• Qualitative: semi-structured interview design, thematic coding, key informant analysis, document review
• Survey design: questionnaire development, sampling, nonresponse bias assessment
• Statistical software: Stata, R, or Python for quantitative work; NVivo or Atlas.ti for qualitative coding

Institutional knowledge:

• OMB Circular A-11 requirements for agency performance management
• Evidence Act (Foundations for Evidence-Based Policymaking Act of 2018) — learning agendas, evidence tiers, and evaluation officer roles
• IRB protocols and informed consent requirements for human subjects research
• Federal data governance: data use agreements, FISMA compliance for research data systems

Preferred experience backgrounds:

• GAO, agency Inspector General, or Congressional Research Service
• Federally-funded research and evaluation centers (Mathematica, Abt Associates, MDRC, Urban Institute, RAND)
• State legislative fiscal offices or state evaluation units
• University research institutes with policy evaluation portfolios

Soft skills:

• Ability to write clearly for non-technical audiences without distorting methodological nuance
• Diplomatic persistence in obtaining data access from reluctant program offices
• Comfort delivering findings that contradict what program sponsors hoped to hear

Demand for rigorous program evaluation has grown steadily since the passage of the Foundations for Evidence-Based Policymaking Act of 2018. The Evidence Act required federal agencies to designate Evaluation Officers, publish learning agendas, and build evaluation capacity — creating a structural hiring driver that didn't exist before. Most CFO Act agencies now have dedicated evaluation units that are still staffing up.

Budget pressure is a double-edged force. In tight fiscal environments, oversight bodies and appropriators want evidence that programs are delivering value — which supports evaluation demand. At the same time, evaluation budgets are discretionary and can be cut when agencies need to trim. The net effect is that evaluation work funded by statute (as in many HHS and Education programs) is more durable than evaluation funded through agency administrative accounts.

The evidence ecosystem beyond the federal government is also expanding. States have significantly increased evaluation capacity under federal matching incentives in Medicaid, SNAP, and child welfare. Philanthropic organizations — particularly those focused on economic mobility, criminal justice reform, and K–12 education — are major funders of independent evaluation. The result is a labor market with meaningful demand from government, nonprofits, and consulting firms simultaneously.

Automation is reshaping the work without eliminating it. Tasks that once consumed evaluation staff time — literature searches, qualitative coding passes, boilerplate report sections — are being accelerated by AI tools. This is compressing timelines and allowing evaluators to take on more projects, but it is also raising expectations for productivity. Evaluators who embrace these tools while maintaining methodological rigor are well-positioned; those who treat methods as the only value they offer will find the role harder to justify at its current cost.

For evaluators with strong causal inference skills and familiarity with administrative data systems, the job market in 2026 is favorable. The career ladder runs from analyst to senior evaluator to evaluation director or Chief Evaluation Officer — a role that now exists by statute at every major federal agency. Consulting paths lead to principal and director-level positions at firms that hold large federal evaluation contracts, where total compensation can substantially exceed the GS pay scale.

Dear Hiring Manager,

I'm applying for the Program Evaluator position at [Agency/Organization]. I have six years of experience designing and conducting evaluations of federally-funded social programs, most recently as a senior analyst at [Firm/Agency] where I led a multi-site impact evaluation of a DOL workforce development demonstration.

That evaluation used a difference-in-differences design with propensity score matching to construct a comparison group from state unemployment insurance wage records — a methodological choice we made after a randomized design was ruled out mid-project due to program expansion. Managing that redesign while keeping the evaluation credible for OMB review was the hardest problem-solving I've done, and it gave me strong working knowledge of administrative data linkage and the data use agreement process across three states.

On the qualitative side, I managed a portfolio of 60 site visit interviews with program operators and participants, supervised coding in NVivo, and wrote the implementation findings chapter that ended up driving the program's mid-course corrections more than the impact findings did. That experience shaped how I think about what different methods are actually useful for — impact estimates tell you whether something worked, implementation analysis tells you why and for whom.

I've followed [Agency]'s learning agenda closely and I think the evaluation gap around [specific program area] you identified in the FY2025 agenda is one I could contribute to directly. I'd welcome the opportunity to discuss how my background fits what your evaluation team is building.

[Your Name]