From analyst to systems builder

I’ve spent more than ten years with the Oregon Department of Human Services, and my role has changed a lot in that time. I started as a Research Analyst 4, leading interagency research projects with partners from the Oregon Health Authority, the Oregon Employment Department, the Oregon Department of Education and Oregon Housing and Community Services.

Along the way, I fell in love with the technical side of the job and started spending my spare time at work building durable systems and workflows for my teammates. I brought GitHub into our process to make our research more reproducible and transparent. In 2017 I led the procurement of our first machine learning server, Microsoft Machine Learning Server, which ran Microsoft’s own distribution of R. That server became the backbone for the predictive models my team built for agency programs. Microsoft retired the product in 2022, and in 2023 I migrated our remaining production model to open-source R so it could keep running.

What I work on now

Today I lead our team’s research infrastructure. I led the procurement of our Posit Team environment, which runs in Snowpark Container Services as a Snowflake Native App, and I’m its administrator. That modern, scalable platform now hosts the production models and APIs our team builds, including some of the work I’m proudest of. I hope to write about it here in more detail.

Off the clock

Outside of work, I’m the lucky father of two young kids. When they aren’t keeping me busy with art projects, outdoor play, Fortnite matches and Minecraft builds, I love reading, creative writing and running tabletop RPGs. For more than a decade, I’ve had the good fortune of leading the same core group of friends through weekly adventures.

What you’ll find here

• Technical: the platforms, pipelines and tools behind our work.
• Gov data: what doing data work inside state government is really like.
• Off the clock: game sessions, worldbuilding and the technical tinkering I do in Foundry Virtual Tabletop (Foundry VTT).
• Neurodivergence: what I’ve learned about working as a neurodivergent adult, and the strategies that help me thrive.

I’ll do my best to keep posts short, interesting and relevant. I’m glad you’re here, and I hope you find something worthwhile.

What administrative data is built to answer

Benefit programs like SNAP and TANF, and services like child welfare, carry heavy federal reporting requirements. Those requirements largely decide what gets recorded, how it’s defined, and how carefully it’s checked. If a field feeds a federal report, it tends to be well defined and fairly complete. If it doesn’t, it may be optional, free text, inconsistently used, or missing entirely.

So administrative data is very good at questions like:

• How many people were served, and for how long?
• Were eligibility decisions made on time?
• How much was paid out, and to whom?

It is much weaker at the questions programs care about most:

• Are families more economically stable a year after they leave the program?
• Did a new service improve child well-being, or only shift when cases close?
• Which parts of the program make a difference, and for whom?

These outcome questions often depend on things the system never captures: someone’s wellbeing, the stability of their housing, their experience with the program, what happened after they left. You can’t analyze a variable that was never recorded.

What quasi-experimental methods can and can’t do

When a randomized trial isn’t possible, which in public programs is most of the time, we turn to quasi-experimental designs: difference-in-differences, regression discontinuity, propensity score methods, interrupted time series, and similar tools. I use them often, and they are powerful. Applied carefully, they can get much closer to a causal estimate than a simple before-and-after comparison.

But they have limits that no amount of statistical skill removes:

• They can only estimate effects on outcomes that were measured. If the outcome a program cares about isn’t in the data, no design recovers it.
• They depend on assumptions the data can’t always test. Parallel trends, no unmeasured confounding, a clean cutoff. Administrative data rarely holds the variables we’d need to check those assumptions properly.
• They inherit every quality problem in the source data. A sophisticated model built on inconsistent records gives a precise-looking answer to a fuzzy question.

Quasi-experimental methods help us get the most out of the data we have. They don’t replace data that was collected on purpose to answer a specific question.

Purposeful data collection has real costs

When a program wants to know whether it’s working, the natural next step is to collect new data: a survey, interviews, focus groups, a structured intake form. This is often the right move. What gets underestimated, almost everywhere in government, is what doing it well costs.

Good data collection needs:

• Planning. Defining the question, deciding what to measure, choosing a sample, and working out how the data will be analyzed before anything is collected. This is the step that most often gets skipped, and skipping it is the main reason surveys end up measuring satisfaction when the real question was impact.
• Instrument design. Writing survey questions that are clear, unbiased, and actually measure the concept you care about takes training and testing. A question that seems obvious to the person writing it can mean three different things to respondents.
• Privacy and ethics review. Human services data is sensitive. Consent, data sharing agreements, and protections for participants all take time and expertise.
• Staffing. Someone has to recruit participants, follow up on non-response, manage the data, and analyze it. Qualitative work needs even more: facilitators, note-takers, transcription, and careful coding by people trained to do it.
• Infrastructure. Data needs to be captured in systems that validate entries, restrict access, track changes, and connect to the rest of the analytic environment.

None of this is exotic. It’s standard research practice. It just isn’t free, and it rarely appears in a program’s budget.

The spreadsheet problem

When the infrastructure isn’t there, people make do. Across government, a lot of program-level data collection happens in Excel. That isn’t carelessness. It’s what’s available, it’s familiar, and it gets the job done in the moment.

But spreadsheets weren’t designed for data collection. There’s no validation on entry, so a date ends up typed three different ways. There’s no controlled vocabulary, so one category shows up under five spellings. Files get copied, renamed, and emailed around, so no one is sure which version is current. Columns change meaning partway through a year. By the time the data reaches an analyst, a large share of the effort goes into cleaning it, and some errors can’t be fixed because the information needed to fix them was never captured.

These are system problems, not people problems. Give the same staff a well-designed form with validation and a shared database, and data quality improves right away.

Why this keeps happening

It’s easy to read all of this as a criticism of the people doing the work. It isn’t. The program staff I work with care a great deal about the families they serve and want to know whether what they’re doing helps. The constraints are structural:

• Funding follows compliance. Federal requirements come with deadlines and consequences. Evaluation usually doesn’t, so it’s often the first thing cut when budgets tighten.
• Data collection is treated as overhead. It’s rarely budgeted as part of running a program, so it competes with direct services, and direct services almost always win.
• Staff capacity is stretched thin. Teams running programs are often already at or past capacity. Designing and running a rigorous data collection effort on top of that isn’t realistic without dedicated support.
• Research methods training isn’t part of most program roles. That’s reasonable. We don’t expect researchers to be eligibility experts either. But it means survey design and measurement often fall to people who were never given the tools for it.

The result is a system very good at counting what it does and much less able to learn whether it’s working.

What would help

I don’t think the answer is a large new research operation in every program. Some smaller shifts would go a long way:

1. Start with the question. Before collecting anything, write down what decision the data will inform and what you’d need to know to make it.
2. Bring research staff in early. An hour of consultation during design saves weeks of cleaning and reanalysis later, and can mean the difference between data that answers the question and data that can’t.
3. Budget for data collection as part of the program. If an outcome matters enough to ask about, it matters enough to fund measuring it properly, including staff time and infrastructure.
4. Collect less, but on purpose. A short, well-designed instrument that measures the right outcome beats a long survey that measures everything loosely.
5. Invest in shared infrastructure. Validated forms, secure databases, and standard definitions benefit every program that uses them and make each new collection cheaper than the last.

Administrative data will always be the backbone of public-sector research, and it should be. It’s comprehensive, longitudinal, and already paid for. But if we want to know whether programs are actually improving people’s lives, we have to be willing to collect the data that question requires, and be honest about what that takes.