StaffTrace

What nursing home ratings miss between inspections

Comment

Inspiration

A study from the National Bureau of Economic Research found that nursing homes can predict when their inspections are coming, and as a result, will staff up near the days or weeks they're expected to arrive. Another finding it produced concluded that when nursing homes put more nurse hours on the floor, fewer residents die.

A study estimates that the hours nursing homes add because an inspection might be coming save about 886 lives a year, compared with a year with no inspections. Residents are more likely to survive in the weeks when a nursing home raises its hours, and more likely to die in the weeks when it cuts them. An inspection is supposed to be a surprise, but a nursing home can often tell roughly when the next one is coming. It cuts nurse hours after the inspectors leave, then raises them again as the next inspection gets closer. Another study finds that same rise and fall in hours.

StaffTrace gives regulators a game-theoretic inspection list inspired by how airport security and Coast Guard patrols are scheduled, so a home cannot tell when the inspectors are coming. A study estimates that this kind of unpredictable schedule could save an additional 92 lives a year.

The core problem with the current rating system for nursing homes is that CMS (Centers for Medicare & Medicaid Services) bases a nursing home’s star rating largely on what inspectors see during that visit when their hours are high. A nursing home that adds hours only while the inspectors are there can get a rating that looks better than the care residents receive after the inspectors leave.

What it does

StaffTrace compares the 14 days up to the day before the inspection ended with 4 to 8 weeks afterward to see which nursing homes raised staffing for the visit. When that increase is clearly larger than at a typical Georgia nursing home, StaffTrace lowers that home's CMS star rating, so it better reflects the care residents get year round. Of 350 scored homes, 30 show this pattern clearly and 22 actually lose a star (with the rest already being at one star).

The Payroll-Based Journal (getting staff hours from payroll) is self-reported, so each home’s result is shown with its uncertainty range to ensure the assessment isn't overconfident. Each home's page also has a plain-language explanation written by Grok, which only rephrases the numbers we give it and never invents new ones. Families can also click Ask Grok to talk with a voice assistant, built on Grok Voice, that looks up any home and explains its rating.

The month's inspection list that StaffTrace builds for state inspectors is made by chance. The inspector enters how many nursing homes the state can visit. Homes that are legally overdue are always included, and StaffTrace draws the rest at random. A home’s chance rises with many factors: the time since the last inspection being the most important, alongside its number of residents, how much it raises nurse hours around inspections, and how often inspectors have already found residents harmed. Those factors point the draw toward a home where missed visits affect more people or hide staffing increases. All nursing homes retain some chance of being chosen to avoid the situation we began with.

We also built Call Clock, a physical device that sits in a resident's room with no camera or microphone. A light sensor watches the call light and a motion sensor watches the doorway. It records how long it takes for someone to arrive after a resident calls for help, and when a call gets cancelled without anyone coming in. Every entry is saved in a hash-chained, signed log, so any edit shows up. Readings stream live to a StaffTrace page, giving a view of everyday care that doesn't depend on self-reported staffing data.

How it was built

StaffTrace runs entirely on public CMS data. Our Python/pandas pipeline takes daily nurse staffing (from CMS's Payroll-Based Journal) for all 356 Georgia nursing homes, about 288,000 facility-days, and lines it up with CMS inspection dates. For each home we run an event study to figure out how much higher nurse hours were per resident around each inspection than a month later. We shrink scores toward the state average when a home has few inspections, and bootstrap a 95% uncertainty range. Our monthly inspection scheduler uses a Stackelberg security game for its generation. Homes with higher risk scores are more likely to be picked, homes that are legally overdue are always picked, and the final list is drawn at random. That way no home can predict its next inspection and staff up just for it.

We used a FastAPI backend and a Next.js frontend with Leaflet for the statewide map, and Recharts for each home's staffing chart. The xAI Grok API writes each home's explanation from a fixed set of facts, with a template fallback if Grok is slow, and Grok Voice powers the Ask Grok line. Call Clock runs on an Arduino Nano with a light sensor and a motion sensor. It signs each event with HMAC and chains it to the previous one, and a Python serial bridge sends the events to our FastAPI backend, which streams them live to the web app. We split the work four ways and ensured we properly managed our code using separate branches, pull requests, and appropriate merge conflict management. On night 1, we froze a shared API contract with sample data, so all four parts could be built in parallel from the beginning. We used Cursor to implement our development for a large portion of the build.

Challenges we ran into

Once the pipeline worked, our hardest problem was deciding when the data justifies taking a star away from a real nursing home. We tried several approaches.

  • One made the penalty bigger the more a home's staffing went up around inspections.
  • We settled on something simpler and stricter: a home loses exactly one star, and only if its whole 95% uncertainty range, over at least two inspections, is above the typical Georgia home's staffing increase.
  • This rule only lowers a rating when the evidence is clear and repeated. It flags homes where staffing consistently rises around inspections, and it doesn't punish homes with little data or normal ups and downs in staffing.

Accomplishments that we're proud of

We're proud that we turned public data nobody was connecting into something families and regulators can use. For families, StaffTrace gives a rating that starts from the CMS stars they already trust and adjusts it when the evidence is strong. The result reflects the care residents get on ordinary days, not just what inspectors see during their visits. For regulators, the scheduler produces a real monthly inspection list that follows Georgia's legal deadlines while making the timing hard for any home to predict. When a home can't guess when inspectors will arrive, it has to staff well all the time, and that is where residents benefit. We're also proud of how carefully we present the results. Every score comes with its uncertainty range, the rating never goes up and only goes down when a pattern repeats across inspections, and families never see when a home's next inspection is likely.

What we learned

We learned that working with public data takes more than downloading it. The CMS files don't always document what their fields are. For example, the inspection date isn't labeled as the start or the end of the inspection, so we had to work it out from the staffing data itself. We also learned how much the comparison point matters. The typical Georgia home already staffs up slightly around inspections, so judging homes against zero would have flagged almost everyone. Judging them against the state average picked out the homes that actually stand out. None of us knew much about how nursing homes are inspected before this weekend, and the biggest surprise was how predictable the timing is. Inspections are supposed to be unannounced, but most homes get one roughly once a year, so a home can make a good guess at when the next one is coming. That matters because the star ratings families rely on are largely based on what inspectors see during those visits. If a home knows roughly when inspectors will show up, the rating can end up reflecting its best weeks instead of a normal one.

What's next for StaffTrace

The obvious next step is going beyond Georgia. The staffing and inspection data we use covers every nursing home in the country, so the same approach could work in any state. On the regulator side, we'd want to test the scheduler with a real state survey agency. Right now our risk weights are judgement calls, and working with inspectors would let us tune them against what inspections actually turn up.

We want a real pilot for Call Clock in nursing homes, with each home's response times shown on its StaffTrace page next to its staffing score. Because every entry is signed and chained, any later edit shows up, so residents and families would have trustworthy evidence of long wait times. It would also give regulators a new, independent view of everyday care that doesn't depend on self-reported staffing data.

Sources

Chen and Dillender, “Government Monitoring of Health Care Quality: Evidence from the Nursing Home Sector,” NBER Working Paper 34037 (2025). https://www.nber.org/papers/w34037

Gandhi, Olenski, and Shi, “Predictably Unpredictable Inspections,” NBER Working Paper 34491 (November 2025, revised June 2026). https://www.nber.org/papers/w34491

NBER Bulletin, “How Inspection Timing Affects Care Quality in US Nursing Homes.” https://www.nber.org/bh/20262/how-inspection-timing-affects-care-quality-us-nursing-homes

HHS Office of Inspector General, “CMS’s Processes Were Not Effective in Ensuring the Accuracy of Staffing Information Reported in the Payroll-Based Journal,” A-09-24-02005 (June 2026). https://oig.hhs.gov/reports/all/2026/cmss-processes-were-not-effective-in-ensuring-the-accuracy-of-staffing-information-reported-in-the-payroll-based-journal

Built With

Share this project:

Updates

Submission history