Skip to content

Foray 324 · Essai 01 · Historical Health portfolio

Did nine projects miss the same measure?

The FDA investment for Mission Accomplishments and Regulatory Compliance Services (MARCS) is linked to nine projects. A reviewer looking up its performance measures sees “Not Met” alongside every one of those projects. Does she need nine recovery updates?

Ask the investment owner for the history of measure 3481. There is one recorded “Not Met”. It belongs to the investment’s measure for the number of imported products screened per year. All nine project records share a path to it through that one investment. The extract gives no individual project result against the measure.

Inspect the two saved exports

9 projects × 5 measures = 45 pairs

Each mark is one project paired with the measure on that row. Four measure records have no recorded outcome.

  • 3479 · Unreported
  • 3480 · Unreported
  • 3481 · Not Met
  • 3482 · Unreported
  • 3483 · Unreported

9 projects × 1 measure = 9 pairs

The same nine project IDs are present. Here there is just one linked measure record, 3481, and it carries the same outcome.

  • 3481 · Not Met

How the extra rows appear

Start with investment 009-000005317. Follow pays_for to each project and judged_by to each measure. Pairing each project with every measure on that investment is a valid way to draw its neighbourhood. Counting those pairs as distinct outcomes would make the investment with the most projects appear to have the most failures. In either export, collapsing the paths to their distinct measure IDs leaves a single reported result.

Bratanič’s multigraph example asks what degree counts when several relationships connect the same endpoints. We adapted that experiment by projecting each project–investment–measure combination back onto its investment and measure. Keeping all paths gives measure 3481 a count of nine. Counting the distinct investment–measure pair gives it a count of one. We implemented the comparison directly over the two published exports in Python. Bratanič’s multigraph article · Original notebook.

What the reviewer takes to the meeting

Bring measure 3481 and its reporting definition to the investment owner. Ask when it was measured, what the target was and whether any project-level evidence links delivery work to that outcome. Check with the data steward why four other measure records appear only in Health and whether their outcomes can be recovered. The file named “Complete” contains fewer measures for this particular investment. Neither file settles why their coverage differs.

Reproduce the figures

The standard-library Python replay verifies the published files by checksum, reads their saved node and relationship statements as text, and compares the nine project IDs across both files. It prints the measure IDs, their recorded outcomes and the number of matching pairs. There is no database setup. Run python3 replay.py, or use python3 replay.py --directory PATH with your own local copies of the two published files. Download the replay.

The inputs are the historical research-and-development snapshots already available in the Library. The comparison covers this investment only. Health extract · Complete export · Source notes and dates.

Choose a question from another department →