pytest.fix_local_tests

Goal

• Given information about failed tests <FAILURE_INFO> run the tests, understand what is the root cause, and fix them

Workflow

Run Tests

• The input <FAILURE_INFO> can be:

  • A failing test in pytest
    • E.g., dev_scripts_helpers/documentation/test/test_notes_to_pdf.py
  • A repro generated by pytest_failed_multi_build.py
    • E.g., tmp.pytest_failed_multi_build.repro.sh

• Gather information about the failing tests by:

  • Reading the code
  • Inspecting the content of <FAILURE_INFO>
  • Running the tests locally, if needed
    • E.g.,

      > pytest_log `<FAILURE_INFO>`
      > tmp.pytest_failed_multi_build.repro.sh
      

Analyze Failures

• Read .claude/skills/pytest.rules.md about how to diagnose issues

• Collect the failing tests from the run

  > i pytest_failed
  ...
  Failed tests: 25/7
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_actions::test1
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_actions::test2
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_edge_cases::test1
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_edge_cases::test2
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_edge_cases::test3
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_edge_cases::test4
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_filters::test1
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_filters::test2
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_filters::test3
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_filters::test4
  dev_scripts_helpers/documentation/test/test_notes_to_pdf.py::Test_notes_to_pdf_filters::test5
  ...
  13:07:36 - WARN  lib_tasks_pytest.py pytest_failed:1745                 To run the failed tests run: tmp.pytest_failed.sh
  ...
  

• For the tests that involve running Docker / Apple containers run the same tests with different builds, as per ## Analyze the Effect of Container Builds in .claude/skills/pytest.rules.md

• Carefully consider if the root cause and the solution is the same for all the 3 builds

Prepare A Plan to Fix Failures

• Prepare a plan following # Format for Unit Test Fixing Plan from .claude/skills/pytest.rules.md
• Save it in the file plan-pytest.fix_local_tests.md in the current dir
• Do not print anything on screen

Implement and Verify the Fix

• If the root cause and the fix are clear, implement the fixes directly
• If the root cause or the correct fix is unclear, stop and ask the user before implementing anything
• Verify that the fixes work correctly as per # Verification of the Fix from .claude/skills/pytest.rules.md

Verification

• [ ] plan-pytest.fix_local_tests.md lists a fix for every failing test
• [ ] Every fix follows # Verification of the Fix in .claude/skills/pytest.rules.md
• [ ] The failing tests pass locally after the fix