Build a new Solution for each Python test case

TK
2h ago
+255-2

Problem and behavior

The Python driver ran every test case against one Solution built before the case loop, so an answer that keeps state on self started each case with what the earlier cases left behind. On Grid Segment Net Yield, the answer in the issue, with self.best set in __init__, showed Test results · 2/6, every case after the second returning 42; it now passes 6 of 6, as it already did in C++. An __init__ that raises, such as self.memo = {}[0], used to read as Could not find bestRouteYield in your code - keep the starter signature.; each case now fails with KeyError: 0.

Approach

The driver keeps its setup check that Solution defines the entry method and builds the instance inside each case's try, as the C++ and Java harnesses in web/compiler-explorer.js already do. Class judges and answers written as a plain function take the same path as before, and a test pins each. Another test makes sure each case runs on an instance __init__ built for it alone: every case reports how many times __init__ has run, whether self is the newest object it ran on, and the attributes self already had. An answer that does heavy work in __init__ now repeats it for every case, inside the same 5 s run budget and counted in that case's timeMs, which is also how C++ and Java construct.

pythonDriver is exported from web/runners.js so tests/browser/python-driver.test.js can run it on CPython from node --test, as suggested on #783. The tests skip only when the interpreter, python3 or the one PYTHON names, is missing; one that is there but does not start, exits nonzero or hangs fails them with what --version showed. A fix for #783 would need the same export, so whichever change lands second rebases onto the other. The fresh-state test also asserts that the judge's cases still expose a shared instance, so reordering the bank's cases cannot leave it passing on the old driver.

Validation

  • node --test tests/browser/python-driver.test.js: # pass 6, # fail 0. Before the fix the three tests of per-case state and construction fail (cases 3 to 6 got 42; a setup error instead of KeyError: 0). A driver that shares one instance, hands each case a copy.copy or copy.deepcopy of it, calls __init__ on it again (with or without clearing its __dict__), builds every instance before the loop, skips __init__, or repeats, stops at or returns the first constructor error fails at least one test. With PYTHON pointing at a missing interpreter all six skip; with a non-executable one or /bin/false all six fail with the probe's message.
  • Pyodide in Chromium through runBrowserTests: the issue's answer passes 6/6 and the raising __init__ gives KeyError: 0 in every case. I also checked both by hand in an offline Practice interview.
  • ./scripts/test.sh browser at the first head (e3d54427, the same web/runners.js): pass 2224, fail 0, with 1393 tests skipped because Playwright 1.65's Chromium was not installed here. The suites that run Python in the page (runners, graph-clone-identity, candidate-cases, task-runner, specialized-tree-cycles) and the new file, rerun with an installed Playwright 1.64.1 Chromium: pass 192, fail 0, 22 Java skips.
  • ./scripts/test.sh portable: review-memory-tests, git-hooks and shell-syntax fail here identically on a clean main (no gh in WSL, commentflow, shfmt and ruff not installed, shellcheck 0.11.0). ./scripts/test.sh rust passes here. Not run: Java (no javac).

Close #799


Summary by cubic

Builds a fresh Solution for each Python test case so answers that keep state on self no longer leak state between cases, matching what C++ and Java already do. On Grid Segment Net Yield an answer that set self.best in __init__ previously passed 2 of 6 cases, returning 42 after the second case; it now passes 6 of 6. A constructor that raises now fails each case with the real error (KeyError: 0) instead of Could not find bestRouteYield in your code - keep the starter signature.

  • Moves Solution() construction inside each case's try; the setup check that the entry method exists is unchanged.
  • One test pins that each case runs on an instance __init__ built for it alone: each case reports the construction count, whether it's the newest instance, and pre-existing attributes — a driver that shares, copies, re-initializes, or prebuilds an instance fails it.
  • Exports pythonDriver from web/runners.js so tests/browser/python-driver.test.js can run it on CPython, skipping when python3 is absent. The new tests also cover the paths the fix keeps: a missing entry method is still a setup error, class judges keep one shared instance per case, and plain-function answers still run.

Closes #799.