fix(compare-baseline): exact baseline matches should pass (+1 more)

[andrewwhitecdw]

This PR addresses 2 issues in bobber/lib/analysis/compare_baseline.py.

Fix 1

fix: exact baseline matches should pass, not fail

Fix: Replace:

    if got > expected:
        return True
    else:
        return False

with:

    if got >= expected:
        return True
    else:
        return False

Fix 2

fix: evaluate_test crashes when nccl or dali results are missing

Fix: Replace:

        elif test_name == 'nccl':
            failures = evaluate_nccl(test_values, results, failures, tolerance)
        elif test_name == 'dali':
            failures = evaluate_dali(test_values,
                                     results['dali'],
                                     test_name,
                                     failures,
                                     tolerance)

with:

        elif test_name == 'nccl':
            if test_name not in results:
                print(f'  {TEST_MAPPING[test_name]}')
                print('    No NCCL results found, skipping...')
                continue
            failures = evaluate_nccl(test_values, results, failures, tolerance)
        elif test_name == 'dali':
            if test_name not in results:
                print(f'  {TEST_MAPPING[test_name]}')
                print('    No DALI results found, skipping...')
                continue
            failures = evaluate_dali(test_values,
                                     results['dali'],
                                     test_name,
                                     failures,
                                     tolerance)

Files changed

• bobber/lib/analysis/compare_baseline.py