Evaluation for mailcom

[kimlee87]

Document evaluation strategy and related results

• Qualitative evaluation
  • mailcom functionalities and misaligned cases
  • mailcom vs. other open-source pseudonymization tools
    • QualiAnon (by their tutorial video)
    • Amnesia (by their playground webpage)
    • Presidio
    • Scrubadub
• Quantitative evaluation (mailcom vs. Presidio)
  • email detection
  • NER
  • number detection
• Summarize findings

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.59%. Comparing base (faf6624) to head (4e8831d).
⚠️ Report is 24 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #118      +/-   ##
==========================================
- Coverage   99.63%   99.59%   -0.04%     
==========================================
  Files          13       13              
  Lines        2191     2494     +303     
==========================================
+ Hits         2183     2484     +301     
- Misses          8       10       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Adds project documentation describing how mailcom was evaluated (qualitative + quantitative) and provides reproducible notebooks/scripts which generated the reported results.

Changes:

• Add an “Evaluation” section to the root README linking to the evaluation write-up and notebooks.
• Introduce evaluation_strategy.md describing qualitative comparisons and quantitative benchmark methodology/results.
• Add two new notebooks for quantitative evaluation and for testing/comparing other tools.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 13 comments.

File	Description
README.md	Adds a short entry point linking readers to evaluation docs and notebooks.
evaluation_strategy.md	Documents evaluation strategy, qualitative observations, quantitative setup, and summarized metrics.
docs/source/notebooks/quantitative_eval.ipynb	Notebook to generate quantitative evaluation datasets and compute metrics.
docs/source/notebooks/test_other_tools.ipynb	Notebook to run Presidio/Scrubadub comparisons and export results.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[iulusoy]

Thank you for all your work on this, it looks great!

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud