Distributed support (rework) - #996

Draft

lrzpellegrini wants to merge 21 commits into

ContinualAI:masterfrom

lrzpellegrini:distributed_support_rework

Commits on Apr 8, 2022

Merge remote-tracking branch 'upstream/master'
lrzpellegrini
committed
6bd297b

Commits on Apr 12, 2022

Reworking distributed support (WIP).
lrzpellegrini
committed
55a5480

Commits on Apr 21, 2022

Working strategy composition and example (naive, replay, scheduler).
lrzpellegrini
committed
b0ce2e3

Commits on Apr 22, 2022

Commits on Jul 19, 2022

Commits on Nov 10, 2022

Partial merge remote-tracking branch 'upstream/master' into distributed_support_rework
lrzpellegrini
committed
f104a0e

Commits on Nov 22, 2022

Integrated distributed training with RNGManager, new collate system. Additional tests. Detection WIP.
lrzpellegrini
committed
88f75a9

Commits on Nov 23, 2022

Commits on Dec 11, 2022

Fix multitask issues. Improve distributed training support and tests.
lrzpellegrini
committed
2a93ad8

Commits on Jan 10, 2023

Added additional unit tests. Issue with all_gather to be fixed.
lrzpellegrini
committed
1174f33

Commits on Jan 16, 2023

Tests for DistributedHelper. Distributed support field in plugins.
lrzpellegrini
committed
6a3dd1f