Hello, my name is Seokjin Oh, and I have a few questions about modifying the base code.
(I also sent it by email, but I'm writing it here as well in case it's difficult to check the email.)
[Replace backbone model]
In your paper, you adopt mbart-large-cc25 as the backbone model of your framework. I'm only interested in multi-source neural machine translation, so I want to modify the backbone to the NLLB(NLLB (huggingface.co)) model. I, of course, changed the model checkpoint and some configurations, but It doesn't work at all. Specifically, the train loss wasn't dropped(stuck at around 25). Could you give me some advice for changing the backbone model?
[Using more than 2 hypotheses]
In your paper, the described MSG tasks only take 2 inputs. But I want to give more than 2 inputs(maybe 3~5) to the framework. I really don't know how to make Fine Encoder work with more than 2 inputs. e.g. If there are 3 inputs, does the Fine Encoder calculate Cross-Att between all combinations of inputs([1,2],[1,3],[2,3]) and mean pooling at the decoder side? And it will be very very helpful if you provide me an advice for modifying this part.(If you have any previously tried experiments, could you please share some materials for me? If possible...)
Thank you in advance for taking the time out of your busy schedule. 😄
Hello, my name is Seokjin Oh, and I have a few questions about modifying the base code.
(I also sent it by email, but I'm writing it here as well in case it's difficult to check the email.)
[Replace backbone model]
In your paper, you adopt mbart-large-cc25 as the backbone model of your framework. I'm only interested in multi-source neural machine translation, so I want to modify the backbone to the NLLB(NLLB (huggingface.co)) model. I, of course, changed the model checkpoint and some configurations, but It doesn't work at all. Specifically, the train loss wasn't dropped(stuck at around 25). Could you give me some advice for changing the backbone model?
[Using more than 2 hypotheses]
In your paper, the described MSG tasks only take 2 inputs. But I want to give more than 2 inputs(maybe 3~5) to the framework. I really don't know how to make Fine Encoder work with more than 2 inputs. e.g. If there are 3 inputs, does the Fine Encoder calculate Cross-Att between all combinations of inputs([1,2],[1,3],[2,3]) and mean pooling at the decoder side? And it will be very very helpful if you provide me an advice for modifying this part.(If you have any previously tried experiments, could you please share some materials for me? If possible...)
Thank you in advance for taking the time out of your busy schedule. 😄