Questions about weight of energy and force loss

[Swiftily]

I used to think the absolute value of the energy_weight and force_weight doesn't matter, what we need is the ratio of these two. But recently I've found that I was wrong. That is if I set these two 1,1 or 10000,10000, training process doesn't go as the same, the former value often makes the optimization be stagnant. I've read the source code(/mace/modules/loss.py), it seems the weight actually determines the value of loss, then the gradient of loss, which is directly relevant to parameter update. Or maybe is there something like regularization to eliminate this issue? And how should I effectively find a reasonable value of weight?

[gabor1]

Can you show some learning curves with the two settings? Do you actually get better models with one of these settings? There are other parameters like the learning rate and the batch size that will interact with the loss weights

[ilyes319]

Hey,

Adam with weight decay is not scale invariant, so it is not surprising that you see an impact on the training dynamics. I recommand you to use something like 10/100 on forces and 1/10 on energy.

[Swiftily]

yeah, I,'ve figured it out. So usually, we should set the force weight much larger than energy?

[gabor1]

Initially yes, high weight on forces. But to get low energy errors (which is essential for a good model, otherwise you are sampling the wrong potential), we use two-stage training and in the second stage you raise the energy weight. The force errors don't go up by much but the energy errors drop significantly.

[ilyes319]

yes and then use stage_two for last 30% of epochs to reduce energy error.

[Swiftily]

Many thanks! I will try it.

[jagriti-iisc]

Initially yes, high weight on forces. But to get low energy errors (which is essential for a good model, otherwise you are sampling the wrong potential), we use two-stage training and in the second stage you raise the energy weight. The force errors don't go up by much but the energy errors drop significantly.

IN the second stage we should increase the energy weights, but what about the force weight, should we keep them same?
ps: i am training a model for md simualtions

mace_run_train --name="Mg_Water_MACE" --train_file="train_fixed.xyz" --valid_file="valid_fixed.xyz" --model="MACE" --num_channels=128 --num_interactions=2 --correlation=3 --max_L=1 --r_max=5.0 --energy_weight=1.0 --forces_weight=1000.0 --swa_energy_weight=1000.0 --swa_forces_weight=10.0 --E0s="{1: -13.5820, 8: -431.1051, 12: -1720.6420}" --energy_key="REF_energy" --forces_key="REF_forces" --batch_size=5 --max_num_epochs=1200 --patience=200 --swa --start_swa=800 --ema --ema_decay=0.99 --amsgrad --lr=0.001 --scheduler_patience=20 --default_dtype="float32" --save_cpu --error_table="PerAtomMAE" --device=cuda --seed=123

i used this,and what i observed 2026-04-25 02:00:38.958 INFO: Epoch 326: head: Default, loss=5.72790098, MAE_E_per_atom= 1.66 meV, MAE_F= 71.04 meV / A
2026-04-25 02:06:17.900 INFO: Epoch 327: head: Default, loss=5.72761869, MAE_E_per_atom= 1.79 meV, MAE_F= 71.03 meV / A
2026-04-25 02:11:56.635 INFO: Epoch 328: head: Default, loss=5.72191715, MAE_E_per_atom= 1.55 meV, MAE_F= 71.04 meV / A
2026-04-25 02:11:56.635 INFO: Stopping optimization after 200 epochs without improvement and starting Stage Two
2026-04-25 02:11:56.635 INFO: Changing loss based on Stage Two Weights
2026-04-25 02:17:35.609 INFO: Epoch 801: head: Default, loss=0.05654204, MAE_E_per_atom= 1.58 meV, MAE_F= 89.75 meV / A
2026-04-25 02:23:14.536 INFO: Epoch 802: head: Default, loss=0.05651229, MAE_E_per_atom= 1.58 meV, MAE_F= 84.59 meV / A
2026-04-25 02:28:53.455 INFO: Epoch 803: head: Default, loss=0.05662346, MAE_E_per_atom= 1.59 meV, MAE_F= 93.13 meV / A
2026-04-25 02:34:32.353 INFO: Epoch 804: head: Default, loss=0.05652142, MAE_E_per_atom= 1.57 meV, MAE_F= 88.95 meV / A
2026-04-25 02:40:11.136 INFO: Epoch 805: head: Default, loss=0.05656323, MAE_E_per_atom= 1.58 meV, MAE_F= 91.15 meV / A
2026-04-25 02:45:49.919 INFO: Epoch 806: head: Default, loss=0.05658132, MAE_E_per_atom= 1.57 meV, MAE_F= 91.81 meV / A
2026-04-25 02:51:28.826 INFO: Epoch 807: head: Default, loss=0.05656414, MAE_E_per_atom= 1.58 meV, MAE_F= 91.19 meV / A