Allow @ShadowSources path to traverse through non-planning variables

[Takkes]

Is your feature request related to a problem? Please describe.
Our domain is basically a vehicle routing problem. In contrast to the "basic" setup each stop is not just a single stop but effectively a fixed tour of multiple stops which we are planning on the vehicles. We had a pretty complicated setup with variable listeners and chained entities and are currently migrating to Timefold v2.1.

As of now the vehicle has a planning list variable of tours and each tour has a fixed list of stops. Each stop has a parent reference to the tour it belongs to and it has a shadow variable drivingTime that is calculated by a supplier. For all stops except the first one the driving time just depends on the vehicle (during solving), because the previous stop is fixed and for the first stop in a tour it also depends on the (last stop of the) previous tour.

The problem we are running into is that to access the vehicle and the previous tour we need to traverse through the parent reference to the tour within the shadow source which is not possible, because it's just a plain reference and not part of the entity meta model.

@PlanningEntity
public class Vehicle {
    @PlanningListVariable
    private List<Tour> tours = new ArrayList<>();
}

@PlanningEntity
public class Tour {
    @PreviousElementShadowVariable(sourceVariableName = "tours")
    private Tour previousTour;
    
    @InverseRelationShadowVariable(sourceVariableName = "tours")
    private Vehicle vehicle;
    
    private List<Stop> stops = new ArrayList<>();
}

@PlanningEntity
public class Stop {
    @ShadowVariable(supplierName = "calculateDrivingTime")
    private Duration drivingTime;

    private Tour tour;

    @ShadowSources({"tour.vehicle", "tour.previousTour"})
    public Duration calculateDrivingTime() {
        //...
    }
}

Describe the solution you'd like
I'm not sure whether this would be the best solution but considering the alternatives I can think of this seems like the best solution to me right now: Allow the @ShadowSources path to traverse through non-planning variables, possibly by marking them somehow as a problem fact?

Describe alternatives you've considered
Alternatives I could think of, which I haven't tested yet as it seems rather hacky to me.

1. Make the tour-stop relation a planning variable and make sure that their values are never actually changed. This would add them to the meta model but it would be really hacky in my opinion, because it would misuse planning variables that should never be planned.
2. Effectively get rid of the tour class and directly plan stops. However, this would need to make sure that all stops of a tour are only moved together. With hard constraints this would introduce a huge amount of useless moves. Therefore this would probably require custom moves to always move all stops together. This might work but from a modeling perspective it doesn't feel great because it's not actually representing the problem and also from a complexity and maintainability perspective it wouldn't be ideal.
3. Have the shadow variable(s) and supplier(s) in the tour instead of the stops and update everything at once. While this probably works for the minimal example above it would probably run into problems with our more complicated actual setup. There each stop can have dependencies to other stops of other tours on other vehicles and it could also happen that two tours reference each other multiple times in a non-cyclic way that requires bouncing back and forth between the two tours when calculating their times. If the listeners would be on the tours it would probably require a lot of redundant calculations, because we have to calculate the whole tour, instead of only the necessary times and I'm not sure whether the built-in cycle detection of Timefold wouldn't abort this back-and-forth.

Additional context

[triceo]

Hello @Takkes, and thanks for reporting. This is a tricky one.

Personally, I'd try to find a different model which does not require this indirection. Having three entity classes in a single problem is already unusual.

Solving this instead by introducing "meta" shadow variables could be made to work, but it feels wrong. It's not a variable, and the only reason you'd be declaring it as such would be to reference it as a shadow source; it's a nail when you already have a hammer.

@Christopher-Chianelli What's your opinion?

[Takkes]

@triceo Thanks for your quick response and sorry for my late response, I got a bit side-tracked.

I understand your assessment but I'm a bit unhappy that there's no direct way to model our problem the way it is represented in reality. However, I do understand that it might not be a trivial change, assuming this would actually be something that is worth adding to the solver as a general feature. As the migration got de-prioritized on our end I'll leave it like that and then hopefully come back with our solution once we have the capacity to continue working on this.

[TomCools]

If I get it correctly, @Takkes your nr 3 would look like the following code example.

If you could clarify a bit what you mean with "dependencies between stops on different tours" and "tours reference each other multiple times". These are 2 things not really clear to me and not really mentioned in your original question.

@PlanningEntity
public class Vehicle {
    @PlanningListVariable
    private List<Tour> tours = new ArrayList<>();
}

@PlanningEntity
public class Tour {
    @PreviousElementShadowVariable(sourceVariableName = "tours")
    private Tour previousTour;
    
    @InverseRelationShadowVariable(sourceVariableName = "tours")
    private Vehicle vehicle;
    
    private List<Stop> stops = new ArrayList<>();

    @ShadowVariable(supplierName = "enhanceStops")
    private List<Stop> enhancedStops;

    @ShadowSources({"vehicle", "previousTour"})
    public List<Stop> enhanceStops() {
        //do your driving time calculation here, using the non-variable field "stops"
        //put all the new stop objects in a list and return it. 
    }
}

public class Stop {
    private Duration drivingTime;

    private Tour tour;
}