I'd like to extend the Levenberg-Marquardt algorithm to support direct function minimization in addition to the existing model fitting approach.

Model-Based vs Function-Based Approaches

Current Model-Based Approach

Currently, the LevenbergMarquardtMinimizer only works with IObjectiveModel which represents a curve fitting problem:

• It minimizes the sum of squared residuals: RSS = Σ w_i * (y_i - f(x_i; p))²
• Where f(x; p) is the model function that predicts y given x and parameters p
• This approach is used when we have observed data points (x_i, y_i) and want to find the best model parameters p

Proposed Function-Based Approach

I'd like to add support for direct objective function minimization:

• Directly minimizes an arbitrary objective function F(p) (which can include RSS as a special case)
• No explicit observed data points are required
• The function's gradient (∇F(p)) and Hessian (∇²F(p)) are used for optimization
• This approach is more general and can represent a wider range of optimization problems
• Despite not having explicit data points, statistical calculations (standard errors, covariance) are still valuable, so a degreeOfFreedom parameter is needed to properly calculate these statistics

Both approaches use the same core Levenberg-Marquardt algorithm, but they require different interfaces and calculations.

Proposed changes:

1. Add IFixableObjectiveFunction interface to allow fixing parameters during optimization
2. Rename existing NonlinearObjectiveFunction (internal class) to NonlinearObjectiveModel for clarity
3. Add new NonlinearObjectiveFunction class for direct function minimization
4. Extend LevenbergMarquardtMinimizer to support IObjectiveFunction
5. Add degreeOfFreedom parameter for IObjectiveFunction to allow proper calculation of covariance and correlation matrices
6. Add DirectMinimizationFunction helper methods to ObjectiveFunction class

Regarding NonlinearFunction methods:

The current NonlinearFunction methods create conceptual confusion because:

• They transform a model-based approach f(x; p) into a function-based approach F(p)
• This blurs the distinction between the two optimization approaches

The current implementation assumes a given f(x; p) is already F(p), which is confusing.
The newly proposed NonlinearLeastSquaresFunction will properly convert a model function f(x; p) into an objective function F(p) = Σ w_i * (y_i - f(x_i; p))².
I suggest marking the existing methods as deprecated to improve API clarity by:

• Maintaining clearer separation between model fitting and direct function minimization
• Directing users to either NonlinearLeastSquaresFunction or NonlinearModel based on their needs
• Providing DirectMinimizationFunction as a clearer alternative for direct function optimization

This extension would make the Levenberg-Marquardt algorithm more versatile, supporting both model fitting and direct function minimization while maintaining backward compatibility except for the deprecated methods.

I'd like to get your thoughts on the proposed deprecation of the current NonlinearFunction methods. Is this approach acceptable, or would you prefer a different solution to address the conceptual confusion?

I plan to submit a PR implementing these changes.