Assertion error hit during optimization

[Ickaser]

I haven't yet been able to boil down my example to an MWE, but I am hitting the below AssertionError that seems like a logic bug rather than numerical bug.
I am using Optim v2.0.1 and LineSearches v7.6.0 or v7.6.2, through OptimizationOptimJL from the SciML ecosystem. The I used here is algorithm BFGS(linesearch=LineSearches.BackTracking()). Is this likely to be an issue with LineSearches instead? Or perhaps with OptimizationOptimJL?

ERROR: AssertionError: !(hasproperty(state, :f_x)) || phi_0 == state.f_x
Stacktrace:
  [1] InitialStatic
    @ C:\Users\iwheeler\.julia\packages\LineSearches\lihz0\src\initialguess.jl:17 [inlined]
  [2] perform_linesearch!(state::Optim.BFGSState{…}, method::BFGS{…}, d::Optim.ManifoldObjective{…})
    @ Optim C:\Users\iwheeler\.julia\packages\Optim\DtV5C\src\utilities\perform_linesearch.jl:48
  [3] update_state!(d::TwiceDifferentiable{…}, state::Optim.BFGSState{…}, method::BFGS{…})
    @ Optim C:\Users\iwheeler\.julia\packages\Optim\DtV5C\src\multivariate\solvers\first_order\bfgs.jl:155
  [4] optimize(d::TwiceDifferentiable{…}, initial_x::Vector{…}, method::BFGS{…}, options::Optim.Options{…}, state::Optim.BFGSState{…})
    @ Optim C:\Users\iwheeler\.julia\packages\Optim\DtV5C\src\multivariate\optimize\optimize.jl:79
  [5] optimize(d::TwiceDifferentiable{…}, initial_x::Vector{…}, method::BFGS{…}, options::Optim.Options{…})
    @ Optim C:\Users\iwheeler\.julia\packages\Optim\DtV5C\src\multivariate\optimize\optimize.jl:45
  [6] __solve(cache::OptimizationCache{…})
    @ OptimizationOptimJL C:\Users\iwheeler\.julia\packages\OptimizationOptimJL\mYdL4\src\OptimizationOptimJL.jl:261
  [7] solve!(cache::OptimizationCache{…})
    @ OptimizationBase C:\Users\iwheeler\.julia\packages\OptimizationBase\y7Tug\src\solve.jl:236
  [8] solve_call(::OptimizationProblem{…}, ::BFGS{…}; merge_callbacks::Bool, kwargshandle::Nothing, kwargs::@Kwargs{…})
    @ OptimizationBase C:\Users\iwheeler\.julia\packages\OptimizationBase\y7Tug\src\solve.jl:282
  [9] solve_up(prob::OptimizationProblem{…}, sensealg::Nothing, u0::Vector{…}, p::Tuple{…}, args::BFGS{…}; originator::SciMLBase.ChainRulesOriginator, kwargs::@Kwargs{…})
    @ OptimizationBase C:\Users\iwheeler\.julia\packages\OptimizationBase\y7Tug\src\solve.jl:265
 [10] solve_up
    @ C:\Users\iwheeler\.julia\packages\OptimizationBase\y7Tug\src\solve.jl:256 [inlined]
 [11] solve(prob::OptimizationProblem{…}, args::BFGS{…}; sensealg::Nothing, u0::Nothing, p::Nothing, wrap::Val{…}, kwargs::@Kwargs{…})
    @ OptimizationBase C:\Users\iwheeler\.julia\packages\OptimizationBase\y7Tug\src\solve.jl:104
 [12] macro expansion
    @ .\timing.jl:689 [inlined]
 [13] top-level scope
    @ c:\Users\iwheeler\OneDrive - purdue.edu\Documents\01_Projects\temp_FulbrightPores\scripts\fulbright_sucrose.jl:446
Some type information was truncated. Use `show(err)` to see complete types.

[pkofod]

Strange. I don't know anything about OptimizationOptimJL I'm afraid. It's saying that either the state does not have f_x (which I think it most certainly does) or phi_0 is not equal to state.f_x which should not happen, but maybe that's the problem. Can you show the actual optimization problem?

[Ickaser]

It's an ODE parameter-fitting problem for a handcoded model of mine and it only happens on some bad parameter guesses (which means in practice I avoid the error now with better guesses), so it will take several hundred lines of code to express but I can try to extract it out into at least a freestanding script if that would be a useful starting point for investigation

[pkofod]

can you show me the code you use to use Optim from Optimization.jl? Assuming you have the objective functions etc all coded up then how do you actually run the optimization using Optimization.jl? If we can convert it to an Optim call I can maybe help, but I cannot support Optimization.jl as I have no idea how it works and I'm not involved in the work.

Maybe if you ask Claude to set it up to optimize directly through Optim.jl?

[Ickaser]

Optimization.jl is a pretty thin wrapper, in this case. Here is the call expressed directly through Optim, which I just tested on my case:

optalg = BFGS(linesearch=LineSearches.BackTracking())
Optim.optimize(objn_optim, [0.0, 0.0, 0.0], optalg; autodiff=AutoForwardDiff())

with a corresponding shorter stack trace:

ERROR: AssertionError: !(hasproperty(state, :f_x)) || phi_0 == state.f_x
Stacktrace:
 [1] InitialStatic
   @ C:\Users\ickas\.julia\packages\LineSearches\2sBPR\src\initialguess.jl:17 [inlined]
 [2] perform_linesearch!(state::Optim.BFGSState{…}, method::BFGS{…}, d::Optim.ManifoldObjective{…})
   @ Optim C:\Users\ickas\.julia\packages\Optim\DtV5C\src\utilities\perform_linesearch.jl:48
 [3] update_state!(d::OnceDifferentiable{…}, state::Optim.BFGSState{…}, method::BFGS{…})
   @ Optim C:\Users\ickas\.julia\packages\Optim\DtV5C\src\multivariate\solvers\first_order\bfgs.jl:155
 [4] optimize(d::OnceDifferentiable{…}, initial_x::Vector{…}, method::BFGS{…}, options::Optim.Options{…}, state::Optim.BFGSState{…})
   @ Optim C:\Users\ickas\.julia\packages\Optim\DtV5C\src\multivariate\optimize\optimize.jl:79
 [5] optimize
   @ C:\Users\ickas\.julia\packages\Optim\DtV5C\src\multivariate\optimize\optimize.jl:45 [inlined]
 [6] optimize(f::Function, x0::Vector{…}, method::BFGS{…}, options::Optim.Options{…}; inplace::Bool, autodiff::AutoForwardDiff{…})
   @ Optim C:\Users\ickas\.julia\packages\Optim\DtV5C\src\multivariate\optimize\interface.jl:225
 [7] optimize
   @ C:\Users\ickas\.julia\packages\Optim\DtV5C\src\multivariate\optimize\interface.jl:216 [inlined]
 [8] top-level scope
   @ c:\Users\ickas\OneDrive - purdue.edu\Documents\01_Projects\FulbrightPores\scripts\fulbright_dextran.jl:239
Some type information was truncated. Use `show(err)` to see complete types.

There's nothing proprietary about the code generating this but it does depend on several CSVs worth of data--probably too much for a single file, and it is sensitive to the initial guess (changing [0.0, 0.0, 0.0] to [-1.0, 1.0, 0.0] for my example leads to a successful solve rather than this error), so it's not obvious to me how to distill it down to a smaller example.

[Ickaser]

After a little more playing around, I suspect this was related to regions where my objective function was producing NaNs: when I made it produce NaNs a little more eagerly (and predictably) the problem also seemed to go away. There might have been something more subtle going on, though.

[pkofod]

After a little more playing around, I suspect this was related to regions where my objective function was producing NaNs: when I made it produce NaNs a little more eagerly (and predictably) the problem also seemed to go away. There might have been something more subtle going on, though.

Indeed, this should not happen which is why it's an assert, so I'm still interested in figuring out why.

[pkofod]

Optimization.jl is a pretty thin wrapper, in this case. Here is the call expressed directly through Optim, which I just tested on my case:

yeah, but I don't know anything about the software which is why I asked for an Optim.jl reproducer and I appreciate that you gave it to me.

[pkofod]

The problem is probably also that on a different system it can maybe even fail to reproduce. But if you can share it with me I can have a look.

[pkofod]

oh wait, did it start with a NaN? Maybe it's just

julia> NaN==NaN
false

[pkofod]

can you check out the specific version of Optim locally and change it to === ?

[pkofod]

Are you always using backtracking? Then I think I found the issue

[Ickaser]

Are you always using backtracking? Then I think I found the issue

Yes, I've been using backtracking for all of this (I had some problem with HagerZhang, can't remember what off the top of my head).

can you check out the specific version of Optim locally and change it to === ?

Is this a separate suggestion? I think I can check if changing the assertion to === fixes the case I had. Or is this a different fix?

[pkofod]

Are you always using backtracking? Then I think I found the issue

Yes, I've been using backtracking for all of this (I had some problem with HagerZhang, can't remember what off the top of my head).

Hager Zhang was probably the "isfinite" assertion. That has recently been improved in LineSearches and I'm putting out a new version of Optim soon that should make that assertion go away and improve things in general.

[pkofod]

These NaNs are a bit of a headache, because the algorithms are not really built for them in the middle of the search region. It's simply to handle if you can backtrack into a region where it's finite, but when the nans are scattered it's hard to do anything sensible, but we try :)

[pkofod]

Did you try in any of the latest versions?