Skip to content

fixed InvalidOperationError bug#418

Open
smartbackwards wants to merge 1 commit intopnxenopoulos:mainfrom
smartbackwards:smartbackwards-SortingBug
Open

fixed InvalidOperationError bug#418
smartbackwards wants to merge 1 commit intopnxenopoulos:mainfrom
smartbackwards:smartbackwards-SortingBug

Conversation

@smartbackwards
Copy link
Copy Markdown

issue was found trying to access any dataframe from map 1 of https://www.hltv.org/matches/2380994/betboom-vs-og-pgl-astana-2025-europe-closed-qualifier

``C:\Users\username\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\polars\lazyframe\frame.py", line 2332, in collect return wrap_df(ldf.collect(engine, callback)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ polars.exceptions.InvalidOperationError: argument in operation 'asof_join' is not sorted, please sort the 'expr/series/column' first

was raised probably due to the tick column not being out of order

issue was fixed when I tested the change in a clean venv

@adisujithkumar
Copy link
Copy Markdown
Collaborator

Here is the entire stack trace of the error I am seeing when I am trying to parse the same demo with the parse_demo.ipynb example notebook:

---------------------------------------------------------------------------
InvalidOperationError                     Traceback (most recent call last)
Cell In[8], line 11
      9 print(f"\nHeader: \n{dem.header}")
     10 print(f"\nRounds: \n{dem.rounds.head(n=3)}")
---> 11 print(f"\nKills: \n{dem.kills.head(n=3)}")
     12 print(f"\nDamages: \n{dem.damages.head(n=3)}")
     13 print(f"\nWeapon Fires: \n{dem.shots.head(n=3)}")

File [~/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python3.11/functools.py:1001](http://localhost:8888/lab/tree/docs/examples/~/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/python3.11/functools.py#line=1000), in cached_property.__get__(self, instance, owner)
    999 val = cache.get(self.attrname, _NOT_FOUND)
   1000 if val is _NOT_FOUND:
-> 1001     val = self.func(instance)
   1002     try:
   1003         cache[self.attrname] = val

File /mnt/d/projects/fps/awpy/awpy/awpy/demo.py:334, in Demo.kills(self)
    332 kills = awpy.parsers.utils.get_event_from_parsed_events(self.events, "player_death")
    333 kills = awpy.parsers.events.parse_kills(kills)
--> 334 return awpy.parsers.rounds.apply_round_num(df=kills, rounds_df=self.rounds, tick_col="tick").filter(
    335     pl.col("round_num").is_not_null()
    336 )

File /mnt/d/projects/fps/awpy/awpy/awpy/parsers/rounds.py:232, in apply_round_num(df, rounds_df, tick_col)
    205 """Assign a round number to each event based on its tick value.
    206 
    207 For each row in `df`, this function finds the round from `rounds_df`
   (...)
    228     matching round is found, "round_num" will be null.
    229 """
    230 # Use join_asof to get the round where round.start <= event.tick.
    231 # This join will add the columns 'round_num', 'start', and 'end' from rounds_df.
--> 232 df_with_round = df.join_asof(
    233     rounds_df.select(["round_num", "start", "official_end"]),
    234     left_on=tick_col,
    235     right_on="start",
    236     strategy="backward",
    237 )
    239 # Validate that the event tick is within the round boundaries.
    240 # If the tick is greater than the round's 'end', then set round_num to null.
    241 df_with_round = df_with_round.with_columns(
    242     pl.when(pl.col(tick_col) <= pl.col("official_end")).then(pl.col("round_num")).otherwise(None).alias("round_num")
    243 )

File /mnt/d/projects/fps/awpy/awpy/.venv/lib/python3.11/site-packages/polars/dataframe/frame.py:7319, in DataFrame.join_asof(self, other, left_on, right_on, on, by_left, by_right, by, strategy, suffix, tolerance, allow_parallel, force_parallel, coalesce, allow_exact_matches, check_sortedness)
   7297         msg = f"expected `right_on` to be str or Expr, got {type(right_on).__name__!r}"
   7298         raise TypeError(msg)
   7300 return (
   7301     self.lazy()
   7302     .join_asof(
   7303         other.lazy(),
   7304         left_on=left_on,
   7305         right_on=right_on,
   7306         on=on,
   7307         by_left=by_left,
   7308         by_right=by_right,
   7309         by=by,
   7310         strategy=strategy,
   7311         suffix=suffix,
   7312         tolerance=tolerance,
   7313         allow_parallel=allow_parallel,
   7314         force_parallel=force_parallel,
   7315         coalesce=coalesce,
   7316         allow_exact_matches=allow_exact_matches,
   7317         check_sortedness=check_sortedness,
   7318     )
-> 7319     .collect(_eager=True)
   7320 )

File /mnt/d/projects/fps/awpy/awpy/.venv/lib/python3.11/site-packages/polars/lazyframe/frame.py:2066, in LazyFrame.collect(self, type_coercion, _type_check, predicate_pushdown, projection_pushdown, simplify_expression, slice_pushdown, comm_subplan_elim, comm_subexpr_elim, cluster_with_columns, collapse_joins, no_optimization, streaming, engine, background, _check_order, _eager, **_kwargs)
   2064 # Only for testing purposes
   2065 callback = _kwargs.get("post_opt_callback", callback)
-> 2066 return wrap_df(ldf.collect(callback))

InvalidOperationError: argument in operation 'asof_join' is not sorted, please sort the 'expr/series/column' first

If the issue gets fixed simply by just ordering the df by ticks column before returning, then it seems to be simple enough to fix. However, I think it would be ideal to really know why this error is popping up. Regardless this PR looks good to me, but I'll let @pnxenopoulos have a chance to give his thoughts as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants