Skip to content

fix: real-time axis, non-UTF-8 file support, and OOM on large logs#12

Open
kaklik wants to merge 1 commit into
calibrationfrom
parser-fix
Open

fix: real-time axis, non-UTF-8 file support, and OOM on large logs#12
kaklik wants to merge 1 commit into
calibrationfrom
parser-fix

Conversation

@kaklik
Copy link
Copy Markdown
Member

@kaklik kaklik commented May 6, 2026

  • UnicodeDecodeError fix: log files are now opened with encoding="ascii", errors="replace", so non-ASCII bytes in file headers no longer crash the parser.

  • Real-time axis: the parser now reads the $TIME record present in AIRDOS04C fw 2.x logs and extracts the rtc_unix_offset field (= Unix timestamp − RTC counter). All timestamps in time_axis and telemetry are converted to Unix epoch. PlotCanvas detects the has_real_time metadata flag and switches to pg.DateAxisItem so graphs show actual UTC date/time instead of relative minutes. A SmartDateAxisItem subclass adds the calendar date to the first tick of each new day, so the date is always visible at midnight transitions (or at the first visible tick after opening a file). Logs without a $TIME record (old format) are unaffected.

  • OOM fix for large files: AirdosV2LogParser now runs a fast pre-scan pass over the file to find the highest channel index used in $E and $STOP records, then allocates hist and every current_hist copy only to that size with dtype=int32. For a file with max channel 7 647 and 11 001 records, the spectral_matrix shrinks from ~5.8 GB (65 536 ch × int64) to ~320 MB (7 648 ch × int32), making files that were previously killed by the OOM killer parseable in under a second.

@kaklik kaklik requested a review from roman-dvorak May 6, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant