Run crunch_artifacts on a worker thread so the GUI stays responsive - #196
Open
OneSixForensics wants to merge 1 commit into
Open
Run crunch_artifacts on a worker thread so the GUI stays responsive#196OneSixForensics wants to merge 1 commit into
OneSixForensics wants to merge 1 commit into
Conversation
The Process button called crunch_artifacts directly on the Tk main thread, so for the whole run the only thing pumping the event loop was log_text.update() inside logfunc. Between two log lines the window is unserviced: a slow artifact that works for seconds without logging leaves Windows painting "(Not Responding)" over a run that is healthy, and the log and progress bar only move when a line happens to be emitted. crunch_artifacts now runs on a daemon worker thread and reports back over a queue that an after() poller drains on the main thread. Tk is not thread-safe, so the worker never touches a widget: while GuiWindow.message_queue is set, logfunc and SetProgressBar put messages on the queue instead, and poll_crunch does every insert and config on the main thread. The queue is cleared when the run ends, so a CLI run and a GUI run that has finished keep the previous direct-write behaviour. initialize_lava() moved from process() into the worker with it. sqlite3 objects may only be used by the thread that created them, so a LAVA connection opened on the main thread raises ProgrammingError on the first artifact search pattern the worker inserts. Opening it on the thread that uses it is the fix; nothing on the main thread touches lava_db. The completion UI moved to finish_crunch() unchanged; process() now ends by starting the worker. An unhandled exception in the worker is caught and sent as a 'failed' message, so a crash surfaces the existing error dialog instead of leaving the poller waiting forever. Log lines are inserted once per poll rather than once per write, which also drops the per-line update() that made a chatty artifact cost more in the widget than in the parser. Measured on a synthetic 12-artifact run where one artifact works 2.5s without logging: worst event-loop stall 2.50s -> 0.17s, and 13 -> 40+ loop slices served over the run. Ported from the iLEAPP change, including the lava thread-affinity fix a real run there surfaced; the twin was read rather than copied blind, and the crunch_artifacts call in this core keeps its own signature. Not yet run against a real image in this core. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
The GUI called
crunch_artifacts()directly on the Tk main thread from the Process button handler, so for the whole length of a run the only thing pumping the event loop waslog_text.update()insidelogfunc(scripts/ilapfuncs.py). Between log lines the window stops answering Windows, which paints "(Not Responding)" over a run that is perfectly healthy. A single slow artifact is enough to trigger it, and an examiner watching a report that looks hung has no way to tell it apart from one that is.The change
crunch_artifactsruns on a daemon worker thread and reports back over aqueue.Queuethat anafter()poller drains on the main thread.GuiWindow.message_queueis set while a run is in flight.logfuncandSetProgressBarcheck it and enqueue instead of touching a widget, so the worker never calls into Tk.log_text.insert()per poll rather than one per line, which is also less work than the old path.finish_crunch(), called by the poller on the main thread.initialize_lava()moved out ofprocess()and into the worker. Asqlite3connection may only be used by the thread that created it, so LAVA's connection has to be opened on the same thread the artifacts run on. Opening it on the main thread raisesSQLite objects created in a thread can only be used in that same threadon the first artifact that writes.The
scripts/ilapfuncs.pyhalf is identical in all five LEAPP cores. The GUI half keeps each core's owncrunch_artifactssignature.Measured
logfuncandSetProgressBarare the real ones, driven through the realmessage_queuegate; only the workload is simulated (12 artifacts, one of them working 2.5 s without logging). Worst event-loop stall, and how many times the loop was serviced:Windows paints "(Not Responding)" at roughly 5 s, so 2.55 s is the visible symptom's floor rather than its ceiling — a genuinely slow artifact stalls for longer.
Verified on a real run
Full GUI run against a real Berla iVe Ford Sync 3 acquisition (the
DCASourceFilesUpload.zipfrom an iVe case folder): 12 artifacts, 6,730 rows, no errors.The other four LEAPP cores carry the same change on a branch of this name; each was verified against its own real data, since each keeps its own
crunch_artifactssignature and passing one does not clear the others.