Skip to content

OOM if you take too many pictures. #140

Description

@mattcaron

Works: 100 positions, 5 focus stacks
Fails: 300 positions, 5 focus stacks.
Fails: 200 positions, 5 focus stacks.
Works: 150 positions, 5 focus stacks.

  • BUT - only the first time. If you scan one, stack the images, then do another scan, the second scan fails.

Camera is the 64MP Hawkeye. Board is 8GB RPi4.

2026-08-07 13:52:52,968 - DEBUG - [scan_task.py:_capture_photos_at_position] - Focus stacking enabled, capturing photo 4 / 5 with focus 13.75
2026-08-07 13:52:52,968 - DEBUG - [settings.py:__setattr__] - Trying to set 'manual_focus' to '13.75'
2026-08-07 13:52:52,969 - DEBUG - [settings.py:__setattr__] - Set 'manual_focus' to '13.75'
2026-08-07 13:52:53,585 - INFO - [picamera2.py:_configure_focus] - Manual focus enabled, current Focus set to: 13.75
2026-08-07 13:52:53,585 - DEBUG - [picamera2.py:_apply_settings_to_hardware] - Applied settings to hardware: {"shutter":50.0,"saturation":1.0,"contrast":1.0,"awbg_red":1.8,"awbg_blue":1.8,"gain":1.0,"jpeg_quality":90,"AF":false,"AF_window":null,"manual_focus":13.75,"crop_width":0,"crop_height":16,"orientation_flag":8,"preview_resolution":null,"photo_resolution":null}
2026-08-07 13:52:53,788 - INFO - [picamera2.py:_configure_focus] - Manual focus enabled, current Focus set to: 13.75
2026-08-07 13:52:53,788 - DEBUG - [picamera2.py:_configure_cropping_for_scalercrop] - Updated ScalerCrop: 
2026-08-07 13:52:53,810 - INFO - [picamera2.py:stop_] - Camera stopped
2026-08-07 13:52:53,814 - INFO - [picamera2.py:configure_] - Configuration successful!
2026-08-07 13:52:53,992 - ERROR - [scan_task.py:_capture_photos_at_position] - Error taking photo at position 2: Failed to start camera: Cannot allocate memory
Traceback (most recent call last):
  File "/opt/openscan3/openscan_firmware/controllers/services/tasks/core/scan_task.py", line 480, in _capture_photos_at_position
    photo_data = await self._ctx.camera_controller.photo_async(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self._ctx.scan.settings.image_format
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/camera.py", line 139, in photo_async
    return await asyncio.to_thread(self.photo, image_format)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/camera.py", line 133, in photo
    return handler[image_format]()
           ~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/picamera2.py", line 525, in capture_jpeg
    cam_metadata = self._picam.switch_mode_and_capture_file(self.photo_config,
                                                        jpeg_data,
                                                        #delay=5,
                                                        format='jpeg',
                                                        exif_data=exif_data)
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1466, in switch_mode_and_capture_file
    return self.dispatch_functions(functions, wait, signal_function, immediate=True)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1362, in dispatch_functions
    return job.get_result(timeout=timeout) if wait else job
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/job.py", line 79, in get_result
    return self._future.result(timeout=timeout)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/job.py", line 48, in execute
    done, result = self._functions[0]()
                   ~~~~~~~~~~~~~~~~~~^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1430, in switch_mode_
    self.start_()
    ~~~~~~~~~~~^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1153, in start_
    self.camera.start(controls)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
RuntimeError: Failed to start camera: Cannot allocate memory
2026-08-07 13:52:54,061 - ERROR - [scan_task.py:_execute_scan_loop] - Error taking photo at position 2: Failed to start camera: Cannot allocate memory
Traceback (most recent call last):
  File "/opt/openscan3/openscan_firmware/controllers/services/tasks/core/scan_task.py", line 340, in _execute_scan_loop
    await self._capture_photos_at_position(current_point, original_index)
  File "/opt/openscan3/openscan_firmware/controllers/services/tasks/core/scan_task.py", line 480, in _capture_photos_at_position
    photo_data = await self._ctx.camera_controller.photo_async(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self._ctx.scan.settings.image_format
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/camera.py", line 139, in photo_async
    return await asyncio.to_thread(self.photo, image_format)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/camera.py", line 133, in photo
    return handler[image_format]()
           ~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/picamera2.py", line 525, in capture_jpeg
    cam_metadata = self._picam.switch_mode_and_capture_file(self.photo_config,
                                                        jpeg_data,
                                                        #delay=5,
                                                        format='jpeg',
                                                        exif_data=exif_data)
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1466, in switch_mode_and_capture_file
    return self.dispatch_functions(functions, wait, signal_function, immediate=True)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1362, in dispatch_functions
    return job.get_result(timeout=timeout) if wait else job
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/job.py", line 79, in get_result
    return self._future.result(timeout=timeout)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/job.py", line 48, in execute
    done, result = self._functions[0]()
                   ~~~~~~~~~~~~~~~~~~^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1430, in switch_mode_
    self.start_()
    ~~~~~~~~~~~^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1153, in start_
    self.camera.start(controls)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
RuntimeError: Failed to start camera: Cannot allocate memory
2026-08-07 13:52:54,079 - ERROR - [scan_task.py:run] - Error during scan 2 for project XCom MEC Trooper: Failed to start camera: Cannot allocate memory
Traceback (most recent call last):
  File "/opt/openscan3/openscan_firmware/controllers/services/tasks/core/scan_task.py", line 205, in run
    async for progress in self._execute_scan_loop(resume_from_step, total):
        yield progress
  File "/opt/openscan3/openscan_firmware/controllers/services/tasks/core/scan_task.py", line 340, in _execute_scan_loop
    await self._capture_photos_at_position(current_point, original_index)
  File "/opt/openscan3/openscan_firmware/controllers/services/tasks/core/scan_task.py", line 480, in _capture_photos_at_position
    photo_data = await self._ctx.camera_controller.photo_async(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        self._ctx.scan.settings.image_format
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/camera.py", line 139, in photo_async
    return await asyncio.to_thread(self.photo, image_format)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/concurrent/futures/thread.py", line 59, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/camera.py", line 133, in photo
    return handler[image_format]()
           ~~~~~~~~~~~~~~~~~~~~~^^
  File "/opt/openscan3/openscan_firmware/controllers/hardware/cameras/picamera2.py", line 525, in capture_jpeg
    cam_metadata = self._picam.switch_mode_and_capture_file(self.photo_config,
                                                        jpeg_data,
                                                        #delay=5,
                                                        format='jpeg',
                                                        exif_data=exif_data)
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1466, in switch_mode_and_capture_file
    return self.dispatch_functions(functions, wait, signal_function, immediate=True)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1362, in dispatch_functions
    return job.get_result(timeout=timeout) if wait else job
           ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/job.py", line 79, in get_result
    return self._future.result(timeout=timeout)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/job.py", line 48, in execute
    done, result = self._functions[0]()
                   ~~~~~~~~~~~~~~~~~~^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1430, in switch_mode_
    self.start_()
    ~~~~~~~~~~~^^
  File "/opt/openscan3/venv/lib/python3.13/site-packages/picamera2/picamera2.py", line 1153, in start_
    self.camera.start(controls)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
RuntimeError: Failed to start camera: Cannot allocate memory

After that, you can't get to the webUI (which is not surprising, given the OOM condition).

My question is - why does the memory usage grow? It should only need to buffer 1 image while it is writing it out to disk.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions