Skip to content

Portrait mode: a third capture mode with the background blurred from depth - #271

Merged
darioalessandro merged 1 commit into
masterfrom
portrait-mode
Nov 9, 2026
Merged

darioalessandro merged 1 commit into
masterfrom
portrait-mode

Conversation

@darioalessandro

Copy link
Copy Markdown
Member

Portrait comes to Remote Shutter as a third mode on the remote, next to Photo, the way Apple's Camera does it. People already know what it means, so there's nothing new to learn.

What you get

  • PHOTO / PORTRAIT / VIDEO on the director. PORTRAIT shows only when every camera in the rig can take it. If one camera can't (an older build, a single-lens phone, a Mac, or the front camera), the tab hides and the rig drops back to Photo.
  • The camera turns on depth and the person matte, shoots at 12MP, and bakes the blur into the still with Core Image's portrait filter at f/4.5. Aspect crop, sync stamp and transfer to the remote work as they do for any still.
  • The shutter keeps the still mode. Before this, every TakePic/ScheduledCapture forced .Photo, which would have turned depth off right before the shot.

Wire changes

  • RecordingModeEnum.Portrait = 4
  • CameraState.supports_portrait: bool: the active camera delivers depth. It moves with device swaps. Old peers don't send it, so they read as unsupported and never see mode 4.
  • No new commands. SetCameraMode already carries the mode.

Not in this PR

  • Front-camera Portrait. The front camera is the plain wide lens, which has no depth, so Portrait hides on selfie. Switching the front to TrueDepth changes device IDs and gets its own PR.
  • Live blur in the preview, an f-stop control, the yellow "Portrait" subject badge, and Watch support.
  • Depth isn't kept in the file, so the blur can't be re-edited in Photos.
  • PHOTO/VIDEO/PORTRAIT on the director aren't localized (PHOTO/VIDEO weren't before either). The camera-phone label is, in all 15 locales.

Verified

  • Simulator: 819 tests, 0 failures, 25 hardware tests skipped. 10 new tests: wire round trip and old-peer default, shutter keeps Portrait (TakePic and scheduled), the rig-wide gate and fallback, tray tiles, view-model fallback, and the no-depth crop fallback.
  • Mac Catalyst builds.
  • iPhone 18 Pro, full CaptureIntegrationTests: 16 of 17 pass, including the 4K30 gate and the new testPortraitStillIsBlurredWithin12MPAndFramesKeepFlowing. Depth and the matte turn on, the zoom range stays 1.0–189, the still comes back at 4032×3024 and re-rendered by the blur, and frames flow both ways.
  • The one failure, testExposureProbeAppliesAndReportsTruth, fails the same way on unchanged master. It's a pre-existing crash: manual exposure on the 18 Pro throws NSInvalidArgumentException because its lens aperture is adjustable. Separate fix.

Not verified yet (why this is a draft)

  • What the blur looks like. The test stills came out black because the phone's camera was covered. Needed: one shot of a person on the 18 Pro.
  • Older phones (iPhone 11 and others) haven't been run.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

📊 Test Coverage Report

✅ 796/820 tests passed · ⏭️ 24 skipped

Metric Coverage
App code (excl. generated + vendored Theater) ██████████░░░░░░░░░░ 48.3% 🔴
Whole target ██████████░░░░░░░░░░ 49.4% 🔴

🎥 Capture & session core

File Coverage Lines
CaptureEngine.swift ██░░░░░░░░░░░░░░░░░░ 11.9% 271/2274 🔴
RecordingPipeline.swift ████████████████░░░░ 80.7% 706/875 🟢
FrameStreamingCoordinator.swift ███░░░░░░░░░░░░░░░░░ 13.0% 32/247 🔴
FrameStreamer.swift ███████████████████░ 92.6% 113/122 🟢
RemoteCmdFlatBuffers.swift ██████████████████░░ 92.0% 1189/1292 🟢
📁 Full report — 103 files
File Coverage Lines
DebugLog.swift █████████████████████ 104.8% 22/21 🟢
JPEGFrameEncoder.swift ████████████████████ 100.0% 12/12 🟢
CameraPreviewMode.swift ████████████████████ 100.0% 13/13 🟢
AppTheme.swift ████████████████████ 100.0% 33/33 🟢
AppActivityMonitor.swift ████████████████████ 100.0% 31/31 🟢
ZoomScale.swift ████████████████████ 100.0% 56/56 🟢
Messages.swift ████████████████████ 100.0% 7/7 🟢
ZoomScaleSeed.swift ████████████████████ 100.0% 14/14 🟢
MonitorChrome.swift ████████████████████ 100.0% 12/12 🟢
PeerSessionCore.swift ████████████████████ 100.0% 26/26 🟢
ClockOffsetEstimator.swift ████████████████████ 100.0% 24/24 🟢
ExposurePolicy.swift ████████████████████ 100.0% 42/42 🟢
StreamingConfig.swift ████████████████████ 100.0% 34/34 🟢
MultipeerMessages.swift ████████████████████ 100.0% 12/12 🟢
FocusedCameraControlState.swift ████████████████████ 100.0% 6/6 🟢
Locked.swift ████████████████████ 100.0% 19/19 🟢
FrameCreditWindow.swift ████████████████████ 100.0% 9/9 🟢
AppDelegate.swift ████████████████████ 100.0% 16/16 🟢
CameraLink.swift ████████████████████ 100.0% 59/59 🟢
HEICFrameEncoder.swift ████████████████████ 100.0% 17/17 🟢
PeerCompatibility.swift ████████████████████ 100.0% 42/42 🟢
RigQualityMenu.swift ████████████████████ 99.3% 141/142 🟢
DeviceScannerViewModel.swift ████████████████████ 98.9% 183/185 🟢
FrameSender.swift ████████████████████ 98.4% 184/187 🟢
FocusPointMapping.swift ████████████████████ 97.6% 40/41 🟢
CaptureSyncMetadata.swift ███████████████████░ 97.0% 65/67 🟢
MultiCamChrome.swift ███████████████████░ 96.9% 31/32 🟢
RemoteCmds.swift ███████████████████░ 95.9% 326/340 🟢
MulticamViewModel.swift ███████████████████░ 94.9% 148/156 🟢
VP9FrameEncoder.swift ███████████████████░ 94.7% 195/206 🟢
DiscoveredPeers.swift ███████████████████░ 94.4% 17/18 🟢
WatchSharedTypes.swift ███████████████████░ 94.0% 236/251 🟢
WatchPreviewStreamer.swift ███████████████████░ 93.6% 44/47 🟢
MulticamController.swift ███████████████████░ 93.0% 1505/1618 🟢
FrameStreamer.swift ███████████████████░ 92.6% 113/122 🟢
RemoteCmdFlatBuffers.swift ██████████████████░░ 92.0% 1189/1292 🟢
WelcomeView.swift ██████████████████░░ 89.3% 492/551 🟢
VideoTransferProgressView.swift █████████████████░░░ 84.0% 295/351 🟢
UIImage+gif.swift ████████████████░░░░ 81.6% 129/158 🟢
CameraDeviceDescriptor.swift ████████████████░░░░ 81.0% 34/42 🟢
RecordingPipeline.swift ████████████████░░░░ 80.7% 706/875 🟢
CameraProgressOverlayView.swift ████████████████░░░░ 78.6% 187/238 🟡
UICmds.swift ████████████████░░░░ 78.1% 75/96 🟡
SessionCoordinator.swift ████████████████░░░░ 77.7% 1870/2406 🟡
FlatBufferSchemas_generated.swift (excluded from app-code metric) ███████████████░░░░░ 77.1% 881/1143 🟡
SettingsView.swift ███████████████░░░░░ 76.2% 593/778 🟡
RolePickerView.swift ███████████████░░░░░ 76.2% 314/412 🟡
StreamLog.swift ███████████████░░░░░ 75.0% 15/20 🟡
WelcomeViewModel.swift ███████████████░░░░░ 74.1% 100/135 🟡
CameraRecordingTimerView.swift ███████████████░░░░░ 73.8% 127/172 🟡
SettingsViewModel.swift ██████████████░░░░░░ 70.6% 89/126 🟡
DeviceScannerView.swift ██████████████░░░░░░ 70.1% 1142/1630 🟡
SwiftConstants.swift █████████████░░░░░░░ 66.7% 12/18 🟡
UIAlertController.swift █████████████░░░░░░░ 66.0% 35/53 🟡
MonitorView.swift █████████████░░░░░░░ 65.0% 514/791 🟡
WelcomeViewController.swift █████████████░░░░░░░ 64.1% 41/64 🟡
ScannerLobby.swift ████████████░░░░░░░░ 62.5% 5/8 🟡
CameraViewModel.swift ████████████░░░░░░░░ 59.9% 160/267 🟡
StoreManager.swift ███████████░░░░░░░░░ 54.6% 83/152 🟡
SceneDelegate.swift ██████████░░░░░░░░░░ 51.9% 14/27 🟡
UIViewController+SwiftUIHosting.swift ██████████░░░░░░░░░░ 48.4% 15/31 🔴
FrameStreamReceiver.swift ██████████░░░░░░░░░░ 47.7% 244/512 🔴
FrameCodecs.swift █████████░░░░░░░░░░░ 47.1% 24/51 🔴
MonitorViewModel.swift █████████░░░░░░░░░░░ 44.0% 11/25 🔴
PurchaseManaging.swift ████████░░░░░░░░░░░░ 42.0% 21/50 🔴
MulticamView.swift ████████░░░░░░░░░░░░ 40.6% 554/1365 🔴
CameraScreenView.swift ████████░░░░░░░░░░░░ 39.3% 262/667 🔴
MulticamViewController.swift ████████░░░░░░░░░░░░ 38.0% 130/342 🔴
EnumExtensions.swift ███████░░░░░░░░░░░░░ 33.9% 40/118 🔴
HEVCFrameEncoder.swift ██████░░░░░░░░░░░░░░ 31.9% 110/345 🔴
ExposureControls.swift ██████░░░░░░░░░░░░░░ 30.8% 94/305 🔴
CaptureTally.swift █████░░░░░░░░░░░░░░░ 25.0% 13/52 🔴
RulerPill.swift █████░░░░░░░░░░░░░░░ 24.5% 152/621 🔴
DeviceScannerViewController.swift █████░░░░░░░░░░░░░░░ 24.3% 117/482 🔴
CameraRig.swift █████░░░░░░░░░░░░░░░ 23.8% 150/629 🔴
WatchSessionManager.swift ████░░░░░░░░░░░░░░░░ 20.8% 37/178 🔴
MediaProcessors.swift ███░░░░░░░░░░░░░░░░░ 17.5% 11/63 🔴
SessionDebugConsole.swift ███░░░░░░░░░░░░░░░░░ 15.0% 74/494 🔴
FrameStreamingCoordinator.swift ███░░░░░░░░░░░░░░░░░ 13.0% 32/247 🔴
CaptureEngine.swift ██░░░░░░░░░░░░░░░░░░ 11.9% 271/2274 🔴
RolePickerController.swift ██░░░░░░░░░░░░░░░░░░ 10.2% 23/226 🔴
PeerLinkStatus.swift █░░░░░░░░░░░░░░░░░░░ 7.1% 10/141 🔴
CameraCountdownTorch.swift █░░░░░░░░░░░░░░░░░░░ 4.9% 3/61 🔴
WatchRemoteCameraController.swift █░░░░░░░░░░░░░░░░░░░ 3.2% 9/279 🔴
SoundManager.swift ░░░░░░░░░░░░░░░░░░░░ 2.1% 1/47 🔴
RecordingTimer.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/52 🔴
ViewfinderGestureLayer.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/165 🔴
CameraPermissionsView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/676 🔴
CameraHostController.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/205 🔴
RemoteShutterHelpView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/1372 🔴
CameraPermissionErrorView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/416 🔴
ZoomPill.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/116 🔴
PreviewSurface.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/100 🔴
PhotosAccessDeniedView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/613 🔴
MicrophonePermissionPromptView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/347 🔴
Photos.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/12 🔴
LocalNetworkPermissionView.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/747 🔴
PermissionManager.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/185 🔴
OrientationUtils.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/18 🔴
PhotosAccessHelper.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/347 🔴
UIAlertPresenter.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/43 🔴
MultipeerService.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/123 🔴
Data+MD5.swift ░░░░░░░░░░░░░░░░░░░░ 0.0% 0/4 🔴

Generated from 7c09887 · workflow run

@darioalessandro
darioalessandro marked this pull request as ready for review November 9, 2026 06:19
…depth

The director shows PHOTO / PORTRAIT / VIDEO when every camera in the rig
reports supports_portrait; one camera that can't (older build, single-lens
phone, Mac, front wide camera) hides the tab and drops the rig to Photo.

Wire: RecordingModeEnum.Portrait = 4 and CameraState.supports_portrait.
No new commands; SetCameraMode carries the mode.

Camera: Portrait is the photo format plus depth and person-matte delivery,
capped at 12MP. The blur is baked by CIDepthBlurEffect at f/4.5 and the
still is cropped and encoded like any other. Shutter paths keep a still
mode instead of forcing Photo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@darioalessandro
darioalessandro merged commit b7d5d72 into master Nov 9, 2026
2 checks passed
@darioalessandro
darioalessandro deleted the portrait-mode branch November 9, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant