Repository navigation
Conversation
youtube-shorts turns a long YouTube video into vertical Shorts: it reads the word-timed transcript (YouTube captions, or local whisper.cpp with DTW timing when there are none), resolves quoted phrases to word-accurate cuts, detects camera cuts on the source for per-shot 9:16 framing, snaps cut edges into pauses and flags likely clipped words, renders 1080x1920 clips with one continuous loudness-normalized audio pass, and writes HyperFrames overlay projects in the channel's own style. Sources are the creator's master, the user's own yt-dlp, or YouTube's 360p stream for review. youtube-shorts-publish checks titles, descriptions, hashtags, tags, and schedules against YouTube's documented limits and writes copy-paste sheets and cover frames for YouTube Studio.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Verdict: changes needed
Reviewed head 04028ec380478e8c0aa461d920c31c0fd17c44cf. GitHub rejects a formal request-changes review through the connected author's account, so this comment records the verdict.
P2: Publishing handoff ignores the burned-caption render
packages/youtube-skills/src/publish/cli.ts:96–98 selects <id>-overlay.mp4 or <id>.mp4, never <id>-captioned.mp4. The Shorts skill makes overlays conditional (step 7) and allows the ordinary burned-caption render to proceed to publishing. On that supported path the upload sheet points at the uncaptioned intermediate and extracts its cover. A creator following the sheet can publish without the captions they just generated.
Reproduced with the committed publish.mjs bundle and an isolated project with valid video/EDL/publish JSON, render/S1.mp4 and render/S1-captioned.mp4, and no overlay. kit returned ok:true, no issues, and both shorts[0].video and the sheet's Video field selected S1.mp4. Prefer the captioned render after an overlay, or expose explicit final-render selection, and add CLI regression coverage.
Verification
- Independently built the library dependency; skill suite with
SHORTS_FFMPEG_TEST=1: 141 passed, 4 skipped, including real FFmpeg render/shot checks. Skill TypeScript check passed. - Exact-head Verify CI 42483512963: all eight jobs successful, including clean CI bundle verification.
- Local bundle check reported youtube/watch/shorts byte differences in the reused dependency environment. Not asserted as a source defect given clean exact-head CI; local reproducibility remains unverified.
- No actual YouTube upload, live extraction, Whisper installation or full HyperFrames render performed.
- A separate review task was interrupted by an automated safety check. Its unfinished security tests are not verified findings; this review is not complete security assurance.
No implementation, push, merge or deployment performed.
overlay now detects faces in each rendered short (macOS Vision through JavaScript for Automation, sampled twice a second; typical face zones per framing elsewhere) and places the hook, name tag, cards, end card, and captions where they cover the least face. Titles and cards use the top band or lower third (or the seam on split screens) and may shift up to 2 s to stay within one camera shot. Captions re-place only when the shot or the titles on screen change, keep a gap from titles and cards, and never cover the wordmark. Remaining overlaps are reported with a percentage.
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Re-review verdict: changes needed
Head 02dcf74b5edc5494b01a58fde609bba0b6dada4d, compared with previously reviewed 04028ec3. The new commit adds face-aware overlay placement; it does not change publishing file selection.
Prior P2 remains: src/publish/cli.ts:96–98 still ignores <id>-captioned.mp4. Re-ran the isolated publishing reproduction with both plain and captioned outputs and no overlay: ok:true, no issues, and the upload sheet still selected plain S1.mp4. Creators taking the supported non-overlay path can upload without their generated burned captions. The previous review contains the repro and suggested selection fix.
Checks: independently reran the source suite with SHORTS_FFMPEG_TEST=1: 148 passed, 4 skipped; TypeScript passed. Reviewed the face-detection fallback and average-overlap placement changes. Changed TypeScript inputs were fetched at the exact new SHA and their Git blob hashes matched. Publishing bundle is unchanged from the prior reviewed head. No additional verified defect is claimed from the new placement work.
Limits: macOS Vision detection itself could not run on Linux; actual HyperFrames rendered placement was not visually verified. Current-head Verify run 42486663671 was still in progress when checked. Current shorts bundle parity was not independently established (the connector's large-file content was incomplete, so source tests were used instead). Prior interrupted security checks remain unverified and were not retried. No full security assurance or production pass is claimed. No code implementation, push, merge or deployment performed.
GitHub's author-account restriction prevents a formal REQUEST_CHANGES review, so this top-level comment records the explicit verdict.
Headings (hook, name tag, cards, end card) now always sit at the top, 48 px below the wordmark. Captions never go at the top: on single-person shots they sit low (1150 px to YouTube's bottom UI), as low as clears the chin, and on split screens they sit on the seam between the two people. Faces now only decide where low captions sit and when name tags and cards show. Heading face reports count only from the brows down, and suggest cropping to the speaker when a heading shows over a split screen. Context cards are documented as optional.
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Re-review verdict: changes needed
Reviewed e9edd322e496c61d7b3882869e1a00cce39551b1 against 02dcf74b. The new commit fixes heading positions at the top and captions low/on the split-screen seam; it does not address the prior publishing defect.
P2 still present: packages/youtube-skills/src/publish/cli.ts:96–98 ignores <id>-captioned.mp4. Fresh reproduction using the current committed publishing bundle (verified blob 662b525ec462f05ee4c6882c658f2dbfffed16ba) with plain and captioned clips but no overlay returned ok:true, no issues, and selected S1.mp4 for the upload sheet. Users taking the optional-no-overlay path can still publish the uncaptioned intermediate. See the original finding.
Independent checks: four new source-level placement assertions passed: crop caption top 1422px, unbranded heading top 236px, split caption top 909px, branded heading top 310px. Used exact-head overlay/transcript source with only the import filename adapted for Node's native TypeScript execution. Inspected the new source, tests and documentation. No further verified defect is claimed.
Limits: the previous full local checkout/dependencies were no longer present, so the full suite and TypeScript check were not rerun for this head; earlier-head totals are not presented as current passes. Current Verify CI 42504540612 was in progress at inspection. No macOS Vision execution, rendered-frame visual verification, full shorts-bundle parity or production test. Prior safety-interrupted security checks remain unverified and were not retried. No implementation, push, merge or deployment.
Recording the explicit verdict in a COMMENT review because GitHub disallows requesting changes on the connected account's own PR.
publish.mjs upload sends each short with resumable videos.insert, using publish.json's title, description (links and hashtags), and tags. It asks only for the youtube.upload scope through Google's loopback flow with PKCE, using the user's own Desktop app OAuth client, and saves the refresh token under ~/.config/youtube-shorts-publish with mode 600. Uploads are unlisted by default or private; public posting stays in YouTube Studio. It checks every short before uploading any, requires the made-for-kids declaration in publish.json, resumes interrupted uploads from the byte YouTube reports, records finished uploads in publish/uploads.json so reruns skip them, and has --dry-run.
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Re-review verdict: changes needed
Reviewed 08481d2ab30a943c6148a2480a05c185128ffa2c and its new upload/OAuth code, tests and publishing workflow against e9edd322.
P2 persists and now affects direct uploads
In packages/youtube-skills/src/publish/cli.ts, both kit's file selector and the new upload candidate list choose overlay then plain MP4, skipping the burned-captioned deliverable. Fresh tests against the exact committed publishing bundle (blob b1130fedc45ec0de5ecbae9638bd40081a582ec2, hash verified) with S1.mp4 plus S1-captioned.mp4, no overlay:
kitreturnsok:trueand points the upload sheet atS1.mp4.upload --dry-runreturnsok:trueand plansS1.mp4.upload --variant captioned --privacy private --dry-runcorrectly selectsS1-captioned.mp4.
The explicit variant is a workaround, not a fix for the default workflow. Users proceeding from an ordinary captioned render without overlays can now directly upload the uncaptioned intermediate. Prefer overlay → captioned → plain consistently for the default selection and add both kit and upload regression tests. This extends the impact of the existing finding, rather than introducing a duplicate issue.
Verification
- Above three selection checks completed locally with a synthetic video and no Google requests.
- Public privacy is rejected with
INVALID_INPUTas intended. - Five independent mock-only upload assertions passed: completed result, resumption byte range, remainder length, private metadata, and quota error classification. All fetches were replaced with in-memory responses; no credentials or uploads were sent.
- Inspected the new unit tests and OAuth state/PKCE/scope handling; did not perform real consent or acquire/store any real credentials.
- Exact-head Verify run 42507979893 was in progress when inspected. Full local suite/typecheck and shorts bundle parity remain unverified in the replacement workspace. Earlier-head suite totals are not current-head results.
- Prior safety-interrupted security checks were not retried and remain unverified. No full security assurance, actual YouTube upload, macOS Vision or rendered HyperFrames visual pass is claimed.
No code implementation, push, merge or deployment. COMMENT records the explicit changes-needed verdict because the connected author's account cannot submit REQUEST_CHANGES on its own PR.
kit and upload each chose the overlay render, then the plain cut, skipping <id>-captioned.mp4, so a captioned render without overlays could upload the uncaptioned intermediate. Both now share one picker: overlay, then captioned, then plain, with --variant overriding. A plain cut, which has no captions on screen, now draws a warning in kit and upload. kit also takes --variant so its sheet and cover match the uploaded file. Regression tests cover the picker order, a variant, and kit and upload --dry-run against real video files.
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Re-review verdict: approve
Reviewed head 68f6600d572ad18b7129c8b4b605ee1175e0f816 against 08481d2a. The prior P2 publishing-selection finding is fixed in this scoped retest. No remaining material finding identified in the reviewed change.
The shared picker in packages/youtube-skills/src/publish/cli.ts:107–122 now consistently prefers overlay → captioned → plain. Both kit and upload use it, and kit accepts the same explicit variant as upload. This prevents the supported non-overlay captioned workflow from silently choosing the uncaptioned intermediate.
Independent verification
- Fetched the exact committed publishing bundle and verified its Git blob hash
ab756c0b8d2814f48e304dcdf81507f84b30cd5f. - 12 independent CLI cases passed against real synthetic MP4 files: captioned-over-plain, overlay preference, plain-only warning, explicit variant, missing variant without fallback, and missing render, each for
kitandupload --dry-run. Also verified kit sheets name the selected file. - Public privacy, empty variant, and an unknown
--onlyID were rejected withINVALID_INPUT. - Inspected the added regression tests, source changes and documentation. No Google requests, real credentials, or uploads were used in these checks.
- Verify run 42516331686 is now successful. Initial Dashboard E2E(other) failed at
platform/test/e2e/admin-access.spec.ts:41because the credit input was not focused after 10 seconds; one targeted retry passed. That is recorded as a transient CI failure, not evidence of a new live defect or a proven root cause. The other initial jobs passed. CI runs on the PR merge ref2977bfeccombining this head with base42177711. - Skill CI logs show 161 passed / 8 skipped, with TypeScript and bundle parity checks passing. These are CI results, not a claim that I reran the full suite locally; the independent CLI matrix covers the new file-selection paths skipped by the FFmpeg-gated CI cases.
Limits
No actual YouTube upload/consent, live extraction, Whisper installation, macOS Vision execution, or full HyperFrames visual verification. Prior safety-interrupted security checks remain unverified and were not retried; this is not full security assurance. Approval is a code-review verdict, not a production verification result.
COMMENT records the explicit approval verdict because the connected account is the PR author. No implementation, push, merge or deployment performed.
overlay.hookImage puts a round photo beside the hook (the hook text drops to 60 px to fit), copied into each overlay project from a local file. The selection reference now covers writing teaser and late-start hook variants per short, wording hooks for strangers (credentials and recognisable roles over bare names, questions for disputed claims), comparing only the openings, and fully rendering just the chosen variants. The overlay reference documents hookImage and its licensing and credit rules.
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Re-review verdict: approve
Reviewed 8ee68ef6f4934ee4d13337c96d7b115985180077 against approved 68f6600d. This delta adds an optional local hook photo, its generated HTML/animation, and hook-variant/licensing guidance. No new material defect identified in the reviewed delta.
Verification
- Inspected the image-copy path in
src/shorts/cli.ts:625–634, overlay generation and new test, surrounding EDL handling, and changed guidance. - 12 independent generated-HTML assertions passed using this head's overlay source through Node's native TypeScript execution (only the runtime transcript import filename adapted). Covered photo markup, 60px side text, image entrance and shared exit animation, absent-photo and absent-hook behavior, escaped alt text, and existing crop/split caption/header positions.
- The publishing bundle remains blob
ab756c0b8d2814f48e304dcdf81507f84b30cd5f, byte-identical to the previously independently verified caption-selection fix. That prior finding remains resolved; the earlier 12 CLI cases were not rerun in this replacement workspace. - Verify run 42571954262: all eight jobs passed, without a retry in this review. CI tested merge ref
7ada4aa4combining this head with base42177711. Skill CI reports 162 passed / 8 skipped, TypeScript and bundle parity passed. These are CI results, distinct from my independent assertions.
Verification limits
The local Chromium preview could not launch: socket() failed: Operation not permitted, including a permitted escalation retry. Therefore actual pixel layout, fonts, long-hook wrapping and photo rendering are not visually verified; HTML assertions are not a rendered-video pass. No actual YouTube upload/consent, live extraction, Whisper installation, macOS Vision execution or full HyperFrames run. Prior safety-interrupted security checks remain unverified and were not retried. No full security assurance or production result is claimed.
COMMENT records the explicit approval because the connected account is the PR author. No implementation, push, merge or deployment performed.
The end card now plays for 2.8 s after the last word instead of covering it: the final frame freezes and blurs, the source video's thumbnail lands, a tap presses its play button, and a chip points at the related-video link. Sound effects come from HyperFrames' bundled library, resolved offline at overlay time, so the skill ships no audio files. Shorts can now list zooms (snap, push, punch) centred on the speaker's face, and scenes: hand-made HyperFrames sub-compositions mounted as a bottom-half panel or a full-frame overlay, optionally growing from one to the other. Captions move to the seam under a bottom scene. Scene files are copied into the project, so rerunning overlay keeps them. references/motion.md sets the rules: animation in the bottom half or over the whole frame, never in a corner; no text cards; about one zoom every 10 s; and the scene-file pitfalls found while porting the case study. Also stops the last caption (and SRT) from picking up the first word of the next sentence when the snapped out-edge reaches it.
… splits
Sound now goes only where it lands: snap zooms are silent unless a zoom
sets "sound": true, punch-ins keep their bass hit, and scenes get a
whoosh only as they open ("enterSound": false drops it, "exitSound":
true adds a closing one). Expanding a bottom scene is silent.
A bottom scene that starts exactly on a cut to a split screen now
appears with the cut instead of sliding in. One that starts just after
such a cut, where the other person would only flash up, is reported
with the time to use: about 2 s after the cut, so the podcast host gets
screen time, or on the cut.
motion.md gains a Sound section and the host-visibility guidance.
devhims
left a comment
devhims
left a comment
There was a problem hiding this comment.
Re-review verdict: changes needed
Reviewed f7ef3317f7e2665a246f7cf457d9191d8f909102, including both commits since approved 8ee68ef6, the updated description, source/tests, motion guidance and bundle.
P2: A zoom admitted before a camera cut continues across the split screen
packages/youtube-skills/src/shorts/overlay.ts:631–649 checks the layout only at startSec + 0.05, then animates the shared #cam wrapper through the whole hold/release. It never clips or resets a zoom at a later cut into stack.
Independent reproduction: call buildOverlay for a 10-second short with a crop segment from 0–2s, a stack segment from 2–10s, and overlay.zooms: [{atSec: 1.5}]. The generated timeline contains:
tl.to("#cam", { scale: 1.16, duration: 0.14, ease: "power3.out" }, 1.5);
tl.to("#cam", { scale: 1, duration: 0.4, ease: "power2.inOut" }, 2.84);There is no zoom/split warning. Thus the split inherits 1.16× scaling from 2–2.84s and remains scaled during release until 3.24s. A control starting the zoom at 2.1s correctly skips it and reports the split. The guard therefore protects only zooms that start on a split, not those overlapping it.
Expected: keep stack segments unzoomed, by rejecting/truncating the intersecting zoom or resetting at the layout boundary, and cover this transition with a regression test. Impact: a normal punchline immediately before a camera cut can crop the carefully framed two-person composition despite the advertised split-screen protection. This is verified in the generated timeline, not a claim that I visually inspected the final rendered video.
Verification and prior findings
- Fresh isolated checkout at the reviewed SHA; independently built the library and ran
SHORTS_FFMPEG_TEST=1 npm testfor the skills: 172 passed, 4 skipped. This includes real FFmpeg rendering/end-frame extraction and mocked thumbnail/sound/upload boundaries. TypeScript and bundle parity checks passed. - Independently reproduced the cross-cut case and the starts-on-split control from exact-head source; only the runtime transcript import filename was adapted for Node native TypeScript execution.
- Publishing selection regressions passed again in the full suite, including captioned-over-plain for kit and dry-run upload. The prior caption-selection P2 remains fixed.
- Verify CI 42597567036 is successful. Green tests do not cover the cross-cut zoom above.
Limits: no full HyperFrames video/visual playback, live YouTube upload/consent, live extraction, macOS Vision or real sound-library resolution. Network-dependent thumbnail/sound/upload tests used supplied mocks. Prior safety-interrupted security checks remain unverified and were not retried. No production pass or full security assurance claimed.
COMMENT records changes-needed because the connected account is the PR author. No implementation, push, merge or deployment.
Summary
Two new published skills. Both bundle from
packages/youtube-skillslikeyoutube-ctx, and neither calls the hosted platform.Overlay placement:
Motion and end card:
references/motion.mdholds the rules from the case-study review: animation only in the bottom half or over the whole frame, no text cards, about one zoom every 10 s, sound only where it lands, and about 2 s of the host on a split before a scene covers them.Sources, best first: the creator's master, the user's yt-dlp (
fetch), YouTube's 360p stream (--from-youtube, review only). The library itself still reads only what YouTube serves without sign-in; PO-token handling stays inside yt-dlp, on the user's machine.Evidence
Case study: the 77-minute episode
dD1zZE15irQ, five shorts rendered at 1080p with overlays.--from-youtubegave 360p review copies. Cut edges came straight from caption times. Each camera change inside a cut re-encoded its own audio, causing a small dip and about 50 ms of drift per segment. No captions meant no workflow.After:
rendersnapped every cut edge into a pause except two where speech runs straight through. Those two are flagged and get a longer fade.publish.mjs kitreturnsok: truefor all five shorts.publish.mjs uploadis tested only against mocked Google endpoints: consent, refresh, resuming after a dropped connection, quota errors, and skip on rerun. It hasn't uploaded to a real channel yet. On the real project it stops, as designed, untilmadeForKidsis declared.edl.jsonfields, two scene files andshorts.mjs overlay. The rebuild matches the hand-made version beat for beat in sampled frames, at the same loudness (−13.9 LUFS, true peak −1.0 dBTP, no clipping). It runs 43.2 s against 43.07 s because the hand-made cut also trimmed the onset of the next sentence's first word; the rebuild no longer captions that word.motion.md: GSAP measures SVG boxes fortransformOriginbefore a scene is laid out, so scaled SVG nodes drifted about one radius off their lines. Nodes built as HTML elements sit exactly on them.npm --prefix packages/youtube-skills run verifyon this branch: 165 passed, 11 skipped, bundles current, typecheck clean. WithSHORTS_FFMPEG_TEST=1: 172 passed, 4 skipped (the live-YouTube and watch-FFmpeg tests).Merge Danger
Door: two-way
New skill folders and new source directories. The
youtube-ctxbundles are byte-identical after rebuilding.Blast Radius: small
The new
uploadcommand posts to a YouTube channel, but only when the user runs it with their own OAuth client and consent, and only as unlisted or private.npx skills add devhims/video2ctxlists two more skills.youtube-shortsneeds FFmpeg and npx/HyperFrames for overlays and missing captions, and it usesyoutube-ctxfor overlay research. Overlay renders fetch GSAP from jsDelivr and fonts from Google Fonts.overlayfetches the source thumbnail fromi.ytimg.comand runsnpx hyperframes media-use resolvefor sound effects; either failing only warns. HyperFrames renders with the first FFmpeg onPATH, which needs libx264 (a minimal Homebrew build fails with "Unrecognized option 'preset'");overlay.mdsays so.brand.endCardchanged from a list of lines to{ heading, channel, cta }, unmerged so far. The docs site (docs/skills/) and the root README don't mention the new skills yet.