Skip to content

Incorporate app-text-slicer functionality into text_document_helper as the app is discontinued #417

Closed

Description

@keighrim

New Feature Summary

After reviewing the implementation of app-text-slicer, I propose deprecating the standalone app and porting its logic (https://github.com/clamsproject/app-text-slicer/blob/269ffd40ee3a220b3cf3664701f0f8ee8e8def39/app.py) directly into text_document_helper within the SDK.

The core text slicing functionality is straightforward to absorb into the helper utility. Slicing text is primarily useful during internal app development rather than serving as an end-user pipeline step, making a standalone CLAMS app unnecessary.

Related

No response

Alternatives

No response

Additional context

No response

Activity

  1. keighrim commented on Nov 5, 2026

    @keighrim
    MemberAuthor

    #418 covers the core feature of app-text-slicer (the transcript text of each TimeFrame) with the existing Mmif.get_annotations_between_time, so the SDK needs no new function. Calling it with at_types=[AnnotationTypes.Token] gives the Tokens of a time range. Selecting TimeFrames by label, which the app did with containLabel, is already Mmif.get_annotations_of_labels. text_document_helper.slice_text, which joins the Tokens of every transcript into one string, is deprecated and will be removed in 2.0.0.

    While doing so, I fixed these bugs in get_annotations_between_time:

    1. Annotations that start or end exactly on the range boundary were left out, so slicing a segment dropped its first and last words (and a short segment could come back empty).
    2. The time unit was read from each view's TimeFrame entry in contains, which fails on a view of TimePoints only, and a view holding both TimeFrames and TimePoints was searched twice, so its matches came back twice.
    3. A Token aligned to more than one anchor in the range (for example, a word TimeFrame plus a forced aligner's TimeFrame) was returned once per anchor.

    I also clarified the behavior:

    1. The range includes its start and excludes its end, and any overlap counts. A zero-length anchor (a TimePoint, or a TimeFrame with equal start and end) belongs to the range it falls in. Ranges that meet end to end share no annotation.
    2. Every time anchor in the MMIF must be in integer milliseconds. One that declares another timeUnit, or has a non-integer time, raises ValueError instead of being converted (the app passed times on as milliseconds whatever the unit).
    3. Query times in units other than milliseconds (the time_unit argument) still work with a DeprecationWarning until 2.0.0; frames raise.
  2. moved this from Todo to Done in infraon Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ✨NNew feature or request

    Type

    No type

    Projects

    • Status
      Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions