diff --git a/clams/app/__init__.py b/clams/app/__init__.py index 3f1bc2b..3c8d2cc 100644 --- a/clams/app/__init__.py +++ b/clams/app/__init__.py @@ -240,8 +240,13 @@ def annotate(self, mmif: Union[str, dict, Mmif], **runtime_params: List[str]) -> t = datetime.now() with warnings.catch_warnings(record=True) as ws: annotated, cuda_profiler = self._profile_cuda_memory(self._annotate)(mmif, **refined) + # turn pending document properties into `Annotation`s now, so a + # view whose only output is document properties is not empty; + # `serialize()` repeats this later, skipping values already written + annotated.generate_capital_annotations() new_views = [v for v in annotated.views if v.id not in existing_view_ids] - is_empty = lambda v: len(v.annotations) == 0 and not v.has_error() and not v.has_warnings() + # an app-set error or warnings view has no annotations, but is kept + is_empty = lambda v: not (v.has_content() or v.has_error() or v.has_warnings()) for v in new_views: if is_empty(v): del annotated.views._items[v.id] # no public API to remove a view (mmif-python) @@ -397,7 +402,9 @@ def sign_view(self, view: View, runtime_conf: dict) -> None: """ view.metadata.app = str(self.metadata.identifier) if self.metadata.app_tags: - view.metadata.set_additional_property('appTags', list(self.metadata.app_tags)) + # item assignment stores the tags in the declared `appTags` field + # when mmif-python has one, and as an additional property otherwise + view.metadata['appTags'] = list(self.metadata.app_tags) if self._RAW_PARAMS_KEY in runtime_conf: for k, v in runtime_conf.items(): if k == self._RAW_PARAMS_KEY: diff --git a/clams/develop/__init__.py b/clams/develop/__init__.py index cb682e4..0bc4601 100644 --- a/clams/develop/__init__.py +++ b/clams/develop/__init__.py @@ -2,12 +2,15 @@ import pathlib import re import shutil +import warnings from string import Template from typing import List import clams update_tmp_suffix = '.tmp' +# Recipes with a ``deprecated`` message are left out of the default recipe +# set and warn when baked on request. available_recipes = { 'app': { 'description': 'Skeleton code for a CLAMS app', @@ -20,12 +23,18 @@ 'targetdir': '.github', }, 'utl-tf': { - 'description': 'Local helper module for iterating TimeFrames and collecting per-TF frame tasks ' + 'description': 'DEPRECATED. Local helper module for iterating TimeFrames and collecting per-TF frame tasks ' '(baked into ``utils/timeframe.py``; backend-agnostic, safe to edit/delete)', 'sourcedir': 'utl-tf', 'targetdir': 'utils', + 'deprecated': 'The `utl-tf` recipe is deprecated and is no longer baked by default. mmif-python now ' + 'provides its functions: use `Mmif.get_annotations_of_labels()` to filter TimeFrames by ' + 'label, and `mmif.utils.video_document_helper.iter_timeframe_images()` to sample images ' + 'per TimeFrame with the TimePoint each image came from.', }, } +default_recipes = [name for name, recipe in available_recipes.items() + if 'deprecated' not in recipe] class CookieCutter(object): @@ -37,7 +46,13 @@ def __init__(self, name: str, outdir: str, recipes: List[str]): if recipes: self.recipes = recipes else: - self.recipes = list(available_recipes.keys()) + self.recipes = list(default_recipes) + for recipe in self.recipes: + message = available_recipes.get(recipe, {}).get('deprecated') + if message: + # FutureWarning, unlike DeprecationWarning, is shown to CLI + # users by default + warnings.warn(message, FutureWarning, stacklevel=2) def tokenize_rawname(self): word_pat = re.compile('[A-Z][a-z]+|[0-9A-Z]+(?=[A-Z][a-z])|[0-9A-Z]{2,}|[a-z0-9]{2,}|[a-zA-Z0-9]') @@ -158,7 +173,7 @@ def prep_argparser(**kwargs): action='store', nargs='+', default=[], - help=f"Pick recipes to bake. DEFAULT: {list(available_recipes.keys())}" + help=f"Pick recipes to bake. DEFAULT: {default_recipes}" ) parser.add_argument( '-n', '--name', diff --git a/clams/develop/templates/utl-tf/timeframe.py.template b/clams/develop/templates/utl-tf/timeframe.py.template index 5dc9585..1a380eb 100644 --- a/clams/develop/templates/utl-tf/timeframe.py.template +++ b/clams/develop/templates/utl-tf/timeframe.py.template @@ -1,6 +1,14 @@ """ TimeFrame iteration / frame-sampling helpers, local to this app. +DEPRECATED: mmif-python now provides these functions. Use +``Mmif.get_annotations_of_labels()`` in place of +:func:`iter_timeframes`, and +``mmif.utils.video_document_helper.iter_timeframe_images()`` in place of +:func:`collect_timeframes_of_interest` (it yields +``(time_frame, images, timepoint_ids)``, and wrapping those into +``clams.app.PromptTask`` is left to the app). + Generated by ``clams develop -r utl-tf``. The code in this file is part of your app, not the SDK; edit it freely, refactor as needed, or delete the whole file if your app does not iterate TimeFrame annotations. diff --git a/pyproject.toml b/pyproject.toml index 28989b7..d0731a5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -19,7 +19,7 @@ classifiers = [ "Programming Language :: Python :: 3 :: Only", ] dependencies = [ - "mmif-python==1.5.4", + "mmif-python==1.6.0", "Flask>=2", "Flask-RESTful>=0.3.9", "gunicorn>=20", diff --git a/tests/test_clamsapp.py b/tests/test_clamsapp.py index a3ea819..829fe23 100644 --- a/tests/test_clamsapp.py +++ b/tests/test_clamsapp.py @@ -336,11 +336,6 @@ def test_annotate(self): self.assertEqual(len(v.metadata.parameters), 1) # 'pretty` parameter was passed out_mmif = Mmif(self.app.annotate(out_mmif)) self.assertEqual(len(out_mmif.views), 4) - views = list(out_mmif.views) - # insertion order is kept - self.assertEqual(views[0].metadata.timestamp, views[1].metadata.timestamp) - self.assertEqual(views[2].metadata.timestamp, views[3].metadata.timestamp) - self.assertTrue(views[1].metadata.timestamp < views[2].metadata.timestamp) def test_run_id(self): # first run @@ -538,6 +533,53 @@ def _annotate(self, mmif, **kwargs): self.assertTrue(view.has_error()) self.assertFalse(view.has_warnings()) + def test_self_reported_warning_view_is_not_pruned(self): + class WarningClamsApp(clams.app.ClamsApp): + def _appmetadata(self): + pass + def _annotate(self, mmif, **kwargs): + new_view = mmif.new_view() + self.sign_view(new_view, kwargs) + new_view.metadata.warnings = ['simulated warning'] + return mmif + out_mmif = Mmif(WarningClamsApp().annotate(self.in_mmif)) + self.assertEqual(len(out_mmif.views), 1) + view = next(iter(out_mmif.views)) + self.assertEqual(view.metadata.warnings, ['simulated warning']) + + def test_document_properties_stay_in_own_view(self): + # an app whose only output is document properties keeps its own view; + # the generated `Annotation`s must not go to an earlier app's view + class DocPropsClamsApp(clams.app.ClamsApp): + def _appmetadata(self): + pass + def _annotate(self, mmif, **kwargs): + new_view = mmif.new_view() + self.sign_view(new_view, kwargs) + mmif['v1'].add_property('duration', 1234) + mmif['t1'].add_property('author', 'me') + return mmif + in_mmif = Mmif(self.in_mmif) + earlier = in_mmif.new_view() + earlier.metadata.app = 'http://apps.clams.ai/earlier-app/v1' + earlier.new_annotation(AnnotationTypes.TimeFrame, start=0, end=10) + app = DocPropsClamsApp() + out_mmif = Mmif(app.annotate(in_mmif.serialize())) + self.assertEqual( + 0, len(list(out_mmif[earlier.id].get_annotations(AnnotationTypes.Annotation)))) + own_views = [v for v in out_mmif.views + if v.metadata.app == str(app.metadata.identifier)] + self.assertEqual(len(own_views), 1) + self.assertFalse(own_views[0].has_warnings()) + # one `Annotation` per document, even though `serialize()` converts + # the same pending properties a second time + anns = list(own_views[0].get_annotations(AnnotationTypes.Annotation)) + self.assertEqual(len(anns), 2) + props = {a.get_property('document'): + {k: v for k, v in a.properties.items() if k not in ('id', 'document')} + for a in anns} + self.assertEqual({'v1': {'duration': 1234}, 't1': {'author': 'me'}}, props) + def test_gpu_mem_fields_default_zero(self): """GPU memory fields default to 0.""" metadata = AppMetadata( diff --git a/tests/test_develop.py b/tests/test_develop.py index 95a8c02..4d58ed0 100644 --- a/tests/test_develop.py +++ b/tests/test_develop.py @@ -7,6 +7,7 @@ import sys import tempfile import unittest +import warnings from pathlib import Path from unittest import mock @@ -16,12 +17,25 @@ from clams.develop import CookieCutter +class TestRecipeDefaults(unittest.TestCase): + + def test_default_recipes_leave_out_deprecated(self): + with warnings.catch_warnings(): + warnings.simplefilter('error') + cutter = CookieCutter( + name='app', outdir=tempfile.gettempdir(), recipes=[]) + self.assertTrue(cutter.recipes) + self.assertNotIn('utl-tf', cutter.recipes) + + class TestUtlTfRecipe(unittest.TestCase): def setUp(self): tmp = tempfile.TemporaryDirectory() self.addCleanup(tmp.cleanup) - with contextlib.redirect_stdout(io.StringIO()): + # `utl-tf` is deprecated, and asking for it warns + with self.assertWarns(FutureWarning), \ + contextlib.redirect_stdout(io.StringIO()): CookieCutter( name='tfapp', outdir=tmp.name, recipes=['utl-tf']).bake() path = Path(tmp.name) / 'tfapp' / 'utils' / 'timeframe.py'