Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion doc/api_reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -80,12 +80,15 @@ Queries
:members:
:show-inheritance:

.. autoclass:: fairgraph.queries.Regex
.. autoclass:: fairgraph.queries.Contains
:show-inheritance:

.. autoclass:: fairgraph.queries.Equals
:show-inheritance:

.. autoclass:: fairgraph.queries.Regex
:show-inheritance:

Utility classes and functions
=============================

Expand Down
24 changes: 16 additions & 8 deletions doc/queries.rst
Original file line number Diff line number Diff line change
Expand Up @@ -177,25 +177,33 @@ For example, to see only datasets whose name contain the phrase 'patch-clamp'::
e.g., ``DatasetVersion.property_names``
or consult the inline help (``help(omcore.DatasetVersion)``).

A plain string matches any property value that *contains* it.
To require the whole value to match, pass an :class:`Equals` instead::
For a property containing text, the filter value says how the text is to be matched:

from fairgraph import Equals
- :class:`Contains`: the property value must *contain* the text. This is the default:
a plain string means the same, so ``full_name="patch-clamp"`` above is short for
``full_name=Contains("patch-clamp")``.
- :class:`Equals`: the whole property value must match.
- :class:`Regex`: the property value must match a regular expression.

In all three cases, case is ignored. For example::

from fairgraph import Contains, Equals

datasets = Dataset.list(client, short_name=Contains("FOO"))
datasets = Dataset.list(client, short_name=Equals("FOO"))

This finds datasets whose short name is "FOO", but not those named "FOO-BAR".
Case is ignored, but whitespace is not, so "FOO " would not match.
The first finds datasets whose short name is "FOO", "foo" or "FOO-BAR"; the second finds those whose short name
is "FOO" or "foo", but not "FOO-BAR". Whitespace is not ignored, so "FOO " would not match :class:`Equals`.
For a case-sensitive search on a property such as ``name``, see :meth:`by_name` below.

For a more flexible search, pass a :class:`Regex` instead of a plain string::
For a more flexible search, use a :class:`Regex`::

from fairgraph import Regex

datasets = Dataset.list(client, full_name=Regex("^Whole cell patch-clamp"))

The KG applies the pattern as a *search*: a pattern with no anchors matches anywhere in the property value,
so ``Regex("patch-clamp")`` finds the same datasets as the plain string ``"patch-clamp"``.
so ``Regex("patch-clamp")`` finds the same datasets as ``Contains("patch-clamp")``.
The ``^`` in the example above is what restricts the match to names that *begin* with the phrase;
``$`` likewise anchors to the end.

Expand Down Expand Up @@ -228,7 +236,7 @@ For example, to find software versions developed by both a Baggins and an Oakens

versions = SoftwareVersion.list(client, developers__family_name=["Baggins", "Oakenshield"])

Each item in the list may be a plain string ("contains"), an :class:`Equals` or a :class:`Regex`,
Each item in the list may be a :class:`Contains` (or a plain string), an :class:`Equals` or a :class:`Regex`,
or, for a linked property, an object or its id::

from fairgraph.openminds.publications import Chapter
Expand Down
5 changes: 5 additions & 0 deletions doc/release_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ Version 0.16.0
New features
------------

- A filter value for a text property can now also be given as a :class:`~fairgraph.queries.Contains`,
alongside :class:`~fairgraph.queries.Equals` and :class:`~fairgraph.queries.Regex`, so that all three ways of
matching text can be written explicitly. A plain string means the same as a :class:`~fairgraph.queries.Contains`,
so existing code is unaffected.

- ``File.download()`` now checks the downloaded file against the hash recorded in the Knowledge Graph
(MD5, SHA1, or any other algorithm available in :mod:`hashlib`), and raises :exc:`ValueError` if it does
not match. Pass ``check_hash=False`` to skip the check, e.g. for very large files.
Expand Down
2 changes: 1 addition & 1 deletion fairgraph/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
from .embedded import KGEmbedded
from .kgproxy import KGProxy
from .kgquery import KGQuery
from .queries import Regex, Equals
from .queries import Regex, Equals, Contains
from .collection import Collection
from . import client, errors, openminds, utility

Expand Down
10 changes: 7 additions & 3 deletions fairgraph/node.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
from .kgproxy import KGProxy
from .kgquery import KGQuery
from .queries import (
Contains,
Equals,
QueryProperty,
get_query_properties,
Expand Down Expand Up @@ -609,9 +610,12 @@ def _existence_query_value(property_name: str, value: Any, existence_match: str)
)
if query_val is None:
raise CannotBuildExistenceQuery(f"Required value for '{property_name}' is missing")
if isinstance(query_val, str) and existence_match == "equals":
# match exactly, so that we don't identify this object with one whose value merely contains ours
query_val = Equals(query_val)
if isinstance(query_val, str):
if existence_match == "equals":
# match exactly, so that we don't identify this object with one whose value merely contains ours
query_val = Equals(query_val)
else:
query_val = Contains(query_val)
return query_val


Expand Down
33 changes: 29 additions & 4 deletions fairgraph/queries.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@

class Regex(str):
"""
A regular expression, for use as a filter value in place of a plain string.
A regular expression, for use as a filter value.

Where a plain string filters with the KG's "CONTAINS" operator, a `Regex` filters with "REGEX".
Where a `Contains` (or a plain string) filters with the KG's "CONTAINS" operator, a `Regex` filters with "REGEX".
The KG applies the pattern as a search rather than a full match,
so a pattern with no anchors matches anywhere in the property value; use "^" and "$" to anchor it.
Matching is always case-insensitive, whatever the pattern says.
Expand Down Expand Up @@ -64,12 +64,32 @@ def __new__(cls, pattern: str):
return super().__new__(cls, pattern)


class Contains(str):
"""
A string, for use as a filter value, that must be contained in the property value.

A `Contains` filters with the KG's "CONTAINS" operator, so `Contains("FOO")` matches "FOO-BAR".
The KG ignores case when matching, so `Contains("foo")` also matches "FOO-BAR".
This is the default: a plain string given as the filter value for a text property means the same.

As with any filter value, a list of `Contains` values matches only if every item is matched.

Args:
value (str): the text the property value must contain.

Example:
>>> import fairgraph.openminds.core as omcore
>>> from fairgraph import Contains
>>> omcore.Dataset.list(client, short_name=Contains("FOO")) # the same as short_name="FOO"
"""


class Equals(str):
"""
A string, for use as a filter value in place of a plain string, that must match the
A string, for use as a filter value, that must match the
property value exactly rather than as a substring.

Where a plain string filters with the KG's "CONTAINS" operator, an `Equals` filters with "EQUALS".
Where a `Contains` (or a plain string) filters with the KG's "CONTAINS" operator, an `Equals` filters with "EQUALS".
The whole property value must match, so `Equals("FOO")` does not match "FOO-BAR".
The KG ignores case when matching, but not whitespace, so `Equals("FOO")` does match "foo",
but not "FOO ".
Expand Down Expand Up @@ -536,6 +556,8 @@ def get_query_filter_property(property, context, filter: Any, name: Optional[str
op = "REGEX"
elif isinstance(filter, Equals):
op = "EQUALS"
elif isinstance(filter, Contains):
op = "CONTAINS"
elif property.types[0] in (int, float, bool, datetime, date):
op = "EQUALS"
else:
Expand Down Expand Up @@ -630,6 +652,9 @@ def is_valid(val):
# todo: consider using client.uri_from_uuid()
# would require passing client as arg
filter_item = f"https://kg.ebrains.eu/api/instances/{item}"
elif type(item) is str and str in property.types:
# a plain string for a text property: make the default explicit
filter_item = Contains(item)
else:
filter_item = item
filter_items.append(filter_item)
Expand Down
4 changes: 2 additions & 2 deletions test/test_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
from fairgraph.errors import CannotBuildExistenceQuery
from fairgraph.base import ErrorHandling
from fairgraph.utility import ActivityLog
from fairgraph.queries import Equals
from fairgraph.queries import Contains, Equals
from .utils import MockKGResponse, mock_client

import pytest
Expand Down Expand Up @@ -489,7 +489,7 @@ def test_build_existence_query__with_lists__contains(self, list_existence_query)
obj = self._construct_object_all_properties()
query = obj._build_existence_query("contains")
assert query["a_required_list_of_strings"] == ["banana", "pear"]
assert not any(isinstance(item, Equals) for item in query["a_required_list_of_strings"])
assert all(isinstance(item, Contains) for item in query["a_required_list_of_strings"])

def test_build_existence_query__with_empty_list(self, list_existence_query):
obj = self._construct_object_all_properties()
Expand Down
52 changes: 51 additions & 1 deletion test/test_queries.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from openminds import IRI
import pytest
from kg_core.request import Stage, Pagination
from fairgraph.queries import Query, QueryProperty, Filter, PathElement, Regex, Equals
from fairgraph.queries import Query, QueryProperty, Filter, PathElement, Regex, Equals, Contains
import fairgraph.openminds.core as omcore
import fairgraph.openminds.controlled_terms as omterms
from .utils import kg_client, mock_client, skip_if_no_connection
Expand Down Expand Up @@ -779,6 +779,56 @@ def test_equals_is_exported_from_the_package():
assert fairgraph.Equals is Equals


def test_contains_marker_filters_with_contains_operator_like_a_plain_string(mock_client):
queries = [
omcore.Dataset.generate_query(client=mock_client, space=None, filters={"short_name": value})
for value in (Contains("FOO"), "FOO")
]
assert queries[0] == queries[1]
filters = [prop["filter"] for prop in queries[0]["structure"] if prop.get("propertyName", None) == "Qshort_name"]
assert filters == [{"op": "CONTAINS", "value": "FOO"}]


def test_contains_is_exported_from_the_package():
import fairgraph

assert fairgraph.Contains is Contains


def test_normalize_filter_makes_plain_strings_for_text_properties_explicit():
instance_id = "https://kg.ebrains.eu/api/instances/00000000-0000-0000-0000-000000000001"
normalized = omcore.DatasetVersion.normalize_filter(
{
"short_name": "FOO",
"version_identifier": Equals("1.0"),
"version_innovation": Regex("^first"),
"is_new_version_of": instance_id,
"release_date": date(2024, 1, 1),
}
)
# a plain string for a text property becomes a Contains; explicit filter values are kept as given
assert type(normalized["short_name"]) is Contains
assert type(normalized["version_identifier"]) is Equals
assert type(normalized["version_innovation"]) is Regex
# an id for a linked node, and a date, are not text, so they are left as plain strings
assert type(normalized["is_new_version_of"]) is str
assert type(normalized["release_date"]) is str


def test_list_of_mixed_filter_values_gives_one_filter_per_item(mock_client):
query = omcore.Dataset.generate_query(
client=mock_client,
space=None,
filters={"short_name": ["a", Contains("b"), Equals("c"), Regex("^d")]},
)
filters = sorted(
(prop["filter"]["value"], prop["filter"]["op"])
for prop in query["structure"]
if prop.get("propertyName", "").startswith("Qshort_name")
)
assert filters == [("^d", "REGEX"), ("a", "CONTAINS"), ("b", "CONTAINS"), ("c", "EQUALS")]


@pytest.mark.parametrize(
"cls, filters, path_expected",
[
Expand Down
Loading