Add few tweaks into the Rules/Languages/hu/definitions.yaml file, Added missing definitions.rs test file the test/Languages/hu directory, and added SimpleSpeak test directory the mroot and menclose tests

[hammera]

Hi @NSoiffer and @MartheGjelstad,

In this week I detected when we generated in april the hungarian first test suite directory, this directory doesn't contains final some newest tests into from the english directory.
Affected english file is the definition.rs file, I already localized these test, and doed few tweaks the hungarian definitions.yaml file with me showed this definitions.rs test.
So, this original test suite file is fantastic too, showed me few failures with I corrected the hungarian definitions.yaml file.

I added into the hungarian SimpleSpeak directory the menclose and mroot.rs files, so I extended the hungarian test case entries (not matter if have more tests the hungarian language, because hungarian language is more complex with english language).

I have got few questions with some definition tests related:
Factory level, the english language empty-set and set-difference tests into the vector part is commented out with a // mark, and the english language with the empty-set related definition is commented out a # part.
Why have this?

If I remove the // markup the following code, basic purpose I think following tests are need generate following hungarian language output, similar with english default commented out translated messages.
Original english tests:

fn set_tests() -> Result<()> {
    let tests = vec![
        ("set", "the set of x"),
        // ("set-difference", "set difference of x and y"),
        ("complement", "complement of x"),
        //("empty-set", "empty set"),

The interesting part is the set difference test, and the empty-set test.
Hungarian language need very similar output this tests, I show affected code:

fn set_tests() -> Result<()> {
    let tests = vec![
        ("set", "a halmaz x"),
        //("set-difference", "x és y halmaz különbsége"),
        ("complement", "komplemens x"),
        //("empty-set", "üres halmaz"),

The problem is following the affected hungarian definition tests if I remove // mark these affected hungarian set-difference and empty-test code lines, I see following failures:

---- Languages::hu::definitions::set_tests stdout ----
Error: MathCAT crash! Please report the following information: 'MathCAT crash! Please report the following information: 'assertion `left == right` failed: 
test with hu/ClearSpeak failed
  left: "x és y halmaz különbsége"
 right: "különbséghalmaz x"' at tests/common/mod.rs:55:0' at tests/common/mod.rs:61:0

Why cutted out the y part from the test part?
The empty-set definition into the hungarian definitions.yaml file contains following modified line, similar with english line:

    "empty-set":"function=üres halmaz",

The set-difference related definition I tried following form, because this definition is worked an another complex set difference test with an another code block the rust test file:

"set-difference":"function=különbséghalmaz || infix=mínusz || postfix=különbséghalmaza", # NOTE: not tested

The another affected set difference test is following, with need the különbséghalmaz form (this test passed right with this showed definition):

fn set_difference_basic() -> Result<()> {
    let expr = r#"
      <math>
        <mrow intent="set-difference($A,$B)">
          <mi arg="A">A</mi>
          <mo>&#x2216;</mo>
          <mi arg="B">B</mi>
        </mrow>
      </math>
    "#;

    test( "hu", "ClearSpeak", expr, "különbséghalmaz nagy a vessző, nagy b")?;

    Ok(())

English set-difference definitions is following in Rules/Languages/en/definitions.yaml file:

    "set-difference":"function=set difference; and || infix=minus", # NOTE: not tested

Attila

[hammera]

If I remove the // mark the empty-set test, I don't no why, but after the üres halmaz part Mathcat put an unneed x letter.
Look the failure:

---- Languages::hu::definitions::set_tests stdout ----
Error: MathCAT crash! Please report the following information: 'MathCAT crash! Please report the following information: 'assertion `left == right` failed: 
test with hu/ClearSpeak failed
  left: "üres halmaz"
 right: "üres halmaz x"' at tests/common/mod.rs:55:0' at tests/common/mod.rs:61:0

The wanted output only the üres halmaz (empty set text), because is an empty set is an empty set. :-):-)

Last, have an interesting crash in the definition root test related, with my knowledge is not enough to diagnose why happens if I remove the // mark the following test part:

fn functions_and_inverses_tests() -> Result<()> {
...
//("root", "gyök x"),

failures:

---- Languages::hu::definitions::functions_and_inverses_tests stdout ----
Error: MathCAT crash! Please report the following information: 'MathCAT crash! Please report the following information: 'Pattern match/replacement failure!
caused by: attempting replacement pattern: "default" for "math".
Replacement
[with:
variables: [name: ClearSpeak_Fractions="IfThenElse($Verbosity='Verbose' and $ClearSpeak_Fractions='Auto', 'EndFrac', $ClearSpeak_Fractions)"],[name: ClearSpeak_AbsoluteValue="IfThenElse($Verbosity='Verbose' and $ClearSpeak_AbsoluteValue='Auto', 'AbsEnd', $ClearSpeak_AbsoluteValue)"],[name: ClearSpeak_Roots="IfThenElse($Verbosity='Verbose' and $ClearSpeak_Roots='Auto', 'RootEnd', $ClearSpeak_Roots)"],[name: ClearSpeak_Matrix="IfThenElse($Verbosity='Verbose' and $ClearSpeak_Matrix='Auto', 'EndMatrix', $ClearSpeak_Matrix)"],[name: MatchingPause="false()"],[name: IsColumnSilent="false()"],
replace: [test: [ if: '"$MathRate = 100"' then:[""] else:[rate: 987654321.5[""]]]
]]
...due to matching the MathML


x


with the pattern
"."
The patterns are in /usr/src/mathcat/hammera_mathcat/mathcat/Rules/Languages/hu/SharedRules/default.yaml.

caused by: replacing inside 'with'
caused by: attempting replacement pattern: "default" for "root".
Replacement

• test: [ if: '"$Verbosity!='Terse'"' then:[t: "a"]]

• test: [ if: '"$ClearSpeak_Roots = 'PosNegSqRoot' or $ClearSpeak_Roots = 'PosNegSqRootEnd'"' then:

• test: [ if: '"parent::[(self::m:minus or self::m:plus) and count()=1]"' then:[bookmark: "parent/@id"
  ]]

• test: [ if: '"parent::m:minus"' then:[t: "negatív"] else:[t: "pozitív"]]

]

• test: [ if: '"[2][self::m:mn and not(contains(., '.'))]"' then _test:test: [ if: '"[2][.='2']"' then:[t: "négyzetgyök"]]
  test: [ if: '"*[2][.='3']"' then:[t: "köbgyök"]]
  test: [ else:
• "ToOrdinal(*[2])"
• t: "gyök"
  ]
  else:
• test: [ if: '"*[2][self::m:mi][string-length(.)=1]"' then:
• "*[2]"
• pronounce: pronounce: [text: '-edik'pronounce: [,ipa: 'ɛdik'pronounce: [,sapi5: 'edik'pronounce: [,eloquence: 'edik']

else:["*[2]"]]

• t: "gyök"
  ]

• test: [ if: '"$Verbosity!='Terse'"' then:[t: ""]]

• "*[1]"

• test: [ if: '"$ClearSpeak_Roots = 'RootEnd' or $ClearSpeak_Roots = 'PosNegSqRootEnd'"' then:

• pause: 200

• t: "gyök vége"

• pause: 400
  else _test:test: [ if: '"IsNode(*[1], 'simple')"' then:[pause: 200] else:[pause: 800]]
  ]

...due to matching the MathML

x

with the pattern
"."
The patterns are in /usr/src/mathcat/hammera_mathcat/mathcat/Rules/Languages/hu/ClearSpeak_Rules.yaml.

caused by: During replacement, no matching element found
' at tests/common/mod.rs:56:0' at tests/common/mod.rs:61:0

English test are produced into the tests/Languages/en/definitions.rs file with "root x" output.
The english definitions.yaml file are contains following entry into the root element related:

"root":"function=root",

The hungarian definitions.yaml file are similar:

"root":"function=root",

Attila

[hammera]

Note: I not fetch and rebase to the upstream main branch to my feature hu branch, I not want risking merge conflicts.
Translator audit output now looks following:

╭──────────────────────────────────────────────────────────────────────────────╮
│ MathCAT Translation Audit: HU                                                │
╰──────────────────────────────────────────────────────────────────────────────╯

  Comparing against en reference files
  Files to check: 12

────────────────────────────────────────────────────────────────────────────────
⚠ ClearSpeak_Rules.yaml
  en: 39 rules  →  hu: 42 rules
────────────────────────────────────────────────────────────────────────────────

  ≠ Rule Issues [3] (grouped by rule and issue type)
      • trig-power-argument-before-exponent-general ([mi, mn])
          Extra in Translation [1]
              • (line 263 in hu)
      • trig-power-grouped-argument-before-exponent-general ([mo, mrow])
          Extra in Translation [1]
              • (line 290 in hu)
      • trig-function-power-base-only (power)
          Extra in Translation [1]
              • (line 349 in hu)

────────────────────────────────────────────────────────────────────────────────
⚠ SimpleSpeak_Rules.yaml
  en: 23 rules  →  hu: 37 rules
────────────────────────────────────────────────────────────────────────────────

  ≠ Rule Issues [14] (grouped by rule and issue type)
      • trig-power-argument-before-exponent-general ([mi, mn])
          Extra in Translation [1]
              • (line 183 in hu)
      • trig-power-grouped-argument-before-exponent-general ([mo, mrow])
          Extra in Translation [1]
              • (line 210 in hu)
      • trig-function-power-base-only (power)
          Extra in Translation [1]
              • (line 269 in hu)
      • AfterPower-nested (power)
          Extra in Translation [1]
              • (line 278 in hu)
      • AfterPower-default (power)
          Extra in Translation [1]
              • (line 291 in hu)
      • nested-squared-or-cubed (power)
          Extra in Translation [1]
              • (line 376 in hu)
      • nested-negative-squared-or-cubed (power)
          Extra in Translation [1]
              • (line 395 in hu)
      • nested-var-squared-or-cubed (power)
          Extra in Translation [1]
              • (line 416 in hu)
      • nested-negative-var-squared-or-cubed (power)
          Extra in Translation [1]
              • (line 439 in hu)
      • default-exponent-power (power)
          Extra in Translation [1]
              • (line 463 in hu)
      • ClearSpeak-intervals ([closed-interval, closed-open-interval, 
open-closed-interval, open-interval])
          Extra in Translation [1]
              • (line 558 in hu)
      • ClearSpeak_Functions_None (mo)
          Extra in Translation [1]
              • (line 606 in hu)
      • no-times (mo)
          Extra in Translation [1]
              • (line 617 in hu)
      • ClearSpeak-default ([mlabeledtr, mtr])
          Extra in Translation [1]
              • (line 643 in hu)

This is equals with the june merged result.

In unicode.yaml file related the audit result is looking following, after latest translation merge:

⚠ unicode.yaml
  en: 121 rules  →  hu: 126 rules
────────────────────────────────────────────────────────────────────────────────

  ≠ Rule Issues [11] (grouped by rule and issue type)
      • "a"
          Missing in Translation [1]
              • (line 5 in en)
      • "b-z"
          Missing in Translation [1]
              • (line 10 in en)
      • "A"
          Missing in Translation [1]
              • (line 20 in en)
      • "B-Z"
          Missing in Translation [1]
              • (line 43 in en)
      • "a-z"
          Extra in Translation [1]
              • (line 2 in hu)
      • "A-Z"
          Extra in Translation [1]
              • (line 14 in hu)

This is good, because Neil merge the a, A, b-z and B-z type rules with the relevant a-z and A-Z type rules in hungarian unicode.yaml file.

@NSoiffer, we need doing anything with following audit part, the english or hungarian unicode.yaml file related?

      • "‴"
          Extra in Translation [1]
              • (line 300 in hu)
      • "∗"
          Extra in Translation [1]
              • (line 421 in hu)
      • "∷"
          Extra in Translation [1]
              • (line 451 in hu)
      • "≦"
          Extra in Translation [1]
              • (line 477 in hu)
      • "≧"
          Extra in Translation [1]
              • (line 478 in hu)

These characters are containing too the hungarian unicode-full.yaml file.
I remove this duplicated characters from hungarian unicode.yaml file, to provide more beautiful translation audit result?
U+2034 have in unicode-full.yaml file the 234TH line, u+2217 character have into the unicode-full.yaml file the 651TH line, u+2237 character have into the unicode-full.yaml file the 691TH line, u+2266TH line have into the unicode-full.yaml file the 817TH line, u+2267TH line have into the unicode-full.yaml file the 818 line.
So:
I remove unicode.yaml file this duplicates only?

Attila

[MartheGjelstad]

@hammera , about the files definitions.yaml and definitons.rs: There have been a lot of updates to the definitons.yaml file to better support the intent attribute. The definitons.rs file is very new, and I think it is only for English at the moment. I haven't looked at the updates for Norwegian yet. So, I think there is no rush to get all the tests in definitons.rs to run. Just make sure all the other tests files work :)

I will look at the files for Norwegian, but right now I don't have the time.

[hammera]

@MartheGjelstad, Of course:
The oldest time with april you and neil doed generated Hungarian tests still work fine, the only change I made was that yesterday in the unicode.yaml file I replaced the a with the a(z) in the intervals, since the a is not always pronounced for vowels, consonants or numbers (that's why I used the more general form in the localization).
So, when begin implementing the definitions.rs hungarian changes and definitions.yaml fixes, other already upstream repo merged tests are not produce failures.

When there was a big change in the English definitions.yaml file before, maybe before the end of June, since the audit tool does not check the translation differences between English and the target language based on the definitions.yaml file, I had to compare the Hungarian file based on the English diff patch to detect what change missing into the hungarian file with english file contains (for example a new definition rule). :-):-)
So, if this file were included in the audited tools in time, similarly to the other yaml files with audit translation tool examining, that would be wonderful future.

Regardless of the fact that this test was probably added recently for the English language, it helped me a lot in showing me the points where I still need to improve the localization of the definitions.yaml file.
Unfortunately, previous tests did not reveal the need for the localization fixes that have now been I fixed in the definitions.yaml file for some reason.

Attila

[hammera]

Don't surprise, because I see merged may 2 state upstream main branch commits into the hu upstream branch, I do a rebase and force-push only my fork hu branch.
All tests of course passed before force-push operation.

[MartheGjelstad]

About the file definitions.yaml and the audit tool: It's being worked on. See issue: #617

[NSoiffer]

I've merged in your PR.

As for the "en" version, that apparently wasn't included in the tests (not part of C:\Users\neils\MathCAT\tests\Languages\en). That was an oversight. There expected strings are wrong in many cases. Also, it was missing the intent properties for the prefix/postfix tests. I'll work on fixing them up.

[NSoiffer]

I had someone add all the intents in the MathML spec to definitions.yaml, but I forgot to tell him that there was a limitation on what that file can do. Things like integrals with the "from" ... "to" syntax don't fit any of the "fixity" properties (prefix, infix, ...) and so need rules and can't be in definitions.yaml. I'll work on getting that cleaned up in the English version over the next day or two.

[hammera]

Neil, thanks the PR merge and all other answers.
If this is possible, please drop me a notification when this englis fixes happening the definitions.yaml, because perhaps me need put this fixes or a variant fix the changed rules related in hungarian definitions.yaml file.
If you have a little time, have possibility to merge this PR to the main branch, and after the main branch merge, merge main to the hu branch?

I see, happened lot of new change, and want dropping a small PR to the unicode-full.yaml file related, I attaching the diff patch with compatible the latest main branch state:

diff --git a/Rules/Languages/hu/unicode-full.yaml b/Rules/Languages/hu/unicode-full.yaml
index 2926422e..4e62bfd5 100644
--- a/Rules/Languages/hu/unicode-full.yaml
+++ b/Rules/Languages/hu/unicode-full.yaml
@@ -951,11 +951,6 @@
  - "⊢": [T: "bizonyítja"]                       	#  0x22a2	(en: 'proves', DeepL translation)
  - "⊣": [T: "nem enged"]                        	#  0x22a3	(en: 'does not yield', DeepL translation)
  - "⊤": [T: "felső"]                            	#  0x22a4	(en: 'top', DeepL translation)
- - "⊥":                                         	#  0x22a5
-     - test: 
-         if: "$Verbosity!='Terse'"
-         then: [T: "a(z)"]                        	# 	(en: 'is', DeepL translation)
-     - T: "alul"                                	# 	(en: 'bottom', DeepL translation)
  - "⊦": [T: "redukálódik"]                      	#  0x22a6	(en: 'reduces to', DeepL translation)
  - "⊧": [T: "modellezi"]                         	#  0x22a7	(en: 'models', DeepL translation)
  - "⊨":                                         	#  0x22a8
@@ -1543,11 +1538,15 @@
  - "➾": [T: "nyitott körvonalú jobbra mutató nyíl"]	#  0x27be	(en: 'open-outlined right arrow', DeepL translation)
  - "⟀": [T: "háromdimenziós szög"]              	#  0x27c0	(en: 'three dimensional angle', DeepL translation)
  - "⟁": [T: "fehér háromszög, benne egy kis fehér háromszög"]	#  0x27c1	(en: 'white triangle containing small white triangle', DeepL translation)
- - "⟂":                                         	#  0x27c2
-     - test: 
-         if: "$Verbosity!='Terse'"
-         then: [T: "a(z)"]                        	# 	(en: 'is', DeepL translation)
-     - T: "merőleges"                           	# 	(en: 'perpendicular to', DeepL translation)
+ - "⟂⊥":                                          # 0x27c2, 22A5 (22A5 wrongly used for perpendicular by most editors)
+     - test:
+        - if: "$SpeechStyle = 'LiteralSpeak' and text()='⊥'"   # 22A5 (up tack/bottom)
+          then: [T: "felfelé mutató merőleges"]
+        - else:
+          - test:
+              if: "$Verbosity!='Terse'"
+              then: [T: "a(z)"]
+          - T: "merőleges"
  - "⟃":                                         	#  0x27c3
      - test: 
          if: "$Verbosity!='Terse'"

If you commit simple this diff patch after the main branch is merged to the hu branch, me full good, my purpose this audit syncing land to the latest main branch after this PR merged to the main branch. :-):-)
Tests are passed, so I already tested this patch.