Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,10 @@ or dropdown gets `answer_order=random`, a multiple choice
`subquestion_order=random`. A randomized `select_multiple` keeps its
exclusive answers at their position (`keep_codes_order`, LimeSurvey 6.17 or
later), which ODK and Kobo don't, so a `randomize-pinned` warning names them.
An option that is not exclusive but belongs at the end ("Sonstiges") is marked
with a `fixed` column (`yes`) on the choices sheet, on a `select_one` or a
`select_multiple`: its code joins the exclusive ones in `keep_codes_order`.
Without `randomize=true` it has no effect; DDI carries it as any choice column.
LimeSurvey's "other" answer is always last. A `seed` is dropped with a
`randomize-seed-dropped` warning, a randomized grid row with
`randomize-dropped`. The reverse paths read `randomize=true` back.
Expand Down
4 changes: 4 additions & 0 deletions codegen/emit_skill.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,10 @@ def _random_order_section(ro: dict) -> list[str]:
"in the `parameters` column to shuffle the options, e.g. to avoid primacy effects. "
"LimeSurvey shuffles them too, and keeps a `select_multiple`'s `exclusive` answers "
"at their position (LimeSurvey 6.17 or later); ODK and Kobo shuffle every option. "
f"To keep another option in place too (\u201cSonstiges\u201d, \u201cKeine Angabe\u201d at the end), "
f"add the `{ro.get('fixedColumn', 'fixed')}` column to the `choices` sheet and set it to "
f"`{ro.get('fixedTrueValues', ['yes'])[0]}` on that choice, on a `select_one` or a "
f"`select_multiple`; it has no effect without `{key}={yes}`. "
f"Don't add a `{ro.get('seedKey', 'seed')}`: LimeSurvey has none and drops it.\n",
]

Expand Down
10 changes: 8 additions & 2 deletions registry/conventions/randomOrder.jsonld
Original file line number Diff line number Diff line change
Expand Up @@ -61,11 +61,17 @@
},
"keepAttribute": "keep_codes_order",
"keepSeparator": ";",
"fixedColumn": "fixed",
"fixedTrueValues": [
"yes",
"true",
"1"
],
"limesurveyEmission": "parameters randomize=true on a select becomes LimeSurvey 6's ordering attribute for its question type: answer_order=random on a list (L) or dropdown (!), subquestion_order=random on a multiple choice (M). The legacy random_order=1 is only read back",
"pinned": "a randomized select_multiple's exclusive choices (convention:exclusiveChoice) also go in keep_codes_order, so LimeSurvey keeps them at their position in the list (usually last) while it shuffles the rest. keep_codes_order needs LimeSurvey 6.17 or later; older versions drop it on import and pin only a single exclude_all_others code. ODK and Kobo shuffle every choice, so a randomize-pinned warning names the pinned codes. LimeSurvey's own other option (convention:other) is always last, as or_other is in ODK",
"pinned": "a randomized select_multiple's exclusive choices (convention:exclusiveChoice) also go in keep_codes_order, so LimeSurvey keeps them at their position in the list (usually last) while it shuffles the rest. So do the choices marked in the optional choices column `fixed` (yes), on a select_one or a select_multiple: an option like 'Sonstiges' that is not exclusive but belongs at the end. The codes keep the list's order, an exclusive and fixed choice appears once. `fixed` has no effect without randomize=true and is carried as an ordinary choice column in DDI (cdl:choice_column) keep_codes_order needs LimeSurvey 6.17 or later; older versions drop it on import and pin only a single exclude_all_others code. ODK and Kobo shuffle every choice, so a randomize-pinned warning names the pinned (exclusive or fixed) codes. LimeSurvey's own other option (convention:other) is always last, as or_other is in ODK",
"seed": "ODK's seed parameter has no LimeSurvey counterpart: it is dropped with a randomize-seed-dropped warning, and LimeSurvey draws an order per respondent",
"unmapped": "a randomize=true LimeSurvey can't order (a select in a table-list grid, any other LimeSurvey type) is dropped with a randomize-dropped warning",
"reverse": "answer_order=random, subquestion_order=random or random_order=1 on a list, dropdown or multiple choice comes back as parameters randomize=true",
"reverse": "answer_order=random, subquestion_order=random or random_order=1 on a list, dropdown or multiple choice comes back as parameters randomize=true; a keep_codes_order code that is not an exclude_all_others code comes back as fixed=yes on its choice",
"ddiEmission": "none beyond the form's parameters cell, kept as a cdl:parameters note (convention:ddiFields)"
}
}
Expand Down
2 changes: 1 addition & 1 deletion skills/cdl-survey-types/references/xlsform-syntax.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Applies to: `select_multiple`. For an answer that rules out all others (“Keine

## Random answer order (`parameters: randomize=true`)

Applies to: `select_one`, `select_multiple` (also from a file). Write `randomize=true` in the `parameters` column to shuffle the options, e.g. to avoid primacy effects. LimeSurvey shuffles them too, and keeps a `select_multiple`'s `exclusive` answers at their position (LimeSurvey 6.17 or later); ODK and Kobo shuffle every option. Don't add a `seed`: LimeSurvey has none and drops it.
Applies to: `select_one`, `select_multiple` (also from a file). Write `randomize=true` in the `parameters` column to shuffle the options, e.g. to avoid primacy effects. LimeSurvey shuffles them too, and keeps a `select_multiple`'s `exclusive` answers at their position (LimeSurvey 6.17 or later); ODK and Kobo shuffle every option. To keep another option in place too (“Sonstiges”, “Keine Angabe” at the end), add the `fixed` column to the `choices` sheet and set it to `yes` on that choice, on a `select_one` or a `select_multiple`; it has no effect without `randomize=true`. Don't add a `seed`: LimeSurvey has none and drops it.

## Long lists from a file (`select_*_from_file`)

Expand Down
18 changes: 17 additions & 1 deletion src/conventions/randomOrder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,12 @@
* shuffles its options. LimeSurvey 6 orders them by a question attribute that
* depends on the question type: `answer_order=random` on a list or dropdown,
* `subquestion_order=random` on a multiple choice. A randomized multiple
* choice's exclusive answers keep their position (`keep_codes_order`).
* choice's exclusive answers keep their position (`keep_codes_order`), and
* so do the choices marked `fixed` (any select).
*/
import conventions from '../generated/conventions.js';
import { parseParameters } from '../utils/parameters.js';
import type { ChoiceRow } from '../xlsform/types.js';
import { FROM_FILE_BASE } from './fromFile.js';

export const RANDOM_ORDER_RULE = conventions.conventions.randomOrder;
Expand Down Expand Up @@ -59,3 +61,17 @@ export function isRandomOrder(
cell(legacy.attribute) === legacy.value
);
}

const FIXED_TRUE_VALUES = new Set<string>(RANDOM_ORDER_RULE.fixedTrueValues);

/** Whether the choice is marked `fixed`: it keeps its position when shuffled. */
export function isFixed(choice: ChoiceRow): boolean {
const value = choice[RANDOM_ORDER_RULE.fixedColumn];
const text =
typeof value === 'string' ||
typeof value === 'number' ||
typeof value === 'boolean'
? String(value).trim().toLowerCase()
: '';
return FIXED_TRUE_VALUES.has(text);
}
10 changes: 8 additions & 2 deletions src/generated/conventions.json
Original file line number Diff line number Diff line change
Expand Up @@ -1022,11 +1022,17 @@
},
"keepAttribute": "keep_codes_order",
"keepSeparator": ";",
"fixedColumn": "fixed",
"fixedTrueValues": [
"yes",
"true",
"1"
],
"limesurveyEmission": "parameters randomize=true on a select becomes LimeSurvey 6's ordering attribute for its question type: answer_order=random on a list (L) or dropdown (!), subquestion_order=random on a multiple choice (M). The legacy random_order=1 is only read back",
"pinned": "a randomized select_multiple's exclusive choices (convention:exclusiveChoice) also go in keep_codes_order, so LimeSurvey keeps them at their position in the list (usually last) while it shuffles the rest. keep_codes_order needs LimeSurvey 6.17 or later; older versions drop it on import and pin only a single exclude_all_others code. ODK and Kobo shuffle every choice, so a randomize-pinned warning names the pinned codes. LimeSurvey's own other option (convention:other) is always last, as or_other is in ODK",
"pinned": "a randomized select_multiple's exclusive choices (convention:exclusiveChoice) also go in keep_codes_order, so LimeSurvey keeps them at their position in the list (usually last) while it shuffles the rest. So do the choices marked in the optional choices column `fixed` (yes), on a select_one or a select_multiple: an option like 'Sonstiges' that is not exclusive but belongs at the end. The codes keep the list's order, an exclusive and fixed choice appears once. `fixed` has no effect without randomize=true and is carried as an ordinary choice column in DDI (cdl:choice_column) keep_codes_order needs LimeSurvey 6.17 or later; older versions drop it on import and pin only a single exclude_all_others code. ODK and Kobo shuffle every choice, so a randomize-pinned warning names the pinned (exclusive or fixed) codes. LimeSurvey's own other option (convention:other) is always last, as or_other is in ODK",
"seed": "ODK's seed parameter has no LimeSurvey counterpart: it is dropped with a randomize-seed-dropped warning, and LimeSurvey draws an order per respondent",
"unmapped": "a randomize=true LimeSurvey can't order (a select in a table-list grid, any other LimeSurvey type) is dropped with a randomize-dropped warning",
"reverse": "answer_order=random, subquestion_order=random or random_order=1 on a list, dropdown or multiple choice comes back as parameters randomize=true",
"reverse": "answer_order=random, subquestion_order=random or random_order=1 on a list, dropdown or multiple choice comes back as parameters randomize=true; a keep_codes_order code that is not an exclude_all_others code comes back as fixed=yes on its choice",
"ddiEmission": "none beyond the form's parameters cell, kept as a cdl:parameters note (convention:ddiFields)"
},
"sanitization": {
Expand Down
10 changes: 8 additions & 2 deletions src/generated/conventions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1031,11 +1031,17 @@ const conventions = {
},
"keepAttribute": "keep_codes_order",
"keepSeparator": ";",
"fixedColumn": "fixed",
"fixedTrueValues": [
"yes",
"true",
"1"
],
"limesurveyEmission": "parameters randomize=true on a select becomes LimeSurvey 6's ordering attribute for its question type: answer_order=random on a list (L) or dropdown (!), subquestion_order=random on a multiple choice (M). The legacy random_order=1 is only read back",
"pinned": "a randomized select_multiple's exclusive choices (convention:exclusiveChoice) also go in keep_codes_order, so LimeSurvey keeps them at their position in the list (usually last) while it shuffles the rest. keep_codes_order needs LimeSurvey 6.17 or later; older versions drop it on import and pin only a single exclude_all_others code. ODK and Kobo shuffle every choice, so a randomize-pinned warning names the pinned codes. LimeSurvey's own other option (convention:other) is always last, as or_other is in ODK",
"pinned": "a randomized select_multiple's exclusive choices (convention:exclusiveChoice) also go in keep_codes_order, so LimeSurvey keeps them at their position in the list (usually last) while it shuffles the rest. So do the choices marked in the optional choices column `fixed` (yes), on a select_one or a select_multiple: an option like 'Sonstiges' that is not exclusive but belongs at the end. The codes keep the list's order, an exclusive and fixed choice appears once. `fixed` has no effect without randomize=true and is carried as an ordinary choice column in DDI (cdl:choice_column) keep_codes_order needs LimeSurvey 6.17 or later; older versions drop it on import and pin only a single exclude_all_others code. ODK and Kobo shuffle every choice, so a randomize-pinned warning names the pinned (exclusive or fixed) codes. LimeSurvey's own other option (convention:other) is always last, as or_other is in ODK",
"seed": "ODK's seed parameter has no LimeSurvey counterpart: it is dropped with a randomize-seed-dropped warning, and LimeSurvey draws an order per respondent",
"unmapped": "a randomize=true LimeSurvey can't order (a select in a table-list grid, any other LimeSurvey type) is dropped with a randomize-dropped warning",
"reverse": "answer_order=random, subquestion_order=random or random_order=1 on a list, dropdown or multiple choice comes back as parameters randomize=true",
"reverse": "answer_order=random, subquestion_order=random or random_order=1 on a list, dropdown or multiple choice comes back as parameters randomize=true; a keep_codes_order code that is not an exclude_all_others code comes back as fixed=yes on its choice",
"ddiEmission": "none beyond the form's parameters cell, kept as a cdl:parameters note (convention:ddiFields)"
},
"sanitization": {
Expand Down
39 changes: 32 additions & 7 deletions src/instrument/fromLstsv.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ import { OTHER_CODE, OTHER_SUFFIX } from '../conventions/other.js';
import { GRID_APPEARANCE } from '../conventions/grid.js';
import { EXCLUSIVE_RULE } from '../conventions/exclusive.js';
import {
KEEP_CODES_ATTRIBUTE,
RANDOM_ORDER_RULE,
RANDOMIZE_PARAMETER,
isRandomOrder,
} from '../conventions/randomOrder.js';
Expand Down Expand Up @@ -234,6 +236,20 @@ function excludes(row: Record<string, unknown>, code: string): boolean {
.some((c) => c.trim() === code);
}

/**
* Whether a Q row's `keep_codes_order` lists `code` and `exclude_all_others`
* doesn't: a `fixed` choice (convention:randomOrder).
*/
function isFixedCode(row: Record<string, unknown>, code: string): boolean {
const value = row[KEEP_CODES_ATTRIBUTE];
return (
!excludes(row, code) &&
(typeof value === 'string' ? value : '')
.split(RANDOM_ORDER_RULE.keepSeparator)
.some((c) => c.trim() === code)
);
}

/**
* An A/SQ row: an option of `owner`'s list. A multiple choice's defaults sit
* on its SQ rows as `Y`.
Expand All @@ -251,17 +267,26 @@ function addChoice(
.join(' ');
}
const code = cell(row, 'name');
// The question's keep_codes_order, as the `fixed` column (a choice column
// the model doesn't lift, so a codebook carries it too).
const fixed =
question && isFixedCode(question.row, code)
? {
[RANDOM_ORDER_RULE.fixedColumn]: RANDOM_ORDER_RULE.fixedTrueValues[0],
}
: null;
(state.lists[owner] ??= []).push({
name: code,
label: text(state, `label:${cls}:${owner}:${code}`),
// The question's exclude_all_others, on the choice as an XLSForm has it.
row:
question && excludes(question.row, code)
? {
...row,
[EXCLUSIVE_RULE.choicesColumn]: EXCLUSIVE_RULE.trueValues[0],
}
: row,
row: {
...row,
...(question && excludes(question.row, code)
? { [EXCLUSIVE_RULE.choicesColumn]: EXCLUSIVE_RULE.trueValues[0] }
: {}),
...fixed,
},
...(fixed ? { columns: fixed } : {}),
});
}

Expand Down
2 changes: 1 addition & 1 deletion src/pipelines/lstsv2xlsform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ CLI: `formtransform lstsv2xlsform <input.tsv>`.
| per-language rows | `label::<lang>` / `hint::<lang>` | direct |
| S rows | settings (`default_language`, `form_title`, `style`, and `limesurvey_<setting>` for each supported survey setting) | convention:surveySettings `limesurveySettings`; `shownoanswer=N`, formtransform's default, isn't read back |
| `relevance`, `em_validation_q` | `relevant`, `constraint` | reverse-transpiled — see below |
| `answer_order=random` (L, `!`), `subquestion_order=random` (M), legacy `random_order=1` | `parameters: randomize=true` | convention:randomOrder; `keep_codes_order` isn't read back (a multiple choice's exclusive answers come back from `exclude_all_others`) |
| `answer_order=random` (L, `!`), `subquestion_order=random` (M), legacy `random_order=1` | `parameters: randomize=true` | convention:randomOrder; a `keep_codes_order` code that isn't in `exclude_all_others` comes back as `fixed=yes` on its choice; exclusive answers come back from `exclude_all_others` |
| `array_filter = q` | `choice_filter: selected(${q}, name)` | convention:choiceFilter. A filter the forward path recoded (`selected(${q}, <column>)`) comes back on the codes LimeSurvey stores, not the list's own |

`label` / `list-nolabel` matrix appearances (the non-`table-list` matrix
Expand Down
24 changes: 20 additions & 4 deletions src/pipelines/xlsform2lstsv/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ import {
KEEP_CODES_ATTRIBUTE,
RANDOM_ORDER_ATTRIBUTES,
RANDOM_ORDER_RULE,
isFixed,
randomizable,
randomizeOf,
} from '../../conventions/randomOrder.js';
Expand Down Expand Up @@ -453,6 +454,21 @@ class Conversion {
.map((c) => this.fieldNameHandler.sanitizeAnswerCode(String(c.name)));
}

/**
* The codes a randomized question keeps in place (keep_codes_order): its
* exclusive answers and the choices marked `fixed` (convention:randomOrder),
* in list order, once each.
*/
private keptCodes(baseType: string, listName: string | null): string[] {
if (!listName) return [];
const exclusive = EXCLUSIVE_RULE.appliesTo.includes(baseType)
? isExclusive
: () => false;
return (this.choiceManager.getChoices(listName) ?? [])
.filter((c) => exclusive(c) || isFixed(c))
.map((c) => this.fieldNameHandler.sanitizeAnswerCode(String(c.name)));
}

/**
* `exclude_all_others` for a select_multiple whose list marks choices
* `exclusive` (convention:exclusiveChoice): their answer codes, `;`-joined.
Expand All @@ -472,8 +488,8 @@ class Conversion {

/**
* `parameters: randomize=true` on a select → LimeSurvey's ordering
* attribute for its type (convention:randomOrder), with its exclusive
* answers kept in place. A seed, an exclusive answer ODK would shuffle, and
* attribute for its type (convention:randomOrder), with its exclusive and
* fixed answers kept in place. A seed, an exclusive answer ODK would shuffle, and
* a type LimeSurvey can't order are each a warning.
*/
private randomOrderAttributes(
Expand Down Expand Up @@ -505,12 +521,12 @@ class Conversion {
),
);
}
const keep = this.exclusiveCodes(baseType, listName);
const keep = this.keptCodes(baseType, listName);
if (keep.length > 0) {
this.warn(
warning(
'randomize-pinned',
`'${questionName}': LimeSurvey keeps the exclusive answer(s) ${keep.join(', ')} at their position while it shuffles the others (${KEEP_CODES_ATTRIBUTE}, LimeSurvey 6.17 or later); ODK and Kobo shuffle them with the rest`,
`'${questionName}': LimeSurvey keeps the exclusive or fixed answer(s) ${keep.join(', ')} at their position while it shuffles the others (${KEEP_CODES_ATTRIBUTE}, LimeSurvey 6.17 or later); ODK and Kobo shuffle them with the rest`,
questionName,
),
);
Expand Down
Loading
Loading