From ce6ce3bf3dbef0381eb84d8049a970766bf05525 Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Sat, 8 Aug 2026 13:38:21 -0600 Subject: [PATCH 1/9] Revise description format in README.md Updated the description section format and content. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 901e74b..f797f10 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ R package for content validity analysis - Email: [diegolivia\@hotmail.com](mailto:diegolivia@hotmail.com) - ORCID: https://orcid.org/0000-0002-2107-3140 -\strong{Description:}\ +## Description: `ValCont` is a dedicated content validity package in R. `ValCont` implement the computation of several coefficients used in content validity studies, with data usually obtained from selected participants such as expert judges or experiential judges. The coefficients calculated by ValCont are: - CVC (Hernandez-Nieto, 2002) From 73935e8eb3d51bc9f65fff9d9125a53a03d41878 Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Sat, 8 Aug 2026 13:41:01 -0600 Subject: [PATCH 2/9] Revise installation and dependencies section in README Updated formatting for installation instructions and dependencies. --- README.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index f797f10..7fa3164 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ Some functions were added to estimate other relevant aspects of the content vali - Basic functions to make graphs of results are also implemented. -\strong{Install:} +## Install: You can install the development version of `ValCont` from GitHub using: ```R @@ -49,7 +49,7 @@ if(!"devtools" %in% row.names(installed.packages())){ } devtools::install_github("Diegolivia/ValCont") ``` -\strong{References:} +## References: - Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40, 955-959. - Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. @@ -66,11 +66,11 @@ devtools::install_github("Diegolivia/ValCont") - Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. - Zou, G., Donner, A. and Qiu, S. (2025). MOVER-R for Confidence Intervals of Ratios. In Wiley StatsRef: Statistics Reference Online (eds N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, F. Ruggeri and J.L. Teugels). -\strong{Depends:} R (\>= 2.10)\ -\strong{Imports:} boots, ggplo2, stats, utils\ -\strong{License:} GPL-3\ -\strong{Encoding:} UTF-8\ -\strong{LazyData:} true\ -\strong{Maintainer:} Diego Livia-Ortiz\ -\strong{URL:} https://github.com/Diegolivia/ValCont/ +*Depends*: R (\>= 2.10)\ +*Imports*: boots, ggplo2, stats, utils\ +*License*: GPL-3\ +*Encoding*: UTF-8\ +*LazyData*: true\ +*Maintainer*: Diego Livia-Ortiz\ +*URL*: https://github.com/Diegolivia/ValCont/ From 7dd9233f7a58af0736af1805323481338a7ad290 Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Mon, 24 Aug 2026 14:57:15 -0600 Subject: [PATCH 3/9] Update: Spanish => English, improving functions --- DESCRIPTION | 2 +- NAMESPACE | 46 +++++---- R/CID.R | 44 ++++----- R/CIDsingle.R | 26 ++--- R/CIR.R | 237 ++++++++++++++++++++++++++++------------------ R/CVC.R | 39 ++++---- R/CVI.R | 46 ++++----- R/CVIR.R | 63 ++++++------ R/HAiken.R | 39 ++++---- R/MDScontent.R | 11 ++- R/MER.R | 18 ++-- R/VAiken.R | 34 +++---- R/Vaikenpub.R | 22 +++-- man/CID.Rd | 18 ++-- man/CIDsingle.Rd | 12 +-- man/CIR.Rd | 89 +++++++---------- man/CVC.Rd | 13 +-- man/CVI.Rd | 24 ++--- man/CVIR.Rd | 20 ++-- man/Haiken.Rd | 11 +-- man/MDScontent.Rd | 11 ++- man/MER.Rd | 18 ++-- man/Vaiken.Rd | 10 +- man/Vaikenpub.Rd | 10 +- 24 files changed, 451 insertions(+), 412 deletions(-) diff --git a/DESCRIPTION b/DESCRIPTION index 75965e6..db072f1 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -33,4 +33,4 @@ Depends: Suggests: testthat (>= 3.0.0) Config/testthat/edition: 3 -Config/roxygen2/version: 8.0.0 +Config/roxygen2/version: 8.1.0 diff --git a/NAMESPACE b/NAMESPACE index c87a42d..446e0bc 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -28,24 +28,30 @@ export(SVALsingle) export(Vaiken) export(Vaikenpub) export(minimumCV) -importFrom(boot,boot) -importFrom(boot,boot.ci) -importFrom(ggplot2,aes) -importFrom(ggplot2,element_text) -importFrom(ggplot2,geom_abline) -importFrom(ggplot2,geom_errorbar) -importFrom(ggplot2,geom_hline) -importFrom(ggplot2,geom_point) -importFrom(ggplot2,ggplot) -importFrom(ggplot2,labs) -importFrom(ggplot2,theme) -importFrom(ggplot2,theme_minimal) -importFrom(stats,complete.cases) -importFrom(stats,na.omit) -importFrom(stats,pbeta) -importFrom(stats,pbinom) -importFrom(stats,pnorm) -importFrom(stats,qnorm) -importFrom(stats,qt) -importFrom(stats,sd) +importFrom(boot, + boot, + boot.ci +) +importFrom(ggplot2, + aes, + element_text, + geom_abline, + geom_errorbar, + geom_hline, + geom_point, + ggplot, + labs, + theme, + theme_minimal +) +importFrom(stats, + complete.cases, + na.omit, + pbeta, + pbinom, + pnorm, + qnorm, + qt, + sd +) importFrom(utils,combn) diff --git a/R/CID.R b/R/CID.R index 99820e7..47e7977 100644 --- a/R/CID.R +++ b/R/CID.R @@ -15,24 +15,22 @@ #'`CID` uses Method of Variance Estimates Recovery (MOVER; Zou, & Donner, 2008). #'Because data produced by judges' judgments tend to be asymmetrically distributed (if the item is rated, on a scale of 1 to 5, as predominantly valid then its values will be > 3), MOVER is appropriate for non-normal distributions. #'MOVER depends on the quality or precision of the confidence intervals calculated for the coefficients in each group. -#'The application of MOVER for content validity coefficient was initially published by Merino-Soto (2018) for the difference between V coefficients (Aiken, 1980, 1985). Later, Merino-Soto (2023) extended this approach for Aiken's V by adding a point estimator of the difference, based on the standardized difference between proportions +#'The application of MOVER for content validity coefficient was initially published by Merino-Soto (2018) for the difference between V coefficients (Aiken, 1980, 1985). Later, Merino-Soto (2023) extended this approach for Aiken's V by adding a point estimator of the difference, based on the standardized difference between proportions. #'The compared content validity coefficients obtained should be of the same type, and the estimated confidence intervals for these coefficients should also come from the same level; for example, at .95 or .90. -#'Singer (2010) observed that at extremely low values (e.g., proportions near .0), the coverage of this method is not as good. In the context of comparing content validity coefficients, treated as proportions, it is rare to find such low coefficients (and their confidence intervals). Unless the items are extremely poor in content. -#' -#'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. +#'Singer (2010) observed that at extremely low values (e.g., proportions near .0), the coverage of this method is not as good. In the context of comparing content validity coefficients, treated as proportions, it is rare to find such very low coefficients (and their confidence intervals). Unless the items are extremely poor in content. #' #'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} #' -#'Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. Anales de Psicologia, 34(3), 587-590. https://doi.org/10.6018/analesps.34.3.283481 +#'Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. \emph{Anales de Psicologia, 34}(3), 587-590. \doi{10.6018/analesps.34.3.283481} #' -#'Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. MHSalud, 20(1), 23-32. https://doi.org/10.15359/mhs.20-1.3 +#'Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. \emph{MHSalud, 20}(1), 23-32. \doi{10.15359/mhs.20-1.3} #' -#'Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702. Statistics in Medicine, 29(16), 1757–1759. https://doi.org/10.1002/sim.3887 +#'Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702. \emph{Statistics in Medicine, 29}(16), 1757–1759. \doi{10.1002/sim.3887} #' -#'Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. Stat. Med. 27, 1693–1702. https://doi.org//10.1002/sim.3095 +#'Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. \emph{Statistics in Medicine, 27}, 1693–1702. \doi{10.1002/sim.3095} #' #'@author #'Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) @@ -99,34 +97,34 @@ #' lwr.col = "lwr.ci", #' upr.col = "upr.ci") #' -#' +#' #'@export CID <- function(group1, group2, coef.col = "coef", lwr.col = "lwr.ci", upr.col = "upr.ci", na.rm = FALSE) { - # Validar que los argumentos son data.frames + # Verify that the arguments are DataFrames if (!is.data.frame(group1) || !is.data.frame(group2)) { - stop("Ambos argumentos 'group1' y 'group2' deben ser data.frames.") + stop("Both the 'group1' and 'group2' arguments must be data.frames.") } - # Detección de valores perdidos + # Detection of Missing Values if (!na.rm) { if (any(is.na(group1)) || any(is.na(group2))) { - stop("Valores perdidos detectados. Usa na.omit() primero o establece na.rm=TRUE.") + stop("Missing values detected. Use na.omit() first, or set na.rm=TRUE.") } } else { group1 <- na.omit(group1) group2 <- na.omit(group2) } - # Validar que las columnas necesarias existen en los data.frames + # Verify that the necessary columns exist in the data frames required_cols <- c(coef.col, lwr.col, upr.col) if (!all(required_cols %in% colnames(group1))) { - stop("El data.frame 'group1' no contiene las columnas necesarias.") + stop("The 'group1' data.frame does not contain the required columns.") } if (!all(required_cols %in% colnames(group2))) { - stop("El data.frame 'group2' no contiene las columnas necesarias.") + stop("The 'group2' data.frame does not contain the required columns.") } - # Combinar los datos para realizar las comparaciones + # Combine the data to make comparisons combined_data <- merge( group1, group2, @@ -136,12 +134,12 @@ CID <- function(group1, group2, coef.col = "coef", lwr.col = "lwr.ci", upr.col = rownames(combined_data) <- combined_data$Row.names combined_data$Row.names <- NULL - # Verificar si hubo exclusiones debido a items ausentes en un grupo + # Check whether there were any exclusions due to missing items in a group if (nrow(combined_data) < nrow(group1) || nrow(combined_data) < nrow(group2)) { - warning("Algunos items no tienen correspondencia entre los grupos y han sido excluidos de la comparacion.") + warning("Some items do not have a match across the groups and have been excluded from the comparison.") } - # Calcular la diferencia de los coeficientes y los intervalos de confianza + # Calculate the difference between the coefficients and the confidence intervals combined_data <- within(combined_data, { Delta <- round(get(paste0(coef.col, "_1")) - get(paste0(coef.col, "_2")), 3) lwr.ci <- round( @@ -152,7 +150,7 @@ CID <- function(group1, group2, coef.col = "coef", lwr.col = "lwr.ci", upr.col = (get(paste0(coef.col, "_2")) - get(paste0(lwr.col, "_2")))^2), 3) }) - # Seleccionar las columnas finales para el reporte y reiniciar nombres de filas + # Select the final columns for the report and reset the row labels result <- combined_data[, c("Delta", "lwr.ci", "upr.ci")] row.names(result) <- NULL diff --git a/R/CIDsingle.R b/R/CIDsingle.R index 087ab98..8e1cc7e 100644 --- a/R/CIDsingle.R +++ b/R/CIDsingle.R @@ -29,17 +29,17 @@ #'in content, it is unlikely to obtain content validity coefficients close to zero. #' #'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} #' -#'Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. Anales de Psicología, 34(3), 587-590. https://doi.org/10.6018/analesps.34.3.283481. +#'Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. \emph{Anales de Psicologia, 34}(3), 587-590. \doi{10.6018/analesps.34.3.283481} #' -#'Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. MHSalud, 20(1), 23-32. https://doi.org/10.15359/mhs.20-1.3 +#'Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. \emph{MHSalud, 20}(1), 23-32. \doi{10.15359/mhs.20-1.3} #' -#'Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702.Statistics in Medicine, 29(16), 1757–1759. https://doi.org/10.1002/sim.3887 +#'Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702. \emph{Statistics in Medicine, 29}(16), 1757–1759. \doi{10.1002/sim.3887} #' -#'Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. Stat. Med. 27, 1693–1702. https://doi.org/10.1002/sim.3095 +#'Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. \emph{Statistics in Medicine, 27}, 1693–1702. \doi{10.1002/sim.3095} #' #'@seealso #'\code{\link[ratesci:moverci]{ratesci::moverci}} for MOVER method ofr ratios @@ -61,23 +61,23 @@ #' #'@export CIDsingle <- function(coef1, coef2, lci1, uci1, lci2, uci2) { - # Validar que todos los argumentos sean numericos + # Verify that all arguments are numeric if (!all(sapply(list(coef1, coef2, lci1, uci1, lci2, uci2), is.numeric))) { - stop("Todos los argumentos deben ser valores numericos.") + stop("All arguments must be numeric values.") } # Validar que los limites sean coherentes - if (!(lci1 <= coef1 && coef1 <= uci1)) stop("El coeficiente 1 no esta dentro de su intervalo de confianza.") - if (!(lci2 <= coef2 && coef2 <= uci2)) stop("El coeficiente 2 no esta dentro de su intervalo de confianza.") + if (!(lci1 <= coef1 && coef1 <= uci1)) stop("The coefficient 1 is not within its confidence interval.") + if (!(lci2 <= coef2 && coef2 <= uci2)) stop("The coefficient 2 is not within its confidence interval.") - # Calcular la diferencia entre los coeficientes + # Calculate the difference between the coefficients difference <- coef1 - coef2 - # Calcular el intervalo de confianza para la diferencia usando MOVER + # Calculate the confidence interval for the difference using MOVER lower_diff <- difference - sqrt((coef1 - lci1)^2 + (uci2 - coef2)^2) upper_diff <- difference + sqrt((uci1 - coef1)^2 + (coef2 - lci2)^2) - # Retornar los resultados + # Return the results return(data.frame( Difference = round(difference, 3), lwr.ci = round(lower_diff, 3), diff --git a/R/CIR.R b/R/CIR.R index 5473681..2025d00 100644 --- a/R/CIR.R +++ b/R/CIR.R @@ -1,80 +1,59 @@ -#'@title Confidence Intervals of Ratios of content validity coefficients -#'@description Calculates confidence interval for the ratio of two content validity coefficients, based on method of variance recovery for ratios (MOVER-R). +#' MOVER-R Confidence Interval for the Ratio of Content Validity Coefficients #' -#'@param group1 Output dataframe 1 of one of the 'ValCont' functions to obtain content validity coefficients. -#'@param group2 Output Output dataframe 2 of one of the 'ValCont' functions to obtain content validity coefficients. -#'@param coef.col Name of the column in the dataframe storing the calculated coefficient -#'@param lwr.col Name of the column in the dataframe that stores the lower bound of the confidence interval -#'@param upr.col Name of the column in the dataframe that stores the upper limit of the confidence interval -#'@param na.rm Logical. If FALSE (default) the function stops when missing values are detected. -#' If TRUE rows with missing values in the relevant columns are removed before processing. +#' Computes the confidence interval for the ratio of two independent +#' content validity coefficients using the MOVER-R (Method of Variance Estimates Recovery for Ratios) +#' closed-form procedure. #' -#'@return -#'dataframe with four columns: label of the items, ratio between the coefficients, the upper and upper limit of the confidence interval of the difference. +#' @param group1 data.frame containing estimates and CIs for the first group. +#' Must include columns specified in \code{coef.col}, \code{lwr.col}, +#' \code{upr.col}, and an \code{"Item"} column for matching. +#' @param group2 data.frame containing estimates and CIs for the second group. +#' Same column requirements as \code{group1}. +#' @param coef.col Character string. Name of the column with the point estimates. +#' @param lwr.col Character string. Name of the column with the lower bounds. +#' @param upr.col Character string. Name of the column with the upper bounds. +#' @param na.rm Logical. If \code{TRUE}, rows with missing values in the relevant +#' columns are removed with a warning. If \code{FALSE}, missing values +#' trigger an error. #' -#'@details -#''CIR' uses method of variance recovery for ratios (MOVER-R; Zou, Donner, & Qiu, 2025), as a general approach for two non-normal quantities. -#'Because data produced by judges' judgments tend to be asymmetrically distributed (if the item is rated, on a scale of 1 to 5, as predominantly valid then its values will be > 3), MOVER-R is appropriate for non-normal distributions. -#'MOVER-R depends on the quality or precision of the confidence intervals calculated for the coefficients in each group. -#'The compared content validity coefficients obtained should be of the same type, and the estimated confidence intervals for these coefficients should also come from the same level; for example, at .95 or .90. -#'If the two dataframes have different numbers of evaluated items (i.e., different numbers of rows), 'CIR' function matches the commonly labeled items, assuming they are the same items. -#'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. +#' @return A data.frame with columns: +#' \item{Item}{Common item names between both groups.} +#' \item{R}{Estimated ratio (coefficient group1 / coefficient group2).} +#' \item{lwr.ci}{Lower bound of the MOVER-R confidence interval.} +#' \item{upr.ci}{Upper bound of the MOVER-R confidence interval.} #' -#'@references -#'Zou, G., Donner, A. & Qiu, S. (2025). MOVER-R for Confidence Intervals of Ratios. In N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, F. Ruggeri and J.L. Teugels (Eds.), Wiley StatsRef: Statistics Reference Online. https://doi.org/10.1002/9781118445112.stat08085 -#' -#' @author -#'Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) -#' -#'@seealso -#'\code{\link[ratesci:moverci]{ratesci::moverci}} -#'\code{\link[ValCont:CID]{ValCont::CID}} -#' -#'@examples -#' -#'### Example 1 ----------- -#' -#'## Random data (Low ratings): 11 items (rows), 4 raters (columns) -#'Data1 <- data.frame( -#' juez1 = sample(1:2, 11, replace = TRUE), -#' juez2 = sample(2:3, 11, replace = TRUE), -#' juez3 = sample(1:2, 11, replace = TRUE), -#' juez4 = sample(1:3, 11, replace = TRUE)) -#' -#'## Random data (High ratings): 10 items (rows), 6 raters (columns) -#'Data2 <- data.frame( -#' obs1 = sample(5:7, 10, replace = TRUE), -#' obs2 = sample(4:7, 10, replace = TRUE), -#' obs3 = sample(4:7, 10, replace = TRUE), -#' obs4 = sample(5:7, 10, replace = TRUE), -#' obs5 = sample(5:7, 10, replace = TRUE), -#' obs6 = sample(6:7, 10, replace = TRUE)) -#' -#'## Saving results from Aiken's V analysis, for each group -#'group1 <- Vaiken(data = Data1, min = 1, max = 7, conf.level = .90) -#'group2 <- Vaiken(data = Data2, min = 1, max = 7, conf.level = .90) -#' -#'CIR(group1 = group1, -#' group2 = group2, -#' coef.col = "V", -#' lwr.col = "lwr.ci", -#' upr.col = "upr.ci") +#' @references +#' Donner, A., & Zou, G. Y. (2012). Closed-form confidence intervals for +#' functions of the normal mean and standard deviation. \emph{Statistical +#' Methods in Medical Research}, 21(4), 347-359. #' +#' Zou, G., Donner, A., & Qiu, S. (2025). MOVER-R for Confidence Intervals +#' of Ratios. In N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, +#' F. Ruggeri and J.L. Teugels (Eds.), \emph{Wiley StatsRef: Statistics +#' Reference Online}. https://doi.org/10.1002/9781118445112.stat08085 #' +#' @examples +#' \dontrun{ +#' # Assuming you have data frames 'g1' and 'g2' with columns 'est', 'low', 'high' +#' result <- CIR(g1, g2, coef.col = "est", lwr.col = "low", upr.col = "high") +#' print(result) +#' } #' @export CIR <- function(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) { - # Validaciones iniciales + + # ------ 1. Validación de columnas ------ if (!all(c(coef.col, lwr.col, upr.col) %in% colnames(group1))) { - stop("group1 debe contener las columnas especificadas en coef.col, lwr.col y upr.col") + stop("group1 must contain the columns specified in coef.col, lwr.col, and upr.col") } if (!all(c(coef.col, lwr.col, upr.col) %in% colnames(group2))) { - stop("group2 debe contener las columnas especificadas en coef.col, lwr.col y upr.col") + stop("group2 must contain the columns specified in coef.col, lwr.col, and upr.col") } - # Detectar valores perdidos en las columnas relevantes + # Columns of interest (including "Item" for the merge) cols_g1 <- intersect(c(coef.col, lwr.col, upr.col, "Item"), colnames(group1)) cols_g2 <- intersect(c(coef.col, lwr.col, upr.col, "Item"), colnames(group2)) + # ------ 2. Handling Missing Values (NA) ------ has_na_g1 <- any(is.na(group1[, cols_g1, drop = FALSE])) has_na_g2 <- any(is.na(group2[, cols_g2, drop = FALSE])) @@ -82,60 +61,132 @@ CIR <- function(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) { if (!na.rm) { stop("Missing values detected. Use na.omit() first or set na.rm=TRUE") } else { - # Eliminar filas con NA en las columnas de coef, lwr o upr (no es necesario Item para complete.cases) - keep_g1 <- complete.cases(group1[, intersect(c(coef.col, lwr.col, upr.col), colnames(group1)), drop = FALSE]) - keep_g2 <- complete.cases(group2[, intersect(c(coef.col, lwr.col, upr.col), colnames(group2)), drop = FALSE]) + + # Only the "Estimate" and "CI" columns (not "Item") for complete cases + est_cols_g1 <- intersect(c(coef.col, lwr.col, upr.col), colnames(group1)) + est_cols_g2 <- intersect(c(coef.col, lwr.col, upr.col), colnames(group2)) + + keep_g1 <- complete.cases(group1[, est_cols_g1, drop = FALSE]) + keep_g2 <- complete.cases(group2[, est_cols_g2, drop = FALSE]) + nrem1 <- sum(!keep_g1) nrem2 <- sum(!keep_g2) + group1 <- group1[keep_g1, , drop = FALSE] group2 <- group2[keep_g2, , drop = FALSE] - warning(sprintf("na.rm=TRUE: removed %d rows with missing values from group1 and %d from group2", nrem1, nrem2)) + + warning(sprintf("na.rm=TRUE: removed %d rows with missing values from group1 and %d from group2", + nrem1, nrem2)) } } - # Identificar items comunes + # ------ 3. Intersection of Items ------ common_items <- intersect(group1$Item, group2$Item) - if (length(common_items) < nrow(group1) || length(common_items) < nrow(group2)) { - warning("Algunos items no coinciden entre group1 y group2. Se procesaran unicamente los items comunes.") + warning("Some items do not match between group1 and group2. Only the common items will be processed..") } - # Filtrar los data.frames por items comunes group1 <- group1[group1$Item %in% common_items, ] group2 <- group2[group2$Item %in% common_items, ] - # Combinar ambos data.frames por Item + # Sort by Item to ensure a match (optional, but safe) + group1 <- group1[order(group1$Item), ] + group2 <- group2[order(group2$Item), ] + + # ------ 4. Mixing (merge) ------ combined <- merge(group1, group2, by = "Item", suffixes = c("_g1", "_g2")) - # Inicializar las columnas de salida + # Pre-assign results results <- data.frame( Item = combined$Item, - R = NA, - lwr.ci = NA, - upr.ci = NA + R = NA_real_, + lwr.ci = NA_real_, + upr.ci = NA_real_, + stringsAsFactors = FALSE ) - # Calcular R y los intervalos de confianza para cada item + # Column names in the combined data.frame + coef_g1 <- paste0(coef.col, "_g1") + coef_g2 <- paste0(coef.col, "_g2") + lwr_g1 <- paste0(lwr.col, "_g1") + upr_g1 <- paste0(upr.col, "_g1") + lwr_g2 <- paste0(lwr.col, "_g2") + upr_g2 <- paste0(upr.col, "_g2") + + # ------ 5. Calculation of the MOVER-R Interval (Donner & Zou, 2012) ------ for (i in 1:nrow(combined)) { - coef_g1 <- combined[[paste0(coef.col, "_g1")]][i] - coef_g2 <- combined[[paste0(coef.col, "_g2")]][i] - lwr_g1 <- combined[[paste0(lwr.col, "_g1")]][i] - upr_g1 <- combined[[paste0(upr.col, "_g1")]][i] - lwr_g2 <- combined[[paste0(lwr.col, "_g2")]][i] - upr_g2 <- combined[[paste0(upr.col, "_g2")]][i] - - R <- coef_g1 / coef_g2 - var_g1 <- (upr_g1 - lwr_g1)^2 / 4 - var_g2 <- (upr_g2 - lwr_g2)^2 / 4 - lwr.ci <- max(0, R - sqrt(var_g1 + var_g2)) - upr.ci <- R + sqrt(var_g1 + var_g2) - - # Guardar los resultados - results$R[i] <- round(R, 3) - results$lwr.ci[i] <- round(lwr.ci, 3) - results$upr.ci[i] <- round(upr.ci,3) + + # Extraer valores + th1 <- combined[[coef_g1]][i] + th2 <- combined[[coef_g2]][i] + l1 <- combined[[lwr_g1]][i] + u1 <- combined[[upr_g1]][i] + l2 <- combined[[lwr_g2]][i] + u2 <- combined[[upr_g2]][i] + + # Point ratio + R <- th1 / th2 + + # --- MOVER-R Components --- + # A = l1*(2*th1 - l1) ; B = u1*(2*th1 - u1) + # C = l2*(2*th2 - l2) ; D = u2*(2*th2 - u2) + A <- l1 * (2 * th1 - l1) + B <- u1 * (2 * th1 - u1) + C <- l2 * (2 * th2 - l2) + D <- u2 * (2 * th2 - u2) + + # Product m = theta1 * theta2 + m <- th1 * th2 + + # Initialize limits as NA + L <- NA_real_ + U <- NA_real_ + + # --- Lower limit --- + # D must be greater than 0, and the radicand must be greater than or equal to 0. + if (D > 0) { + rad_low <- m^2 - A * D + if (rad_low < 0) { + # If the radicand is slightly negative due to rounding, we set it to 0 + if (abs(rad_low) < 1e-12) rad_low <- 0 + } + if (rad_low >= 0) { + L <- (m - sqrt(rad_low)) / D + # By the definition of Aiken's V (ratio), it should not be negative, + # but if it is, we set it to 0 (conservative approach). + if (L < 0 && L > -1e-10) L <- 0 + } else { + warning(sprintf("Negative value found for the lower bound in Item '%s'. NA is assigned.", + combined$Item[i])) + } + } else { + warning(sprintf("D <= 0 for the item '%s'. The lower bound cannot be calculated (D = %f).", + combined$Item[i], D)) + } + + # --- Upper limit --- + # C > 0 and the square root is >= 0 are required + if (C > 0) { + rad_up <- m^2 - B * C + if (rad_up < 0) { + if (abs(rad_up) < 1e-12) rad_up <- 0 + } + if (rad_up >= 0) { + U <- (m + sqrt(rad_up)) / C + } else { + warning(sprintf("Negative value found for the upper limit in Item '%s'. NA is assigned.", + combined$Item[i])) + } + } else { + warning(sprintf("C <= 0 for the item '%s'. The upper bound cannot be calculated (C = %f).", + combined$Item[i], C)) + } + + # Assign results (rounded to 3 decimal places) + results$R[i] <- round(R, 3) + results$lwr.ci[i] <- round(L, 3) + results$upr.ci[i] <- round(U, 3) } return(results) } - diff --git a/R/CVC.R b/R/CVC.R index 8d53199..b55e84b 100644 --- a/R/CVC.R +++ b/R/CVC.R @@ -5,19 +5,20 @@ #'@param conf.level confidence level for confidence intervals (eg., .90, .95, .99). #'@param na.rm Logical. If FALSE (default) the function stops when missing values are detected. #' If TRUE rows with missing values in the relevant columns are removed before processing. -#' +#' #'@return dataframe with CVC coefficients and confidence intervals. #' #'@details #'This function calculates the content validity coefficient CVC (Hernandez-Nieto, 2002). Asymmetric confidence intervals are also calculated (Wilson, 1927; Penfield & Giacobbi, 2004). CVC' is the second coefficient that adjusts for possible random response of the raters, while another proposal for the CVI coefficient was created by Polit, & Beck (2007). #' -#'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. -#' #'@references -#'Hernandez-Nieto, R. A. (2002). Contributions to Statistical Analysis. Merida, Venezuela: Universidad de Los Andes. -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 -#'Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health. 2007;30(4):459-67. https://doi.org/10.1002/nur.20199 -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +#'Hernandez-Nieto, R. A. (2002). \emph{Contributions to Statistical Analysis}. Merida, Venezuela: Universidad de Los Andes. +#' +#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} +#' +#'Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. \emph{Research in Nursing & Health, 30}, 459-467. \doi{10.1002/nur.20199} +#' +#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} #' #'@seealso #'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval @@ -25,8 +26,6 @@ #' @author #' Cesar Merino-Soto (\email{sikayax@yahoo.com.ar}) #' -#' @export -#' #'@examples #'### Example 1 #'### Fictitious data Ej1: 15 judges, and 15 items evaluated by the judges @@ -59,16 +58,14 @@ #'## Run CVC #'CVC(random_data,max = 5, conf.level = .90) #' - - -# Funcion principal CVC +#' @export CVC <- function(data, max, conf.level, na.rm = FALSE) { - # Verificar si el data.frame contiene solo valores numericos + # Check whether the data.frame contains only numeric values if (!all(sapply(data, is.numeric))) { - stop("El data.frame debe contener solo valores numericos.") + stop("The data.frame must contain only numeric values.") } - # Detección de valores perdidos + # Detection of Missing Values if (!na.rm) { if (any(is.na(data))) { stop("Valores perdidos detectados. Usa na.omit() primero o establece na.rm=TRUE.") @@ -77,19 +74,19 @@ CVC <- function(data, max, conf.level, na.rm = FALSE) { data <- na.omit(data) } - # Numero de jueces (columnas) + # Number of judges (columns) num_jueces <- ncol(data) - # Calcular Pe segun la formula dada + # Calculate Pe using the given formula Pe <- (1 / num_jueces) ^ num_jueces - # Calcular la media de cada item (fila) + # Calculate the average for each item (row) medias_items <- rowMeans(data) - # Calcular el CVC para cada item + # Calculate the CVC for each item CVC_items <- round((medias_items - Pe) / max, 3) - # Calcular los intervalos de confianza de Wilson para cada CVC + # Calculate Wilson's confidence intervals for each CVC get_wilson_CI <- function(x, n, conf.level) { p_hat <- x @@ -104,7 +101,7 @@ CVC <- function(data, max, conf.level, na.rm = FALSE) { } intervalos_CI <- t(sapply(CVC_items, get_wilson_CI, n = num_jueces, conf.level = conf.level)) - # Crear un data.frame con los resultados + # data.frame with the results resultado_df <- data.frame( Item = 1:nrow(data), CVC = round(CVC_items, 3), diff --git a/R/CVI.R b/R/CVI.R index b6b9e12..9a30cf6 100644 --- a/R/CVI.R +++ b/R/CVI.R @@ -1,4 +1,4 @@ -#'@title Content Validity Index +#'@title Content Validity Index (CVI) #'@description Calculate the Content Validity Index (CVI), without adjustment for random agreement. #'@param data dataframe, with the columns assigned to each judge, and the rows assigned to each assessed item. #'@param cut Specific cut-off point, from which the item is considered valid (or relevant, clear, etc.) @@ -17,26 +17,25 @@ #'The usual CVI cut-off point for identifying valid from invalid items is generally in the top two ratings (3 or higher on a relevance scale of 1 to 4; Beck & Gable, 2001; Grant & Davis, 1997). 'cut' dichotomizes the judges' responses to calculate CVI. #'The \strong{CVI} function can be used for rated items with any rating range, and any chosen cut point ('cut'). #'Asymmetric confidence intervals use Wilson's (1927) approach, as used for Aiken's V coefficient (Penfield, & Giacobbi, 2004). -#'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. #' #'@references -#'Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. Applied Nursing Research, 5, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 +#'Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. \emph{Applied Nursing Research, 5}, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 #' -#'Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. Research in Nursing & Health, 20, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g +#'Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. \emph{Research in Nursing & Health, 20}, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g #' -#'Lynn, M.R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382-385. +#'Lynn, M.R. (1986). Determination and quantification of content validity. \emph{Nursing Research, 35}, 382-385. #' -#'Martuza, V.R. (1977). Applying norm-referenced and criterion-referenced measurement in education. Boston: Allyn & Bacon +#'Martuza, V.R. (1977). \emph{Applying norm-referenced and criterion-referenced measurement in education}. Boston: Allyn & Bacon #' -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} #' -#'Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health, 29(5), 489-497. https://doi.org/10.1002/nur.20147 +#'Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. \emph{Research in Nursing & Health, 29}(5), 489-497. \doi{10.1002/nur.20147} #' -#'Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health, 30: 459-467. https://doi.org/10.1002/nur.20199 +#'Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. \emph{Research in Nursing & Health, 30}, 459-467. \doi{10.1002/nur.20199} #' -#'Waltz, C.F., & Bausell, R.B. (1981). Nursing research: Design, statistics, and computer analysis. Philadelphia: F. A. Davis. +#'Waltz, C.F., & Bausell, R.B. (1981). \emph{Nursing research: Design, statistics, and computer analysis}. Philadelphia: F. A. Davis. #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} #' #'@seealso #'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval @@ -64,14 +63,9 @@ #'# Run CVI #'CVI(data = Ej1, cut = 4, conf.level = .90) #' -#' @author -#'Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) -#' #'@export -#' -#' CVI <- function(data, cut, conf.level, na.rm = FALSE) { - # Funcion interna para calcular el intervalo de confianza de Wilson + # Built-in function for calculating the Wilson confidence interval get_wilson_CI <- function(x, n, conf.level) { p_hat <- x SE_hat_sq <- p_hat * (1 - p_hat) / n @@ -84,29 +78,29 @@ CVI <- function(data, cut, conf.level, na.rm = FALSE) { return(CI) } - # Verificar si los datos son numericos + # Check whether the data is numeric if (!all(sapply(data, is.numeric))) { - stop("Todas las columnas deben contener datos numericos.") + stop("All columns must contain numeric data.") } - # Detección de valores perdidos + # Detection of Missing Values if (!na.rm) { if (any(is.na(data))) { - stop("Valores perdidos detectados. Usa na.omit() primero o establece na.rm=TRUE.") + stop("Missing values detected. Use na.omit() first, or set na.rm=TRUE.") } } else { data <- na.omit(data) } - # Numero total de jueces + # Total number of judges num_jueces <- ncol(data) - # Inicializar listas para almacenar resultados + # Initialize lists to store results CVI_vals <- numeric(nrow(data)) lwr_cis <- numeric(nrow(data)) upp_cis <- numeric(nrow(data)) - # Calcular CVI, porcentaje de acuerdo y CI de Wilson para cada item + # Calculate CVI, agreement percentage, and Wilson's CI for each item for (i in 1:nrow(data)) { Na <- sum(data[i, ] >= cut) Nna <- num_jueces - Na @@ -114,13 +108,13 @@ CVI <- function(data, cut, conf.level, na.rm = FALSE) { CVI_vals[i] <- Na / N - # Calcular intervalo de confianza de Wilson + # Calculate the Wilson confidence interval CI <- get_wilson_CI(CVI_vals[i], N, conf.level) lwr_cis[i] <- CI['lower'] upp_cis[i] <- CI['upper'] } - # Crear un data frame con los resultados y redondear a 3 decimales + # Data frame with the results, rounded to 3 decimal places resultado <- data.frame( Item = 1:nrow(data), CVI = round(CVI_vals, 3), diff --git a/R/CVIR.R b/R/CVIR.R index 0aefe06..7082ae8 100644 --- a/R/CVIR.R +++ b/R/CVIR.R @@ -1,4 +1,4 @@ -#'@title Content Validity Index Revised (with random agreement adjustment) +#'@title Content Validity Index Revised CVI-R (with random agreement adjustment) #'@description Calculate the CVI coefficient with random agreement adjustment (Polit & Beck, 2007). #'@param data dataframe, with the columns assigned to each judge, and the rows assigned to each evaluated item. #'@param cut Specific cut-off point in the response rating, from which the item is considered valid (or relevant, clear, etc.). @@ -17,23 +17,23 @@ #'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. #' #'@references -#'Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. Applied Nursing Research, 5, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 +#'Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. \emph{Applied Nursing Research, 5}, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 #' -#'Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. Research in Nursing & Health, 20, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g +#'Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. \emph{Research in Nursing & Health, 20}, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g #' -#'Lynn, M.R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382-385. https://doi.org/10.1097/00006199-198611000-00017 +#'Lynn, M.R. (1986). Determination and quantification of content validity. \emph{Nursing Research, 35}, 382-385. #' -#'Martuza, V.R. (1977). Applying norm-referenced and criterion-referenced measurement in education. Boston: Allyn & Bacon +#'Martuza, V.R. (1977). \emph{Applying norm-referenced and criterion-referenced measurement in education}. Boston: Allyn & Bacon #' -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} #' -#'Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health, 29(5), 489-497. https://doi.org/10.1002/nur.20147 +#'Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. \emph{Research in Nursing & Health, 29}(5), 489-497. \doi{10.1002/nur.20147} #' -#'Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health, 30: 459-467. https://doi.org/10.1002/nur.20199 +#'Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. \emph{Research in Nursing & Health, 30}, 459-467. \doi{10.1002/nur.20199} #' -#'Waltz, C.F., & Bausell, R.B. (1981). Nursing research: Design, statistics, and computer analysis. Philadelphia: F. A. Davis. +#'Waltz, C.F., & Bausell, R.B. (1981). \emph{Nursing research: Design, statistics, and computer analysis}. Philadelphia: F. A. Davis. #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. doi: 10.2307/2276774 +#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} #' #'@seealso #'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval @@ -68,7 +68,7 @@ #' #'@export CVIR <- function(data, cut, conf.level) { - # FunciOn interna para calcular el intervalo de confianza de Wilson + # Built-in function for calculating the Wilson confidence interval get_wilson_CI <- function(x, n, conf.level) { p_hat <- x SE_hat_sq <- p_hat * (1 - p_hat) / n @@ -80,48 +80,55 @@ CVIR <- function(data, cut, conf.level) { 'upper' = omega * (A + B)) return(CI) } - # FunciOn interna para calcular P_C usando la fOrmula combinatoria + # Internal function to calculate Pc using the combinatorial formula calculate_pc <- function(N, A) { if (A > N || A < 0) { - return(0) # Evita errores si A es mayor que N o menor que 0 + return(0) # Avoid errors if A is greater than N or less than 0 } - # Aplicamos la fOrmula combinatoria + # We apply the combinatorial formula Pc <- (factorial(N) / (factorial(A) * factorial(N - A))) * (0.5^N) return(Pc) } - # Verificar si los datos son numEricos + # Check whether the data is numeric if (!all(sapply(data, is.numeric))) { - stop("Todas las columnas deben contener datos numEricos.") + stop("All columns must contain numeric data.") } - # NUmero total de jueces + # Total number of judges num_jueces <- ncol(data) - # Inicializar listas para almacenar resultados + + # Initialize lists to store results cvipb_vals <- numeric(nrow(data)) lwr_cis <- numeric(nrow(data)) upr_cis <- numeric(nrow(data)) - # Calcular CVI ajustado para cada Item + + # Calculate the adjusted CVI for each item for (i in 1:nrow(data)) { - Na <- sum(data[i, ] >= cut) # Jueces en acuerdo - Nna <- num_jueces - Na # Jueces en desacuerdo - N <- Na + Nna # Total de jueces + Na <- sum(data[i, ] >= cut) # Judges in agreement + Nna <- num_jueces - Na # Judges in disagree + N <- Na + Nna # Total number of judges CVI_val <- Na / N - # Calcular P_C usando la nueva fOrmula combinatoria dentro de la funciOn + + # Calculate Pc using the new combinatorial formula within the function P_C <- calculate_pc(N, Na) - # Calcular kappa de Lynn + + # Calculate Lynn's kappa* (CVI-R) cvipb_vals[i] <- (CVI_val - P_C) / (1 - P_C) - # Calcular intervalo de confianza de Wilson para kappa usando el valor absoluto + + # Calculate the Wilson confidence interval for kappa using the absolute value abs_kappa <- abs(cvipb_vals[i]) CI_kappa <- get_wilson_CI(abs_kappa, N, conf.level) - # Si kappa es negativo, devolver el signo negativo a los lImites del CI + + # If kappa is negative, return the negative sign to the CI limits if (!is.na(cvipb_vals[i]) && cvipb_vals[i] < 0) { - lwr_cis[i] <- -CI_kappa['upper'] # Intercambia el lImite superior e inferior + lwr_cis[i] <- -CI_kappa['upper'] # Swap the upper and lower limits upr_cis[i] <- -CI_kappa['lower'] } else { lwr_cis[i] <- CI_kappa['lower'] upr_cis[i] <- CI_kappa['upper'] } } - # Crear un data frame con los resultados y redondear a 3 decimales + + # Create a data frame with the results and round to 3 decimal places resultado <- data.frame( Item = 1:nrow(data), CVIR = round(cvipb_vals, 3), diff --git a/R/HAiken.R b/R/HAiken.R index b2d8e88..6bee249 100644 --- a/R/HAiken.R +++ b/R/HAiken.R @@ -6,26 +6,25 @@ #'@param conf.level confidence level for the confidence intervals (eg., .90, .95, .99) #'@param na.rm Logical. If FALSE (default) the function stops when missing values are detected. #' If TRUE rows with missing values in the relevant columns are removed before processing. -#' +#' #'@return #'dataframe with H coefficients for all items analyzed, and their confidence intervals. #' #'@details #'Compute the H coefficient (Aiken, 1980, 1985) to estimate the homogeneity of response of the judges/scorers to the items. #'To maintain consistency with the methods usually associated with content validity, 'HAiken' is proposed as an option. -#''HAiken' also compute asymmetric confidence intervals use the method of Wilson (1927). and adapted by Penfield and Giacobbi (2004) for Aiken's V coefficient. +#''HAiken' also compute asymmetric confidence intervals use the method of Wilson (1927), and adapted by Penfield and Giacobbi (2004) for Aiken's V coefficient. #'The H coefficient, or equivalent coefficients, should complement the results of the content validity coefficients. #'Other methods for estimating judges' agreement or homogeneity of response may also be useful. -#'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. #' #'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} #' -#'Penfield, R. D., & Miller, J. M. (2004). Improving Content Validation Studies Using an Asymmetric Confidence Interval for the Mean of Expert Ratings. Applied Measurement in Education, 17(4), 359–370. https://doi.org/10.1207/s15324818ame1704_2 +#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} #' #'@seealso #'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval @@ -59,23 +58,22 @@ #' #'@export Haiken <- function(data, ncat, conf.level, na.rm = FALSE) { - - # Detección de valores perdidos + # Detection of Missing Values if (!na.rm) { if (any(is.na(data))) { - stop("Valores perdidos detectados. Usa na.omit() primero o establece na.rm=TRUE.") + stop("There are missing values. Use na.omit() first, or set na.rm=TRUE.") } } else { data <- na.omit(data) } - # Numero de filas + # Number of rows num_filas <- nrow(data) - # Calcular j segun si ncol(data) es par o impar + # Calculate j depending on whether ncol(data) is even or odd j <- ifelse(ncol(data) %% 2 == 0, 0, 1) - # data frame para almacenar resultados + # data frame for storing results resultados <- data.frame( Item = 1:num_filas, H = numeric(num_filas), @@ -84,22 +82,22 @@ Haiken <- function(data, ncat, conf.level, na.rm = FALSE) { n.subj = numeric(num_filas) ) - # Calcular las diferencias por pares (diferencias absolutas) y los intervalos de confianza para cada fila + # Calculate the pairwise differences (absolute differences) and confidence intervals for each row for (i in 1:num_filas) { fila_actual <- data[i, ] - # Calcular las diferencias por pares como diferencias absolutas y desempaquetarlas + # Calculate the pairwise differences as absolute differences and unpack them diff_abs <- combn(colnames(fila_actual), 2, function(pair) { col1 <- fila_actual[pair[1]] col2 <- fila_actual[pair[2]] diff_abs <- abs(col1 - col2) return(diff_abs) - }, simplify = FALSE) # Utilizar simplify = FALSE para obtener una lista + }, simplify = FALSE) # Use `simplify = FALSE` to get a list - # Desempaquetar las diferencias absolutas de la lista + # Unpack the absolute differences from the list diff_abs_unlist <- unlist(diff_abs) - # Calcular el resultado para la fila actual usando ncat, diff_abs_unlist y j + # Calculate the result for the current row using ncat, diff_abs_unlist, and j resultado_actual <- 1 - (4 * sum(diff_abs_unlist) / ((ncat - 1) * (ncol(data)^2) - j)) get_wilson_CI <- function(x, n, conf.level) { @@ -114,10 +112,11 @@ Haiken <- function(data, ncat, conf.level, na.rm = FALSE) { 'upper' = omega * (A + B)) return(CI) } - # Calcular el intervalo de confianza con la funcion get_wilson_CI y el nivel de confianza alpha + + # Calculate the confidence interval using the get_wilson_CI function and the alpha confidence level wilson_interval <- get_wilson_CI(resultado_actual, ncol(data), conf.level) - # Almacenar los resultados en el data frame + # Store the results in the data frame resultados[i, "H"] <- resultado_actual resultados[i, "lwr.ci"] <- wilson_interval['lower'] resultados[i, "upr.ci"] <- wilson_interval['upper'] diff --git a/R/MDScontent.R b/R/MDScontent.R index a15dc2f..bbfac47 100644 --- a/R/MDScontent.R +++ b/R/MDScontent.R @@ -44,6 +44,13 @@ #' } #' #' @details +#' ## Operational rationale +#' The MDS method requires a comparative design for obtaining the judges' ratings +#' for each item. Each item must be evaluated against several attributes (not just one), +#' and the judge must rate how well each item corresponds to each attribute. This format +#' is consistent with the substantive validity method (\code{\link[ValCont:CIDsingle]{ValCont::CIDsingle}}) +#' and Hinkey’s approach \code{\link[ValCont:HT]{ValCont::HT}} +#' #' ## Conceptual rationale #' The function provides a **geometric visualization** of content validity structure. #' Ratings from judges are first aggregated into an Items × Traits profile matrix. @@ -63,7 +70,7 @@ #' ## Interpretation #' This function is intended primarily for **visual diagnostic purposes**. #' It does not replace quantitative content validity coefficients (e.g., Aiken's V, -#' CVI, SVAL, or Hit-based approaches), but complements them by examining +#' Polit's CVI, Anderson-Gerbing's, or Colquit's approaches), but complements them by examining #' structural coherence. #' #' The map may be interpreted as follows: @@ -94,7 +101,7 @@ #'# was evaluated for its correspondence to four attributes. The database should #'# be structured as follows: #'# -#'# item item1.judge1 item1.judge2 item1.judge3 item1.judge4 item2.judge1 item2.judge2 ... +#'# item item1.jdg1 item1.jde2 item1.jde3 item1.jde4 item2.jde1 item2.jde2 ... #'# #'# For this example: #' diff --git a/R/MER.R b/R/MER.R index ee79bc5..7ee9571 100644 --- a/R/MER.R +++ b/R/MER.R @@ -1,5 +1,5 @@ #'@title Mean of expert ratings (MER) -#'@description Calculate the average score, with asymmetric confidence intervals. +#'@description Calculate the Mean of expert ratings (MER), with asymmetric confidence intervals. #'@param data dataframe, with the columns assigned to each judge, and the rows assigned to each evaluated item. #'@param ncat Number of possible values or categories used in rating #'@param start Minimum possible value (0 or 1) @@ -16,21 +16,21 @@ #'The results should be supplemented with an estimator of inter-judge variability or agreement. #' #'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} #' -#'Merino-Soto, C., & Livia-Segovia, J. (2022). Rating mean of expert judges and asymmetric confidence intervals in content validity: an SPSS syntax. Anales de Psicologia, 38(2), 395-398. https://doi.org/10.6018/analesps.489431 +#'Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} #' -#'Miller, J. M., & Penfield, R. D. (2005). Using the score method to construct asymmetric confidence intervals: An SAS program for content validation in scale development. Behavior Research Methods, 37, 450-452. https://doi.org/10.3758/BF03192713 +#'Miller, J. M., & Penfield, R. D. (2005). Using the score method to construct asymmetric confidence intervals: An SAS program for content validation in scale development. \emph{Behavior Research Methods, 37}, 450-452. \doi{10.3758/BF03192713} #' -#'Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals for the mean of a rating scale item. Psychological methods, 8(2), 149-163. https://doi.org/10.1037/1082-989x.8.2.149 +#'Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals for the mean of a rating scale item. \emph{Psychological methods, 8}(2), 149-163. \doi{10.1037/1082-989x.8.2.149} #' -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} #' -#'Penfield, R. D., & Miller, J. M. (2004). Improving Content Validation Studies Using an Asymmetric Confidence Interval for the Mean of Expert Ratings. Applied Measurement in Education, 17(4), 359–370. https://doi.org/10.1207/s15324818ame1704_2 +#'Penfield, R. D., & Miller, J. M. (2004). Improving Content Validation Studies Using an Asymmetric Confidence Interval for the Mean of Expert Ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. \doi{10.1207/s15324818ame1704_2} #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} #' #'@seealso #'\code{\link[ValCont:Haiken]{ValCont::HAiken}} for a homogeneity coefficient diff --git a/R/VAiken.R b/R/VAiken.R index 76d7f7f..6ef19c0 100644 --- a/R/VAiken.R +++ b/R/VAiken.R @@ -7,7 +7,7 @@ #' @param conf.level confidence level for confidence intervals (ex., .90, .95, .99) #' @param na.rm Logical. If FALSE (default) the function stops when missing values are detected. #' If TRUE rows with missing values in the relevant columns are removed before processing. -#' +#' #'@return #'dataframe with V coefficients for all items analyzed, and confidence intervals #' @@ -19,15 +19,15 @@ #' #' #'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} #' -#'Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. Anales de Psicologia, 25(1), 169-171. https://revistas.um.es/analesps/article/view/71631 +#'Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} #' -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} #' #'@seealso #'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval @@ -52,10 +52,10 @@ Vaiken <- function(data, min, max, conf.level = 0.95, na.rm = FALSE) { - # Detección de valores perdidos + # Detection of Missing Values if (!na.rm) { if (any(is.na(data))) { - stop("Valores perdidos detectados. Usa na.omit() primero o establece na.rm=TRUE.") + stop("Missing values detected. Use na.omit() first, or set na.rm=TRUE.") } } else { data <- na.omit(data) @@ -63,33 +63,33 @@ Vaiken <- function(data, min, max, conf.level = 0.95, na.rm = FALSE) { # Validation: data if (!is.data.frame(data)) { - stop("El argumento 'data' debe ser un data.frame.") + stop("The 'data' argument must be a data.frame.") } if (!is.numeric(as.matrix(data))) { - stop("El 'data' debe contener solo valores numericos.") + stop("The 'data' field must contain only numeric values.") } # Validation: min, max if (!is.numeric(min) || !is.numeric(max)) { - stop("'min' y 'max' deben ser valores numericos.") + stop("'min' and 'max' must be numeric values.") } if (min >= max) { - stop("'min' debe ser menor que 'max'.") + stop("'min' must be less than 'max'.") } # Validation: confidence level if (!is.numeric(conf.level) || conf.level <= 0 || conf.level >= 1) { - stop("'conf.level' debe estar entre 0 y 1 (excluyendo los extremos).") + stop("'conf.level' must be between 0 and 1 (excluding the extremes).") } # Verification: NA if (any(is.na(data))) { - stop("'data' contiene valores NA. Por favor, limpiar los datos antes de usar la funcion.") + stop("'data' contains NA values. Please clean the data before using the function.") } # Verification: valid range of data if (any(data < min | data > max)) { - stop("Todas las puntuaciones en 'data' deben estar entre 'min' y 'max'.") + stop("All scores in 'data' must be between 'min' and 'max'.") } # N judges @@ -100,8 +100,8 @@ Vaiken <- function(data, min, max, conf.level = 0.95, na.rm = FALSE) { # Calculate V Aiken and confidence interval calcular_valores <- function(puntajes) { - M <- mean(puntajes) # Media de las calificaciones - V <- (M - min) / (max - min) # C?lculo de V de Aiken + M <- mean(puntajes) # Average of the scores + V <- (M - min) / (max - min) # Calculation of Aiken's V # lower and upper limits, method Wilson's score nk <- n * (max - min) diff --git a/R/Vaikenpub.R b/R/Vaikenpub.R index 2905e17..cbc719c 100644 --- a/R/Vaikenpub.R +++ b/R/Vaikenpub.R @@ -9,17 +9,19 @@ #' @param n Integer. Number of judges or raters who rated each item. #' @param lo Numeric. Minimum value of the rating scale. #' @param hi Numeric. Maximum value of the rating scale. -#' @param conf.level Confidence level for the Wilson interval (default = 0.95). +#' @param conf.level Confidence level for the Wilson interval (default = 0.90). #' @param item.names Optional. Vector of item names. If NULL, defaults to Item1, Item2, etc. #' #' @return A data.frame with item names, V values, and lower and upper confidence intervals. #' #' @references -#' Penfield, R. D., & Giacobbi, P. R., Jr. (2004). Applying a score confidence interval to Aiken’s item content-relevance index. -#' \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213–225. #' -#' Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. -#' \emph{Journal of the American Statistical Association, 22}, 209–212. +#' Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} +#' +#' Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} +#' +#' Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} +#' #' #' @examples #' \donttest{ @@ -41,21 +43,21 @@ Vaikenpub <- function(V, n, lo, hi, conf.level = 0.90, item.names = NULL) { B <- crit * sqrt(p_hat * (1 - p_hat) / N + crit^2 / (4 * N^2)) c(lower = omega * (A - B), upper = omega * (A + B)) } - + if (any(V < 0 | V > 1)) stop("All V values must be between 0 and 1.") if (is.null(item.names)) item.names <- paste0("Item", seq_along(V)) - + k <- hi - lo N <- n * k - + lwr <- upr <- numeric(length(V)) - + for (i in seq_along(V)) { CI <- get_wilson_CI(V[i], N, conf.level) lwr[i] <- CI["lower"] upr[i] <- CI["upper"] } - + data.frame( Item = item.names, V = round(V, 3), diff --git a/man/CID.Rd b/man/CID.Rd index 7be00b4..0280da0 100644 --- a/man/CID.Rd +++ b/man/CID.Rd @@ -37,11 +37,9 @@ Calculates confidence interval for the difference of content validity coefficien `CID` uses Method of Variance Estimates Recovery (MOVER; Zou, & Donner, 2008). Because data produced by judges' judgments tend to be asymmetrically distributed (if the item is rated, on a scale of 1 to 5, as predominantly valid then its values will be > 3), MOVER is appropriate for non-normal distributions. MOVER depends on the quality or precision of the confidence intervals calculated for the coefficients in each group. -The application of MOVER for content validity coefficient was initially published by Merino-Soto (2018) for the difference between V coefficients (Aiken, 1980, 1985). Later, Merino-Soto (2023) extended this approach for Aiken's V by adding a point estimator of the difference, based on the standardized difference between proportions +The application of MOVER for content validity coefficient was initially published by Merino-Soto (2018) for the difference between V coefficients (Aiken, 1980, 1985). Later, Merino-Soto (2023) extended this approach for Aiken's V by adding a point estimator of the difference, based on the standardized difference between proportions. The compared content validity coefficients obtained should be of the same type, and the estimated confidence intervals for these coefficients should also come from the same level; for example, at .95 or .90. -Singer (2010) observed that at extremely low values (e.g., proportions near .0), the coverage of this method is not as good. In the context of comparing content validity coefficients, treated as proportions, it is rare to find such low coefficients (and their confidence intervals). Unless the items are extremely poor in content. - -Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. +Singer (2010) observed that at extremely low values (e.g., proportions near .0), the coverage of this method is not as good. In the context of comparing content validity coefficients, treated as proportions, it is rare to find such very low coefficients (and their confidence intervals). Unless the items are extremely poor in content. } \examples{ @@ -104,17 +102,17 @@ CID(group1 = output.V2, } \references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} -Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. Anales de Psicologia, 34(3), 587-590. https://doi.org/10.6018/analesps.34.3.283481 +Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. \emph{Anales de Psicologia, 34}(3), 587-590. \doi{10.6018/analesps.34.3.283481} -Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. MHSalud, 20(1), 23-32. https://doi.org/10.15359/mhs.20-1.3 +Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. \emph{MHSalud, 20}(1), 23-32. \doi{10.15359/mhs.20-1.3} -Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702. Statistics in Medicine, 29(16), 1757–1759. https://doi.org/10.1002/sim.3887 +Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702. \emph{Statistics in Medicine, 29}(16), 1757–1759. \doi{10.1002/sim.3887} -Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. Stat. Med. 27, 1693–1702. https://doi.org//10.1002/sim.3095 +Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. \emph{Statistics in Medicine, 27}, 1693–1702. \doi{10.1002/sim.3095} } \seealso{ \code{\link[ratesci:moverci]{ratesci::moverci}} diff --git a/man/CIDsingle.Rd b/man/CIDsingle.Rd index 15d0153..27d06b5 100644 --- a/man/CIDsingle.Rd +++ b/man/CIDsingle.Rd @@ -54,17 +54,17 @@ CIDsingle( } \references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} -Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. Anales de Psicología, 34(3), 587-590. https://doi.org/10.6018/analesps.34.3.283481. +Merino-Soto, C. (2018) Confidence interval for difference between coefficients of content validity (Aiken's V): a SPSS syntax. \emph{Anales de Psicologia, 34}(3), 587-590. \doi{10.6018/analesps.34.3.283481} -Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. MHSalud, 20(1), 23-32. https://doi.org/10.15359/mhs.20-1.3 +Merino-Soto, C. (2023). Coeficientes V de Aiken: diferencias en los juicios de validez de contenido. \emph{MHSalud, 20}(1), 23-32. \doi{10.15359/mhs.20-1.3} -Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702.Statistics in Medicine, 29(16), 1757–1759. https://doi.org/10.1002/sim.3887 +Singer, J. (2010). Construction of confidence limits about effect measures: A general approach, by G. Y. Zou and A. Donner, Statistics in Medicine 2008; 27:1693-1702. \emph{Statistics in Medicine, 29}(16), 1757–1759. \doi{10.1002/sim.3887} -Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. Stat. Med. 27, 1693–1702. https://doi.org/10.1002/sim.3095 +Zou, G.Y. and Donner, A. (2008) Construction of confidence limits about effect measures: a general approach. \emph{Statistics in Medicine, 27}, 1693–1702. \doi{10.1002/sim.3095} } \seealso{ \code{\link[ratesci:moverci]{ratesci::moverci}} for MOVER method ofr ratios diff --git a/man/CIR.Rd b/man/CIR.Rd index 32bf54d..79e8375 100644 --- a/man/CIR.Rd +++ b/man/CIR.Rd @@ -2,77 +2,54 @@ % Please edit documentation in R/CIR.R \name{CIR} \alias{CIR} -\title{Confidence Intervals of Ratios of content validity coefficients} +\title{MOVER-R Confidence Interval for the Ratio of Content Validity Coefficients} \usage{ CIR(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) } \arguments{ -\item{group1}{Output dataframe 1 of one of the 'ValCont' functions to obtain content validity coefficients.} +\item{group1}{data.frame containing estimates and CIs for the first group. +Must include columns specified in \code{coef.col}, \code{lwr.col}, +\code{upr.col}, and an \code{"Item"} column for matching.} -\item{group2}{Output Output dataframe 2 of one of the 'ValCont' functions to obtain content validity coefficients.} +\item{group2}{data.frame containing estimates and CIs for the second group. +Same column requirements as \code{group1}.} -\item{coef.col}{Name of the column in the dataframe storing the calculated coefficient} +\item{coef.col}{Character string. Name of the column with the point estimates.} -\item{lwr.col}{Name of the column in the dataframe that stores the lower bound of the confidence interval} +\item{lwr.col}{Character string. Name of the column with the lower bounds.} -\item{upr.col}{Name of the column in the dataframe that stores the upper limit of the confidence interval} +\item{upr.col}{Character string. Name of the column with the upper bounds.} -\item{na.rm}{Logical. If FALSE (default) the function stops when missing values are detected. -If TRUE rows with missing values in the relevant columns are removed before processing.} +\item{na.rm}{Logical. If \code{TRUE}, rows with missing values in the relevant +columns are removed with a warning. If \code{FALSE}, missing values +trigger an error.} } \value{ -dataframe with four columns: label of the items, ratio between the coefficients, the upper and upper limit of the confidence interval of the difference. +A data.frame with columns: + \item{Item}{Common item names between both groups.} + \item{R}{Estimated ratio (coefficient group1 / coefficient group2).} + \item{lwr.ci}{Lower bound of the MOVER-R confidence interval.} + \item{upr.ci}{Upper bound of the MOVER-R confidence interval.} } \description{ -Calculates confidence interval for the ratio of two content validity coefficients, based on method of variance recovery for ratios (MOVER-R). -} -\details{ -'CIR' uses method of variance recovery for ratios (MOVER-R; Zou, Donner, & Qiu, 2025), as a general approach for two non-normal quantities. -Because data produced by judges' judgments tend to be asymmetrically distributed (if the item is rated, on a scale of 1 to 5, as predominantly valid then its values will be > 3), MOVER-R is appropriate for non-normal distributions. -MOVER-R depends on the quality or precision of the confidence intervals calculated for the coefficients in each group. -The compared content validity coefficients obtained should be of the same type, and the estimated confidence intervals for these coefficients should also come from the same level; for example, at .95 or .90. -If the two dataframes have different numbers of evaluated items (i.e., different numbers of rows), 'CIR' function matches the commonly labeled items, assuming they are the same items. -Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. +Computes the confidence interval for the ratio of two independent +content validity coefficients using the MOVER-R (Method of Variance Estimates Recovery for Ratios) +closed-form procedure. } \examples{ - -### Example 1 ----------- - -## Random data (Low ratings): 11 items (rows), 4 raters (columns) -Data1 <- data.frame( - juez1 = sample(1:2, 11, replace = TRUE), - juez2 = sample(2:3, 11, replace = TRUE), - juez3 = sample(1:2, 11, replace = TRUE), - juez4 = sample(1:3, 11, replace = TRUE)) - -## Random data (High ratings): 10 items (rows), 6 raters (columns) -Data2 <- data.frame( - obs1 = sample(5:7, 10, replace = TRUE), - obs2 = sample(4:7, 10, replace = TRUE), - obs3 = sample(4:7, 10, replace = TRUE), - obs4 = sample(5:7, 10, replace = TRUE), - obs5 = sample(5:7, 10, replace = TRUE), - obs6 = sample(6:7, 10, replace = TRUE)) - -## Saving results from Aiken's V analysis, for each group -group1 <- Vaiken(data = Data1, min = 1, max = 7, conf.level = .90) -group2 <- Vaiken(data = Data2, min = 1, max = 7, conf.level = .90) - -CIR(group1 = group1, - group2 = group2, - coef.col = "V", - lwr.col = "lwr.ci", - upr.col = "upr.ci") - - +\dontrun{ +# Assuming you have data frames 'g1' and 'g2' with columns 'est', 'low', 'high' +result <- CIR(g1, g2, coef.col = "est", lwr.col = "low", upr.col = "high") +print(result) } -\references{ -Zou, G., Donner, A. & Qiu, S. (2025). MOVER-R for Confidence Intervals of Ratios. In N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, F. Ruggeri and J.L. Teugels (Eds.), Wiley StatsRef: Statistics Reference Online. https://doi.org/10.1002/9781118445112.stat08085 } -\seealso{ -\code{\link[ratesci:moverci]{ratesci::moverci}} -\code{\link[ValCont:CID]{ValCont::CID}} -} -\author{ -Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) +\references{ +Donner, A., & Zou, G. Y. (2012). Closed-form confidence intervals for +functions of the normal mean and standard deviation. \emph{Statistical +Methods in Medical Research}, 21(4), 347-359. + +Zou, G., Donner, A., & Qiu, S. (2025). MOVER-R for Confidence Intervals +of Ratios. In N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, +F. Ruggeri and J.L. Teugels (Eds.), \emph{Wiley StatsRef: Statistics +Reference Online}. https://doi.org/10.1002/9781118445112.stat08085 } diff --git a/man/CVC.Rd b/man/CVC.Rd index 7801342..8948785 100644 --- a/man/CVC.Rd +++ b/man/CVC.Rd @@ -24,8 +24,6 @@ Calculates the content validity coefficient (CVC; Hernandez-Nieto, 2002). CVC ma } \details{ This function calculates the content validity coefficient CVC (Hernandez-Nieto, 2002). Asymmetric confidence intervals are also calculated (Wilson, 1927; Penfield & Giacobbi, 2004). CVC' is the second coefficient that adjusts for possible random response of the raters, while another proposal for the CVI coefficient was created by Polit, & Beck (2007). - -Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. } \examples{ ### Example 1 @@ -61,10 +59,13 @@ CVC(random_data,max = 5, conf.level = .90) } \references{ -Hernandez-Nieto, R. A. (2002). Contributions to Statistical Analysis. Merida, Venezuela: Universidad de Los Andes. -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 -Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health. 2007;30(4):459-67. https://doi.org/10.1002/nur.20199 -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +Hernandez-Nieto, R. A. (2002). \emph{Contributions to Statistical Analysis}. Merida, Venezuela: Universidad de Los Andes. + +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} + +Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. \emph{Research in Nursing & Health, 30}, 459-467. \doi{10.1002/nur.20199} + +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } \seealso{ \code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval diff --git a/man/CVI.Rd b/man/CVI.Rd index 02f022e..0c6e3e0 100644 --- a/man/CVI.Rd +++ b/man/CVI.Rd @@ -2,7 +2,7 @@ % Please edit documentation in R/CVI.R \name{CVI} \alias{CVI} -\title{Content Validity Index} +\title{Content Validity Index (CVI)} \usage{ CVI(data, cut, conf.level, na.rm = FALSE) } @@ -31,7 +31,6 @@ The usual labels to evaluate the relevance for each item are: not relevant, some The usual CVI cut-off point for identifying valid from invalid items is generally in the top two ratings (3 or higher on a relevance scale of 1 to 4; Beck & Gable, 2001; Grant & Davis, 1997). 'cut' dichotomizes the judges' responses to calculate CVI. The \strong{CVI} function can be used for rated items with any rating range, and any chosen cut point ('cut'). Asymmetric confidence intervals use Wilson's (1927) approach, as used for Aiken's V coefficient (Penfield, & Giacobbi, 2004). -Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. } \examples{ @@ -58,27 +57,24 @@ CVI(data = Ej1, cut = 4, conf.level = .90) } \references{ -Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. Applied Nursing Research, 5, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 +Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. \emph{Applied Nursing Research, 5}, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 -Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. Research in Nursing & Health, 20, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g +Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. \emph{Research in Nursing & Health, 20}, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g -Lynn, M.R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382-385. +Lynn, M.R. (1986). Determination and quantification of content validity. \emph{Nursing Research, 35}, 382-385. -Martuza, V.R. (1977). Applying norm-referenced and criterion-referenced measurement in education. Boston: Allyn & Bacon +Martuza, V.R. (1977). \emph{Applying norm-referenced and criterion-referenced measurement in education}. Boston: Allyn & Bacon -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} -Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health, 29(5), 489-497. https://doi.org/10.1002/nur.20147 +Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. \emph{Research in Nursing & Health, 29}(5), 489-497. \doi{10.1002/nur.20147} -Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health, 30: 459-467. https://doi.org/10.1002/nur.20199 +Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. \emph{Research in Nursing & Health, 30}, 459-467. \doi{10.1002/nur.20199} -Waltz, C.F., & Bausell, R.B. (1981). Nursing research: Design, statistics, and computer analysis. Philadelphia: F. A. Davis. +Waltz, C.F., & Bausell, R.B. (1981). \emph{Nursing research: Design, statistics, and computer analysis}. Philadelphia: F. A. Davis. -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } \seealso{ \code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval } -\author{ -Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) -} diff --git a/man/CVIR.Rd b/man/CVIR.Rd index 4acb59b..ffec15f 100644 --- a/man/CVIR.Rd +++ b/man/CVIR.Rd @@ -2,7 +2,7 @@ % Please edit documentation in R/CVIR.R \name{CVIR} \alias{CVIR} -\title{Content Validity Index Revised (with random agreement adjustment)} +\title{Content Validity Index Revised CVI-R (with random agreement adjustment)} \usage{ CVIR(data, cut, conf.level) } @@ -54,23 +54,23 @@ CVIR(data = Ej1, cut = 4, conf.level = .90) } \references{ -Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. Applied Nursing Research, 5, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 +Davis, L.L. (1992). Instrument review: Getting the most from your panel of experts. \emph{Applied Nursing Research, 5}, 194-197. https://doi.org/10.1016/S0897-1897(05)80008-4 -Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. Research in Nursing & Health, 20, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g +Grant, J.S., & Davis, L.T. (1997). Selection and use of content experts in instrument development. \emph{Research in Nursing & Health, 20}, 269-274. https://doi.org/10.1002/(sici)1098-240x(199706)20:3<269::aid-nur9>3.0.co;2-g -Lynn, M.R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382-385. https://doi.org/10.1097/00006199-198611000-00017 +Lynn, M.R. (1986). Determination and quantification of content validity. \emph{Nursing Research, 35}, 382-385. -Martuza, V.R. (1977). Applying norm-referenced and criterion-referenced measurement in education. Boston: Allyn & Bacon +Martuza, V.R. (1977). \emph{Applying norm-referenced and criterion-referenced measurement in education}. Boston: Allyn & Bacon -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken's item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} -Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. Research in nursing & health, 29(5), 489-497. https://doi.org/10.1002/nur.20147 +Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what's being reported? Critique and recommendations. \emph{Research in Nursing & Health, 29}(5), 489-497. \doi{10.1002/nur.20147} -Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res. Nurs. Health, 30: 459-467. https://doi.org/10.1002/nur.20199 +Polit, D.F., Beck, C.T. and Owen, S.V. (2007), Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. \emph{Research in Nursing & Health, 30}, 459-467. \doi{10.1002/nur.20199} -Waltz, C.F., & Bausell, R.B. (1981). Nursing research: Design, statistics, and computer analysis. Philadelphia: F. A. Davis. +Waltz, C.F., & Bausell, R.B. (1981). \emph{Nursing research: Design, statistics, and computer analysis}. Philadelphia: F. A. Davis. -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. doi: 10.2307/2276774 +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } \seealso{ \code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval diff --git a/man/Haiken.Rd b/man/Haiken.Rd index 63b8875..ab619a7 100644 --- a/man/Haiken.Rd +++ b/man/Haiken.Rd @@ -25,10 +25,9 @@ Calculate the coefficient of homogeneity of response for each item (Aiken, 1980, \details{ Compute the H coefficient (Aiken, 1980, 1985) to estimate the homogeneity of response of the judges/scorers to the items. To maintain consistency with the methods usually associated with content validity, 'HAiken' is proposed as an option. -'HAiken' also compute asymmetric confidence intervals use the method of Wilson (1927). and adapted by Penfield and Giacobbi (2004) for Aiken's V coefficient. +'HAiken' also compute asymmetric confidence intervals use the method of Wilson (1927), and adapted by Penfield and Giacobbi (2004) for Aiken's V coefficient. The H coefficient, or equivalent coefficients, should complement the results of the content validity coefficients. Other methods for estimating judges' agreement or homogeneity of response may also be useful. -Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. } \examples{ ### Example 1 -------------- @@ -55,13 +54,13 @@ Haiken(as.data.frame(t(Ej2)), ncat = 5, conf.level = .90) } \references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} -Penfield, R. D., & Miller, J. M. (2004). Improving Content Validation Studies Using an Asymmetric Confidence Interval for the Mean of Expert Ratings. Applied Measurement in Education, 17(4), 359–370. https://doi.org/10.1207/s15324818ame1704_2 +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } \seealso{ \code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval diff --git a/man/MDScontent.Rd b/man/MDScontent.Rd index 032619b..8c4b95e 100644 --- a/man/MDScontent.Rd +++ b/man/MDScontent.Rd @@ -82,6 +82,13 @@ correspondence between each item and each trait (attribute). The function: and (6) plots either the main map, a biplot, or both. } \details{ +## Operational rationale +The MDS method requires a comparative design for obtaining the judges' ratings +for each item. Each item must be evaluated against several attributes (not just one), +and the judge must rate how well each item corresponds to each attribute. This format +is consistent with the substantive validity method (\code{\link[ValCont:CIDsingle]{ValCont::CIDsingle}}) +and Hinkey’s approach \code{\link[ValCont:HT]{ValCont::HT}} + ## Conceptual rationale The function provides a **geometric visualization** of content validity structure. Ratings from judges are first aggregated into an Items × Traits profile matrix. @@ -101,7 +108,7 @@ The resulting map represents: ## Interpretation This function is intended primarily for **visual diagnostic purposes**. It does not replace quantitative content validity coefficients (e.g., Aiken's V, -CVI, SVAL, or Hit-based approaches), but complements them by examining +Polit's CVI, Anderson-Gerbing's, or Colquit's approaches), but complements them by examining structural coherence. The map may be interpreted as follows: @@ -132,7 +139,7 @@ that complements coefficient-based evidence. # was evaluated for its correspondence to four attributes. The database should # be structured as follows: # -# item item1.judge1 item1.judge2 item1.judge3 item1.judge4 item2.judge1 item2.judge2 ... +# item item1.jdg1 item1.jde2 item1.jde3 item1.jde4 item2.jde1 item2.jde2 ... # # For this example: diff --git a/man/MER.Rd b/man/MER.Rd index bb62f20..74511fb 100644 --- a/man/MER.Rd +++ b/man/MER.Rd @@ -22,7 +22,7 @@ If TRUE rows with missing values in the relevant columns are removed before proc dataframe with MERs for all items analyzed, and confidence intervals. } \description{ -Calculate the average score, with asymmetric confidence intervals. +Calculate the Mean of expert ratings (MER), with asymmetric confidence intervals. } \details{ Calculate the average rating of the judges for each item, based on the proposal of Penfield & Miller (2004), and asymmetric confidence intervals (Wilson, 1927; Penfield, 2003; Penfield & Miller, 2004). MER' is a modification of the two syntax previous (Merino-Soto and Livia-Segovia, 2022; Penfield, & Miller, 2004). @@ -57,21 +57,21 @@ MER(data = Ej1, ncat = 6, start = 1, conf.level = .90) } \references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} -Merino-Soto, C., & Livia-Segovia, J. (2022). Rating mean of expert judges and asymmetric confidence intervals in content validity: an SPSS syntax. Anales de Psicologia, 38(2), 395-398. https://doi.org/10.6018/analesps.489431 +Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} -Miller, J. M., & Penfield, R. D. (2005). Using the score method to construct asymmetric confidence intervals: An SAS program for content validation in scale development. Behavior Research Methods, 37, 450-452. https://doi.org/10.3758/BF03192713 +Miller, J. M., & Penfield, R. D. (2005). Using the score method to construct asymmetric confidence intervals: An SAS program for content validation in scale development. \emph{Behavior Research Methods, 37}, 450-452. \doi{10.3758/BF03192713} -Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals for the mean of a rating scale item. Psychological methods, 8(2), 149-163. https://doi.org/10.1037/1082-989x.8.2.149 +Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals for the mean of a rating scale item. \emph{Psychological methods, 8}(2), 149-163. \doi{10.1037/1082-989x.8.2.149} -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} -Penfield, R. D., & Miller, J. M. (2004). Improving Content Validation Studies Using an Asymmetric Confidence Interval for the Mean of Expert Ratings. Applied Measurement in Education, 17(4), 359–370. https://doi.org/10.1207/s15324818ame1704_2 +Penfield, R. D., & Miller, J. M. (2004). Improving Content Validation Studies Using an Asymmetric Confidence Interval for the Mean of Expert Ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. \doi{10.1207/s15324818ame1704_2} -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } \seealso{ \code{\link[ValCont:Haiken]{ValCont::HAiken}} for a homogeneity coefficient diff --git a/man/Vaiken.Rd b/man/Vaiken.Rd index bbcb47b..5fa69ae 100644 --- a/man/Vaiken.Rd +++ b/man/Vaiken.Rd @@ -43,15 +43,15 @@ J6 = c(4, 1, 3, 5, 5)) Vaiken(data = data2Tst, min = 1, max = 5, conf.level = .90) } \references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and. Psychological Measurement, 40, 955-959. https://doi.org/10.1177/001316448004000419 +Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45, 131-142. https://doi.org/10.1177/0013164485451012 +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} -Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. Anales de Psicologia, 25(1), 169-171. https://revistas.um.es/analesps/article/view/71631 +Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. https://doi.org/10.1207/s15327841mpee0804_3 +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22, 209-212. https://doi.org/10.2307/2276774 +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } \seealso{ \code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval diff --git a/man/Vaikenpub.Rd b/man/Vaikenpub.Rd index fac014a..3ae6070 100644 --- a/man/Vaikenpub.Rd +++ b/man/Vaikenpub.Rd @@ -15,7 +15,7 @@ Vaikenpub(V, n, lo, hi, conf.level = 0.9, item.names = NULL) \item{hi}{Numeric. Maximum value of the rating scale.} -\item{conf.level}{Confidence level for the Wilson interval (default = 0.95).} +\item{conf.level}{Confidence level for the Wilson interval (default = 0.90).} \item{item.names}{Optional. Vector of item names. If NULL, defaults to Item1, Item2, etc.} } @@ -39,9 +39,9 @@ Vaikenpub( } \references{ -Penfield, R. D., & Giacobbi, P. R., Jr. (2004). Applying a score confidence interval to Aiken’s item content-relevance index. -\emph{Measurement in Physical Education and Exercise Science, 8}(4), 213–225. +Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. -\emph{Journal of the American Statistical Association, 22}, 209–212. +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} + +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} } From 3f9eb0e032891ea76150094ab4dfb4183dbade4b Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Mon, 24 Aug 2026 16:59:15 -0600 Subject: [PATCH 4/9] update CIR --- R/CIR.R | 256 ++++++++++++++++++++++++++++++++++------------------- man/CIR.Rd | 81 ++++++++++++----- 2 files changed, 224 insertions(+), 113 deletions(-) diff --git a/R/CIR.R b/R/CIR.R index 2025d00..a4f64b7 100644 --- a/R/CIR.R +++ b/R/CIR.R @@ -1,26 +1,58 @@ -#' MOVER-R Confidence Interval for the Ratio of Content Validity Coefficients +#' MOVER-R Confidence Interval for the Ratio of Two Bounded Proportion Estimates #' -#' Computes the confidence interval for the ratio of two independent -#' content validity coefficients using the MOVER-R (Method of Variance Estimates Recovery for Ratios) -#' closed-form procedure. +#' Computes the confidence interval for the ratio of two independent bounded +#' proportion estimates (e.g., Aiken's V coefficients) using the MOVER-R +#' (Method of Variance Estimates Recovery for Ratios) closed-form procedure. #' -#' @param group1 data.frame containing estimates and CIs for the first group. -#' Must include columns specified in \code{coef.col}, \code{lwr.col}, -#' \code{upr.col}, and an \code{"Item"} column for matching. -#' @param group2 data.frame containing estimates and CIs for the second group. -#' Same column requirements as \code{group1}. -#' @param coef.col Character string. Name of the column with the point estimates. -#' @param lwr.col Character string. Name of the column with the lower bounds. -#' @param upr.col Character string. Name of the column with the upper bounds. -#' @param na.rm Logical. If \code{TRUE}, rows with missing values in the relevant -#' columns are removed with a warning. If \code{FALSE}, missing values -#' trigger an error. +#' @param group1 A \code{data.frame} containing the point estimates and their +#' confidence limits for the first group. Must include columns specified in +#' \code{coef.col}, \code{lwr.col}, \code{upr.col}, and an \code{"Item"} +#' column for matching. +#' @param group2 A \code{data.frame} containing the point estimates and their +#' confidence limits for the second group. Same column requirements as +#' \code{group1}. +#' @param coef.col Character string indicating the column name for the point +#' estimates. +#' @param lwr.col Character string indicating the column name for the lower +#' confidence limits. +#' @param upr.col Character string indicating the column name for the upper +#' confidence limits. +#' @param na.rm Logical. If \code{TRUE}, rows containing missing values in the +#' relevant columns are removed with a warning. If \code{FALSE}, missing +#' values trigger an error. #' -#' @return A data.frame with columns: -#' \item{Item}{Common item names between both groups.} -#' \item{R}{Estimated ratio (coefficient group1 / coefficient group2).} -#' \item{lwr.ci}{Lower bound of the MOVER-R confidence interval.} -#' \item{upr.ci}{Upper bound of the MOVER-R confidence interval.} +#' @return A \code{data.frame} with the following columns: +#' \item{Item}{Common item identifiers present in both groups.} +#' \item{R}{The estimated ratio (coefficient of group1 divided by +#' coefficient of group2).} +#' \item{lwr.ci}{The lower bound of the MOVER-R confidence interval.} +#' \item{upr.ci}{The upper bound of the MOVER-R confidence interval.} +#' +#' @details +#' The MOVER-R interval is computed using the closed-form solution proposed by +#' Donner and Zou (2012). Given two independent estimates \eqn{\hat{\theta}_1} +#' and \eqn{\hat{\theta}_2} with corresponding confidence limits +#' \eqn{(l_1, u_1)} and \eqn{(l_2, u_2)}, the interval for the ratio +#' \eqn{R = \theta_1 / \theta_2} is obtained as: +#' +#' \deqn{L = \frac{m - \sqrt{m^2 - A \cdot D}}{D}, \quad +#' U = \frac{m + \sqrt{m^2 - B \cdot C}}{C}} +#' +#' where \eqn{m = \hat{\theta}_1 \hat{\theta}_2}, +#' \eqn{A = l_1(2\hat{\theta}_1 - l_1)}, +#' \eqn{B = u_1(2\hat{\theta}_1 - u_1)}, +#' \eqn{C = l_2(2\hat{\theta}_2 - l_2)}, and +#' \eqn{D = u_2(2\hat{\theta}_2 - u_2)}. +#' +#' For bounded proportion estimates (e.g., Aiken's V, which lies in [0, 1]), +#' it is common for the lower confidence limit to equal exactly 0 or the upper +#' limit to equal exactly 1. This yields \eqn{C = 0} or \eqn{D = 0}, +#' respectively, causing a singular division. To maintain numerical stability, +#' the function applies a small perturbation (\eqn{\epsilon = 10^{-10}}) to +#' any zero-valued auxiliary terms \eqn{C} or \eqn{D}. A warning is issued +#' whenever such a correction is applied. Users should consider this +#' adjustment carefully and, if necessary, resort to alternative methods +#' (e.g., bootstrap) for items with boundary confidence limits. #' #' @references #' Donner, A., & Zou, G. Y. (2012). Closed-form confidence intervals for @@ -30,39 +62,46 @@ #' Zou, G., Donner, A., & Qiu, S. (2025). MOVER-R for Confidence Intervals #' of Ratios. In N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, #' F. Ruggeri and J.L. Teugels (Eds.), \emph{Wiley StatsRef: Statistics -#' Reference Online}. https://doi.org/10.1002/9781118445112.stat08085 +#' Reference Online}. doi:10.1002/9781118445112.stat08085 #' #' @examples #' \dontrun{ -#' # Assuming you have data frames 'g1' and 'g2' with columns 'est', 'low', 'high' -#' result <- CIR(g1, g2, coef.col = "est", lwr.col = "low", upr.col = "high") -#' print(result) +#' # Suppose 'group1' and 'group2' contain columns 'est', 'low', and 'high' +#' res <- CIR(group1, group2, coef.col = "est", lwr.col = "low", upr.col = "high") +#' print(res) #' } +#' #' @export CIR <- function(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) { - # ------ 1. Validación de columnas ------ + # ---------------------------------------------------------------------- + # 1. Column validation + # ---------------------------------------------------------------------- + + # Ensure that the required columns exist in both data frames if (!all(c(coef.col, lwr.col, upr.col) %in% colnames(group1))) { - stop("group1 must contain the columns specified in coef.col, lwr.col, and upr.col") + stop("'group1' must contain the columns specified in 'coef.col', 'lwr.col', and 'upr.col'.") } if (!all(c(coef.col, lwr.col, upr.col) %in% colnames(group2))) { - stop("group2 must contain the columns specified in coef.col, lwr.col, and upr.col") + stop("'group2' must contain the columns specified in 'coef.col', 'lwr.col', and 'upr.col'.") } - # Columns of interest (including "Item" for the merge) + # Identify the relevant columns (including the matching key "Item") cols_g1 <- intersect(c(coef.col, lwr.col, upr.col, "Item"), colnames(group1)) cols_g2 <- intersect(c(coef.col, lwr.col, upr.col, "Item"), colnames(group2)) - # ------ 2. Handling Missing Values (NA) ------ + # ---------------------------------------------------------------------- + # 2. Handling of missing values + # ---------------------------------------------------------------------- + has_na_g1 <- any(is.na(group1[, cols_g1, drop = FALSE])) has_na_g2 <- any(is.na(group2[, cols_g2, drop = FALSE])) if (has_na_g1 || has_na_g2) { if (!na.rm) { - stop("Missing values detected. Use na.omit() first or set na.rm=TRUE") + stop("Missing values detected. Use 'na.omit()' first or set 'na.rm = TRUE'.") } else { - - # Only the "Estimate" and "CI" columns (not "Item") for complete cases + # Subset to the estimation columns (excluding "Item") for complete.cases est_cols_g1 <- intersect(c(coef.col, lwr.col, upr.col), colnames(group1)) est_cols_g2 <- intersect(c(coef.col, lwr.col, upr.col), colnames(group2)) @@ -75,37 +114,48 @@ CIR <- function(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) { group1 <- group1[keep_g1, , drop = FALSE] group2 <- group2[keep_g2, , drop = FALSE] - warning(sprintf("na.rm=TRUE: removed %d rows with missing values from group1 and %d from group2", - nrem1, nrem2)) + warning(sprintf( + "na.rm = TRUE: removed %d rows with missing values from 'group1' and %d from 'group2'.", + nrem1, nrem2 + )) } } - # ------ 3. Intersection of Items ------ + # ---------------------------------------------------------------------- + # 3. Matching items across groups + # ---------------------------------------------------------------------- + common_items <- intersect(group1$Item, group2$Item) + if (length(common_items) < nrow(group1) || length(common_items) < nrow(group2)) { - warning("Some items do not match between group1 and group2. Only the common items will be processed..") + warning( + "Some items in 'group1' and 'group2' do not match. ", + "Only the common items will be processed." + ) } + # Retain only the common items and sort for deterministic merging group1 <- group1[group1$Item %in% common_items, ] group2 <- group2[group2$Item %in% common_items, ] - - # Sort by Item to ensure a match (optional, but safe) group1 <- group1[order(group1$Item), ] group2 <- group2[order(group2$Item), ] - # ------ 4. Mixing (merge) ------ + # ---------------------------------------------------------------------- + # 4. Combine the two data frames + # ---------------------------------------------------------------------- + combined <- merge(group1, group2, by = "Item", suffixes = c("_g1", "_g2")) - # Pre-assign results + # Pre-allocate the results data frame results <- data.frame( - Item = combined$Item, - R = NA_real_, + Item = combined$Item, + R = NA_real_, lwr.ci = NA_real_, upr.ci = NA_real_, stringsAsFactors = FALSE ) - # Column names in the combined data.frame + # Construct column names for easier access within the loop coef_g1 <- paste0(coef.col, "_g1") coef_g2 <- paste0(coef.col, "_g2") lwr_g1 <- paste0(lwr.col, "_g1") @@ -113,10 +163,16 @@ CIR <- function(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) { lwr_g2 <- paste0(lwr.col, "_g2") upr_g2 <- paste0(upr.col, "_g2") - # ------ 5. Calculation of the MOVER-R Interval (Donner & Zou, 2012) ------ - for (i in 1:nrow(combined)) { + # ---------------------------------------------------------------------- + # 5. MOVER-R interval computation (Donner & Zou, 2012) + # ---------------------------------------------------------------------- - # Extraer valores + # Numerical tolerance for detecting boundary cases + EPS <- 1e-10 + + for (i in seq_len(nrow(combined))) { + + # Retrieve the estimates and their confidence limits th1 <- combined[[coef_g1]][i] th2 <- combined[[coef_g2]][i] l1 <- combined[[lwr_g1]][i] @@ -124,65 +180,87 @@ CIR <- function(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) { l2 <- combined[[lwr_g2]][i] u2 <- combined[[upr_g2]][i] - # Point ratio + # Point estimate of the ratio R <- th1 / th2 - # --- MOVER-R Components --- - # A = l1*(2*th1 - l1) ; B = u1*(2*th1 - u1) - # C = l2*(2*th2 - l2) ; D = u2*(2*th2 - u2) + # Auxiliary quantities for MOVER-R + # A = l1 * (2*th1 - l1) + # B = u1 * (2*th1 - u1) + # C = l2 * (2*th2 - l2) + # D = u2 * (2*th2 - u2) A <- l1 * (2 * th1 - l1) B <- u1 * (2 * th1 - u1) C <- l2 * (2 * th2 - l2) D <- u2 * (2 * th2 - u2) + # -------------------------------------------------------------------- + # Small correction for singularities (C or D equal to zero) + # This frequently occurs when the confidence limits of a bounded + # proportion (e.g., Aiken's V) are exactly 0 or 1. + # -------------------------------------------------------------------- + if (C <= EPS) { + warning(sprintf( + "Item '%s': auxiliary term C = %.10f (l2 = %.4f, th2 = %.4f). ", + "Setting C to %.1e to avoid division by zero.", + combined$Item[i], C, l2, th2, EPS + )) + C <- EPS + } + + if (D <= EPS) { + warning(sprintf( + "Item '%s': auxiliary term D = %.10f (u2 = %.4f, th2 = %.4f). ", + "Setting D to %.1e to avoid division by zero.", + combined$Item[i], D, u2, th2, EPS + )) + D <- EPS + } + # Product m = theta1 * theta2 m <- th1 * th2 - # Initialize limits as NA - L <- NA_real_ - U <- NA_real_ - - # --- Lower limit --- - # D must be greater than 0, and the radicand must be greater than or equal to 0. - if (D > 0) { - rad_low <- m^2 - A * D - if (rad_low < 0) { - # If the radicand is slightly negative due to rounding, we set it to 0 - if (abs(rad_low) < 1e-12) rad_low <- 0 - } - if (rad_low >= 0) { - L <- (m - sqrt(rad_low)) / D - # By the definition of Aiken's V (ratio), it should not be negative, - # but if it is, we set it to 0 (conservative approach). - if (L < 0 && L > -1e-10) L <- 0 - } else { - warning(sprintf("Negative value found for the lower bound in Item '%s'. NA is assigned.", - combined$Item[i])) - } + # -------------------- Lower bound (L) -------------------- + # L = (m - sqrt(m^2 - A*D)) / D, provided that A*D <= m^2 + rad_low <- m^2 - A * D + + # Correct for minuscule negative radicands due to rounding + if (rad_low < 0 && abs(rad_low) < 1e-12) { + rad_low <- 0 + } + + if (rad_low < 0) { + warning(sprintf( + "Item '%s': negative radicand (%.10f) for the lower bound. Setting lwr.ci to NA.", + combined$Item[i], rad_low + )) + L <- NA_real_ } else { - warning(sprintf("D <= 0 for the item '%s'. The lower bound cannot be calculated (D = %f).", - combined$Item[i], D)) + L <- (m - sqrt(rad_low)) / D + # If the lower bound is trivially negative (due to rounding), set to 0 + if (!is.na(L) && L < 0 && abs(L) < 1e-10) L <- 0 + } + + # -------------------- Upper bound (U) -------------------- + # U = (m + sqrt(m^2 - B*C)) / C, provided that B*C <= m^2 + rad_up <- m^2 - B * C + + if (rad_up < 0 && abs(rad_up) < 1e-12) { + rad_up <- 0 } - # --- Upper limit --- - # C > 0 and the square root is >= 0 are required - if (C > 0) { - rad_up <- m^2 - B * C - if (rad_up < 0) { - if (abs(rad_up) < 1e-12) rad_up <- 0 - } - if (rad_up >= 0) { - U <- (m + sqrt(rad_up)) / C - } else { - warning(sprintf("Negative value found for the upper limit in Item '%s'. NA is assigned.", - combined$Item[i])) - } + if (rad_up < 0) { + warning(sprintf( + "Item '%s': negative radicand (%.10f) for the upper bound. Setting upr.ci to NA.", + combined$Item[i], rad_up + )) + U <- NA_real_ } else { - warning(sprintf("C <= 0 for the item '%s'. The upper bound cannot be calculated (C = %f).", - combined$Item[i], C)) + U <- (m + sqrt(rad_up)) / C } - # Assign results (rounded to 3 decimal places) + # -------------------------------------------------------------------- + # Store the results with rounding to three decimal places + # -------------------------------------------------------------------- results$R[i] <- round(R, 3) results$lwr.ci[i] <- round(L, 3) results$upr.ci[i] <- round(U, 3) diff --git a/man/CIR.Rd b/man/CIR.Rd index 79e8375..7b54ecc 100644 --- a/man/CIR.Rd +++ b/man/CIR.Rd @@ -2,46 +2,79 @@ % Please edit documentation in R/CIR.R \name{CIR} \alias{CIR} -\title{MOVER-R Confidence Interval for the Ratio of Content Validity Coefficients} +\title{MOVER-R Confidence Interval for the Ratio of Two Bounded Proportion Estimates} \usage{ CIR(group1, group2, coef.col, lwr.col, upr.col, na.rm = FALSE) } \arguments{ -\item{group1}{data.frame containing estimates and CIs for the first group. -Must include columns specified in \code{coef.col}, \code{lwr.col}, -\code{upr.col}, and an \code{"Item"} column for matching.} +\item{group1}{A \code{data.frame} containing the point estimates and their +confidence limits for the first group. Must include columns specified in +\code{coef.col}, \code{lwr.col}, \code{upr.col}, and an \code{"Item"} +column for matching.} -\item{group2}{data.frame containing estimates and CIs for the second group. -Same column requirements as \code{group1}.} +\item{group2}{A \code{data.frame} containing the point estimates and their +confidence limits for the second group. Same column requirements as +\code{group1}.} -\item{coef.col}{Character string. Name of the column with the point estimates.} +\item{coef.col}{Character string indicating the column name for the point +estimates.} -\item{lwr.col}{Character string. Name of the column with the lower bounds.} +\item{lwr.col}{Character string indicating the column name for the lower +confidence limits.} -\item{upr.col}{Character string. Name of the column with the upper bounds.} +\item{upr.col}{Character string indicating the column name for the upper +confidence limits.} -\item{na.rm}{Logical. If \code{TRUE}, rows with missing values in the relevant -columns are removed with a warning. If \code{FALSE}, missing values -trigger an error.} +\item{na.rm}{Logical. If \code{TRUE}, rows containing missing values in the +relevant columns are removed with a warning. If \code{FALSE}, missing +values trigger an error.} } \value{ -A data.frame with columns: - \item{Item}{Common item names between both groups.} - \item{R}{Estimated ratio (coefficient group1 / coefficient group2).} - \item{lwr.ci}{Lower bound of the MOVER-R confidence interval.} - \item{upr.ci}{Upper bound of the MOVER-R confidence interval.} +A \code{data.frame} with the following columns: + \item{Item}{Common item identifiers present in both groups.} + \item{R}{The estimated ratio (coefficient of group1 divided by + coefficient of group2).} + \item{lwr.ci}{The lower bound of the MOVER-R confidence interval.} + \item{upr.ci}{The upper bound of the MOVER-R confidence interval.} } \description{ -Computes the confidence interval for the ratio of two independent -content validity coefficients using the MOVER-R (Method of Variance Estimates Recovery for Ratios) -closed-form procedure. +Computes the confidence interval for the ratio of two independent bounded +proportion estimates (e.g., Aiken's V coefficients) using the MOVER-R +(Method of Variance Estimates Recovery for Ratios) closed-form procedure. +} +\details{ +The MOVER-R interval is computed using the closed-form solution proposed by +Donner and Zou (2012). Given two independent estimates \eqn{\hat{\theta}_1} +and \eqn{\hat{\theta}_2} with corresponding confidence limits +\eqn{(l_1, u_1)} and \eqn{(l_2, u_2)}, the interval for the ratio +\eqn{R = \theta_1 / \theta_2} is obtained as: + +\deqn{L = \frac{m - \sqrt{m^2 - A \cdot D}}{D}, \quad + U = \frac{m + \sqrt{m^2 - B \cdot C}}{C}} + +where \eqn{m = \hat{\theta}_1 \hat{\theta}_2}, +\eqn{A = l_1(2\hat{\theta}_1 - l_1)}, +\eqn{B = u_1(2\hat{\theta}_1 - u_1)}, +\eqn{C = l_2(2\hat{\theta}_2 - l_2)}, and +\eqn{D = u_2(2\hat{\theta}_2 - u_2)}. + +For bounded proportion estimates (e.g., Aiken's V, which lies in [0, 1]), +it is common for the lower confidence limit to equal exactly 0 or the upper +limit to equal exactly 1. This yields \eqn{C = 0} or \eqn{D = 0}, +respectively, causing a singular division. To maintain numerical stability, +the function applies a small perturbation (\eqn{\epsilon = 10^{-10}}) to +any zero-valued auxiliary terms \eqn{C} or \eqn{D}. A warning is issued +whenever such a correction is applied. Users should consider this +adjustment carefully and, if necessary, resort to alternative methods +(e.g., bootstrap) for items with boundary confidence limits. } \examples{ \dontrun{ -# Assuming you have data frames 'g1' and 'g2' with columns 'est', 'low', 'high' -result <- CIR(g1, g2, coef.col = "est", lwr.col = "low", upr.col = "high") -print(result) +# Suppose 'group1' and 'group2' contain columns 'est', 'low', and 'high' +res <- CIR(group1, group2, coef.col = "est", lwr.col = "low", upr.col = "high") +print(res) } + } \references{ Donner, A., & Zou, G. Y. (2012). Closed-form confidence intervals for @@ -51,5 +84,5 @@ Methods in Medical Research}, 21(4), 347-359. Zou, G., Donner, A., & Qiu, S. (2025). MOVER-R for Confidence Intervals of Ratios. In N. Balakrishnan, T. Colton, B. Everitt, W. Piegorsch, F. Ruggeri and J.L. Teugels (Eds.), \emph{Wiley StatsRef: Statistics -Reference Online}. https://doi.org/10.1002/9781118445112.stat08085 +Reference Online}. doi:10.1002/9781118445112.stat08085 } From 95d3b6d24a229e4ff0c4feee711dc96e8c23eece Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Fri, 28 Aug 2026 11:19:02 -0600 Subject: [PATCH 5/9] update 2: Vaiken Adding total V --- R/VAiken.R | 255 ++++++++++++++++++++++++++++++++++---------------- man/Vaiken.Rd | 112 ++++++++++++++++------ 2 files changed, 258 insertions(+), 109 deletions(-) diff --git a/R/VAiken.R b/R/VAiken.R index 6ef19c0..6a5a360 100644 --- a/R/VAiken.R +++ b/R/VAiken.R @@ -7,124 +7,217 @@ #' @param conf.level confidence level for confidence intervals (ex., .90, .95, .99) #' @param na.rm Logical. If FALSE (default) the function stops when missing values are detected. #' If TRUE rows with missing values in the relevant columns are removed before processing. +#' @param overall Logical. If TRUE, adds a row with the overall V coefficient. +#' @param overall.method Character. Method for the overall V: +#' \itemize{ +#' \item \code{"global"}: treats the entire matrix as a single "super-item" (all ratings pooled). +#' \item \code{"Aiken"}: computes V for each judge (based on all items) and then averages them. +#' } +#' @param overall.ci Character. Method for the overall confidence interval: +#' \itemize{ +#' \item \code{"Wilson"}: Wilson score interval (Penfield & Giacobbi, 2004). +#' \item \code{"MER"}: score interval for the mean of ratings, then transformed to V (Penfield, 2003). +#' } +#' Note: When \code{overall.method = "Aiken"}, only \code{"Wilson"} is used (others ignored). #' -#'@return -#'dataframe with V coefficients for all items analyzed, and confidence intervals +#' @return dataframe with V coefficients for all items, and confidence intervals. +#' If overall = TRUE, an extra row with the overall index is included. #' -#'@details -#'Calculate the V coefficient (Aiken, 1980, 1985), with the formula of Penfield & Giacobbi (2004). It also calculates asymmetric confidence intervals (Wilson, 1927; Penfield & Giacobbi, 2004). The results should be complemented by an estimator of variability or inter-judge agreement. The function uses the modified formula presented by Penfield & Giacobbi (2004). -#'This function substantially improves on Vaiken (Merino, & Livia, 2009) because it calculates for multiple items and any confidence level. +#' @details +#' The overall V provides a global estimate of content validity for the whole instrument. +#' Two aggregation methods are available: +#' \itemize{ +#' \item \code{"global"}: all ratings are pooled (treating the matrix as one super-item). +#' The V is the mean of all transformed scores. +#' \item \code{"Aiken"}: V is computed for each judge (using their ratings across all items) +#' and then averaged. This follows the large-sample procedure described by Aiken (1985). +#' } +#' For confidence intervals, the Wilson score method (Penfield & Giacobbi, 2004) is available for +#' both approaches. For the \code{"global"} method, the MER method (Penfield, 2003) is also offered, +#' which constructs the CI on the mean of ratings and then transforms to V. #' -#'Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. +#' **Overall V and its confidence interval** #' +#' When `overall = TRUE`, the function treats the entire matrix of ratings (all items × all judges) +#' as a single "super-item". The overall V coefficient is computed as the mean of all transformed +#' scores `(rating - min) / (max - min)`, which is equivalent to `(mean(all_ratings) - min) / (max - min)`. +#' This provides a global estimate of content validity for the whole instrument. +#' Two methods are available to construct the asymmetric confidence interval for this overall V: #' -#'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} +#' - **`overall_method = "Wilson"`** (default): Applies the Wilson score interval directly to the overall +#' proportion V, following the logic of Penfield & Giacobbi (2004). The effective sample size +#' is `n_judges × (max - min)`, respecting the independence of judges. This method is the +#' most direct extension of the standard item-level V confidence interval to the global index, and +#' is consistent with the original proposal by Aiken and later developments. #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} +#' - **`overall_method = "mer"`**: Implements the score confidence interval for the **mean** of +#' the ratings (MER; Penfield, 2003; Penfield & Miller, 2004) in the original scale, and then +#' transforms the lower and upper bounds to the V metric. This approach first computes an +#' asymmetric CI for the mean `M` of all ratings, and then applies the linear +#' transformation `(CI - min) / (max - min)`. By using the mean as the primary parameter, this +#' method explicitly models the variability among judges and does not rely on the binomial expansion +#' used in the Wilson method. #' -#'Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} +#' Both methods yield a V total that is identical in point estimate, but the confidence intervals may +#' differ slightly. In either case, it is recommended to report the method used and, +#' if possible, to provide both intervals in supplementary materials for transparency. #' -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} +#' @references +#' Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. +#' Educational and Psychological Measurement, 45, 131-142. +#' Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals +#' for the mean of a rating scale item. Psychological Methods, 8(2), 149-163. +#' Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval +#' to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. +#' Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. +#' Journal of the American Statistical Association, 22, 209-212. #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} -#' -#'@seealso -#'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval +#' @export #' -#' @author -#' Diego Livia-Ortiz (\email{diegolivia@hotmail.com}) -#' Cesar Merino-Soto (\email{sikayax@yahoo.com.ar}) +#' @examples +#' data2Tst <- data.frame( +#' J1 = c(4, 1, 1, 1, 4), +#' J2 = c(4, 1, 2, 2, 3), +#' J3 = c(4, 1, 3, 3, 5), +#' J4 = c(4, 1, 4, 5, 5), +#' J5 = c(4, 1, 5, 5, 5), +#' J6 = c(4, 1, 3, 5, 5)) #' -#' @export +#' # Original: item-level V with Wilson CI +#' Vaiken(data2Tst, min = 1, max = 5, conf.level = .90) #' -#'@examples +#' # Overall V with global pooling, Wilson CI +#' Vaiken(data2Tst, min = 1, max = 5, overall = TRUE, overall.method = "global", overall.ci = "Wilson") #' -#'### Example 2 -#'data2Tst <- data.frame( -#'J1 = c(4, 1, 1, 1, 4), -#'J2 = c(4, 1, 2, 2, 3), -#'J3 = c(4, 1, 3, 3, 5), -#'J4 = c(4, 1, 4, 5, 5), -#'J5 = c(4, 1, 5, 5, 5), -#'J6 = c(4, 1, 3, 5, 5)) -#'Vaiken(data = data2Tst, min = 1, max = 5, conf.level = .90) - -Vaiken <- function(data, min, max, conf.level = 0.95, na.rm = FALSE) { - - # Detection of Missing Values +#' # Overall V with Aiken's method (average of judge V's), Wilson CI +#' Vaiken(data2Tst, min = 1, max = 5, overall = TRUE, overall.method = "Aiken") +Vaiken <- function(data, min, max, conf.level = 0.95, na.rm = FALSE, + overall = FALSE, + overall.method = c("global", "Aiken"), + overall.ci = c("Wilson", "MER")) { + + # ---- Missing values ---- if (!na.rm) { if (any(is.na(data))) { - stop("Missing values detected. Use na.omit() first, or set na.rm=TRUE.") + stop("Missing values detected. Use na.omit() first or set na.rm=TRUE.") } } else { data <- na.omit(data) } - # Validation: data - if (!is.data.frame(data)) { - stop("The 'data' argument must be a data.frame.") - } - if (!is.numeric(as.matrix(data))) { - stop("The 'data' field must contain only numeric values.") - } - - # Validation: min, max - if (!is.numeric(min) || !is.numeric(max)) { - stop("'min' and 'max' must be numeric values.") - } - if (min >= max) { - stop("'min' must be less than 'max'.") - } - - # Validation: confidence level - if (!is.numeric(conf.level) || conf.level <= 0 || conf.level >= 1) { - stop("'conf.level' must be between 0 and 1 (excluding the extremes).") - } - - # Verification: NA - if (any(is.na(data))) { - stop("'data' contains NA values. Please clean the data before using the function.") + # ---- Input validation ---- + if (!is.data.frame(data)) stop("'data' must be a data.frame.") + if (!is.numeric(as.matrix(data))) stop("'data' must contain only numeric values.") + if (!is.numeric(min) || !is.numeric(max)) stop("'min' and 'max' must be numeric.") + if (min >= max) stop("'min' must be less than 'max'.") + if (!is.numeric(conf.level) || conf.level <= 0 || conf.level >= 1) + stop("'conf.level' must be between 0 and 1.") + if (any(data < min | data > max)) + stop("All ratings must be between 'min' and 'max'.") + + overall.method <- match.arg(overall.method) + overall.ci <- match.arg(overall.ci) + + # If overall.method == "Aiken", force overall.ci = "Wilson" (with a message) + if (overall.method == "Aiken" && overall.ci != "Wilson") { + warning("For overall.method = 'Aiken', only 'Wilson' CI is available. Switching to 'Wilson'.") + overall.ci <- "Wilson" } - # Verification: valid range of data - if (any(data < min | data > max)) { - stop("All scores in 'data' must be between 'min' and 'max'.") - } - - # N judges - n <- ncol(data) - - # Critic value Z for confidence level + n_jueces <- ncol(data) + n_items <- nrow(data) z <- qnorm(1 - (1 - conf.level) / 2) + k <- max - min # range of the scale - # Calculate V Aiken and confidence interval + # ---- Internal function for item-level V and Wilson CI (unchanged) ---- calcular_valores <- function(puntajes) { - M <- mean(puntajes) # Average of the scores - V <- (M - min) / (max - min) # Calculation of Aiken's V - - # lower and upper limits, method Wilson's score - nk <- n * (max - min) + M <- mean(puntajes) + V <- (M - min) / k + nk <- length(puntajes) * k # n_jueces * k term1 <- 2 * nk * V + z^2 term2 <- z * sqrt(4 * nk * V * (1 - V) + z^2) denom <- 2 * (nk + z^2) - - lwr.ci <- (term1 - term2) / denom - upr.ci <- (term1 + term2) / denom - + lwr.ci <- max(0, min(1, (term1 - term2) / denom)) + upr.ci <- max(0, min(1, (term1 + term2) / denom)) return(c(V = V, lwr.ci = lwr.ci, upr.ci = upr.ci)) } - # Aiken V for every item (row in the data frame) resultados <- t(apply(data, 1, calcular_valores)) - - # Data frame for the output resultados_df <- data.frame( - Item = 1:nrow(data), + Item = 1:n_items, V = round(resultados[, "V"], 3), lwr.ci = round(resultados[, "lwr.ci"], 3), upr.ci = round(resultados[, "upr.ci"], 3) ) + # ---- Overall calculation (if requested) ---- + if (overall) { + + if (overall.method == "global") { + # ---- Global super-item approach (pool all ratings) ---- + all_scores <- as.vector(as.matrix(data)) + V_total <- mean((all_scores - min) / k) # mean of transformed scores + + if (overall.ci == "Wilson") { + # Wilson CI directly on V_total, using n_jueces * k as effective N + nk_total <- n_jueces * k + term1_t <- 2 * nk_total * V_total + z^2 + term2_t <- z * sqrt(4 * nk_total * V_total * (1 - V_total) + z^2) + denom_t <- 2 * (nk_total + z^2) + lwr_total <- max(0, min(1, (term1_t - term2_t) / denom_t)) + upr_total <- max(0, min(1, (term1_t + term2_t) / denom_t)) + etiqueta <- "Total (global, Wilson)" + + } else { # overall.ci == "MER" + # MER: Wilson CI on mean of ratings, then transform to V + M_total <- mean(all_scores) + p <- V_total # same as (M - min)/k + p <- max(1e-10, min(1 - 1e-10, p)) + + n <- n_jueces + term1_mer <- 2 * p * n * k + z^2 + term2_mer <- z * sqrt(4 * n * k * p * (1 - p) + z^2) + denom_mer <- 2 * (n * k + z^2) + pi_L <- (term1_mer - term2_mer) / denom_mer + pi_U <- (term1_mer + term2_mer) / denom_mer + + LCL_mean <- M_total - z * sqrt(k * pi_L * (1 - pi_L) / n) + UCL_mean <- M_total + z * sqrt(k * pi_U * (1 - pi_U) / n) + LCL_mean <- max(min, min(max, LCL_mean)) + UCL_mean <- max(min, min(max, UCL_mean)) + + lwr_total <- max(0, min(1, (LCL_mean - min) / k)) + upr_total <- max(0, min(1, (UCL_mean - min) / k)) + etiqueta <- "Total (global, MER)" + } + + } else { # overall.method == "Aiken" + # ---- Aiken's method: average of V's computed for each judge ---- + V_por_juez <- apply(data, 2, function(col) { + (mean(col) - min) / k + }) + V_total <- mean(V_por_juez) + + # Wilson CI on the mean V (treating it as a proportion) + # Effective N = n_jueces * k + nk_total <- n_jueces * k + term1_t <- 2 * nk_total * V_total + z^2 + term2_t <- z * sqrt(4 * nk_total * V_total * (1 - V_total) + z^2) + denom_t <- 2 * (nk_total + z^2) + lwr_total <- max(0, min(1, (term1_t - term2_t) / denom_t)) + upr_total <- max(0, min(1, (term1_t + term2_t) / denom_t)) + etiqueta <- "Total (Aiken, Wilson)" + } + + # Add overall row to results + fila_total <- data.frame( + Item = etiqueta, + V = round(V_total, 3), + lwr.ci = round(lwr_total, 3), + upr.ci = round(upr_total, 3) + ) + resultados_df <- rbind(resultados_df, fila_total) + } + return(resultados_df) } diff --git a/man/Vaiken.Rd b/man/Vaiken.Rd index 5fa69ae..11ffbf1 100644 --- a/man/Vaiken.Rd +++ b/man/Vaiken.Rd @@ -4,7 +4,16 @@ \alias{Vaiken} \title{Aiken's V coefficient} \usage{ -Vaiken(data, min, max, conf.level = 0.95, na.rm = FALSE) +Vaiken( + data, + min, + max, + conf.level = 0.95, + na.rm = FALSE, + overall = FALSE, + overall.method = c("global", "Aiken"), + overall.ci = c("Wilson", "MER") +) } \arguments{ \item{data}{dataframe, with the columns assigned to each rater, and the rows assigned to each evaluated item.} @@ -17,46 +26,93 @@ Vaiken(data, min, max, conf.level = 0.95, na.rm = FALSE) \item{na.rm}{Logical. If FALSE (default) the function stops when missing values are detected. If TRUE rows with missing values in the relevant columns are removed before processing.} + +\item{overall}{Logical. If TRUE, adds a row with the overall V coefficient.} + +\item{overall.method}{Character. Method for the overall V: +\itemize{ + \item \code{"global"}: treats the entire matrix as a single "super-item" (all ratings pooled). + \item \code{"Aiken"}: computes V for each judge (based on all items) and then averages them. +}} + +\item{overall.ci}{Character. Method for the overall confidence interval: +\itemize{ + \item \code{"Wilson"}: Wilson score interval (Penfield & Giacobbi, 2004). + \item \code{"MER"}: score interval for the mean of ratings, then transformed to V (Penfield, 2003). +} +Note: When \code{overall.method = "Aiken"}, only \code{"Wilson"} is used (others ignored).} } \value{ -dataframe with V coefficients for all items analyzed, and confidence intervals +dataframe with V coefficients for all items, and confidence intervals. + If overall = TRUE, an extra row with the overall index is included. } \description{ Calculate the V coefficient, known as Aiken's V. } \details{ -Calculate the V coefficient (Aiken, 1980, 1985), with the formula of Penfield & Giacobbi (2004). It also calculates asymmetric confidence intervals (Wilson, 1927; Penfield & Giacobbi, 2004). The results should be complemented by an estimator of variability or inter-judge agreement. The function uses the modified formula presented by Penfield & Giacobbi (2004). -This function substantially improves on Vaiken (Merino, & Livia, 2009) because it calculates for multiple items and any confidence level. - -Note: The function has not yet been prepared to resolve missing values, so the user must remove or impute any missing values. +The overall V provides a global estimate of content validity for the whole instrument. +Two aggregation methods are available: +\itemize{ + \item \code{"global"}: all ratings are pooled (treating the matrix as one super-item). + The V is the mean of all transformed scores. + \item \code{"Aiken"}: V is computed for each judge (using their ratings across all items) + and then averaged. This follows the large-sample procedure described by Aiken (1985). } -\examples{ +For confidence intervals, the Wilson score method (Penfield & Giacobbi, 2004) is available for +both approaches. For the \code{"global"} method, the MER method (Penfield, 2003) is also offered, +which constructs the CI on the mean of ratings and then transforms to V. -### Example 2 -data2Tst <- data.frame( -J1 = c(4, 1, 1, 1, 4), -J2 = c(4, 1, 2, 2, 3), -J3 = c(4, 1, 3, 3, 5), -J4 = c(4, 1, 4, 5, 5), -J5 = c(4, 1, 5, 5, 5), -J6 = c(4, 1, 3, 5, 5)) -Vaiken(data = data2Tst, min = 1, max = 5, conf.level = .90) -} -\references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} +**Overall V and its confidence interval** -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} +When `overall = TRUE`, the function treats the entire matrix of ratings (all items × all judges) +as a single "super-item". The overall V coefficient is computed as the mean of all transformed +scores `(rating - min) / (max - min)`, which is equivalent to `(mean(all_ratings) - min) / (max - min)`. +This provides a global estimate of content validity for the whole instrument. +Two methods are available to construct the asymmetric confidence interval for this overall V: -Merino, C., & Livia, J. (2009). Intervalos de confianza asimetricos para el indice de validez de contenido: un programa Visual Basic para la V de Aiken. \emph{Anales de Psicologia, 25}(1), 169-171. \url{https://revistas.um.es/analesps/article/view/71631} + - **`overall_method = "Wilson"`** (default): Applies the Wilson score interval directly to the overall + proportion V, following the logic of Penfield & Giacobbi (2004). The effective sample size + is `n_judges × (max - min)`, respecting the independence of judges. This method is the + most direct extension of the standard item-level V confidence interval to the global index, and + is consistent with the original proposal by Aiken and later developments. -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} + - **`overall_method = "mer"`**: Implements the score confidence interval for the **mean** of + the ratings (MER; Penfield, 2003; Penfield & Miller, 2004) in the original scale, and then + transforms the lower and upper bounds to the V metric. This approach first computes an + asymmetric CI for the mean `M` of all ratings, and then applies the linear + transformation `(CI - min) / (max - min)`. By using the mean as the primary parameter, this + method explicitly models the variability among judges and does not rely on the binomial expansion + used in the Wilson method. -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} + Both methods yield a V total that is identical in point estimate, but the confidence intervals may + differ slightly. In either case, it is recommended to report the method used and, + if possible, to provide both intervals in supplementary materials for transparency. } -\seealso{ -\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval +\examples{ +data2Tst <- data.frame( + J1 = c(4, 1, 1, 1, 4), + J2 = c(4, 1, 2, 2, 3), + J3 = c(4, 1, 3, 3, 5), + J4 = c(4, 1, 4, 5, 5), + J5 = c(4, 1, 5, 5, 5), + J6 = c(4, 1, 3, 5, 5)) + +# Original: item-level V with Wilson CI +Vaiken(data2Tst, min = 1, max = 5, conf.level = .90) + +# Overall V with global pooling, Wilson CI +Vaiken(data2Tst, min = 1, max = 5, overall = TRUE, overall.method = "global", overall.ci = "Wilson") + +# Overall V with Aiken's method (average of judge V's), Wilson CI +Vaiken(data2Tst, min = 1, max = 5, overall = TRUE, overall.method = "Aiken") } -\author{ -Diego Livia-Ortiz (\email{diegolivia@hotmail.com}) -Cesar Merino-Soto (\email{sikayax@yahoo.com.ar}) +\references{ +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. + Educational and Psychological Measurement, 45, 131-142. +Penfield, R. D. (2003). A score method of constructing asymmetric confidence intervals + for the mean of a rating scale item. Psychological Methods, 8(2), 149-163. +Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval + to Aiken’s item content-relevance index. Measurement in Physical Education and Exercise Science, 8(4), 213-225. +Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. + Journal of the American Statistical Association, 22, 209-212. } From b85d53514295e4154d20dade1bf2665b216a7ce6 Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Mon, 31 Aug 2026 14:44:17 -0600 Subject: [PATCH 6/9] update HAiken Overall H, IC-boot --- R/HAiken.R | 342 ++++++++++++++++++++++++++++++++++---------------- man/Haiken.Rd | 107 +++++++++------- 2 files changed, 296 insertions(+), 153 deletions(-) diff --git a/R/HAiken.R b/R/HAiken.R index 6bee249..e2c66a8 100644 --- a/R/HAiken.R +++ b/R/HAiken.R @@ -1,129 +1,259 @@ -#'@title Coefficient of homogeneity of response -#'@description Calculate the coefficient of homogeneity of response for each item (Aiken, 1980, 1985). +#' Coefficient of Homogeneity of Response (Aiken's H) #' -#'@param data dataframe, with the columns assigned to each judge, and the rows assigned to each evaluated item. -#'@param ncat number of response categories or options used in the rating -#'@param conf.level confidence level for the confidence intervals (eg., .90, .95, .99) -#'@param na.rm Logical. If FALSE (default) the function stops when missing values are detected. -#' If TRUE rows with missing values in the relevant columns are removed before processing. +#' @description +#' Calculates Aiken's H coefficient of homogeneity for each item and, optionally, +#' an overall H (total) across all items. The function uses bootstrap resampling +#' of judges to obtain confidence intervals. #' -#'@return -#'dataframe with H coefficients for all items analyzed, and their confidence intervals. +#' @param data A data frame with judges in columns and items in rows. +#' @param ncat Number of response categories. +#' @param conf.level Confidence level for the intervals (e.g., .90, .95). +#' @param na.rm Logical. If TRUE, rows with missing values are removed. +#' @param overall Logical. If TRUE (default), the overall H (total) is added as the last row. +#' @param B Integer. Number of bootstrap resamples (default = 1000). +#' @param ci.type Character: "logit" (default, recommended), "perc" (percentile), or "norm" (normal approximation). +#' The "logit" method transforms the bootstrap H values to the logit scale, +#' computes percentiles there, and back-transforms, ensuring the CI lies within [0,1]. #' -#'@details -#'Compute the H coefficient (Aiken, 1980, 1985) to estimate the homogeneity of response of the judges/scorers to the items. -#'To maintain consistency with the methods usually associated with content validity, 'HAiken' is proposed as an option. -#''HAiken' also compute asymmetric confidence intervals use the method of Wilson (1927), and adapted by Penfield and Giacobbi (2004) for Aiken's V coefficient. -#'The H coefficient, or equivalent coefficients, should complement the results of the content validity coefficients. -#'Other methods for estimating judges' agreement or homogeneity of response may also be useful. +#' @details +#' The procedure for obtaining confidence intervals with ci.type = "logit" is as follows: +#' \enumerate{ +#' \item Compute the point estimate of H for each item and for the total (if overall = TRUE). +#' \item Generate B bootstrap samples by resampling the judges (columns) with replacement. +#' For each bootstrap sample, recompute H for each item and the total. +#' \item Apply the logit transformation to each bootstrap H value: +#' \deqn{L = \log(H / (1 - H))}. +#' To avoid infinities when H = 0 or H = 1, a small constant \eqn{\varepsilon = 10^{-6}} +#' is added (or subtracted) so that \eqn{H^* = \max(\varepsilon, \min(1-\varepsilon, H))}. +#' \item Obtain the percentiles of the logit-transformed bootstrap distribution: +#' \eqn{L_{\text{inf}} = \text{percentile}_{\alpha/2}(L)}, \eqn{L_{\text{sup}} = \text{percentile}_{1-\alpha/2}(L)}. +#' \item Back-transform to the original scale: +#' \eqn{H_{\text{inf}} = \exp(L_{\text{inf}}) / (1 + \exp(L_{\text{inf}}))}, +#' \eqn{H_{\text{sup}} = \exp(L_{\text{sup}}) / (1 + \exp(L_{\text{sup}}))}. +#' } +#' This ensures that the confidence interval respects the [0,1] bounds and is asymmetric when appropriate. #' -#'@references -#'Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} +#' The methods "perc" and "norm" use the bootstrap percentiles or normal approximation directly on H, +#' which may produce limits outside [0,1] in extreme cases; they are provided for comparison but are not recommended. #' -#'Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} +#' @return A data frame with columns: +#' \item{Item}{Item number, or "Total" for the overall coefficient.} +#' \item{H}{Aiken's H coefficient.} +#' \item{lwr.ci}{Lower bound of the confidence interval.} +#' \item{upr.ci}{Upper bound of the confidence interval.} +#' \item{n.jueces}{Number of judges (same for all rows).} #' -#'Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} +#' @references +#' Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. +#' \emph{Educational and Psychological Measurement, 40}, 955-959. #' -#'Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} +#' Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. +#' \emph{Educational and Psychological Measurement, 45}, 131-142. #' -#'@seealso -#'\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval -#' -#'@author -#'Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) +#' @examples +#' \dontrun{ +#' # Sample data: 8 items, 18 judges, ratings 1-4 +#' datos <- data.frame(t(file1[, c("claA6", "claA18", "claA2", "claA16", +#' "claA4", "claA12", "claA9", "claA21")])) +#' Haiken(datos, ncat = 4, conf.level = .90, overall = TRUE, B = 1000, ci.type = "logit") +#' } #' +#' @export +Haiken <- function(data, ncat, conf.level = .95, na.rm = FALSE, + overall = TRUE, B = 1000, + ci.type = c("logit", "perc", "norm")) { -#'@examples -#'### Example 1 -------------- -#' -#'#Load data -#'Ej2 <- data.frame( -#' j1 = c(4, 1, 1, 1, 4), -#' j2 = c(4, 1, 2, 2, 3), -#' j3 = c(4, 1, 3, 3, 5), -#' j4 = c(4, 1, 4, 5, 5), -#' j5 = c(4, 1, 5, 5, 5), -#' j6 = c(4, 1, 3, 5, 5) -#') -#' -#'# Run HAiken -#'Haiken(Ej2, ncat = 5, conf.level = .90) -#' -#'### Example 2 ---------------- -#'# In a dataframe where the rows are the items and the columns are the raters, -#'# H can be calculated for the raters (columns) by simply transposing the data -#'# and entering it as a data frame. -#' -#'Haiken(as.data.frame(t(Ej2)), ncat = 5, conf.level = .90) -#' -#'@export -Haiken <- function(data, ncat, conf.level, na.rm = FALSE) { - # Detection of Missing Values - if (!na.rm) { - if (any(is.na(data))) { - stop("There are missing values. Use na.omit() first, or set na.rm=TRUE.") + ci.type <- match.arg(ci.type) + + if (!is.data.frame(data)) data <- as.data.frame(data) + + if (!na.rm && anyNA(data)) { + stop("Missing values found. Set na.rm=TRUE or handle them first.") + } + if (na.rm) data <- na.omit(data) + + n_items <- nrow(data) + m <- ncol(data) + if (n_items < 1 || m < 2) stop("Need at least 1 item and 2 judges.") + + delta <- ifelse(m %% 2 == 0, 1, 0) + denom_item <- (ncat - 1) * (m^2 - delta) + denom_total <- (ncat - 1) * n_items * (m^2 - delta) + + # --- Internal function to compute H (individual and total) --- + calc_H <- function(mat) { + ni <- nrow(mat) + mi <- ncol(mat) + delta_i <- ifelse(mi %% 2 == 0, 1, 0) + denom_i <- (ncat - 1) * (mi^2 - delta_i) + denom_total_i <- (ncat - 1) * ni * (mi^2 - delta_i) + + H_ind <- numeric(ni) + sum_diffs_total <- 0 + all_valid <- TRUE + + for (i in 1:ni) { + # Convert row to numeric robustly + row_i <- as.numeric(mat[i, ]) + # If any NA, try to convert from factor/character + if (anyNA(row_i)) { + row_i <- as.numeric(as.character(mat[i, ])) + } + # If still any NA, impute with median of non-NA values (or skip) + if (anyNA(row_i)) { + row_i[is.na(row_i)] <- median(row_i, na.rm = TRUE) + } + + # Compute pairwise absolute differences + diffs <- tryCatch( + combn(row_i, 2, function(p) abs(p[1] - p[2])), + error = function(e) rep(NA, choose(mi, 2)) + ) + + if (anyNA(diffs)) { + all_valid <- FALSE + H_ind[i] <- NA + sum_diffs_total <- NA + break + } else { + sum_diffs <- sum(diffs, na.rm = TRUE) + sum_diffs_total <- sum_diffs_total + sum_diffs + H_ind[i] <- 1 - (4 * sum_diffs) / denom_i + } } - } else { - data <- na.omit(data) + + H_total <- if (anyNA(H_ind) || !all_valid) { + NA + } else { + 1 - (4 * sum_diffs_total) / denom_total_i + } + + list(H_individual = H_ind, H_total = H_total, valid = all_valid) } - # Number of rows - num_filas <- nrow(data) + # --- Point estimates --- + est <- calc_H(data) + H_ind <- est$H_individual + H_total <- est$H_total - # Calculate j depending on whether ncol(data) is even or odd - j <- ifelse(ncol(data) %% 2 == 0, 0, 1) + if (anyNA(H_ind)) stop("Point estimates contain NA. Check data and ncat.") + if (overall && is.na(H_total)) stop("Total H is NA. Check data.") - # data frame for storing results - resultados <- data.frame( - Item = 1:num_filas, - H = numeric(num_filas), - lwr.ci = numeric(num_filas), - upr.ci = numeric(num_filas), - n.subj = numeric(num_filas) - ) + # --- Bootstrap over judges (columns) with resampling until valid --- + boot_H_ind <- matrix(NA, nrow = B, ncol = n_items) + boot_H_total <- numeric(B) - # Calculate the pairwise differences (absolute differences) and confidence intervals for each row - for (i in 1:num_filas) { - fila_actual <- data[i, ] - - # Calculate the pairwise differences as absolute differences and unpack them - diff_abs <- combn(colnames(fila_actual), 2, function(pair) { - col1 <- fila_actual[pair[1]] - col2 <- fila_actual[pair[2]] - diff_abs <- abs(col1 - col2) - return(diff_abs) - }, simplify = FALSE) # Use `simplify = FALSE` to get a list - - # Unpack the absolute differences from the list - diff_abs_unlist <- unlist(diff_abs) - - # Calculate the result for the current row using ncat, diff_abs_unlist, and j - resultado_actual <- 1 - (4 * sum(diff_abs_unlist) / ((ncat - 1) * (ncol(data)^2) - j)) - - get_wilson_CI <- function(x, n, conf.level) { - n <- n - p_hat <- x - SE_hat_sq <- p_hat * (1 - p_hat) / n - crit <- qnorm(1 - conf.level / 2) - omega <- n / (n + crit^2) - A <- p_hat + crit^2 / (2 * n) - B <- crit * sqrt(SE_hat_sq + crit^2 / (4 * n^2)) - CI <- c('lower' = omega * (A - B), - 'upper' = omega * (A + B)) - return(CI) + for (b in 1:B) { + valid_boot <- FALSE + attempts <- 0 + while (!valid_boot && attempts < 10) { + idx_col <- sample(1:m, size = m, replace = TRUE) + boot_data <- data[, idx_col, drop = FALSE] + boot_est <- calc_H(boot_data) + if (boot_est$valid && !anyNA(boot_est$H_individual) && !is.na(boot_est$H_total)) { + valid_boot <- TRUE + boot_H_ind[b, ] <- boot_est$H_individual + boot_H_total[b] <- boot_est$H_total + } + attempts <- attempts + 1 } + if (!valid_boot) { + # Fallback: use point estimates + boot_H_ind[b, ] <- H_ind + boot_H_total[b] <- H_total + } + } + + # --- Helper to compute confidence intervals (robust) --- + get_ci <- function(boot_vals, point_est, conf.level, ci.type) { + boot_vals <- boot_vals[!is.na(boot_vals)] + if (length(boot_vals) == 0) { + return(c(lwr = point_est, upr = point_est)) + } + if (length(unique(boot_vals)) == 1) { + return(c(lwr = point_est, upr = point_est)) + } + + alpha <- 1 - conf.level - # Calculate the confidence interval using the get_wilson_CI function and the alpha confidence level - wilson_interval <- get_wilson_CI(resultado_actual, ncol(data), conf.level) + if (ci.type == "perc") { + lwr <- quantile(boot_vals, probs = alpha/2, na.rm = TRUE, names = FALSE) + upr <- quantile(boot_vals, probs = 1 - alpha/2, na.rm = TRUE, names = FALSE) + if (is.na(lwr)) lwr <- min(boot_vals) + if (is.na(upr)) upr <- max(boot_vals) + return(c(lwr = lwr, upr = upr)) - # Store the results in the data frame - resultados[i, "H"] <- resultado_actual - resultados[i, "lwr.ci"] <- wilson_interval['lower'] - resultados[i, "upr.ci"] <- wilson_interval['upper'] - resultados[i, "n.subj"] <- num_filas + } else if (ci.type == "norm") { + se <- sd(boot_vals, na.rm = TRUE) + z <- qnorm(1 - alpha/2) + lwr <- point_est - z * se + upr <- point_est + z * se + return(c(lwr = lwr, upr = upr)) + } else { # "logit" (default) + eps <- 1e-6 + vals <- boot_vals + vals[vals <= 0] <- eps + vals[vals >= 1] <- 1 - eps + + logit_vals <- log(vals / (1 - vals)) + lwr_logit <- quantile(logit_vals, probs = alpha/2, na.rm = TRUE, names = FALSE) + upr_logit <- quantile(logit_vals, probs = 1 - alpha/2, na.rm = TRUE, names = FALSE) + if (is.na(lwr_logit)) lwr_logit <- min(logit_vals) + if (is.na(upr_logit)) upr_logit <- max(logit_vals) + + lwr <- exp(lwr_logit) / (1 + exp(lwr_logit)) + upr <- exp(upr_logit) / (1 + exp(upr_logit)) + return(c(lwr = lwr, upr = upr)) + } } - return(round(resultados, 3)) -} + # --- Compute CIs for each item --- + lwr_ind <- numeric(n_items) + upr_ind <- numeric(n_items) + for (i in 1:n_items) { + ci <- get_ci(boot_H_ind[, i], H_ind[i], conf.level, ci.type) + lwr_ind[i] <- ci["lwr"] + upr_ind[i] <- ci["upr"] + } + # --- CI for the total H (if overall = TRUE) --- + if (overall) { + ci_total <- get_ci(boot_H_total, H_total, conf.level, ci.type) + lwr_total <- ci_total["lwr"] + upr_total <- ci_total["upr"] + } else { + lwr_total <- NA + upr_total <- NA + } + + # --- Build results data.frame --- + resultados <- data.frame( + Item = 1:n_items, + H = round(H_ind, 3), + lwr.ci = round(lwr_ind, 3), + upr.ci = round(upr_ind, 3), + n.jueces = m, + stringsAsFactors = FALSE, + row.names = NULL + ) + + if (overall) { + fila_total <- data.frame( + Item = "Total", + H = round(H_total, 3), + lwr.ci = if (is.na(lwr_total)) NA else round(lwr_total, 3), + upr.ci = if (is.na(upr_total)) NA else round(upr_total, 3), + n.jueces = m, + stringsAsFactors = FALSE, + row.names = NULL + ) + resultados <- rbind(resultados, fila_total) + } + + attr(resultados, "ci.type") <- ci.type + attr(resultados, "B") <- B + attr(resultados, "conf.level") <- conf.level + + return(resultados) +} diff --git a/man/Haiken.Rd b/man/Haiken.Rd index ab619a7..93428a5 100644 --- a/man/Haiken.Rd +++ b/man/Haiken.Rd @@ -2,69 +2,82 @@ % Please edit documentation in R/HAiken.R \name{Haiken} \alias{Haiken} -\title{Coefficient of homogeneity of response} +\title{Coefficient of Homogeneity of Response (Aiken's H)} \usage{ -Haiken(data, ncat, conf.level, na.rm = FALSE) +Haiken( + data, + ncat, + conf.level = 0.95, + na.rm = FALSE, + overall = TRUE, + B = 1000, + ci.type = c("logit", "perc", "norm") +) } \arguments{ -\item{data}{dataframe, with the columns assigned to each judge, and the rows assigned to each evaluated item.} +\item{data}{A data frame with judges in columns and items in rows.} + +\item{ncat}{Number of response categories.} + +\item{conf.level}{Confidence level for the intervals (e.g., .90, .95).} + +\item{na.rm}{Logical. If TRUE, rows with missing values are removed.} -\item{ncat}{number of response categories or options used in the rating} +\item{overall}{Logical. If TRUE (default), the overall H (total) is added as the last row.} -\item{conf.level}{confidence level for the confidence intervals (eg., .90, .95, .99)} +\item{B}{Integer. Number of bootstrap resamples (default = 1000).} -\item{na.rm}{Logical. If FALSE (default) the function stops when missing values are detected. -If TRUE rows with missing values in the relevant columns are removed before processing.} +\item{ci.type}{Character: "logit" (default, recommended), "perc" (percentile), or "norm" (normal approximation). +The "logit" method transforms the bootstrap H values to the logit scale, +computes percentiles there, and back-transforms, ensuring the CI lies within [0,1].} } \value{ -dataframe with H coefficients for all items analyzed, and their confidence intervals. +A data frame with columns: + \item{Item}{Item number, or "Total" for the overall coefficient.} + \item{H}{Aiken's H coefficient.} + \item{lwr.ci}{Lower bound of the confidence interval.} + \item{upr.ci}{Upper bound of the confidence interval.} + \item{n.jueces}{Number of judges (same for all rows).} } \description{ -Calculate the coefficient of homogeneity of response for each item (Aiken, 1980, 1985). +Calculates Aiken's H coefficient of homogeneity for each item and, optionally, +an overall H (total) across all items. The function uses bootstrap resampling +of judges to obtain confidence intervals. } \details{ -Compute the H coefficient (Aiken, 1980, 1985) to estimate the homogeneity of response of the judges/scorers to the items. -To maintain consistency with the methods usually associated with content validity, 'HAiken' is proposed as an option. -'HAiken' also compute asymmetric confidence intervals use the method of Wilson (1927), and adapted by Penfield and Giacobbi (2004) for Aiken's V coefficient. -The H coefficient, or equivalent coefficients, should complement the results of the content validity coefficients. -Other methods for estimating judges' agreement or homogeneity of response may also be useful. +The procedure for obtaining confidence intervals with ci.type = "logit" is as follows: +\enumerate{ + \item Compute the point estimate of H for each item and for the total (if overall = TRUE). + \item Generate B bootstrap samples by resampling the judges (columns) with replacement. + For each bootstrap sample, recompute H for each item and the total. + \item Apply the logit transformation to each bootstrap H value: + \deqn{L = \log(H / (1 - H))}. + To avoid infinities when H = 0 or H = 1, a small constant \eqn{\varepsilon = 10^{-6}} + is added (or subtracted) so that \eqn{H^* = \max(\varepsilon, \min(1-\varepsilon, H))}. + \item Obtain the percentiles of the logit-transformed bootstrap distribution: + \eqn{L_{\text{inf}} = \text{percentile}_{\alpha/2}(L)}, \eqn{L_{\text{sup}} = \text{percentile}_{1-\alpha/2}(L)}. + \item Back-transform to the original scale: + \eqn{H_{\text{inf}} = \exp(L_{\text{inf}}) / (1 + \exp(L_{\text{inf}}))}, + \eqn{H_{\text{sup}} = \exp(L_{\text{sup}}) / (1 + \exp(L_{\text{sup}}))}. } -\examples{ -### Example 1 -------------- - -#Load data -Ej2 <- data.frame( - j1 = c(4, 1, 1, 1, 4), - j2 = c(4, 1, 2, 2, 3), - j3 = c(4, 1, 3, 3, 5), - j4 = c(4, 1, 4, 5, 5), - j5 = c(4, 1, 5, 5, 5), - j6 = c(4, 1, 3, 5, 5) -) - -# Run HAiken -Haiken(Ej2, ncat = 5, conf.level = .90) +This ensures that the confidence interval respects the [0,1] bounds and is asymmetric when appropriate. -### Example 2 ---------------- -# In a dataframe where the rows are the items and the columns are the raters, -# H can be calculated for the raters (columns) by simply transposing the data -# and entering it as a data frame. - -Haiken(as.data.frame(t(Ej2)), ncat = 5, conf.level = .90) +The methods "perc" and "norm" use the bootstrap percentiles or normal approximation directly on H, +which may produce limits outside [0,1] in extreme cases; they are provided for comparison but are not recommended. +} +\examples{ +\dontrun{ +# Sample data: 8 items, 18 judges, ratings 1-4 +datos <- data.frame(t(file1[, c("claA6", "claA18", "claA2", "claA16", + "claA4", "claA12", "claA9", "claA21")])) +Haiken(datos, ncat = 4, conf.level = .90, overall = TRUE, B = 1000, ci.type = "logit") +} } \references{ -Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. \emph{Educational and. Psychological Measurement, 40}, 955-959. \doi{10.1177/001316448004000419} +Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. + \emph{Educational and Psychological Measurement, 40}, 955-959. -Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. \emph{Educational and Psychological Measurement, 45}, 131-142. \doi{10.1177/0013164485451012} - -Penfield, R. D. & Giacobbi, P. R., Jr. (2004) Applying a score confidence interval to Aiken’s item content-relevance index. \emph{Measurement in Physical Education and Exercise Science, 8}(4), 213-225. \doi{10.1207/s15327841mpee0804_3} - -Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. \emph{Journal of the American Statistical Association, 22}, 209-212. \doi{10.2307/2276774} -} -\seealso{ -\code{\link[PropCIs:scoreci]{PropCIs::scoreci}} for score method confidence interval -} -\author{ -Cesar Merino-Soto (\email{sikayax@yahoo.cam.ar}) +Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. + \emph{Educational and Psychological Measurement, 45}, 131-142. } From 58348dee925020283b84fd9cb5218f1485693a8a Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Tue, 1 Sep 2026 12:27:12 -0600 Subject: [PATCH 7/9] Update: change HT to ColquittHT Improving confidence intervals for HTC, HTD More extensive documentation about confidence intervals. Name change: HT.R to ColquittHT.R --- NAMESPACE | 2 +- R/ColquittHT.R | 363 ++++++++++++++++++++++++++++++++++++++ R/HT.R | 440 ---------------------------------------------- man/ColquittHT.Rd | 127 +++++++++++++ man/HTmult.Rd | 191 -------------------- 5 files changed, 491 insertions(+), 632 deletions(-) create mode 100644 R/ColquittHT.R delete mode 100644 R/HT.R create mode 100644 man/ColquittHT.Rd delete mode 100644 man/HTmult.Rd diff --git a/NAMESPACE b/NAMESPACE index 446e0bc..026785c 100644 --- a/NAMESPACE +++ b/NAMESPACE @@ -15,8 +15,8 @@ export(CVRcut.Ayres) export(CVRcut.Bag) export(CVRcut.Wilson) export(CVplot) +export(ColquittHT) export(D2) -export(HTmult) export(Haiken) export(LuAgree) export(MDScontent) diff --git a/R/ColquittHT.R b/R/ColquittHT.R new file mode 100644 index 0000000..cd1309b --- /dev/null +++ b/R/ColquittHT.R @@ -0,0 +1,363 @@ +#' Hinkin–Tracey Content Validity Indices with Wilson Confidence Intervals +#' +#' @description +#' Computes Hinkin and Tracey (1999) content validity indices for multiple items +#' using the standardized operationalizations and empirical benchmarks proposed by +#' Colquitt et al. (2019). For each item, the function computes: +#' \itemize{ +#' \item Descriptive means of ratings across all evaluated constructs. +#' \item \code{htc}: Hinkin–Tracey correspondence index based on the mean rating of the +#' target construct. +#' \item \code{htd}: Hinkin–Tracey distinctiveness index based on the rescaled mean +#' difference between ratings of the target construct and orbiting constructs. +#' \item \code{htd} computed pairwise between the target construct and each individual +#' orbiting construct. +#' } +#' +#' Asymmetric confidence intervals are constructed using Wilson's score interval method +#' for proportions (Penfield & Miller, 2004), eliminating out-of-bounds limits and +#' providing realistic coverage even under zero sample variance or small expert sample sizes. +#' +#' @param data A data frame or matrix in wide format, where each column +#' corresponds to an item–construct rating. Column names must follow the pattern +#' \code{"item.construct"} (e.g., \code{"item1.c1"}, \code{"item1.c2"}). +#' @param items Character vector with the base names of the items (e.g., +#' \code{c("item1", "item2")}). +#' @param constructs Character vector with construct labels (e.g., +#' \code{c("c1", "c2", "c3")}). +#' @param key Named character vector mapping each item to its target construct. +#' Names must match \code{items} and values must belong to \code{constructs} +#' (e.g., \code{c(item1 = "c2", item2 = "c1")}). +#' @param anchors Integer. Number of response scale options (e.g., 5 or 7). Ratings +#' are assumed to range from 1 to \code{anchors}. +#' @param ci Logical. If \code{TRUE}, asymmetric Wilson score confidence intervals are +#' computed for \code{htc}, \code{htd}, and pairwise \code{htd}. Default is \code{FALSE}. +#' @param conf.level Confidence level for intervals (e.g., \code{0.90}, \code{0.95}). +#' Default is \code{0.90}. +#' @param na.rm Logical. If \code{TRUE} (default), missing values are removed pair-wise or +#' list-wise depending on the sub-index calculation. +#' @param nd Integer. Number of decimal places for rounding output values. Default is \code{3}. +#' +#' @details +#' \strong{Empirical Benchmarks (Colquitt et al., 2019)} +#' +#' Based on empirical decile distributions across extensive content validation studies, +#' Colquitt et al. (2019) suggest the following evaluation criteria: +#' \itemize{ +#' \item \strong{Definitional Correspondence (\code{htc}):} +#' \itemize{ +#' \item Strong: \eqn{\ge 0.86} +#' \item Moderate: \eqn{0.78} to \eqn{0.85} +#' \item Weak: \eqn{\le 0.77} +#' } +#' \item \strong{Definitional Distinctiveness (\code{htd}):} +#' \itemize{ +#' \item Strong: \eqn{\ge 0.27} +#' \item Moderate: \eqn{0.21} to \eqn{0.26} +#' \item Weak: \eqn{\le 0.20} +#' } +#' } +#' +#' \strong{Rationale and Construction of Confidence Intervals} +#' +#' Standard Studentized \emph{t}-intervals often fail in content validity tasks because rating +#' distributions near scale bounds produce zero sample variance (e.g., perfect agreement across judges), +#' yielding artificially collapsed zero-width intervals. To overcome this, confidence intervals +#' for \code{htc} and \code{htd} are computed via Wilson's score interval (Penfield & Miller, 2004) for mean of +#' rating expert: +#' \itemize{ +#' \item \strong{\code{htc} Transformation:} The raw target mean \eqn{\bar{X}} (bounded in \eqn{[1, a]}) +#' is mapped to a proportion \eqn{p = (\bar{X} - 1) / (a - 1)}. Wilson's score interval is +#' computed on \eqn{p} and then rescaled back to the \code{htc} metric (\eqn{[1/a, 1.0]}). +#' \item \strong{\code{htd} Transformation:} The distinctiveness index \eqn{htd} naturally ranges in +#' \eqn{[-1, 1]}. To apply score-based estimation, it is linearly mapped to a pseudo-proportion +#' space \eqn{p_{htd} = (htd + 1) / 2 \in [0, 1]}. Wilson limits are derived for \eqn{p_{htd}} +#' and then back-transformed via \eqn{htd_{\text{limit}} = (p_{\text{wilson}} \times 2) - 1}. +#' } +#' +#' \strong{Methodological Note on Comparisons:} +#' Confidence intervals should be used to evaluate the precision of individual coefficients or to +#' compare items within the \emph{same} metric type (e.g., comparing \code{htc} between Item 1 and Item 2). +#' Comparing an \code{htc} interval directly against an \code{htd} interval is methodologically invalid, +#' as they capture fundamentally different theoretical constructs and variance structures. +#' +#' @return A list with three data frames: +#' \describe{ +#' \item{\code{Item.descriptive}}{Item names, target constructs, judge counts (\code{nj}), and mean ratings.} +#' \item{\code{Item.criteria}}{Global \code{htc} and \code{htd} indices with corresponding Wilson CIs.} +#' \item{\code{Pairwise.criteria}}{Pairwise \code{htd} indices comparing target vs. each orbiting construct.} +#' } +#' +#' @references +#' Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019). Content validation guidelines: +#' Evaluation criteria for definitional correspondence and definitional distinctiveness. +#' \emph{Journal of Applied Psychology, 104}(10), 1243–1265. +#' +#' Hinkin, T. R., & Tracey, J. B. (1999). An analysis of variance approach to content validation. +#' \emph{Organizational Research Methods, 2}(2), 175–186. +#' +#' Penfield, R. D., & Miller, J. M. (2004). Improving content validation studies using an asymmetric +#' confidence interval for the mean of expert ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. +#' +#' @export +ColquittHT <- function( + data, + items, + constructs, + key, + anchors, + ci = FALSE, + conf.level = 0.95, + na.rm = TRUE, + nd = 3 +) { + # --- Basic argument validation --- + if (!is.data.frame(data)) { + data <- as.data.frame(data) + } + + if (!is.character(items) || length(items) == 0) { + stop("'items' must be a non-empty character vector.") + } + + if (!is.character(constructs) || length(constructs) < 2) { + stop("'constructs' must be a character vector with at least two constructs.") + } + + if (missing(key) || is.null(key)) { + stop("'key' must be provided as a named character vector mapping items to target constructs.") + } + + if (is.null(names(key)) || any(names(key) == "")) { + stop("'key' must have names corresponding to item names.") + } + + if (!all(items %in% names(key))) { + stop("All 'items' must appear as names in 'key'.") + } + + if (!all(key[items] %in% constructs)) { + stop("All target constructs in 'key' must be included in 'constructs'.") + } + + if (!is.numeric(anchors) || length(anchors) != 1 || anchors <= 1) { + stop("'anchors' must be a numeric value > 1 (e.g., 5 or 7).") + } + + if (!is.logical(ci) || length(ci) != 1) { + stop("'ci' must be a single logical value (TRUE/FALSE).") + } + + if (!is.numeric(conf.level) || conf.level <= 0 || conf.level >= 1) { + stop("'conf.level' must be a numeric value between 0 and 1.") + } + + # --- Internal Helper: Wilson Score Confidence Interval for Proportions --- + # Computes asymmetric Wilson CI for a proportion p with sample size n. + wilson_ci <- function(p, n, conf.level) { + if (is.na(p) || is.na(n) || n <= 0) { + return(list(lwr = NA_real_, upr = NA_real_)) + } + # Bound input proportion to [0, 1] to ensure mathematical stability + p <- max(0, min(1, p)) + + alpha <- 1 - conf.level + z <- stats::qnorm(1 - alpha / 2) + z2 <- z^2 + + denominator <- 1 + z2 / n + center <- (p + z2 / (2 * n)) / denominator + spread <- (z * sqrt((p * (1 - p) / n) + (z2 / (4 * n^2)))) / denominator + + lwr <- max(0, center - spread) + upr <- min(1, center + spread) + + list(lwr = lwr, upr = upr) + } + + item_desc_list <- list() + item_crit_list <- list() + pairwise_list <- list() + + idx_desc <- 1L + idx_crit <- 1L + idx_pair <- 1L + + # --- Loop over items --- + for (item in items) { + + target <- key[[item]] + + # Construct standard column names for the current item + item_cols <- paste0(item, ".", constructs) + + # Verify column existence in input dataset + missing_cols <- setdiff(item_cols, colnames(data)) + if (length(missing_cols) > 0) { + stop( + "For item '", item, "', the following columns are missing in 'data': ", + paste(missing_cols, collapse = ", ") + ) + } + + # Extract construct rating vectors + ratings_list <- lapply(item_cols, function(nm) data[[nm]]) + names(ratings_list) <- constructs + + target_vec <- ratings_list[[target]] + + # Determine effective judge sample size on target construct + nj <- if (na.rm) sum(!is.na(target_vec)) else length(target_vec) + + # --- 1. Descriptive statistics row --- + means_c <- sapply( + ratings_list, + function(v) if (na.rm) mean(v, na.rm = TRUE) else mean(v) + ) + + desc_row <- data.frame( + item = item, + target = target, + nj = nj, + t(means_c), + check.names = FALSE + ) + mean_col_names <- paste0("M.", constructs) + colnames(desc_row)[(ncol(desc_row) - length(constructs) + 1):ncol(desc_row)] <- mean_col_names + + item_desc_list[[idx_desc]] <- desc_row + idx_desc <- idx_desc + 1L + + # --- 2. Global Criteria: HTC and HTD --- + mean_target <- means_c[target] + htc <- mean_target / anchors + + htc_lci <- htc_uci <- NA_real_ + if (ci && nj > 0) { + # Map mean target rating from [1, anchors] to proportion p_target in [0, 1] + p_target <- (mean_target - 1) / (anchors - 1) + ci_target <- wilson_ci(p = p_target, n = nj, conf.level = conf.level) + + # Convert Wilson bounds back from proportion space to target mean, then divide by anchors + mean_lwr <- ci_target$lwr * (anchors - 1) + 1 + mean_upr <- ci_target$upr * (anchors - 1) + 1 + htc_lci <- mean_lwr / anchors + htc_uci <- mean_upr / anchors + } + + # Global HTD computation (judge-level mean difference across orbiting constructs) + orbiting_constructs <- setdiff(constructs, target) + D_vec <- rep(NA_real_, length(target_vec)) + + for (i in seq_along(target_vec)) { + t_i <- target_vec[i] + if (is.na(t_i)) next + orbit_vals <- vapply( + orbiting_constructs, + function(cc) ratings_list[[cc]][i], + FUN.VALUE = NA_real_ + ) + if (na.rm) { + orbit_vals <- orbit_vals[!is.na(orbit_vals)] + } + if (length(orbit_vals) == 0) next + D_vec[i] <- t_i - mean(orbit_vals) + } + + D_mean <- if (na.rm) mean(D_vec, na.rm = TRUE) else mean(D_vec) + htd <- D_mean / (anchors - 1) + + htd_lci <- htd_uci <- NA_real_ + if (ci) { + # Effective number of judges evaluating difference scores + n_D <- if (na.rm) sum(!is.na(D_vec)) else length(D_vec) + if (n_D > 0) { + # Rescale htd from [-1, 1] to pseudo-proportion p_htd in [0, 1] + p_htd <- (htd + 1) / 2 + ci_htd <- wilson_ci(p = p_htd, n = n_D, conf.level = conf.level) + + # Back-transform Wilson bounds to original htd metric [-1, 1] + htd_lci <- (ci_htd$lwr * 2) - 1 + htd_uci <- (ci_htd$upr * 2) - 1 + } + } + + crit_row <- data.frame( + item = item, + htc = htc, + htc.lci = htc_lci, + htc.uci = htc_uci, + htd = htd, + htd.lci = htd_lci, + htd.uci = htd_uci, + check.names = FALSE + ) + + item_crit_list[[idx_crit]] <- crit_row + idx_crit <- idx_crit + 1L + + # --- 3. Pairwise Criteria: HTD per orbiting construct --- + for (cc in orbiting_constructs) { + vec_c <- ratings_list[[cc]] + + if (na.rm) { + idx_valid <- !is.na(target_vec) & !is.na(vec_c) + d_c <- target_vec[idx_valid] - vec_c[idx_valid] + } else { + d_c <- target_vec - vec_c + } + + D_c_mean <- if (na.rm) mean(d_c, na.rm = TRUE) else mean(d_c) + htd_c <- D_c_mean / (anchors - 1) + + htd_c_lci <- htd_c_uci <- NA_real_ + if (ci) { + n_dc <- length(d_c) + if (n_dc > 0) { + # Rescale pairwise htd from [-1, 1] to pseudo-proportion space [0, 1] + p_htd_c <- (htd_c + 1) / 2 + ci_dc <- wilson_ci(p = p_htd_c, n = n_dc, conf.level = conf.level) + + # Back-transform limits to original htd metric [-1, 1] + htd_c_lci <- (ci_dc$lwr * 2) - 1 + htd_c_uci <- (ci_dc$upr * 2) - 1 + } + } + + pair_row <- data.frame( + item = item, + target = target, + orbiting = cc, + htd = htd_c, + htd.lci = htd_c_lci, + htd.uci = htd_c_uci, + check.names = FALSE + ) + + pairwise_list[[idx_pair]] <- pair_row + idx_pair <- idx_pair + 1L + } + } + + # Combine rows across items + Item.descriptive <- do.call(rbind, item_desc_list) + Item.criteria <- do.call(rbind, item_crit_list) + Pairwise.criteria <- do.call(rbind, pairwise_list) + + # Round numeric output columns + round_numeric_df <- function(df, digits) { + is_num <- vapply(df, is.numeric, logical(1)) + df[is_num] <- lapply(df[is_num], round, digits = digits) + df + } + + Item.descriptive <- round_numeric_df(Item.descriptive, nd) + Item.criteria <- round_numeric_df(Item.criteria, nd) + Pairwise.criteria <- round_numeric_df(Pairwise.criteria, nd) + + list( + Item.descriptive = Item.descriptive, + Item.criteria = Item.criteria, + Pairwise.criteria = Pairwise.criteria + ) +} diff --git a/R/HT.R b/R/HT.R deleted file mode 100644 index 297be5e..0000000 --- a/R/HT.R +++ /dev/null @@ -1,440 +0,0 @@ -#' Hinkin–Tracey Content Validity Indices for Multiple Items -#' -#' @description -#' Computes Hinkin and Tracey (1999) content validity indices for multiple items -#' when judges rate how well each item corresponds to several construct -#' definitions in the same judgement task. For each item, the function returns: -#' \itemize{ -#' \item Descriptive means of ratings for each construct. -#' \item \code{htc}: correspondence index based on the mean rating of the -#' target construct. -#' \item \code{htd}: distinctiveness index based on the mean rating -#' difference between the target construct and all orbiting constructs. -#' \item \code{htd} computed pairwise between the target construct and each -#' orbiting construct. -#' } -#' -#' Ratings are assumed to be given on a discrete Likert-type scale from -#' 1 to \code{anchors}. Confidence intervals for the indices are based on -#' a t-based interval for the mean, truncated to the possible rating range, -#' and then rescaled to the \code{htc}/\code{htd} metrics. -#' -#' @param data A data frame or matrix in wide format, where each column -#' corresponds to one item–construct combination. Columns are expected -#' to follow the pattern \code{"item.construct"}, e.g., -#' \code{"item1.c1"}, \code{"item1.c2"}, \code{"item1.c3"}. -#' @param items Character vector with the base names of the items to be -#' analyzed (i.e., the prefixes before the dot), e.g. -#' \code{c("item1", "item2", "item3")}. -#' @param constructs Character vector with the construct labels (suffixes -#' after the dot), e.g. \code{c("c1", "c2", "c3")}. The function will -#' search for columns \code{paste0(item, ".", construct)} for every -#' combination of \code{items} and \code{constructs}. -#' @param key Named character vector mapping each item to its target -#' construct. The names must match \code{items}, and the values must be -#' one of the \code{constructs}. For example: -#' \code{c(item1 = "c2", item2 = "c1", item3 = "c3")}. -#' @param anchors Integer. Number of response options in the rating scale -#' (e.g., 5 or 7). Ratings are assumed to range from 1 to \code{anchors}. -#' @param ci Logical. If \code{TRUE}, asymmetric confidence intervals are -#' computed for \code{htc}, \code{htd} and the pairwise \code{htd} indices. -#' Default is \code{FALSE}. -#' @param conf.level Confidence level for the confidence intervals (e.g., -#' \code{0.90}, \code{0.95}, \code{0.99}). Used only if \code{ci = TRUE}. -#' Default is \code{0.95}. -#' @param na.rm Logical. If \code{TRUE} (default), missing values are removed -#' when computing means and differences. If \code{FALSE}, missing values -#' will propagate \code{NA} in the corresponding statistics. -#' @param nd Integer. Number of decimal places to use when rounding -#' the numeric results. Default is \code{3}. -#' -#' @details -#' For each item, the function extracts the ratings for each construct from -#' columns named \code{"item.construct"}. The target construct is specified -#' via the \code{key} argument. -#' -#' \strong{Item-level descriptive statistics} -#' -#' For each item, \code{$Item.descriptive} reports: -#' \itemize{ -#' \item \code{item}: item name (base). -#' \item \code{target}: target construct for that item. -#' \item \code{nj}: number of judges with a non-missing rating on the -#' target construct. -#' \item \code{M.}: mean rating for each construct. -#' } -#' -#' \strong{Item-level criteria} -#' -#' For each item, \code{$Item.criteria} reports: -#' \itemize{ -#' \item \code{htc}: the correspondence index, defined as the mean rating -#' on the target construct divided by \code{anchors}. -#' \item \code{htd}: the distinctiveness index, defined as: -#' \eqn{\text{htd} = \bar{D} / (a - 1)}, where \eqn{\bar{D}} is the -#' mean (over judges) of the average difference between the rating -#' on the target construct and the ratings on all orbiting -#' constructs, and \eqn{a} is \code{anchors}. -#' \item If \code{ci = TRUE}, lower and upper confidence limits -#' (\code{htc.lci}, \code{htc.uci}, \code{htd.lci}, \code{htd.uci}) -#' based on a t-interval for the mean truncated to the possible -#' rating range and then rescaled. -#' } -#' -#' \strong{Pairwise distinctiveness} -#' -#' For each item and each orbiting construct (i.e., each construct other -#' than the target), \code{$Pairwise.criteria} reports: -#' \itemize{ -#' \item \code{item}: item name. -#' \item \code{target}: target construct for that item. -#' \item \code{orbiting}: the orbiting construct. -#' \item \code{htd}: a pairwise distinctiveness index defined as: -#' \eqn{\text{htd}_{c} = \bar{D}_c / (a - 1)}, where \eqn{\bar{D}_c} -#' is the mean (over judges) of the difference between the rating on -#' the target construct and the rating on the orbiting construct -#' \eqn{c}. -#' \item If \code{ci = TRUE}, \code{htd.lci} and \code{htd.uci} as -#' truncated t-based confidence intervals rescaled to the -#' \code{htd} metric. -#' } -#' -#' This function is intended for the Hinkin and Tracey (1999) method of -#' content validation, where judges rate how well each item corresponds -#' to each construct definition. The same wide format can also be useful -#' for implementing criteria inspired by Colquitt et al. (2019). -#' -#' Ratings are assumed to be coded from 1 to \code{anchors}. If another -#' coding scheme is used, users should recode the ratings before calling -#' the function. -#' -#' @return -#' A list with three components: -#' \describe{ -#' \item{\code{Item.descriptive}}{A data frame with one row per item, -#' containing item-level descriptive statistics: -#' \code{item}, \code{target}, \code{nj}, and the construct means -#' (\code{M.} columns).} -#' \item{\code{Item.criteria}}{A data frame with one row per item, -#' containing the global Hinkin–Tracey indices: -#' \code{item}, \code{htc}, \code{htc.lci}, \code{htc.uci}, -#' \code{htd}, \code{htd.lci}, \code{htd.uci}. If \code{ci = FALSE}, -#' the confidence interval columns are filled with \code{NA}.} -#' \item{\code{Pairwise.criteria}}{A data frame in long format, with one -#' row per item–orbiting construct combination, containing: -#' \code{item}, \code{target}, \code{orbiting}, \code{htd}, -#' \code{htd.lci}, \code{htd.uci}. If \code{ci = FALSE}, the confidence -#' interval columns are filled with \code{NA}.} -#' } -#' -#' @references -#' Hinkin, T. R., & Tracey, J. B. (1999). An analysis of variance approach -#' to content validation. \emph{Organizational Research Methods, 2}(2), 175–186. -#' -#' Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019). -#' Content validation guidelines: Evaluation criteria for definitional -#' correspondence and definitional distinctiveness. \emph{Journal of Applied -#' Psychology, 104}(10), 1243–1265. -#' -#' Penfield, R. D., & Miller, J. M. (2004). Improving content validation -#' studies using an asymmetric confidence interval for the mean of expert -#' ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. -#' -#' @examples -#' \dontrun{ -#' ## Example structure (toy data) -#' set.seed(123) -#' HTmult.data <- data.frame( -#' item1.c1 = sample(1:5, 50, replace = TRUE), -#' item1.c2 = sample(1:5, 50, replace = TRUE), -#' item1.c3 = sample(1:5, 50, replace = TRUE), -#' item2.c1 = sample(1:5, 50, replace = TRUE), -#' item2.c2 = sample(1:5, 50, replace = TRUE), -#' item2.c3 = sample(1:5, 50, replace = TRUE) -#' ) -#' -#'HTmult( -#' data = dat, -#' items = c("item1", "item2"), -#' constructs = c("c1", "c2", "c3"), -#' key = c(item1 = "c2", item2 = "c3"), -#' anchors = 5, -#' ci = TRUE, -#' conf.level = 0.95) -#' } -#' -#' @export -HTmult <- function( - data, - items, - constructs, - key, - anchors, - ci = FALSE, - conf.level = 0.95, - na.rm = TRUE, - nd = 3 -) { - # --- Basic checks and setup --- - if (!is.data.frame(data)) { - data <- as.data.frame(data) - } - - if (!is.character(items) || length(items) == 0) { - stop("'items' must be a non-empty character vector.") - } - - if (!is.character(constructs) || length(constructs) < 2) { - stop("'constructs' must be a character vector with at least two constructs.") - } - - if (missing(key) || is.null(key)) { - stop("'key' must be provided as a named character vector mapping items to target constructs.") - } - - if (is.null(names(key)) || any(names(key) == "")) { - stop("'key' must have names corresponding to item names.") - } - - if (!all(items %in% names(key))) { - stop("All 'items' must appear as names in 'key'.") - } - - if (!all(key[items] %in% constructs)) { - stop("All target constructs in 'key' must be included in 'constructs'.") - } - - if (!is.numeric(anchors) || length(anchors) != 1 || anchors <= 1) { - stop("'anchors' must be a numeric value > 1 (e.g., 5 or 7).") - } - - if (!is.logical(ci) || length(ci) != 1) { - stop("'ci' must be a single logical value (TRUE/FALSE).") - } - - if (!is.numeric(conf.level) || conf.level <= 0 || conf.level >= 1) { - stop("'conf.level' must be a numeric value between 0 and 1.") - } - - # Helper: t-based CI for mean with truncation to [lower, upper] - mean_ci_truncated <- function(x, lower, upper, conf.level) { - x <- x[!is.na(x)] - n <- length(x) - if (n == 0) { - return(list(mean = NA_real_, lwr = NA_real_, upr = NA_real_)) - } - m <- mean(x) - if (n == 1 || is.na(sd(x)) || sd(x) == 0) { - # No variability or single observation: CI collapses to the truncated mean - lwr <- max(min(m, upper), lower) - upr <- lwr - } else { - alpha <- 1 - conf.level - se <- sd(x) / sqrt(n) - tcrit <- stats::qt(1 - alpha / 2, df = n - 1) - lwr <- m - tcrit * se - upr <- m + tcrit * se - # truncate to possible range - lwr <- max(lwr, lower) - upr <- min(upr, upper) - } - list(mean = m, lwr = lwr, upr = upr) - } - - item_desc_list <- list() - item_crit_list <- list() - pairwise_list <- list() - - idx_desc <- 1L - idx_crit <- 1L - idx_pair <- 1L - - # --- Loop over items --- - for (item in items) { - - target <- key[[item]] - - # Build column names for this item - item_cols <- paste0(item, ".", constructs) - - # Check that all required columns exist - missing_cols <- setdiff(item_cols, colnames(data)) - if (length(missing_cols) > 0) { - stop( - "For item '", item, "', the following columns are missing in 'data': ", - paste(missing_cols, collapse = ", ") - ) - } - - # Extract ratings per construct - ratings_list <- lapply(item_cols, function(nm) data[[nm]]) - names(ratings_list) <- constructs - - # Target ratings - target_vec <- ratings_list[[target]] - - # Number of judges (non-missing in target) - if (na.rm) { - nj <- sum(!is.na(target_vec)) - } else { - nj <- length(target_vec) - } - - # --- Item.descriptive row --- - means_c <- sapply( - ratings_list, - function(v) if (na.rm) mean(v, na.rm = TRUE) else mean(v) - ) - - desc_row <- data.frame( - item = item, - target = target, - nj = nj, - t(means_c), - check.names = FALSE - ) - # Rename construct mean columns to M. - mean_col_names <- paste0("M.", constructs) - colnames(desc_row)[(ncol(desc_row) - length(constructs) + 1):ncol(desc_row)] <- mean_col_names - - item_desc_list[[idx_desc]] <- desc_row - idx_desc <- idx_desc + 1L - - # --- Item.criteria row: HTC and HTD global --- - # HTC - mean_target <- means_c[target] - htc <- mean_target / anchors - - htc_lci <- htc_uci <- NA_real_ - if (ci) { - # Ratings assumed 1..anchors - ci_target <- mean_ci_truncated( - x = target_vec, - lower = 1, - upper = anchors, - conf.level = conf.level - ) - if (!is.na(ci_target$lwr)) { - htc_lci <- ci_target$lwr / anchors - htc_uci <- ci_target$upr / anchors - } - } - - # HTD global: average difference target - orbiting (over constructs), - # then averaged over judges and normalized by (anchors - 1) - orbiting_constructs <- setdiff(constructs, target) - # For each judge, compute mean difference across orbiting constructs - D_vec <- rep(NA_real_, length(target_vec)) - for (i in seq_along(target_vec)) { - t_i <- target_vec[i] - if (is.na(t_i)) next - orbit_vals <- vapply( - orbiting_constructs, - function(cc) ratings_list[[cc]][i], - FUN.VALUE = NA_real_ - ) - if (na.rm) { - orbit_vals <- orbit_vals[!is.na(orbit_vals)] - } - if (length(orbit_vals) == 0) next - D_vec[i] <- t_i - mean(orbit_vals) - } - - # Range of D: [-(anchors-1), +(anchors-1)] - D_mean <- if (na.rm) mean(D_vec, na.rm = TRUE) else mean(D_vec) - htd <- D_mean / (anchors - 1) - - htd_lci <- htd_uci <- NA_real_ - if (ci) { - ci_D <- mean_ci_truncated( - x = D_vec, - lower = -(anchors - 1), - upper = (anchors - 1), - conf.level = conf.level - ) - if (!is.na(ci_D$lwr)) { - htd_lci <- ci_D$lwr / (anchors - 1) - htd_uci <- ci_D$upr / (anchors - 1) - } - } - - crit_row <- data.frame( - item = item, - htc = htc, - htc.lci = htc_lci, - htc.uci = htc_uci, - htd = htd, - htd.lci = htd_lci, - htd.uci = htd_uci, - check.names = FALSE - ) - - item_crit_list[[idx_crit]] <- crit_row - idx_crit <- idx_crit + 1L - - # --- Pairwise.criteria: HTD per orbiting construct --- - for (cc in orbiting_constructs) { - vec_c <- ratings_list[[cc]] - - # Differences per judge for this pair - if (na.rm) { - idx_valid <- !is.na(target_vec) & !is.na(vec_c) - d_c <- target_vec[idx_valid] - vec_c[idx_valid] - } else { - d_c <- target_vec - vec_c - } - - D_c_mean <- if (na.rm) mean(d_c, na.rm = TRUE) else mean(d_c) - htd_c <- D_c_mean / (anchors - 1) - - htd_c_lci <- htd_c_uci <- NA_real_ - if (ci) { - ci_dc <- mean_ci_truncated( - x = d_c, - lower = -(anchors - 1), - upper = (anchors - 1), - conf.level = conf.level - ) - if (!is.na(ci_dc$lwr)) { - htd_c_lci <- ci_dc$lwr / (anchors - 1) - htd_c_uci <- ci_dc$upr / (anchors - 1) - } - } - - pair_row <- data.frame( - item = item, - target = target, - orbiting = cc, - htd = htd_c, - htd.lci = htd_c_lci, - htd.uci = htd_c_uci, - check.names = FALSE - ) - - pairwise_list[[idx_pair]] <- pair_row - idx_pair <- idx_pair + 1L - } - } - - # Bind results - Item.descriptive <- do.call(rbind, item_desc_list) - Item.criteria <- do.call(rbind, item_crit_list) - Pairwise.criteria <- do.call(rbind, pairwise_list) - - # Round numeric columns - round_numeric_df <- function(df, digits) { - is_num <- vapply(df, is.numeric, logical(1)) - df[is_num] <- lapply(df[is_num], round, digits = digits) - df - } - - Item.descriptive <- round_numeric_df(Item.descriptive, nd) - Item.criteria <- round_numeric_df(Item.criteria, nd) - Pairwise.criteria <- round_numeric_df(Pairwise.criteria, nd) - - list( - Item.descriptive = Item.descriptive, - Item.criteria = Item.criteria, - Pairwise.criteria = Pairwise.criteria - ) -} diff --git a/man/ColquittHT.Rd b/man/ColquittHT.Rd new file mode 100644 index 0000000..eee3595 --- /dev/null +++ b/man/ColquittHT.Rd @@ -0,0 +1,127 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/ColquittHT.R +\name{ColquittHT} +\alias{ColquittHT} +\title{Hinkin–Tracey Content Validity Indices with Wilson Confidence Intervals} +\usage{ +ColquittHT( + data, + items, + constructs, + key, + anchors, + ci = FALSE, + conf.level = 0.95, + na.rm = TRUE, + nd = 3 +) +} +\arguments{ +\item{data}{A data frame or matrix in wide format, where each column +corresponds to an item–construct rating. Column names must follow the pattern +\code{"item.construct"} (e.g., \code{"item1.c1"}, \code{"item1.c2"}).} + +\item{items}{Character vector with the base names of the items (e.g., +\code{c("item1", "item2")}).} + +\item{constructs}{Character vector with construct labels (e.g., +\code{c("c1", "c2", "c3")}).} + +\item{key}{Named character vector mapping each item to its target construct. +Names must match \code{items} and values must belong to \code{constructs} +(e.g., \code{c(item1 = "c2", item2 = "c1")}).} + +\item{anchors}{Integer. Number of response scale options (e.g., 5 or 7). Ratings +are assumed to range from 1 to \code{anchors}.} + +\item{ci}{Logical. If \code{TRUE}, asymmetric Wilson score confidence intervals are +computed for \code{htc}, \code{htd}, and pairwise \code{htd}. Default is \code{FALSE}.} + +\item{conf.level}{Confidence level for intervals (e.g., \code{0.90}, \code{0.95}). +Default is \code{0.90}.} + +\item{na.rm}{Logical. If \code{TRUE} (default), missing values are removed pair-wise or +list-wise depending on the sub-index calculation.} + +\item{nd}{Integer. Number of decimal places for rounding output values. Default is \code{3}.} +} +\value{ +A list with three data frames: +\describe{ + \item{\code{Item.descriptive}}{Item names, target constructs, judge counts (\code{nj}), and mean ratings.} + \item{\code{Item.criteria}}{Global \code{htc} and \code{htd} indices with corresponding Wilson CIs.} + \item{\code{Pairwise.criteria}}{Pairwise \code{htd} indices comparing target vs. each orbiting construct.} +} +} +\description{ +Computes Hinkin and Tracey (1999) content validity indices for multiple items +using the standardized operationalizations and empirical benchmarks proposed by +Colquitt et al. (2019). For each item, the function computes: +\itemize{ + \item Descriptive means of ratings across all evaluated constructs. + \item \code{htc}: Hinkin–Tracey correspondence index based on the mean rating of the + target construct. + \item \code{htd}: Hinkin–Tracey distinctiveness index based on the rescaled mean + difference between ratings of the target construct and orbiting constructs. + \item \code{htd} computed pairwise between the target construct and each individual + orbiting construct. +} + +Asymmetric confidence intervals are constructed using Wilson's score interval method +for proportions (Penfield & Miller, 2004), eliminating out-of-bounds limits and +providing realistic coverage even under zero sample variance or small expert sample sizes. +} +\details{ +\strong{Empirical Benchmarks (Colquitt et al., 2019)} + +Based on empirical decile distributions across extensive content validation studies, +Colquitt et al. (2019) suggest the following evaluation criteria: +\itemize{ + \item \strong{Definitional Correspondence (\code{htc}):} + \itemize{ + \item Strong: \eqn{\ge 0.86} + \item Moderate: \eqn{0.78} to \eqn{0.85} + \item Weak: \eqn{\le 0.77} + } + \item \strong{Definitional Distinctiveness (\code{htd}):} + \itemize{ + \item Strong: \eqn{\ge 0.27} + \item Moderate: \eqn{0.21} to \eqn{0.26} + \item Weak: \eqn{\le 0.20} + } +} + +\strong{Rationale and Construction of Confidence Intervals} + +Standard Studentized \emph{t}-intervals often fail in content validity tasks because rating +distributions near scale bounds produce zero sample variance (e.g., perfect agreement across judges), +yielding artificially collapsed zero-width intervals. To overcome this, confidence intervals +for \code{htc} and \code{htd} are computed via Wilson's score interval (Penfield & Miller, 2004) for mean of +rating expert: +\itemize{ + \item \strong{\code{htc} Transformation:} The raw target mean \eqn{\bar{X}} (bounded in \eqn{[1, a]}) + is mapped to a proportion \eqn{p = (\bar{X} - 1) / (a - 1)}. Wilson's score interval is + computed on \eqn{p} and then rescaled back to the \code{htc} metric (\eqn{[1/a, 1.0]}). + \item \strong{\code{htd} Transformation:} The distinctiveness index \eqn{htd} naturally ranges in + \eqn{[-1, 1]}. To apply score-based estimation, it is linearly mapped to a pseudo-proportion + space \eqn{p_{htd} = (htd + 1) / 2 \in [0, 1]}. Wilson limits are derived for \eqn{p_{htd}} + and then back-transformed via \eqn{htd_{\text{limit}} = (p_{\text{wilson}} \times 2) - 1}. +} + +\strong{Methodological Note on Comparisons:} +Confidence intervals should be used to evaluate the precision of individual coefficients or to +compare items within the \emph{same} metric type (e.g., comparing \code{htc} between Item 1 and Item 2). +Comparing an \code{htc} interval directly against an \code{htd} interval is methodologically invalid, +as they capture fundamentally different theoretical constructs and variance structures. +} +\references{ +Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019). Content validation guidelines: +Evaluation criteria for definitional correspondence and definitional distinctiveness. +\emph{Journal of Applied Psychology, 104}(10), 1243–1265. + +Hinkin, T. R., & Tracey, J. B. (1999). An analysis of variance approach to content validation. +\emph{Organizational Research Methods, 2}(2), 175–186. + +Penfield, R. D., & Miller, J. M. (2004). Improving content validation studies using an asymmetric +confidence interval for the mean of expert ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. +} diff --git a/man/HTmult.Rd b/man/HTmult.Rd deleted file mode 100644 index 54a6a6c..0000000 --- a/man/HTmult.Rd +++ /dev/null @@ -1,191 +0,0 @@ -% Generated by roxygen2: do not edit by hand -% Please edit documentation in R/HT.R -\name{HTmult} -\alias{HTmult} -\title{Hinkin–Tracey Content Validity Indices for Multiple Items} -\usage{ -HTmult( - data, - items, - constructs, - key, - anchors, - ci = FALSE, - conf.level = 0.95, - na.rm = TRUE, - nd = 3 -) -} -\arguments{ -\item{data}{A data frame or matrix in wide format, where each column -corresponds to one item–construct combination. Columns are expected -to follow the pattern \code{"item.construct"}, e.g., -\code{"item1.c1"}, \code{"item1.c2"}, \code{"item1.c3"}.} - -\item{items}{Character vector with the base names of the items to be -analyzed (i.e., the prefixes before the dot), e.g. -\code{c("item1", "item2", "item3")}.} - -\item{constructs}{Character vector with the construct labels (suffixes -after the dot), e.g. \code{c("c1", "c2", "c3")}. The function will -search for columns \code{paste0(item, ".", construct)} for every -combination of \code{items} and \code{constructs}.} - -\item{key}{Named character vector mapping each item to its target -construct. The names must match \code{items}, and the values must be -one of the \code{constructs}. For example: -\code{c(item1 = "c2", item2 = "c1", item3 = "c3")}.} - -\item{anchors}{Integer. Number of response options in the rating scale -(e.g., 5 or 7). Ratings are assumed to range from 1 to \code{anchors}.} - -\item{ci}{Logical. If \code{TRUE}, asymmetric confidence intervals are -computed for \code{htc}, \code{htd} and the pairwise \code{htd} indices. -Default is \code{FALSE}.} - -\item{conf.level}{Confidence level for the confidence intervals (e.g., -\code{0.90}, \code{0.95}, \code{0.99}). Used only if \code{ci = TRUE}. -Default is \code{0.95}.} - -\item{na.rm}{Logical. If \code{TRUE} (default), missing values are removed -when computing means and differences. If \code{FALSE}, missing values -will propagate \code{NA} in the corresponding statistics.} - -\item{nd}{Integer. Number of decimal places to use when rounding -the numeric results. Default is \code{3}.} -} -\value{ -A list with three components: -\describe{ - \item{\code{Item.descriptive}}{A data frame with one row per item, - containing item-level descriptive statistics: - \code{item}, \code{target}, \code{nj}, and the construct means - (\code{M.} columns).} - \item{\code{Item.criteria}}{A data frame with one row per item, - containing the global Hinkin–Tracey indices: - \code{item}, \code{htc}, \code{htc.lci}, \code{htc.uci}, - \code{htd}, \code{htd.lci}, \code{htd.uci}. If \code{ci = FALSE}, - the confidence interval columns are filled with \code{NA}.} - \item{\code{Pairwise.criteria}}{A data frame in long format, with one - row per item–orbiting construct combination, containing: - \code{item}, \code{target}, \code{orbiting}, \code{htd}, - \code{htd.lci}, \code{htd.uci}. If \code{ci = FALSE}, the confidence - interval columns are filled with \code{NA}.} -} -} -\description{ -Computes Hinkin and Tracey (1999) content validity indices for multiple items -when judges rate how well each item corresponds to several construct -definitions in the same judgement task. For each item, the function returns: -\itemize{ - \item Descriptive means of ratings for each construct. - \item \code{htc}: correspondence index based on the mean rating of the - target construct. - \item \code{htd}: distinctiveness index based on the mean rating - difference between the target construct and all orbiting constructs. - \item \code{htd} computed pairwise between the target construct and each - orbiting construct. -} - -Ratings are assumed to be given on a discrete Likert-type scale from -1 to \code{anchors}. Confidence intervals for the indices are based on -a t-based interval for the mean, truncated to the possible rating range, -and then rescaled to the \code{htc}/\code{htd} metrics. -} -\details{ -For each item, the function extracts the ratings for each construct from -columns named \code{"item.construct"}. The target construct is specified -via the \code{key} argument. - -\strong{Item-level descriptive statistics} - -For each item, \code{$Item.descriptive} reports: -\itemize{ - \item \code{item}: item name (base). - \item \code{target}: target construct for that item. - \item \code{nj}: number of judges with a non-missing rating on the - target construct. - \item \code{M.}: mean rating for each construct. -} - -\strong{Item-level criteria} - -For each item, \code{$Item.criteria} reports: -\itemize{ - \item \code{htc}: the correspondence index, defined as the mean rating - on the target construct divided by \code{anchors}. - \item \code{htd}: the distinctiveness index, defined as: - \eqn{\text{htd} = \bar{D} / (a - 1)}, where \eqn{\bar{D}} is the - mean (over judges) of the average difference between the rating - on the target construct and the ratings on all orbiting - constructs, and \eqn{a} is \code{anchors}. - \item If \code{ci = TRUE}, lower and upper confidence limits - (\code{htc.lci}, \code{htc.uci}, \code{htd.lci}, \code{htd.uci}) - based on a t-interval for the mean truncated to the possible - rating range and then rescaled. -} - -\strong{Pairwise distinctiveness} - -For each item and each orbiting construct (i.e., each construct other -than the target), \code{$Pairwise.criteria} reports: -\itemize{ - \item \code{item}: item name. - \item \code{target}: target construct for that item. - \item \code{orbiting}: the orbiting construct. - \item \code{htd}: a pairwise distinctiveness index defined as: - \eqn{\text{htd}_{c} = \bar{D}_c / (a - 1)}, where \eqn{\bar{D}_c} - is the mean (over judges) of the difference between the rating on - the target construct and the rating on the orbiting construct - \eqn{c}. - \item If \code{ci = TRUE}, \code{htd.lci} and \code{htd.uci} as - truncated t-based confidence intervals rescaled to the - \code{htd} metric. -} - -This function is intended for the Hinkin and Tracey (1999) method of -content validation, where judges rate how well each item corresponds -to each construct definition. The same wide format can also be useful -for implementing criteria inspired by Colquitt et al. (2019). - -Ratings are assumed to be coded from 1 to \code{anchors}. If another -coding scheme is used, users should recode the ratings before calling -the function. -} -\examples{ -\dontrun{ -## Example structure (toy data) -set.seed(123) -HTmult.data <- data.frame( - item1.c1 = sample(1:5, 50, replace = TRUE), - item1.c2 = sample(1:5, 50, replace = TRUE), - item1.c3 = sample(1:5, 50, replace = TRUE), - item2.c1 = sample(1:5, 50, replace = TRUE), - item2.c2 = sample(1:5, 50, replace = TRUE), - item2.c3 = sample(1:5, 50, replace = TRUE) -) - -HTmult( - data = dat, - items = c("item1", "item2"), - constructs = c("c1", "c2", "c3"), - key = c(item1 = "c2", item2 = "c3"), - anchors = 5, - ci = TRUE, - conf.level = 0.95) -} - -} -\references{ -Hinkin, T. R., & Tracey, J. B. (1999). An analysis of variance approach -to content validation. \emph{Organizational Research Methods, 2}(2), 175–186. - -Colquitt, J. A., Sabey, T. B., Rodell, J. B., & Hill, E. T. (2019). -Content validation guidelines: Evaluation criteria for definitional -correspondence and definitional distinctiveness. \emph{Journal of Applied -Psychology, 104}(10), 1243–1265. - -Penfield, R. D., & Miller, J. M. (2004). Improving content validation -studies using an asymmetric confidence interval for the mean of expert -ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. -} From 433b6d0f1066f2f3df82e3b30171f73a39ea59c5 Mon Sep 17 00:00:00 2001 From: Cesar Merino-Soto Date: Thu, 3 Sep 2026 11:41:45 -0600 Subject: [PATCH 8/9] Update ColquittHT.R --- R/ColquittHT.R | 112 ++++++++++++++++++++++++++++++------------------- 1 file changed, 68 insertions(+), 44 deletions(-) diff --git a/R/ColquittHT.R b/R/ColquittHT.R index cd1309b..82922ac 100644 --- a/R/ColquittHT.R +++ b/R/ColquittHT.R @@ -1,4 +1,4 @@ -#' Hinkin–Tracey Content Validity Indices with Wilson Confidence Intervals +#' Hinkin–Tracey Content Validity Indices with Confidence Intervals #' #' @description #' Computes Hinkin and Tracey (1999) content validity indices for multiple items @@ -15,8 +15,9 @@ #' } #' #' Asymmetric confidence intervals are constructed using Wilson's score interval method -#' for proportions (Penfield & Miller, 2004), eliminating out-of-bounds limits and -#' providing realistic coverage even under zero sample variance or small expert sample sizes. +#' for proportions for \code{htc} (Penfield & Miller, 2004), and Willink's (2005) asymmetric +#' interval incorporating the third moment and Cornish-Fisher expansion for \code{htd}, +#' eliminating out-of-bounds limits and providing realistic coverage even under small expert sample sizes. #' #' @param data A data frame or matrix in wide format, where each column #' corresponds to an item–construct rating. Column names must follow the pattern @@ -30,10 +31,10 @@ #' (e.g., \code{c(item1 = "c2", item2 = "c1")}). #' @param anchors Integer. Number of response scale options (e.g., 5 or 7). Ratings #' are assumed to range from 1 to \code{anchors}. -#' @param ci Logical. If \code{TRUE}, asymmetric Wilson score confidence intervals are -#' computed for \code{htc}, \code{htd}, and pairwise \code{htd}. Default is \code{FALSE}. +#' @param ci Logical. If \code{TRUE}, asymmetric confidence intervals are +#' computed for \code{htc} (Wilson score) and \code{htd} (Willink Cornish-Fisher). Default is \code{FALSE}. #' @param conf.level Confidence level for intervals (e.g., \code{0.90}, \code{0.95}). -#' Default is \code{0.90}. +#' Default is \code{0.95}. #' @param na.rm Logical. If \code{TRUE} (default), missing values are removed pair-wise or #' list-wise depending on the sub-index calculation. #' @param nd Integer. Number of decimal places for rounding output values. Default is \code{3}. @@ -61,18 +62,13 @@ #' \strong{Rationale and Construction of Confidence Intervals} #' #' Standard Studentized \emph{t}-intervals often fail in content validity tasks because rating -#' distributions near scale bounds produce zero sample variance (e.g., perfect agreement across judges), -#' yielding artificially collapsed zero-width intervals. To overcome this, confidence intervals -#' for \code{htc} and \code{htd} are computed via Wilson's score interval (Penfield & Miller, 2004) for mean of -#' rating expert: +#' distributions near scale bounds produce zero sample variance or skewness. To overcome this: #' \itemize{ -#' \item \strong{\code{htc} Transformation:} The raw target mean \eqn{\bar{X}} (bounded in \eqn{[1, a]}) -#' is mapped to a proportion \eqn{p = (\bar{X} - 1) / (a - 1)}. Wilson's score interval is -#' computed on \eqn{p} and then rescaled back to the \code{htc} metric (\eqn{[1/a, 1.0]}). -#' \item \strong{\code{htd} Transformation:} The distinctiveness index \eqn{htd} naturally ranges in -#' \eqn{[-1, 1]}. To apply score-based estimation, it is linearly mapped to a pseudo-proportion -#' space \eqn{p_{htd} = (htd + 1) / 2 \in [0, 1]}. Wilson limits are derived for \eqn{p_{htd}} -#' and then back-transformed via \eqn{htd_{\text{limit}} = (p_{\text{wilson}} \times 2) - 1}. +#' \item \strong{\code{htc} Transformation:} Computed via Wilson's score interval (Penfield & Miller, 2004) +#' after mapping the raw target mean to proportion space \eqn{p \in [0, 1]}. +#' \item \strong{\code{htd} Transformation:} Computed via Willink's (2005) asymmetric interval method, +#' which corrects for sample skewness using the third central moment and Cornish-Fisher expansion, +#' with post-hoc truncation to the natural domain \eqn{[-1, 1]}. #' } #' #' \strong{Methodological Note on Comparisons:} @@ -84,7 +80,7 @@ #' @return A list with three data frames: #' \describe{ #' \item{\code{Item.descriptive}}{Item names, target constructs, judge counts (\code{nj}), and mean ratings.} -#' \item{\code{Item.criteria}}{Global \code{htc} and \code{htd} indices with corresponding Wilson CIs.} +#' \item{\code{Item.criteria}}{Global \code{htc} and \code{htd} indices with corresponding confidence intervals.} #' \item{\code{Pairwise.criteria}}{Pairwise \code{htd} indices comparing target vs. each orbiting construct.} #' } #' @@ -99,6 +95,9 @@ #' Penfield, R. D., & Miller, J. M. (2004). Improving content validation studies using an asymmetric #' confidence interval for the mean of expert ratings. \emph{Applied Measurement in Education, 17}(4), 359–370. #' +#' Willink, R. (2005). A confidence interval for a mean with a correction for skewness. +#' \emph{Metrika}, 61(2), 159–173. +#' #' @export ColquittHT <- function( data, @@ -152,13 +151,11 @@ ColquittHT <- function( stop("'conf.level' must be a numeric value between 0 and 1.") } - # --- Internal Helper: Wilson Score Confidence Interval for Proportions --- - # Computes asymmetric Wilson CI for a proportion p with sample size n. + # --- Internal Helper: Wilson Score Confidence Interval for Proportions (HTC) --- wilson_ci <- function(p, n, conf.level) { if (is.na(p) || is.na(n) || n <= 0) { return(list(lwr = NA_real_, upr = NA_real_)) } - # Bound input proportion to [0, 1] to ensure mathematical stability p <- max(0, min(1, p)) alpha <- 1 - conf.level @@ -175,6 +172,48 @@ ColquittHT <- function( list(lwr = lwr, upr = upr) } + # --- Internal Helper: Willink Asymmetric Confidence Interval for HTD (2005) --- + willink_ci_htd <- function(x, conf.level) { + x <- x[!is.na(x)] + n <- length(x) + if (n < 3) { + return(list(lwr = NA_real_, upr = NA_real_)) + } + + x_bar <- mean(x) + s <- stats::sd(x) + alpha <- (1 - conf.level) / 2 + + if (s == 0) { + return(list(lwr = x_bar, upr = x_bar)) + } + + # Unbiased third central moment + mu3_hat <- (n / ((n - 1) * (n - 2))) * sum((x - x_bar)^3) + + # Scaled skewness coefficient (Willink, 2005) + a <- (mu3_hat / (s^3)) / (6 * sqrt(n)) + + t_low <- stats::qt(alpha, df = n - 1) + t_high <- stats::qt(1 - alpha, df = n - 1) + + G <- function(r) { + if (abs(a) < 1e-6) return(r) + inner <- 1 + 6 * a * (r - a) + if (inner < 0) inner <- 0 + (1 / (2 * a)) * (inner^(1/3) - 1) + } + + lwr <- x_bar - G(t_high) * (s / sqrt(n)) + upr <- x_bar - G(t_low) * (s / sqrt(n)) + + # Truncate to natural bounds of htd [-1, 1] + lwr <- max(-1, lwr) + upr <- min(1, upr) + + list(lwr = lwr, upr = upr) + } + item_desc_list <- list() item_crit_list <- list() pairwise_list <- list() @@ -234,11 +273,9 @@ ColquittHT <- function( htc_lci <- htc_uci <- NA_real_ if (ci && nj > 0) { - # Map mean target rating from [1, anchors] to proportion p_target in [0, 1] p_target <- (mean_target - 1) / (anchors - 1) ci_target <- wilson_ci(p = p_target, n = nj, conf.level = conf.level) - # Convert Wilson bounds back from proportion space to target mean, then divide by anchors mean_lwr <- ci_target$lwr * (anchors - 1) + 1 mean_upr <- ci_target$upr * (anchors - 1) + 1 htc_lci <- mean_lwr / anchors @@ -269,17 +306,10 @@ ColquittHT <- function( htd_lci <- htd_uci <- NA_real_ if (ci) { - # Effective number of judges evaluating difference scores - n_D <- if (na.rm) sum(!is.na(D_vec)) else length(D_vec) - if (n_D > 0) { - # Rescale htd from [-1, 1] to pseudo-proportion p_htd in [0, 1] - p_htd <- (htd + 1) / 2 - ci_htd <- wilson_ci(p = p_htd, n = n_D, conf.level = conf.level) - - # Back-transform Wilson bounds to original htd metric [-1, 1] - htd_lci <- (ci_htd$lwr * 2) - 1 - htd_uci <- (ci_htd$upr * 2) - 1 - } + judge_htd <- D_vec / (anchors - 1) + ci_htd <- willink_ci_htd(judge_htd, conf.level = conf.level) + htd_lci <- ci_htd$lwr + htd_uci <- ci_htd$upr } crit_row <- data.frame( @@ -312,16 +342,10 @@ ColquittHT <- function( htd_c_lci <- htd_c_uci <- NA_real_ if (ci) { - n_dc <- length(d_c) - if (n_dc > 0) { - # Rescale pairwise htd from [-1, 1] to pseudo-proportion space [0, 1] - p_htd_c <- (htd_c + 1) / 2 - ci_dc <- wilson_ci(p = p_htd_c, n = n_dc, conf.level = conf.level) - - # Back-transform limits to original htd metric [-1, 1] - htd_c_lci <- (ci_dc$lwr * 2) - 1 - htd_c_uci <- (ci_dc$upr * 2) - 1 - } + judge_htd_c <- d_c / (anchors - 1) + ci_dc <- willink_ci_htd(judge_htd_c, conf.level = conf.level) + htd_c_lci <- ci_dc$lwr + htd_c_uci <- ci_dc$upr } pair_row <- data.frame( From 38239331375bb3e13e3b8249103f37c0b5667c65 Mon Sep 17 00:00:00 2001 From: "diego.livia" Date: Sat, 12 Sep 2026 18:05:33 -0500 Subject: [PATCH 9/9] Update NEWS.md for version 0.2.0: add new functions, enhance content validity calculations, and improve documentation --- NEWS.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/NEWS.md b/NEWS.md index 8709505..e924afb 100644 --- a/NEWS.md +++ b/NEWS.md @@ -6,6 +6,8 @@ - `MDScontent()` for multidimensional scaling maps of item-trait correspondence. - `LuAgree()` for estimating Lu's agreement coefficient. +- `ColquittHT()` for Colquitt's content validity approach. +- `Haiken()` for Aiken's coefficient of homogeneity with bootstrap confidence intervals. - `HTmult()` for Hinkin-Tracey indices. - `minimumCV()` for Wilson-based minimum sample-size or critical-value calculations. - Additional public and supporting functions for content validity analyses, including `CVIpub()` and `Vaikenpub()`. @@ -14,15 +16,17 @@ ### Changed - Added explicit validation for missing data across the package functions. -- Refactored MER confidence-interval calculations and improved code consistency. +- Refactored confidence-interval calculations for MER, Aiken's V, CVI, CVR, and related coefficients. +- Expanded and improved content validity calculations across CVC, CIR, CVI, CVIR, CVR, and SVAL functions. - Improved plotting support using `ggplot2` and added MDS plotting functionality. - Expanded and regenerated package documentation. -- Updated package metadata, namespace exports, dependencies, and repository documentation. +- Updated package metadata, namespace exports, dependencies, README, and repository documentation. ### Fixed - Corrected documentation, namespace, and package-structure issues identified during CRAN-style validation. - Improved handling of package examples and test execution. +- Added package build exclusions and validation checks for clean source-package archives. ## ValCont 0.1.0