en.osm.town is one of the many independent Mastodon servers you can use to participate in the fediverse.
An independent, community of OpenStreetMap people on the Fediverse/Mastodon. Funding graciously provided by the OpenStreetMap Foundation.

Server stats:

248
active users

#testscores

0 posts0 participants0 posts today
Replied in thread

@TomJewell

I argue that there is some understanding in the #psychometrics and professional communities that use #TestScores that evidence of quality is tentative as any statistical result.

And in my personal view, the different methodologies #COSMIN has developed (e.g., link.springer.com/article/10.1) or is currently pushing forward (doi.org/10.1186/s13643-022-019) are tools to aggregate available evidence exactly for this reason: each individual study and estimate offers an incomplete picture.

SpringerLinkCOSMIN guideline for systematic reviews of patient-reported outcome measures - Quality of Life ResearchPurpose Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs. Methods Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs. Results A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1–4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5–8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review. Conclusions The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.
Replied in thread

@TomJewell

To which degree) the term "validity" includes all quality criteria and qualities one can consider to describe the use of #TestScores is a matter of heated discussion in fields related to the philosophy of measurement. But it is probably safe to assume this for the stuff covered in the "Standards".

For a look into the discussion I recommend Kathleen Slaney's book on Construct Validity (link.springer.com/book/10.1057) and several of her papers deal with this topic.

SpringerLinkValidating Psychological Constructs
Replied in thread

@TomJewell

3) _It's about #TestScores_

Not always discussed explicitly in relation to all quality indicators and most frequently in the relation to "#validity", the process of validation produces evidence for the validity of #TestScores (as opposed to the validity of the test!).

The "Standards" make that very clear for the term "Validity" [2014, p. 11]:
"...the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests"

Replied in thread

@TomJewell

The introduction to Chapter 2 (reliability/precision) acknowledges that the sampling process can lead to significant differences in the determined coefficients,
that they therefore [2014, p. 37] "should be estimated separately for all relevant subgroups"
and that this is also important to assess potential consequences for #TestFairness.

And in Chapter 5 focuses especially in the discussion of #Norming of #TestScores on such aspects, including relating to our previous discussion...