DALIA

SubDataThe SubData tool, implemented as a Python library, evaluates the alignment between large language models (LLMs) and human perspectives in subjective annotation tasks. It is particularly relevant during data preprocessing, where it helps to improve data quality by harmonizing heterogeneous datasets, providing standardized keyword mappings and taxonomies, and enabling theory-driven analyses of perspective alignment. In addition, SubData includes ten curated datasets that can be used to identify model biases, test generalizability, and contextualize hate speech datasets within the broader literature. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Code Notebook, Tutorial

File Format

QMD

Languages

English

Media Type

Code, Text

Publication Date

2026-01-08

License

Authors

Siyu Zhang | Leon Fröhling

Keywords

ValiTextThe ValiText provides a framework and practical guidance for validating text-based measures of social constructs. It mitigates measurement errors by outlining key validation evidence, defining tailored validation steps, and offering a checklist for transparent documentation. By guiding researchers through a systematic process, ValiText helps ensure that text-derived indicators accurately capture the intended constructs. As an example, it has been applied to the “Call me sexist, but…” dataset by Samory et al. (2021) to demonstrate practical validation and documentation. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Case Study, Tutorial

File Format

QMD

Languages

English

Media Type

Text

Publication Date

2024-11-22

License

Authors

Lukas Birkenmaier

Keywords

TextPrepThe TextPrep tool, implemented in R, provides text preprocessing and comparative strategies to improve the quality of social media data. It supports common techniques, such as automated translation, minor text operations, and stopword filtering, while allowing systematic comparisons of alternative approaches. The tool helps to assess how different procedures may affect analytical outcomes and provides metrics to quantify differences, helping researchers evaluate choices transparently. As an illustrative use case, a synthetic dataset of social media posts about the 2024 Summer Olympics is processed using different configurations to compare their impact on the resulting text. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Code Notebook, Tutorial

File Format

QMD

Languages

English

Media Type

Code, Text

Publication Date

2024-12-18

License

Authors

Yannik Peters | Kunjan Shah

Keywords

TES-DThe Total Errors Sheet for Datasets (TES-D) is a template-based approach for documenting datasets from online sources such as social media or Wikipedia, focusing on potential errors throughout the research process. It provides a structured catalogue of questions to guide researchers in reflecting critically on data collection and potential measurement or representation errors. Completed TES-D documentation is intended to accompany the dataset, enhancing transparency, reproducibility, and responsible reuse, as illustrated by the “Call me sexist, but…” dataset by Samory et al. (2021), where structured documentation enhances interpretability and reuse. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Case Study, Tutorial

File Format

QMD

Languages

English

Media Type

Text

Publication Date

2025-02-24

License

Authors

Leon Fröhling

Keywords

Delab TreesThe Delab Trees tool is a Python library for analyzing network data, with a particular focus on social media conversation structures. Within the TED-On framework, it is especially relevant at the construct definition stage, where it helps researchers assess the validity of individual nodes in conversation trees. It can also be used during data preprocessing and data analysis, for example to handle large-scale conversation trees, account for deleted posts, study interaction dynamics between authors, and support discussion mining. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Code Notebook, Tutorial

File Format

IPYNB

Languages

English

Media Type

Code, Text

Publication Date

2024-12-17

License

Authors

Julian Dehne

Keywords

SQPThe Survey Quality Predictor (SQP) helps researchers evaluate the expected measurement quality of survey questions before data collection even begins. By coding formal and linguistic features, such as wording, response scale, and mode of administration, SQP predicts key quality components like reliability, validity, and overall measurement quality. By doing so, SQP helps researchers design survey questions that are both clearer and more reliable to address measurement error prior to collecting data. The SQP tool demonstrates this with an applied example that compares two versions of the same question on satisfaction with the country’s economy using different response scales, showing how design choices lead to different predicted quality. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Case Study, Tutorial

File Format

QMD

Languages

English

Media Type

Text

Publication Date

2025-02-11

License

Authors

Lydia Repke | Lukas Birkenmaier | Cem Eryar

Keywords

#survey data

ScaleArchScaleArch is a tool guiding through the basic analyses needed to evaluate and document the validity of a scale as a research instrument. Using an example scale with continuous indicators, it provides hands-on guidance for computing descriptive statistics, exploring the dimensionality of items, fitting confirmatory factor models, and estimating reliability coefficients. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Code Notebook, Tutorial

File Format

QMD

Languages

English

Media Type

Code, Text

Publication Date

2025-01-17

License

Authors

Piotr Koc | Julian Urban | David Grüning | Zoe Greer

Keywords

#survey data

ResquinResquin is a tool for assessing response quality and careless responding in multi-item scales. This tool gives an overview on the assessment of response quality in multi-item scales and guides you through the quality analysis with replicable R-code. It introduces key response quality indicators along with code examples for their calculation, shows how to interpret the different response quality indicators, and assesses their suitability for different question types and response scales. The tool also briefly evaluates different caveats of selected response quality indicators. Further information: This tool was developed as part of the KODAQS project, a partnership between GESIS, the University of Mannheim, and LMU Munich.

Learning Resource Type

Assessment, Code Notebook, Tutorial

File Format

QMD

Languages

English

Media Type

Code, Text

Publication Date

2026-03-04

License

Authors

Fabienne Kraemer | Arjin Eser | Çağla Yıldız | Matthias Roth

Keywords

#survey data