Search NASA⌕ Search

SEARCH · Search NASA

Results for “Psychometrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Psychometric property study of the Affective Lability Scale-short form in Chinese patients with mood disorders

Introduction This study aimed to investigate the psychometric properties of the Affective Lability Scale-short form (ALS-SF) among Chinese patients with mood disorders, and to compare ALS-SF subscale scores between patients with major depressive disorder (MDD) and patients with bipolar disorder (BD) depression. Methods A total of 344 patients with mood disorders were included in our study. Participants were measured through a set of questionnaires including the Chinese version of ALS-SF, Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder 7-item (GAD-7), and NEO-Five Factor Inventory (NEO-FFI). Exploratory factor analysis and confirmatory factor analysis were applied to examine the psychometric properties of ALS-SF. Besides, correlation and regression analyses were performed to explore the relationship between affective lability and depression, anxiety, and neuroticism. Independent samples t -tests were used to compare the subscale scores of ALS-SF between the MDD and BD depression groups. Results Results of factor analysis indicated that the model of ALS-SF was consistent with ALS-SF. The ALS-SF showed a solid validity and high internal consistency (Cronbach’s alpha = 0.861). In addition, each subscale of ALS-SF was significantly correlated with PHQ-9, GAD-7, and NEO-FFI neuroticism subscale, except for the anger subscale showed no significant correlation with PHQ-9. Besides, the depression/elation and anger factor scores in patients with BD depression were higher than in patients with MDD. Conclusion Our study suggests that the Chinese version of ALS-SF has good reliability and validity for measuring affective lability in Chinese patients with mood disorders. Assessing affective lability would assist clinicians to distinguish between MDD and BP depression and may decrease the risks of misdiagnosis.

Ma, Mohan↗
NASA NTRS1983

QUEST - A Bayesian adaptive psychometric method

An adaptive psychometric procedure that places each trial at the current most probable Bayesian estimate of threshold is described. The procedure takes advantage of the common finding that the human psychometric function is invariant in form when expressed as a function of log intensity. The procedure is simple, fast, and efficient, and may be easily implemented on any computer.

Watson, A. B.↗
NASA NTRS1989

Teleoperator comfort and psychometric stability: Criteria for limiting master-controller forces of operation and feedback during telemanipulation

The following question is addressed: How much force should operators exert, or experience, when operating a telemanipulator master-controller for sustained periods without encountering significant fatigue and discomfort, and without loss of stability in psychometric perception of force. The need to minimize exertion demands to avoid fatigue is diametrically opposed by the need to present a wide range of force stimuli to enhance perception of applied or reflected forces. For 104 minutes subjects repetitiously performed a series of 15 s isometric pinch grasps; controlled at 5, 15, and 25 percent of their maximum voluntary strength. Cyclic pinch grasps were separated by rest intervals of 7.5 and 15 s. Upon completion of every 10 minute period, subjects interrupted grasping activities to gage the intensity of fatigue and discomfort in the hand and forearm using a cross-modal matching technique. A series of psychometric tests were then conducted to determine accuracy and stability in the subject's perception of force experienced. Results showed that onset of sensations of discomfort and fatigue were dependent upon the magnitude of grasp force, work/rest ratio, and progression of task. Declines in force magnitude estimation slopes, indicating a reduction in force perception sensitivity, occurred with increased grasp force when work/rest ratios were greater than 1.0. Specific recommendations for avoiding discomfort and shifts in force perception, by limiting pinch grasp force required for master-controller operation and range of force reflection or work/rest ratios, are provided.

Wiker, Steven F.↗
DOE OSTI2018

Assessing the Effectiveness and Interactions of Instructional Coaches: Initial Psychometric Evidence for the Instructional Coaching Assessments–Teacher Forms

The conceptual foundation and initial psychometric evidence are provided for the Instructional Coaching Rating Scales and Interaction Style Scales–Teacher Forms. These forms are part of a multicomponent online assessment system designed to evaluate the effectiveness of coaching skills and interactions that support the needs of teachers and students. Specifically, the article presents the theory, evidence, and measurement framework for the system. Findings indicate that the Rating Scales and Interaction Style Scales–Teacher Forms have very good internal structures based on multiple fit statistics for confirmatory factor analyses, high internal consistency, good item-to-scale total correlations, and freedom from item bias. Collectively, this promising statistical evidence is supportive of valid score inferences. Study limitations and directions for research are discussed.

Reddy, Linda A.↗
DOE OSTI2020

Measuring Intelligence with the Sandia Matrices: Psychometric Review and Recommendations for Free Raven-Like Item Sets

The Sandia Matrices are a free alternative to the Raven’s Progressive Matrices (RPMs). This study offers a psychometric review of Sandia Matrices items focused on two of the most commonly investigated issues regarding the RPMs: (a) dimensionality and (b) sex differences. Model-data fit of three alternative factor structures are compared using confirmatory multidimensional item response theory (IRT) analyses, and measurement equivalence analyses are conducted to evaluate potential sex bias. Although results are somewhat inconclusive regarding factor structure, results do not show evidence of bias or mean differences by sex. Finally, although the Sandia Matrices software can generate infinite items, editing and validating items may be infeasible for many researchers. Further, to aide implementation of the Sandia Matrices, we provide scoring materials for two brief static tests and a computer adaptive test. Implications and suggestions for future research using the Sandia Matrices are discussed.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

Exploring the Psychometrics of Common Post-Scenario Human Factors Questionnaires of Workload, Situation Awareness, and Perceived Difficulty

Abstract: The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program has the primary mission of sustaining operation of the existing commercial nuclear power plants (NPPs) through a multi-pathway approach in conducting research and development (R&D). In this effort, the role of human factors engineering (HFE) has been an integral in ensuring new digital upgrades that are part of the overall main control room modernization strategy for various U.S. NPPs optimize human-system performance and do not introduce new human failure modes. In this process, testing and evaluation of concept (i.e., prototype) human machine interfaces (HMIs) are done in operator-in-the-loop studies at various stages of the HFE modification process, such as during design and verification and validation (V&V) [1]. During these studies, licensed operators go through select scenarios using a testbed, such as a full-scale simulator or at a NPP’s training simulator, to identify potential usability issues and provide general design feedback regarding the new digital HMIs. Human-system performance is measured and evaluated through a range of objective and self-report measures that capture aspects of plant performance, task performance, workload, situation awareness, and anthropometric and ergonomic concerns [2]. The use of post-scenario questionnaires is an acceptable method for gathering measures of workload and situation awareness. To this end, the NASA-Task Load Index (TLX) [e.g., 3, 4, & 5] and Situation Awareness Rating Technique (SART) [e.g., 6 & 7] are common standardized questionnaires used for evaluating workload and situation awareness, respectively. When combined, these questionnaires consist of a total of 15 questions, each using a different rating scale. Debriefs are also done through interviews with operators to gather design insight after each scenario. Since operator- and training simulator-time is limited, a concern during this post-scenario period of operator-in-the-loop studies is the limited time available to run scenarios, administer post-scenario questionnaires, and debrief within the allotted simulator time. To address this potential problem, this paper explores the psychometric characteristics of NASA-TLX, SART, and a common one-question post-scenario usability questionnaire (i.e., the single ease question – SEQ [8]) in effort to inform future development of a condensed questionnaire that captures aspects of workload and situation awareness. This exploratory work uses data from four different operator-in-the-loop studies completed under prior LWRS operator-in-the-loop studies. Findings from this paper should support future development of a consolidated questionnaire that may be used in operator-in-the-loop studies, when time is compressed, that conforms to the HFE scientific and regulatory requirements set for NPP control room modernization.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS2000

Logistical Consideration in Computer-Based Screening of Astronaut Applicants

This presentation reviews the logistical, ergonomic, and psychometric issues and data related to the development and operational use of a computer-based system for the psychological screening of astronaut applicants. The Behavioral Health and Performance Group (BHPG) at the Johnson Space Center upgraded its astronaut psychological screening and selection procedures for the 1999 astronaut applicants and subsequent astronaut selection cycles. The questionnaires, tests, and inventories were upgraded from a paper-and-pencil system to a computer-based system. Members of the BHPG and a computer programmer designed and developed needed interfaces (screens, buttons, etc.) and programs for the astronaut psychological assessment system. This intranet-based system included the user-friendly computer-based administration of tests, test scoring, generation of reports, the integration of test administration and test output to a single system, and a complete database for past, present, and future selection data. Upon completion of the system development phase, four beta and usability tests were conducted with the newly developed system. The first three tests included 1 to 3 participants each. The final system test was conducted with 23 participants tested simultaneously. Usability and ergonomic data were collected from the system (beta) test participants and from 1999 astronaut applicants who volunteered the information in exchange for anonymity. Beta and usability test data were analyzed to examine operational, ergonomic, programming, test administration and scoring issues related to computer-based testing. Results showed a preference for computer-based testing over paper-and -pencil procedures. The data also reflected specific ergonomic, usability, psychometric, and logistical concerns that should be taken into account in future selection cycles. Conclusion. Psychological, psychometric, human and logistical factors must be examined and considered carefully when developing and using a computer-based system for psychological screening and selection.

Galarza, Laura↗
DOE OSTI2018

Personnel Certifications for the Design, Installation and Maintenance of Photovoltaic Systems. Final Technical Report

The North American Board of Certified Energy Practitioners (NABCEP) is the most respected, well-established, and widely recognized national certification organization for professionals in the field of renewable energy. NABCEP’s certifications were created to facilitate consumer protection, improve quality assurance, and help employers judge the skills of existing or potential employees. The certifications also allow individuals to differentiate themselves from others in their field and advance their careers while fostering increased recognition of the profession. Since NABCEP’s founding in 2002, all of its certification exams have been administered in paper-and-pencil format twice per year at 30 exam sites around the United States. The biannual test administration and relatively low number of exam sites make it challenging for many qualified renewable energy professionals to take NABCEP’s certification exams. The rigorous nature of the exam, which leads to many candidates failing the exam on their first and sometimes second attempts, combined with the low number of annual test administrations means that anyone who fails will need to wait another six months for the chance to retake the exam. These factors limited the potential growth of NABCEP’s certification programs and the resultant quality assurance within the renewable energy industry. Thanks to a generous grant by the Department of Energy’s Solar Training and Education for Professional (STEP) program, NABCEP recently began updating its Photovoltaic (PV) Installation Professional Exam and developing three new specialty certifications in PV design, PV installation, and PV commissioning & maintenance. One of the primary objectives of this STEP-funded project was to develop the exams using a computer-based testing (CBT) model to increase the number of exam administrations and available exam sites for NABCEP’s certification exams. To successfully complete this project, NABCEP partnered with Castle Worldwide, Inc. With more than 25 years of experience in test development, test administration, analysis of examinations, and has a staff of 70 experienced personnel, NABCEP was confident that Castle’s team was more than qualified to help NABCEP make the transition to CBT examinations. NABCEP and Castle Worldwide jointly developed a project management plan that included key deliverables, scope management, a work breakdown structure, roles and responsibilities, and a project schedule with milestones. The project schedule was carefully monitored to ensure that activities along the critical path were not delayed and, when deemed necessary, the project was crashed to add resources so that activities could occur in parallel. NABCEP and Castle Worldwide staff met weekly to discuss any potential schedule variances and activities which might delay the closing of the project. NABCEP also recruited 46 subject matter experts (SMEs) to work with Castle’s staff on this project. NABCEP staff, SME’s, and Castle staff worked together to develop the Job Task Analyses that form the basis for the certification exams, create new examination forms, conduct passing score studies, and complete a psychometrical analysis of potential exam items. NABCEP’s SMEs utilized Castle’s item development platform during face-to-face meetings, webinars, and remote item authoring to write, review, and approve exam items. Castle also used their software to psychometrically analyze existing exam items to confirm the reliability and validity of the NABCEP’s certification exams. On January 3rd, 2018, NABCEP’s began offering its PV Installation Professional certification exam via CBT. The exam was available at Castle’s nearly 600 exam sites in the United States. A large number of sites will help to ensure better access for solar professionals in rural and remote areas, which drives down the implicit costs of taking the certification exams. Thanks to the STEP grant and the robust partnership between NABCEP and Castle Worldwide, Inc., personnel certifications will be much more accessible and affordable for renewable energy professionals.

14 SOLAR ENERGY↗
DOE OSTI2022

How do perceptions of safety and car ownership importance affect autonomous vehicle adoption?

In this study, we explicitly modeled how individuals' perceptions of automated vehicle (AV) safety and the importance they place on car ownership affect mode choices involving conventional and automated vehicles in the context of privately owned cars and ridehailing services. We adopted psychometric questions to capture these two latent variables and designed a stated preference survey based on the participants' actual travel patterns. Then, we quantified the impact of these latent variables on mode choices using an integrated choice and latent variable (ICLV) model. We found that both latent variables have a statistically significant effect on mode choices. The results show that car ownership importance has the most potent effect on privately owned cars (conventional car and self-driving car), followed by driverless ridehailing and conventional ridehailing. We also found that changes in safety perception are equivalent to sizable changes in price. In addition, we further investigated the impact of improvements in safety perception through four scenarios. The scenario testing results show that as the distribution of perceived safety is compressed toward positive safety perception, the market share of AVs spikes and dominates regular cars. Our results demonstrate that based on our respondents' current understanding of AVs, even if AV prices were comparable to regular cars, we cannot expect widespread use of AVs. However, improvements in AVs' safety and, consequently, consumer safety perception can considerably expand AVs' market share, and may offset the high cost of using the technology.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Automating ridehailing services would reduce pooling, especially among women

Here, this study investigates how autonomous vehicles (AVs) could transform pooled (shared) ridehailing services, focusing on the impacts of fare reductions, the absence of drivers/staff, and psychological attributes such as trust in other passengers and privacy concerns. We distinguish between the automation of driving tasks and the removal of human driver/staff from the vehicle, providing novel insights into the factors influencing AV ridehailing adoption. Using a national survey with stated preference (SP) choice experiments and psychometric questions, we analyze the complex interactions of ridehailing fare, pooled ridehailing service quality, and latent attitudes on ridehailing choices. Our findings suggest that the elimination of drivers/staff from fully autonomous ridehailing could lead to a shift from pooled to solo rides, particularly among female travelers who may have greater concerns about trust and safety in unstaffed AVs. This study highlights the importance of addressing trust and comfort beyond fare discounts to ensure the inclusivity and widespread adoption of pooled AV ridehailing. These insights underscore the need for ridehailing providers and policymakers to prioritize trust-building measures, user-centered AV design that offers greater privacy, and dynamic pricing strategies, to ensure inclusive and widespread adoption of pooled AV services.

Autonomous vehicle↗
DOE OSTI2022

Disentangling Alzheimer’s disease neurodegeneration from typical brain ageing using machine learning

Abstract Neuroimaging biomarkers that distinguish between changes due to typical brain ageing and Alzheimer’s disease are valuable for determining how much each contributes to cognitive decline. Supervised machine learning models can derive multivariate patterns of brain change related to the two processes, including the Spatial Patterns of Atrophy for Recognition of Alzheimer’s Disease (SPARE-AD) and of Brain Aging (SPARE-BA) scores investigated herein. However, the substantial overlap between brain regions affected in the two processes confounds measuring them independently. We present a methodology, and associated results, towards disentangling the two. T1-weighted MRI scans of 4054 participants (48–95 years) with Alzheimer’s disease, mild cognitive impairment (MCI), or cognitively normal (CN) diagnoses from the Imaging-based coordinate SysTem for AGIng and NeurodeGenerative diseases (iSTAGING) consortium were analysed. Multiple sets of SPARE scores were investigated, in order to probe imaging signatures of certain clinically or molecularly defined sub-cohorts. First, a subset of clinical Alzheimer’s disease patients (n = 718) and age- and sex-matched CN adults (n = 718) were selected based purely on clinical diagnoses to train SPARE-BA1 (regression of age using CN individuals) and SPARE-AD1 (classification of CN versus Alzheimer’s disease) models. Second, analogous groups were selected based on clinical and molecular markers to train SPARE-BA2 and SPARE-AD2 models: amyloid-positive Alzheimer’s disease continuum group (n = 718; consisting of amyloid-positive Alzheimer’s disease, amyloid-positive MCI, amyloid- and tau-positive CN individuals) and amyloid-negative CN group (n = 718). Finally, the combined group of the Alzheimer’s disease continuum and amyloid-negative CN individuals was used to train SPARE-BA3 model, with the intention to estimate brain age regardless of Alzheimer’s disease-related brain changes. The disentangled SPARE models, SPARE-AD2 and SPARE-BA3, derived brain patterns that were more specific to the two types of brain changes. The correlation between the SPARE-BA Gap (SPARE-BA minus chronological age) and SPARE-AD was significantly reduced after the decoupling (r = 0.56–0.06). The correlation of disentangled SPARE-AD was non-inferior to amyloid- and tau-related measurements and to the number of APOE ε4 alleles but was lower to Alzheimer’s disease-related psychometric test scores, suggesting the contribution of advanced brain ageing to the latter. The disentangled SPARE-BA was consistently less correlated with Alzheimer’s disease-related clinical, molecular and genetic variables. By employing conservative molecular diagnoses and introducing Alzheimer’s disease continuum cases to the SPARE-BA model training, we achieved more dissociable neuroanatomical biomarkers of typical brain ageing and Alzheimer’s disease.

Hwang, Gyujoon↗
DOE OSTI2024

W2VPCA: A Machine Learning Method for Measuring Attitudes With Natural Language

Company strategy influences many decisions in freight transportation. Behavioral models of company decision-making therefore could benefit from including strategy variables. However, strategy is difficult to observe and quantify. Attitudinal surveys of company executives can be used to collect measurements of latent strategy to use in quantitative models. However, surveys are costly and burdensome. Text mining methods to collect measurements overcome these issues somewhat, but typically require manual intervention and ignore the context of words, which can be problematic. This study introduces a new machine learning method to generate strategy measurement data from existing big text data. The new method, called W2VPCA, combines Natural Language Processing and Principal Components Analysis. W2VPCA produces measurement data that serve as quantitative indicators of latent strategy in behavioral models. W2VPCA is unsupervised, data-driven, and uses information on word context. We apply W2VPCA to generate measurements of latent strategies using readily available, large-scale text data: annual company reports. The empirical measurements are used successfully to associate two latent strategies, one focusing on distribution and the other on products, with truck fleet and distribution center outsourcing decisions. The main empirical outcome is that the W2VPCA measurements outperform Bag-of-Words measurements in a psychometric analysis of latent firm strategies. While this study focuses on freight behavioral models, W2VPCA may also have applications in behavioral modeling in other domains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Assessing the nature of large language models: A caution against anthropocentrism.

Generative AI models garnered a large amount of public attention and speculation with the release of OpenAI’s chatbot, ChatGPT in November of 2022. At least two opinion camps exist – one that is excited about the possibilities these models offer for fundamental changes to human tasks, and another that is highly concerned about the power these models seem to have – especially since the release of GPT-4, which was trained on multimodal data and has ~1.7 trillion (T) parameters. We evaluated some concerns regarding these models’ power by assessing GPT-3.5 using standard, normed, and validated cognitive and personality measures. These measures come from the tradition of psychometrics in experimental psychology and have a long history of providing valuable insights and predictive distinctions in humans. For this seedling project, we developed a battery of tests that allowed us to estimate the boundaries of some of these models’ capabilities, how stable those capabilities are over a short period of time, and how they compare to humans.

97 MATHEMATICS AND COMPUTING↗