Search NASA⌕ Search

SEARCH · Search NASA

Results for “Practitioner”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Experimental Observations of the Topology of Convolutional Neural Network Activations

Topological data analysis (TDA) is a branch of computational mathematics, bridging algebraic topology and data science, that provides compact, noise-robust representations of complex structures. Deep neural networks (DNNs) learn millions of parameters associated with a series of transformations defined by the model architecture resulting in high-dimensional, difficult to interpret internal representations of input data. As DNNs become more ubiquitous across multiple sectors of our society, there is increasing recognition that mathematical methods are needed to aid analysts, researchers, and practitioners in understanding and interpreting how these models' internal representations relate to the final classification. In this paper we apply cutting edge techniques from TDA with the goal of gaining insight towards interpretability of convolutional neural networks used for image classification. We use two common TDA approaches to explore several methods for modeling hidden layer activations as high-dimensional point clouds, and provide experimental evidence that these point clouds capture valuable structural information about the model's process. First, we demonstrate that a distance metric based on persistent homology can be used to quantify meaningful differences between layers and discuss these distances in the broader context of existing representational similarity metrics for neural network interpretability. Second, we show that a mapper graph can provide semantic insight as to how these models organize hierarchical class knowledge at each layer. These observations demonstrate that TDA is a useful tool to help deep learning practitioners unlock the hidden structures of their models.

topological data analysis, deep learning↗

From Bricks to Clicks: Mapping the White Space in Building Innovation

It is a critical national imperative to transform the buildings sector, yet innovation is impeded by deployment failures that leave promising technologies stranded. Conventional market reports and techno-economic analysis provide an insufficient understanding of markets and resource allocation for emerging building technologies. They omit crucial commercialization factors such as ecosystem maturity and adoption friction, where the coordinated participation of a network of suppliers, contractors, financiers, regulators, and integrators is required to scale solutions. This study addresses these gaps by introducing an evaluation framework grounded in front-line data from six years of the DOE's IMPEL incubator, comprising experience from 300 building-sector innovators and the adjacent, complex ecosystem. Our methodology synthesizes top-down market analysis with bottom-up, practitioner-level data across five megatrends: (M1) Affordable materials and industrialized construction; (M2) Healthy and efficient mechanical systems; (M3) Intelligent building operations; (M4) Buildings as grid assets; and (M5) High-density power and cooling for data centers and therein identify twelve "white space" technology opportunities. Next, we develop a multi-criteria scoring rubric to rank these opportunities based on parameters, i.e., Affordability, Quality of Life, Reliability, and Security, yielding composite ‘Demand’ and ‘Maturity’ indices. Our results indicate that the most significant white spaces may not be incremental products but a new class of ‘Ecosystem Enablers’, such as logistics platforms, orchestration layers, and automated compliance software that solve structural deployment gaps. This paper summarizes this transparent, evidence-based, practitioner-informed evaluation framework for policymakers and investors to re-evaluate policy and resource allocation and unlock scalable market transformation.

Singh, Reshma↗

2011 Performance Assessment Community of Practice Technical Exchange – Summary

The Performance Assessment Community of Practice (PA CoP) was developed in 2008 to improve consistency and quality in the preparation of performance assessments (PAs) and risk assessments across the Department of Energy (DOE) Complex. The term, PA, is used to represent all of these modeling applications in this report. The PA CoP goals are to foster the exchange of information among PA practitioners and to share lessons learned from PAs conducted for DOE, commercial disposal facilities, and international entities. Technical exchanges and workshops are a cornerstone of PA CoP activities. Previous technical exchanges have addressed Engineered Barriers (2009 - http://www.cresp.org/education/workshops/pacop/), the Advanced Simulation Capability for Environmental Management and the Cementitious Barriers Partnership (2010 - http://srnl.doe.gov/copexchange/links.htm). Each technical exchange also includes summary presentations regarding activities at DOE, the Nuclear Regulatory Commission (NRC) and other organizations (e.g., International Atomic Energy Agency (IAEA)) as well as a number of presentations from selected sites to provide insight and perspective from on-going modeling activities. Through the deployment of PA Assistance Teams, the PA CoP has also been engaged in the development of new PAs across the DOE Complex. As a way of improving consistency in the preparation of new PAs, the teams provide technical advice and share experiences, noteworthy practices, and lessons learned from previous Low-Level Waste Disposal Facility Federal Review Group (LFRG) reviews. Teams have provided support for PAs at Hanford, Idaho, Paducah and Portsmouth. The third annual PA CoP Technical Exchange was held on May 25-26, 2011 in Atlanta, GA. The PA CoP Steering Committee Meeting held its first meeting on May 24 prior to the Technical Exchange. Decision making using models and software quality assurance were the topical emphasis for the exchange. A new feature at the 2011 technical exchange was the use of panel discussions to solicit feedback from regulators and practitioners. This report summarizes discussions and recommendations from the steering committee meeting and presentations and feedback obtained at the technical exchange. Appendix I includes the steering committee meeting agenda and Appendix II includes the agenda for the technical exchange and a screenshot of the presentations and video files that are available online.

99 GENERAL AND MISCELLANEOUS↗

DEVELOPMENT OF A SOFTWARE DESIGN TOOL FOR HYBRID SOLAR-GEOTHERMAL HEAT PUMP SYSTEMS IN HEATING- AND COOLING-DOMINATED BUILDINGS

This project provides an easy-to-use, menu-driven, software tool for designing hybrid solar-geothermal heat pump systems (GHP) for both heating- and cooling-dominated buildings. No such design tool currently exists. In heating-dominated buildings, the design approach takes advantage of glazed solar collectors to effectively balance the annual thermal loads on the ground with renewable solar energy. In cooling-dominated climates, the design approach takes advantage of relatively low-cost, unglazed solar collectors as the heat rejecting component. The primary benefit of hybrid GHPs is the reduced initial cost of the ground heat exchanger (GHX). Furthermore, solar thermal collectors can be used to balance the ground loads over the annual cycle, thus making the GHX fully sustainable; in heating-dominated buildings, the hybrid energy source (i.e., solar) is renewable, in contrast to a typical fossil fuel boiler or electric resistance as the hybrid component; in cooling-dominated buildings, use of unglazed solar collectors as a heat rejecter allows for passive heat rejection, in contrast to a cooling tower that consumes a significant amount of energy to operate, and hybrid GHPs can expand the market by allowing reduced GHX footprint in both heating- and cooling-dominated climates. The design tool allows for the straight-forward design of innovative GHP systems that currently pose a significant design challenge. The project lays the foundations for proper and reliable design of hybrid GHP systems, overcoming a series of difficult and cumbersome steps without the use of a system simulation approach, and without an automated optimization scheme. As new technologies and design concepts emerge, sophisticated design tools and methodologies must accompany them and be made usable for practitioners. Lack of reliable design tools results in reluctance of practitioners to implement more complex systems. A menu-driven software tool for the design of hybrid solar GHP systems is provided that is based on mathematically robust, validated models. An automated optimization tool is used to balance ground loads and incorporated into the simulation engine. With knowledge of the building loads, thermal properties of the ground, the borehole heat exchanger configuration, the heat pump peak hourly and seasonal COP for heating and cooling, the critical heat pump design entering fluid temperature, and the thermal performance of a solar collector, the total GHX length can be calculated along with the area of a supplemental solar collector array and the corresponding reduced GHX length. An economic analysis module allows for the calculation of the lowest capital cost combination of solar collector area and GHX length. ACKNOWLEDGMENTS This project was funded by the United States Department of Energy DOE-DE-FOA-0000116, Recovery Act Geothermal Technologies Program: Ground Source Heat Pumps. The lead contractor, The University of Hartford, was supported by The University of Dayton, and the Oak Ridge National Laboratories. All funding and support for this project as well as contributions of graduate and undergraduate students from the contributing institutions are gratefully acknowledged.

14 SOLAR ENERGY↗

Columbia Estuary Ecosystem Restoration Program: Restoration Design Challenges for Topographic Mounds, Channel Outlets, and Reed Canarygrass

The purpose of this study was to provide science-based information to practitioners and managers of restoration projects in the Columbia Estuary Ecosystem Restoration Program (CEERP) regarding aspects of restoration techniques that currently pose known challenges and uncertainties. The CEERP is a program of the Bonneville Power Administration (BPA) and the U.S. Army Corps of Engineers (Corps), Portland District, in collaboration with the National Marine Fisheries Service and five estuary sponsors implementing restoration. The estuary sponsors are Columbia Land Trust, Columbia River Estuary Study Taskforce, Cowlitz Tribe, Lower Columbia Estuary Partnership, and Washington Department of Fish and Wildlife. The scope of the research conducted during federal fiscal year 2015 included three aspects of hydrologic reconnection that were selected based on available scientific information and feedback from restoration practitioners during project reviews: the design of mounds (also called hummocks, peninsulas, or berms); the control of reed canarygrass (Phalaris arundinaceae); and aspects of channel network design related to habitat connectivity for juvenile salmonids.

54 ENVIRONMENTAL SCIENCES↗

Improving Insider Threat Training Awareness and Mitigation Programs at Nuclear Facilities.

In recent years, insider threat programs have become an important aspect of nuclear security, and nuclear security training courses. However, many nuclear security insider threat programs fail to address the insider threat attack and monitoring potential that exists on information technology (IT) systems. This failure is critical because of the importance of information technology and networks in today’s world. IT systems offer an opportunity to perpetrate dangerous insider attacks, but they also present an opportunity to monitor for them and prevent them. This paper suggests a number of best practices for monitoring and preventing insider attacks on IT systems, and proposes the development of a new IT insider threat tabletop that can be used to help train nuclear security practitioners on how best to implement IT insider threat prevention best practices. The development of IT insider threat best practices and a practical tabletop exercise will allow nuclear security practitioners to improve nuclear security trainings as it integrates a critical part of insider threat prevention into the broader nuclear security system.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

User-Oriented Modeling Tools for Advanced Hybrid and Climate-Appropriate Rooftop Air Conditioners

Hybrid unitary air conditioning systems offer a pathway to substantially reduce energy use and peak electrical demand for cooling, heating, and ventilation in commercial buildings. Hybrid air conditioners incorporate multiple subsystems that are carefully orchestrated to provide climate- and application-specific efficiency advantages. There are a multitude of hybrid system architectures, but common subsystems include: heat recovery ventilation, indirect evaporative cooling, desiccant dehumidification, variable speed fans, modulating dampers, and multi-stage or variable-speed vapor compression cooling. Categorically, hybrid systems can operate in numerous discrete modes. For example: indirect evaporative cooling may operate for periods when the subsystem provides adequate sensible cooling, then vapor compression cooling will be included when more cooling or dehumidification is necessary. Laboratory assessments, field studies, and simulations have demonstrated that hybrid unitary air conditioners could reduce energy use for cooling and ventilation by 30-90% depending on climate and application. Heretofore, it has been challenging - if not impossible - for practitioners to model hybrid air conditioners as part of building energy simulations; and the limitation has severely obstructed broader adoption of technologies in this class. In this project, we developed a new feature for EnergyPlus that enables modeling hybrid unitary air conditioning equipment for building energy simulations. This is a significant advancement for both theory and practice, and confers public benefit by enabling practitioners to evaluate this compelling efficiency technology as a part of building energy simulations. The feature is a black-box model that requires extensive performance data for each hybrid unitary product. In parallel, we also developed new features for the Technology Performance Exchange to enable manufacturers to submit performance data in a standard format that can be used with the hybrid unitary model in EnergyPlus. Additionally, through this project we expanded university educational resources, and university- manufacturing industry collaborations in the field of energy efficiency technology. Over two years, we involved 20 undergraduate students in ambitious research projects focused on modeling complex multi-mode mechanical systems, supported three mechanical engineering bachelor theses, established undergraduate apprenticeships with multiple industry partners, and involved those partners in the process of design, validation, and debugging for the new EnergyPlus feature. The EnergyPlus feature is described and discussed in an academic article, as well as in an engineering reference, and input/output reference documentation for EnergyPlus. The Technology Performance Exchange features are live and publicly accessible, our manufacturer partners are primed to submit initial product information and performance data to the exchange, and the EnergyPlus feature is scheduled for public release in Spring 2018 as a part of EnergyPlus v8.9.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Transmission Planning Studies for the Renewable Energy Zone (REZ) Process

Transmission planning studies help practitioners identify the most attractive transmission options to cost-effectively connect Renewable Energy Zones (REZs) to load while sustaining the balance between generation and demand in a variety of potential operating conditions and yielding the most benefits at minimum cost. This document focuses on the transmission planning studies of the REZ Transmission Planning process to help practitioners understand the analyses involved in studying potential transmission upgrade and expansion options, the general data requirements, and the associated modeling considerations. The REZ process is an approach to plan, approve, and build transmission infrastructure that connects REZs - geographic areas that have high-quality RE resources, suitable topography and land-use designations, and demonstrated developer interest - to the power system. The REZ process helps to increase the share of solar photovoltaic (PV), wind, and other resources while also maintaining reliability and economics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Community Energy Planning: Best Practices and Lessons Learned in NREL's Work with Communities

Whether driven by local goals and actions, external market forces, or both, the clean energy transition is accelerating. The associated increase in clean energy deployment occurs on the ground in communities. As a result, communities increasingly need technical expertise and assistance in planning for and managing the energy transition. Building on decades of work with state, local, and tribal jurisdictions, NREL's work providing modeling, analysis, and technical expertise to enable more data-driven community energy planning is expanding. To inform and enhance NREL's capabilities in community energy planning and provide a resource for others working in this space, NREL developed this best-practices document through interviews with seasoned NREL practitioners and a review of the literature on equitable community planning. Findings include five best practices for community energy planning that NREL practitioners can apply to increase the impact of their work.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Assessment of Existing OpenStudio Measures: Reviews, Interviews, and Future Developments

OpenStudio Measure development is continuously in progress and greatly propelled by the collaborative efforts within the building energy modeling community. To ensure the widespread adoption and benefit of OpenStudio Measures, developers must understand the current status of Measure development and the needs of OpenStudio Measure users. The first step involves a comprehensive review of existing content to prevent redundancy and gain insights into how OpenStudio Measures are used in the building energy modeling community. This knowledge can then be integrated into the Measure development process, and the expertise of practitioners and OpenStudio Measure users can be leveraged to shape future Measures. Numerous OpenStudio Measures have been created and shared on the Building Component Library (BCL). The BCL is an open-source repository housing various OpenStudio-related resources, including building component blocks, descriptive metadata, and Measures describing modifications to building energy models. The OpenStudio Measures in BCL encompass a wide range of energy conservation Measures from basic lighting power reduction to complex HVAC model transformation. They also enable users to generate customized reports and facilitate the integration of energy simulation with other analytical processes. This report presents review of 272 currently available OpenStudio Measures in BCL. The OpenStudio Measures were reviewed by category and subcategory. These Measures are summarized by their functionalities and keywords. To gain insights into how OpenStudio Measures are used in building energy modeling community, interviews were conducted. A total of 12 interview responses were collected from 6 individuals in the industry and 6 individuals in academia. The knowledge acquired from reviewing the existing Measures and interview results will be integrated into the Measure development process, and the expertise of practitioners and OpenStudio Measure users will be leveraged to shape future Measures.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Questionnaire for Radioisotope Identification and Estimation from Gamma Spectra using PyRIID v2

Accurate targeting of radioisotope classifiers and estimators requires an understanding of the target problem space. In order to facilitate clear communication on expected model behavior and performance between practitioners and stakeholders on their problems, this questionnaire was created. Stakeholder responses form the basis of a trained model as well as the start of usage requirements for the model as it is integrated with analysis processes or detection systems. This questionnaire may also be useful to machine learning practitioners and gamma spectroscopists developing new algorithms as a starting point for characterizing their problem space, especially if they are using PyRIID.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

WETO Software Stack Best Practices

Wind energy researchers typically share one key characteristic: a passion for increasing wind energy in the global energy mix. The U.S. Department of Energy (DOE) supports this mission in a number of ways including allocating funding directly to various aspects of wind energy research through the Office of Energy Efficiency and Renewable Energy (EERE) via the Wind Energy Technologies Office (WETO). While the traditional output of research is academic publication, software development efforts are increasingly a major focus. Software tools in the research environment allow researchers to describe an idea and quickly increase the scope and scale as they study it further. As a product of research, these tools represent a direct pipeline from researcher to industry practitioners since they are the implementation of ideas described in academic publications. Given this vital role in wind energy research and commercial development, the broad research software portfolio supported by WETO must maintain a minimum level of quality to support the wind energy field in the growing transition to renewable energy. This report outlines a series o f best practices to be adopted by all WETO-supported software projects, as well as expectations that the communities interacting with these projects should have of the developers and tools themselves. Wind energy research software has a unique standing in the field of scientific software. The stakeholders are varied with a subset being: (1) DOE EERE leadership, (2) DOE WETO leadership and program managers, (3) National lab leadership, (4) Associated project principle investigators, (5) Research software engineers, (6) Wind energy researchers in academia (including graduate students, post docs, and national lab staff), (7) Industry researchers and practitioners, (8) Commercial software developers, and (9) The general public interested in wind energy. These software are typically the end-user of other generic software libraries, so the funding cycles are often tied to applied research rather than the development of the software itself. Since the developers are also wind energy researchers, these tools are typically designed in a way that closely resembles the application in which they're used. Additionally, the expertise and incentives for the developers have a high variability, and often neither are aligned with software engineering or computer science. Given the unique environment in which wind energy research software is produced and consumed, it is critical for model owners to understand the context of their software. A framework for developing this understanding is to answer the following questions of a given software project: What is it's purpose? What is its role in the field of wind energy? What is the profile of the expected users? For how long will it be relevant? What is the expected impact? These questions allow model owners to identify the appropriate methods for the design, development, and long term maintenance of their software. Additionally, the answer provide context for future planners to understand why particular decisions were made and discern the consequences of changing course. The information is aggregated from experience within WETO-supported software development groups as well as external organizations and efforts to define the craft of research software engineering. These best practices aim to make the collaborative development process efficient and effective while improving the model understanding across stakeholders. Additionally, the general adoption of a common framework for software quality ensures that the end users of WETO software can trust these tools and accurately understand the risks to workflow integration.

17 WIND ENERGY↗

2024 Stor4Build Annual Meeting: Exploring Challenges and Opportunities in Thermal Energy Storage for Buildings

In late August 2024, Stor4Build brought nearly 80 stakeholders from the thermal energy storage industry to Oak Ridge National Laboratory (ORNL), including researchers, startups, electric utilities, nonprofits, implementers, state energy efficiency offices, and original equipment manufacturers. During the two-day Stor4Build Annual Meeting, participants engaged in vital discussions about the current challenges and opportunities for scaling thermal energy storage solutions in buildings. The event featured panel discussions led by leading technology experts and industry practitioners, as well as updates on Stor4Build–funded projects from national laboratories, highlighting advancements in the thermal energy storage field crucial to achieving the consortium’s mission.

25 ENERGY STORAGE↗

Workshop Summary Report on Using AI Tools to Improve the Efficiency and Outcomes of the NEPA Process: AI for Permitting Workshop at the 2025 National Association of Environmental Professionals (NAEP) Annual Conference

On April 29, 2025, the U.S. Department of Energy and Pacific Northwest National Laboratory hosted a workshop at the National Association of Environmental Professionals 2025 Conference and Training Symposium in Charleston, South Carolina, titled, “Effective and Responsible Use of Customized AI Tools to Improve the Efficiency and Outcomes of the NEPA Process.” The objectives of this workshop were to make environmental practitioners aware of the potential for using artificial intelligence in the National Environmental Policy Act process, demonstrate examples of how artificial intelligence can be integrated effectively to improve efficiency and outcomes and solicit questions and feedback from practitioners. This report summarizes the key points from all talks and case studies, as well as audience questions and feedback on the presentation topics and the broader topic of "AI in permitting". The report concludes by highlighting the key barriers and opportunities for the implementation of AI in permitting, as discussed during the workshop.

54 ENVIRONMENTAL SCIENCES↗

Making an s1G Self-Preserving Environmental DNA (eDNA) Filter from Sustainable Materials

Environmental DNA (eDNA) sampling of aquatic environments is a powerful new tool for resource managers to use in detecting the presence of rare and endangered species. The use of eDNA for species detection is quickly becoming an industry-standard methodology in environmental science, and the technology sector is moving quickly to create purpose-built tools for eDNA sampling. Smith-Root, Inc. pioneered the new technology of self-preserving eDNA filters that automatically preserve the DNA captured on filters by desiccating the internal filter membrane. This capability saves the sampling technician significant time in the field and drastically reduces the potential for sample contamination. One main constraint of this new technology, however, is the necessity of single-use plastics for eDNA sampling. Single-use designs are employed because sampling equipment must be devoid of potential contaminating DNA, and the time required to sterilize equipment is often not cost-effective for practitioners. Thus, this technology is creating a significant new source of plastic waste—something that eDNA practitioners, and society, would like to avoid. This project explores the use of alternative bio based polymers and composites as replacements for incumbent industrial standards for eDNA filtration systems, with an emphasis on ease of processing and biodegradable options. 8 candidate materials, 3 for the top filter housing and 5 for the bottom filter housing, were investigated to replace incumbent housing materials. Of the 8 candidates, 2 materials were identified for future investigation which met performance requirements, were bio-based and could potentially offer better disposal options like soil degradation or composting, features sought by the customers of these products. The top housing candidate, material 1, has comparable performance to the incumbent material and equally manufacturable. The bottom housing candidate, material 6, improved desiccation performance by 81% after 1 hour in comparison to the incumbent material, is 100% bio-based, compostable and can be sourced domestically throughout the U.S.A.

36 MATERIALS SCIENCE↗

Final Technical Report

The work under this grant consisted of encouraging community activities for the development of parallel programming patterns. This work was in conjunction with the research performed under the Pmodels award - where research was pursued on the development for new parallel programming models. Work on programming patterns does not have, as a goal, the invention of new technology. Rather, it is about codifying existing practice, so as to provide practitioners with a common language. This facilitates education and communication between practitioners. In addition, it helps in the design of new parallel frameworks and languages. One major issue in their design is expressiveness: To what extent does the language or framework facilitates the expression of common programming patterns. In order to assess expressiveness in any useful way, it is necessary to identify those common patterns.

97 MATHEMATICS AND COMPUTING↗

Sustainability metrics for assessing water resource recovery facilities of the future

Abstract The recovery of water, energy, and nutrients from water resource recovery facilities ( WRRF s) is needed to address significant global challenges, such as increasing water demand and decreasing availability of nonrenewable resources. To meet these challenges, innovative technological developments must lead to increased adoption of water and resource recovery processes, while addressing stakeholder needs (e.g., innovators, practitioners, regulators). A test bed network of over 90 partner facilities within the United States and abroad will help accelerate innovation and widespread adoption of novel processes through multiscale testing and demonstration of technologies. In this paper, we define a common set of environmental, economic, technical, and social performance metrics for innovative technologies, that will meet the needs of multiple stakeholders in the decision‐making process. These triple bottom line performance metrics can be used to track the sustainability of technologies in a consistent and transparent manner, while aiding the decision‐making process for WRRF s. Practitioner points The Facilities Accelerating Science and Technology (FAST) Water Network includes over 90 test bed facilities dedicated to accelerating innovation and adoption of water energy, and nutrient recovery systems. A common set of environmental, economic, technical, and social performance metrics should be measured and reported when a new technology is evaluated in the FAST Water Network. Performance metrics can aid sustainable decision‐making at WRRF, while meeting the needs of multiple stakeholders.

Cornejo, Pablo K.↗

User-Centered Evaluation of Visual Analytics

Visual analytics systems are becoming very popular. More domains now use interactive visualizations to analyze the ever-increasing amount and heterogeneity of data. More novel visualizations are being developed for more tasks and users. We need to ensure that these systems can be evaluated to determine that they are both useful and usable. A user-centered evaluation for visual analytics needs to be developed for these systems. While many of the typical human-computer interaction (HCI) evaluation methodologies can be applied as is, others will need modification. Additionally, new functionality in visual analytics systems needs new evaluation methodologies. There is a difference between usability evaluations and user-centered evaluations. Usability looks at the efficiency, effectiveness, and user satisfaction of users carrying out tasks with software applications. User-centered evaluation looks more specifically at the utility provided to the users by the software. This is reflected in the evaluations done and in the metrics used. In the visual analytics domain this is very challenging as users are most likely experts in a particular domain, the tasks they do are often not well defined, the software they use needs to support large amounts of different kinds of data, and often the tasks last for months. These difficulties are discussed more in the section on User-centered Evaluation. Our goal is to provide a discussion of user-centered evaluation practices for visual analytics, including existing practices that can be carried out and new methodologies and metrics that need to be developed and agreed upon by the visual analytics community. The material provided here should be of use for both researchers and practitioners in the field of visual analytics. Researchers and practitioners in HCI and interested in visual analytics will find this information useful as well as a discussion on changes that need to be made to current HCI practices to make them more suitable to visual analytics. A history of analysis and analysis techniques and problems is provided as well as an introduction to user-centered evaluation and various evaluation techniques for readers from different disciplines. The understanding of these techniques is imperative if we wish to support analysis in the visual analytics software we develop. Currently the evaluations that are conducted and published for visual analytics software are very informal and consist mainly of comments from users or potential users. Our goal is to help researchers in visual analytics to conduct more formal user-centered evaluations. While these are time-consuming and expensive to carryout, the outcomes of these studies will have a defining impact on the field of visual analytics and help point the direction for future features and visualizations to incorporate. While many researchers view work in user-centered evaluation as a less-than-exciting area to work, the opposite is true. First of all, the goal is user-centered evaluation is to help visual analytics software developers, researchers, and designers improve their solutions and discover creative ways to better accommodate their users. Working with the users is extremely rewarding as well. While we use the term “users” in almost all situations there are a wide variety of users that all need to be accommodated. Moreover, the domains that use visual analytics are varied and expanding. Just understanding the complexities of a number of these domains is exciting. Researchers are trying out different visualizations and interactions as well. And of course, the size and variety of data are expanding rapidly. User-centered evaluation in this context is rapidly changing. There are no standard processes and metrics and thus those of us working on user-centered evaluation must be creative in our work with both the users and with the researchers and developers.

intelligence analysis↗