Search NASA⌕ Search

SEARCH · Search NASA

Results for “performance benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51

Survival, profits and resource utilization

The intertwining of CAB regulation and airline operations makes it impossible to use operating and financial data from the regulated airlines to determine the extent to which regulation affects costs. The essentially nonregulated, but otherwise, similar, intrastate airlines operating wholly within California (and thus beyond the CAB's jurisdiction) provide a benchmark with which to compare the performance of regulated airlines and thus measure the effects of CAB regulation. This comparison, shows that the nonregulated environment within California has been a much sterner disciplinarian of airline operations than the regulated environment of interstate air transportation. Issues of survival, profits, efficiency, fares, and the impacts of CAB regulation on costs are examined.

Jordan, W.↗

Branch recovery with compiler-assisted multiple instruction retry

In processing systems where rapid recovery from transient faults is important, schemes for multiple instruction rollback recovery may be appropriate. Multiple instruction retry has been implemented in hardware by researchers and also in mainframe computers. This paper extends compiler-assisted instruction retry to a broad class of code execution failures. Five benchmarks were used to measure the performance penalty of hazard resolution. Results indicate that the enhanced pure software approach can produce performance penalties consistent with existing hardware techniques. A combined compiler/hardware resolution strategy is also described and evaluated. Experimental results indicate a lower performance penalty than with either a totally hardware or totally software approach.

Alewine, N. J.↗

Controlling state explosion during automatic verification of delay-insensitive and delay-constrained VLSI systems using the POM verifier

Delay-insensitive VLSI systems have a certain appeal on the ground due to difficulties with clocks; they are even more attractive in space. We answer the question, is it possible to control state explosion arising from various sources during automatic verification (model checking) of delay-insensitive systems? State explosion due to concurrency is handled by introducing a partial-order representation for systems, and defining system correctness as a simple relation between two partial orders on the same set of system events (a graph problem). State explosion due to nondeterminism (chiefly arbitration) is handled when the system to be verified has a clean, finite recurrence structure. Backwards branching is a further optimization. The heart of this approach is the ability, during model checking, to discover a compact finite presentation of the verified system without prior composition of system components. The fully-implemented POM verification system has polynomial space and time performance on traditional asynchronous-circuit benchmarks that are exponential in space and time for other verification systems. We also sketch the generalization of this approach to handle delay-constrained VLSI systems.

Probst, D.↗

The application of compiler-assisted multiple instruction retry to VLIW architectures

Very Long Instruction Word (VLIW) architectures enhance performance by exploiting fine-grained instruction level parallelism. We describe the development of two compiler assisted multiple instruction word retry schemes for VLIW architectures. The first scheme utilizes the compiler techniques previously developed for processors with single functional units. A compiler generated hazard-free code with different degrees of rollback capability for uniprocessors is compacted by a modified VLIW trace scheduling algorithm. Nops are then inserted in the scheduled code words to resolve data hazards for VLIW architectures. Performance is compared under three parameters: the rollback distance for uni-processors; the number of functional units; and the rollback distance for VLIW architectures. The second scheme employs a hardware read buffer to resolve frequently occurring data hazards, and utilizes the compiler to resolve the remaining hazards. Performance results are shown for six benchmark programs.

Chen, Shyh-Kwei↗

The P-Mesh: A Commodity-based Scalable Network Architecture for Clusters

We designed a new network architecture, the P-Mesh which combines the scalability and fault resilience of a torus with the performance of a switch. We compare the scalability, performance, and cost of the hub, switch, torus, tree, and P-Mesh architectures. The latter three are capable of scaling to thousands of nodes, however, the torus has severe performance limitations with that many processors. The tree and P-Mesh have similar latency, bandwidth, and bisection bandwidth, but the P-Mesh outperforms the switch architecture (a lower bound for tree performance) on 16-node NAB Parallel Benchmark tests by up to 23%, and costs 40% less. Further, the P-Mesh has better fault resilience characteristics. The P-Mesh architecture trades increased management overhead for lower cost, and is a good bridging technology while the price of tree uplinks is expensive.

Nitzberg, Bill↗

Strain Measurement System Developed for Biaxially Loaded Cruciform Specimens

A new extensometer system developed at the NASA Glenn Research Center at Lewis Field measures test area strains along two orthogonal axes in flat cruciform specimens. This system incorporates standard axial contact extensometers to provide a cost-effective high-precision instrument. The device was validated for use by extensive testing of a stainless steel specimen, with specimen temperatures ranging from room temperature to 1100 F. In-plane loading conditions included several static biaxial load ratios, plus cyclic loadings of various waveform shapes, frequencies, magnitudes, and durations. The extensometer system measurements were compared with strain gauge data at room temperature and with calculated strain values for elevated-temperature measurements. All testing was performed in house in Glenn's Benchmark Test Facility in-plane biaxial load frame.

Krause, David L.↗

A Sulfur Dioxide Covariance-Based Retrieval Algorithm (COBRA): Application to TROPOMI Reveals New Emission Sources

Sensitive and accurate detection of sulfur dioxide (SO 2 ) from space is important for monitoring and estimating global sulfur emissions. Inspired by detection methods applied in the thermal infrared, we present here a new scheme to retrieve SO 2 columns from satellite observations of ultraviolet back-scattered radiances. The retrieval is based on a measurement error covariance matrix to fully represent the SO 2 -free radiance variability, so that the SO 2 slant column density is the only retrieved parameter of the algorithm. We demonstrate this approach, named COBRA, on measurements from the TROPOspheric Monitoring Instrument (TROPOMI) aboard the Sentinel-5 Precursor (S-5P) satellite. We show that the method reduces significantly both the noise and biases present in the current TROPOMI operational DOAS SO 2 retrievals. The performance of this technique is also benchmarked against that of the Principal Component Algorithm (PCA) approach. We find that the quality of the data is similar and even slightly better with the proposed COBRA approach. The ability of the algorithm to retrieve SO 2 accurately is also further supported by comparison with ground-based observations. We illustrate the great sensitivity of the method with a high-resolution global SO 2 map, considering two and a half years of TROPOMI data. In addition to the known sources, we detect many new SO 2 emission hotspots worldwide. For the largest sources, we use the COBRA data to estimate SO 2 emission rates. Results are comparable to other recently published TROPOMI-based SO 2 emissions estimates, but the associated uncertainties are significantly lower than with the operational data. Next, for a limited number of weak sources, we demonstrate the potential of our data for quantifying SO 2 emissions with a detection limit of about 8 kt yr -1 , a factor of 4 better than the emissions derived from the Ozone Monitoring Instrument (OMI). We anticipate that the systematic use of our TROPOMI COBRA SO 2 column data set at a global scale will allow identifying and quantifying missing sources, and help improving SO 2 emission inventories.

SO2↗

Evaluating Cell Temperature Models and the Effect of Wind Speed in PV System Capacity Testing: Preprint

Capacity testing is a routine procedure for assessing a photovoltaic system's performance relative to expectations. The most common test method involves fitting a regression model that predicts system output power using operating weather conditions including wind speed. Structural modifications to the regression model to incorporate wind in different ways improved the model's ability to fit measured system performance, but the observed improvements were small and unlikely to change the result of a capacity test. However, the results showed that the choice of reporting wind speed and inclusion or exclusion of wind speed in the performance model used as the test benchmark can significantly change the test result.

41 EE - Solar Energy Technologies Office (EE-4S)↗

Closing the Loop on Automotive Carbon Fiber Prepreg Manufacturing Scrap

The project demonstrated how to “close the loop” on carbon fiber by integrating industrial carbon fiber scrap into new functional components in an automotive lightweighting application. The project serves as a validation of discontinuous recycled carbon fiber in a commercial context, while generating comprehensive material data throughout the production chain. To this end, the project exhibited increasing complexity as material evaluation progressed from benchtop to commercial scale through full-scale part production, with key material properties thoroughly characterized throughout the process. Of particular focus was the form of the fiber that was fed into compounding, as recycled fiber has historically been problematic to feed at commercial-scale. Carbon fiber is energy-intensive to manufacture, so reuse of existing fiber material can reduce costs and increase sustainability. Additionally, by integrating recovered short fiber into a thermoplastic, regrind processes can be used to provide feedstock for later generations of product. While regrind plastics are not “infinitely recyclable”, reusing the manufacturing scrap over several generations of products can greatly increase material sustainability and lower the fractional embodied energy of each successive product. As such, this project supports the IACMI technical goals of (1) 25% lower carbon fiber-reinforced polymer (CFRP) cost, (2) 50% reduction in CFRP embodied energy, and (3) 80% composite recyclability into useful products. The initial stage of the project involved down selecting surface treatment (sizing) chemistries. Sizing evaluations were performed on Vartega’s chemically recycled intermediate modulus fiber along with standard modulus dry scrap which was oversized with sizing provided by Michelman. More dramatic improvements from sizing were found on the standard rather than the intermediate modulus fibers. The strength of the chemically recycled individual fibers were evaluated by Michelman and ORNL through single fiber testing and found to be comparable to similarly evaluated virgin fibers. UDRI’s mechanical testing on injection molded test specimens identified similar mechanical properties and fiber distribution relative to benchmark specimens. Additional surface chemistry tests and visualizations were performed by the Colorado School of Mines to confirm close conformance between the benchmark and recycled-fiber specimens. As the mechanical test results exceeded the 80% threshold established as the go/no-go(GNG), the project scale was increased to use commercial-scale equipment that would both better characterize the manufacturing utility of the target product format and allow qualitative assessment of a complex commercial part. An upscaled compounding evaluation was performed with a 27 mm twin-screw compounding extruding using oversized standard modulus fibers that were formatted to improve bulk solids transfer. The project team anticipated that milestone mechanical benchmarks could be achieved given the favorable performance of the sized standard modulus material identified in the initial micro-compounding trials. While the mechanical performance did meet the milestone target for that phase of the project, mechanical properties for this standard modulus-based compound were still less than those of the Ford specification. To compare the performance, the project team oversized intermediate modulus dry fibers and compounded them with the project resin at BASF using a 40 mm compounder. Test specimen mechanical performance exceeded the targets laid out in both the project milestone and the Ford specification. A series of prototype parts were successfully molded, albeit with instances of short shot components due to the high thermal conductivity of the carbon fiber compared to glass fiber for which the prototype tool was designed for. The project demonstrated that recycled carbon fiber is a viable option in fiber reinforced compound, providing greatly increased strength and modulus for applications that require them. The “agglomerated” format that facilitated effective bulk solids transfer of recovered fiber showed promise for industrial application.

36 MATERIALS SCIENCE↗

The Structural Evaluation Test Unit (SETU) Benchmark Test Results

A series of extensively instrumented tests was performed on the Structural Evaluation Test Unit in the early 1990s. The purpose of these tests was to determine the response of a minimally designed cask to impacts that were more severe than the design basis impact. This test series provides an excellent opportunity for benchmarking explicit dynamic finite element analysis programs for behaviors that may be experienced by casks during regulatory and extra-regulatory impact events. This report provides the results of the four tests that were conducted. It is meant to go along with a companion report that defines the benchmark problem and gives the locations for the instrumentation and inspection points.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

The Structural Evaluation Test Unit (SETU) Benchmark Problem Statement

A series of extensively instrumented tests was performed on the Structural Evaluation Test Unit in the early 1990s. The purpose of these tests was to determine the response of a minimally designed cask to impacts that were more severe than the design basis impact. This test series provides an excellent opportunity for benchmarking explicit dynamic finite element analysis programs for behaviors that may be experienced by casks during regulatory and extra-regulatory impact events. This report provides the parameters of the test unit, the locations of instrumentation, the locations of inspection points, and the parameters of the four tests that were conducted. A companion report provides the results of the tests.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Using Grid Benchmarks for Dynamic Scheduling of Grid Applications

Navigation or dynamic scheduling of applications on computational grids can be improved through the use of an application-specific characterization of grid resources. Current grid information systems provide a description of the resources, but do not contain any application-specific information. We define a GridScape as dynamic state of the grid resources. We measure the dynamic performance of these resources using the grid benchmarks. Then we use the GridScape for automatic assignment of the tasks of a grid application to grid resources. The scalability of the system is achieved by limiting the navigation overhead to a few percent of the application resource requirements. Our task submission and assignment protocol guarantees that the navigation system does not cause grid congestion. On a synthetic data mining application we demonstrate that Gridscape-based task assignment reduces the application tunaround time.

Frumkin, Michael↗

GHG policy impacts for Seattle’s buildings: targets, timing, and scope

Many US cities are addressing climate change by setting goals to reduce their greenhouse gas (GHG) emissions by a specified amount within a specified time period. In order to achieve these goals, reducing emissions from existing buildings is crucial. Many cities are passing legislation to target existing buildings through benchmarking, auditing, retuning, or energy or emissions performance standards. As cities design legislation, they must consider the timing of the policies, how to prioritize building types and sizes, and how these design decisions will impact the city’s emissions. This paper addresses these questions for one particular US city: Seattle, Washington. A model of Seattle’s building stock was created with benchmarking and tax assessor data. It was then used to predict GHG emissions reductions due to different policy implementations for existing commercial and multifamily buildings. Key findings are: (1) the proposed emissions policy is expected to reduce cumulative emissions from buildings by 19% between 2020 and 2050; (2) delaying the implementation of the policy by five years could limit savings to 12%; and (3) including smaller buildings in the policy could increase savings to 34%. The lessons learned and how this can be used by other cities are discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

The Surface-Topography Challenge: A Multi-Laboratory Benchmark Study to Advance the Characterization of Topography

Surface performance is critically influenced by topography in virtually all real-world applications. The current standard practice is to describe topography using one of a few industry-standard parameters. The most commonly reported number is Ra, the average absolute deviation of the height from the mean line (at some, not necessarily known or specified, lateral length scale). However, other parameters, particularly those that are scale-dependent, influence surface and interfacial properties; for example the local surface slope is critical for visual appearance, friction, and wear. The present Surface-Topography Challenge was launched to raise awareness for the need of a multi-scale description, but also to assess the reliability of different metrology techniques. In the resulting international collaborative effort, 153 scientists and engineers from 64 research groups and companies across 20 countries characterized statistically equivalent samples from two different surfaces: a “rough” and a “smooth” surface. The results of the 2088 measurements constitute the most comprehensive surface description ever compiled. We find wide disagreement across measurements and techniques when the lateral scale of the measurement is ignored. Consensus is established through scale-dependent parameters while removing data that violates an established resolution criterion and deviates from the majority measurements at each length scale. Our findings suggest best practices for characterizing and specifying topography. The public release of the accumulated data and presented analyses enables global reuse for further scientific investigation and benchmarking.

42 ENGINEERING↗

An Application-Based Performance Characterization of the Columbia Supercluster

Columbia is a 10,240-processor supercluster consisting of 20 Altix nodes with 512 processors each, and currently ranked as the second-fastest computer in the world. In this paper, we present the performance characteristics of Columbia obtained on up to four computing nodes interconnected via the InfiniBand and/or NUMAlink4 communication fabrics. We evaluate floating-point performance, memory bandwidth, message passing communication speeds, and compilers using a subset of the HPC Challenge benchmarks, and some of the NAS Parallel Benchmarks including the multi-zone versions. We present detailed performance results for three scientific applications of interest to NASA, one from molecular dynamics, and two from computational fluid dynamics. Our results show that both the NUMAlink4 and the InfiniBand hold promise for application scaling to a large number of processors.

Biswas, Rupak↗

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASA's Space Launch System

The engineering development of the new Space Launch System (SLS) launch vehicle requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The characteristics of these spacecraft systems must be matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large and complex system engineering challenge, which is being addressed in part by focusing on the specific subsystems involved in the handling of off-nominal mission and fault tolerance with response management. Using traditional model based system and software engineering design principles from the Unified Modeling Language (UML) and Systems Modeling Language (SysML), the Mission and Fault Management (M&FM) algorithms for the vehicle are crafted and vetted in specialized Integrated Development Teams (IDTs) composed of multiple development disciplines such as Systems Engineering (SE), Flight Software (FSW), Safety and Mission Assurance (S&MA) and the major subsystems and vehicle elements such as Main Propulsion Systems (MPS), boosters, avionics, Guidance, Navigation, and Control (GNC), Thrust Vector Control (TVC), and liquid engines. These model based algorithms and their development lifecycle from inception through Flight Software certification are an important focus of this development effort to further insure reliable detection and response to off-nominal vehicle states during all phases of vehicle operation from pre-launch through end of flight. NASA formed a dedicated M&FM team for addressing fault management early in the development lifecycle for the SLS initiative. As part of the development of the M&FM capabilities, this team has developed a dedicated testbed that integrates specific M&FM algorithms, specialized nominal and off-nominal test cases, and vendor-supplied physics-based launch vehicle subsystem models. Additionally, the team has developed processes for implementing and validating these algorithms for concept validation and risk reduction for the SLS program. The flexibility of the Vehicle Management End-to-end Testbed (VMET) enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the developed algorithms utilizing actual subsystem models such as MPS. The intent of VMET is to validate the M&FM algorithms and substantiate them with performance baselines for each of the target vehicle subsystems in an independent platform exterior to the flight software development infrastructure and its related testing entities. In any software development process there is inherent risk in the interpretation and implementation of concepts into software through requirements and test cases into flight software compounded with potential human errors throughout the development lifecycle. Risk reduction is addressed by the M&FM analysis group working with other organizations such as S&MA, Structures and Environments, GNC, Orion, the Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission and Loss of Crew probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and associated detection and responses that can be tested in VMET to ensure that failures can be detected, and confirm that responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - ARINC 653 partitioned OS, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM such as telemetry packing and processing. The baseline plan for use of VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by Flight Software. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure the effectiveness of M&FM algorithms performance in the FSW development and test processes.

Trevino, Luis↗

A Vehicle Management End-to-End Testing and Analysis Platform for Validation of Mission and Fault Management Algorithms to Reduce Risk for NASA's Space Launch System

The development of the Space Launch System (SLS) launch vehicle requires cross discipline teams with extensive knowledge of launch vehicle subsystems, information theory, and autonomous algorithms dealing with all operations from pre-launch through on orbit operations. The characteristics of these systems must be matched with the autonomous algorithm monitoring and mitigation capabilities for accurate control and response to abnormal conditions throughout all vehicle mission flight phases, including precipitating safing actions and crew aborts. This presents a large complex systems engineering challenge being addressed in part by focusing on the specific subsystems handling of off-nominal mission and fault tolerance. Using traditional model based system and software engineering design principles from the Unified Modeling Language (UML), the Mission and Fault Management (M&FM) algorithms are crafted and vetted in specialized Integrated Development Teams composed of multiple development disciplines. NASA also has formed an M&FM team for addressing fault management early in the development lifecycle. This team has developed a dedicated Vehicle Management End-to-End Testbed (VMET) that integrates specific M&FM algorithms, specialized nominal and off-nominal test cases, and vendor-supplied physics-based launch vehicle subsystem models. The flexibility of VMET enables thorough testing of the M&FM algorithms by providing configurable suites of both nominal and off-nominal test cases to validate the algorithms utilizing actual subsystem models. The intent is to validate the algorithms and substantiate them with performance baselines for each of the vehicle subsystems in an independent platform exterior to flight software test processes. In any software development process there is inherent risk in the interpretation and implementation of concepts into software through requirements and test processes. Risk reduction is addressed by working with other organizations such as S&MA, Structures and Environments, GNC, Orion, the Crew Office, Flight Operations, and Ground Operations by assessing performance of the M&FM algorithms in terms of their ability to reduce Loss of Mission and Loss of Crew probabilities. In addition, through state machine and diagnostic modeling, analysis efforts investigate a broader suite of failure effects and detection and responses that can be tested in VMET and confirm that responses do not create additional risks or cause undesired states through interactive dynamic effects with other algorithms and systems. VMET further contributes to risk reduction by prototyping and exercising the M&FM algorithms early in their implementation and without any inherent hindrances such as meeting FSW processor scheduling constraints due to their target platform - ARINC 653 partitioned OS, resource limitations, and other factors related to integration with other subsystems not directly involved with M&FM. The plan for VMET encompasses testing the original M&FM algorithms coded in the same C++ language and state machine architectural concepts as that used by Flight Software. This enables the development of performance standards and test cases to characterize the M&FM algorithms and sets a benchmark from which to measure the effectiveness of M&FM algorithms performance in the FSW development and test processes. This paper is outlined in a systematic fashion analogous to a lifecycle process flow for engineering development of algorithms into software and testing. Section I describes the NASA SLS M&FM context, presenting the current infrastructure, leading principles, methods, and participants. Section II defines the testing philosophy of the M&FM algorithms as related to VMET followed by section III, which presents the modeling methods of the algorithms to be tested and validated in VMET. Its details are then further presented in section IV followed by Section V presenting integration, test status, and state analysis. Finally, section VI addresses the summary and forward directions followed by the appendices presenting relevant information on terminology and documentation.

Trevino, Luis↗

An automated and portable method for selecting an optimal GPU frequency

Power consumption poses a significant challenge in current and emerging graphics processing unit (GPU) enabled high-performance computing systems. In modern GPUs, dynamic voltage frequency scaling (DVFS) appears to be a reliable control to regulate power consumption and performance. However, the DVFS design space is large - hence, brute-force approaches are infeasible to select the optimal frequency. Furthermore, no single frequency can be universally optimal for applications with varying computational intensities. Thus, the application's complexity and the availability of a wide range of frequency settings are a challenge in selecting the optimal frequency configuration for a given GPU workload. To that end, this paper proposes a systematic approach that consists of three steps. The feature characterization study identifies the fine-grain GPU utilization metrics that influence the power consumption and execution time of a given workload. To understand the performance, power, and energy consumption behaviors of a workload across GPU's DVFS design space, we derived analytical power and performance models using the identified fine-grain features. Here, it is shown that the same set of GPU utilization metrics can estimate both the power consumption and execution time while being agnostic of changes to frequency and input sizes. Applying a power control with the single objective of reducing power may cause performance degradation, leading to more energy consumption. A multi-objective approach is proposed to select the optimal GPU DVFS configuration for a workload that reduces power consumption with negligible degradation in performance. The evaluation was conducted using SPEC ACCEL benchmarks and three real applications - NAMD LAMMPS, and LSTM on NVIDIA GV100, GA100, and AMD MI210 GPUs. On average, real applications showed 29.6% energy savings with a performance loss of 5.2% on GA100 and 22.6% energy savings with a performance loss of 4.7% on GV100. Moreover, the proposed models are portable to real applications, GPU architectures, and vendors, and require metric collection at only the default frequency rather than all supported DVFS configurations. Additionally, we conducted a comparison between our models and the GPU assembly instructions (PTX)-based static models. The results revealed a significant reduction in the average error rates, with a decrease from 19.7% to 3.1% for power models and from 29.4% to 5.2% for performance models.

97 MATHEMATICS AND COMPUTING↗