Search NASA⌕ Search

Engineering topics

Harris, Chris

Publications and source records attributed to Harris, Chris.

10 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Streaming Large-Scale Microscopy Data to a Supercomputing Facility

Data management is a critical component of modern experimental workflows. As data generation rates increase, transferring data from acquisition servers to processing servers via conventional file-based methods is becoming increasingly impractical. The 4D Camera at the National Center for Electron Microscopy generates data at a nominal rate of 480 Gbit s -1 (87,000 frames s -1 ⁠), producing a 700 GB dataset in 15 s. To address the challenges associated with storing and processing such quantities of data, we developed a streaming workflow that utilizes a high-speed network to connect the 4D Camera’s data acquisition system to supercomputing nodes at the National Energy Research Scientific Computing Center, bypassing intermediate file storage entirely. In this work, we demonstrate the effectiveness of our streaming pipeline in a production setting through an hour-long experiment that generated over 10 TB of raw data, yielding high-quality datasets suitable for advanced analyses. Additionally, we compare the efficacy of this streaming workflow against the conventional file-transfer workflow by conducting a postmortem analysis on historical data from experiments performed by real users. Our findings show that the streaming workflow significantly improves data turnaround time, enables real-time decision-making, and minimizes the potential for human error by eliminating manual user interactions.

4D-STEM↗
DOE OSTI2024

Towards Autonomous Experiments by Connecting High Performance Microscopy with High Performance Computing

The digitization of controls, data, and analysis in microscopy is bringing the idea of autonomous microscopes closer to reality than ever before. Automated transmission electron microscopy (TEM) is already fairly routine for some experiments the only require simple repetitive tasks such as imaging biological macromolecules for single particle cryoEM [1], tilt series for electron tomography [2], and movies for crystallography [3]. The vast majority of TEM experiments are conducted completely by human operators who choose the regions of interest, optimize experimental parameters, and make decisions about data quality visually during an experiment. The field is still a long way from having completely autonomous TEMs that can adapt to sample difficulties and tune experimental parameters based on data quality and desired experimental outcomes. Part of the issue is the lack of capability for feeding information learned from on-line, live data analysis back into the on-going experiment [4]. Furthermore, this presentation will discuss current capabilities for large scale data reduction and analysis using high performance computing (i.e. supercomputing) and progress towards developing a true feed-back loop that places data analysis and theory in the experimental loop.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

sfapi_client v1.0

This software is a client designed to interact with the Superfacility API developed at NERSC. It allows users to easily access the api in python and programmatically interact with the compute resources available at NERSC. Other implementations are ad-hoc and made by our user base, the goal of the project is to encourage more users to adopt the API by making it easier to start building complex HPC workflows.

Tyler, Nicholas↗
DOE OSTI2022

Real-time 3D analysis during electron tomography using tomviz

The demand for high-throughput electron tomography is rapidly increasing in biological and material sciences. However, this 3D imaging technique is computationally bottlenecked by alignment and reconstruction which runs from hours to days. We demonstrate real-time tomography with dynamic 3D tomographic visualization to enable rapid interpretation of specimen structure immediately as data is collected on an electron microscope. Using geometrically complex chiral nanoparticles, we show volumetric interpretation can begin in less than 10 minutes and a high-quality tomogram is available within 30 minutes. Real-time tomography is integrated into tomviz, an open-source and cross-platform 3D data analysis tool that contains intuitive graphical user interfaces (GUI), to enable any scientist to characterize biological and material structure in 3D.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Mitigation of SARS-CoV-2 transmission at a large public university

In Fall 2020, universities saw extensive transmission of SARS-CoV-2 among their populations, threatening health of the university and surrounding communities, and viability of in-person instruction. Here we report a case study at the University of Illinois at Urbana-Champaign, where a multimodal “SHIELD: Target, Test, and Tell” program, with other non-pharmaceutical interventions, was employed to keep classrooms and laboratories open. The program included epidemiological modeling and surveillance, fast/frequent testing using a novel low-cost and scalable saliva-based RT-qPCR assay for SARS-CoV-2 that bypasses RNA extraction, called covidSHIELD, and digital tools for communication and compliance. In Fall 2020, we performed >1,000,000 covidSHIELD tests, positivity rates remained low, we had zero COVID-19-related hospitalizations or deaths amongst our university community, and mortality in the surrounding Champaign County was reduced more than 4-fold relative to expected. This case study shows that fast/frequent testing and other interventions mitigated transmission of SARS-CoV-2 at a large public university.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2021

The Exascale Framework for High Fidelity coupled Simulations (EFFIS): Enabling whole device modeling in fusion science

We present the Exascale Framework for High Fidelity coupled Simulations (EFFIS), a workflow and code coupling framework developed as part of the Whole Device Modeling Application (WDMApp) in the Exascale Computing Project. EFFIS consists of a library, command line utilities, and a collection of run-time daemons. Together, these software products enable users to easily compose and execute workflows that include: strong or weak coupling, in situ (or offline) analysis/visualization/monitoring, command-and-control actions, remote dashboard integration, and more. We describe WDMApp physics coupling cases and computer science requirements that motivate the design of the EFFIS framework. Furthermore, we explain the essential enabling technology that EFFIS leverages: ADIOS for performant data movement, PerfStubs/TAU for performance monitoring, and an advanced COUPLER for transforming coupling data from its native format to the representation needed by another application. Finally, we demonstrate EFFIS using coupled multi-simulation WDMApp workflows and exemplify how the framework supports the project’s needs. We show that EFFIS and its associated services for data movement, visualization, and performance collection does not introduce appreciable overhead to the WDMApp workflow and that the resource-dominant application’s idle time while waiting for data is minimal.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Highly Extensible X-ray Diffraction Toolkit

This code has been previous released as open source under LLNL-CODE-529294. The HEXRD program comprises a library of X-ray diffraction analysis tools and a stand-alone GUI application. While there are many software packages that provide similar capabilities, HEXRD was designed to provide an extremely flexible and extensible basis for describing a broad array of X-ray diffraction instruments; this includes both mono- and poly-chromatic modalities as well as poly- and single-crystal samples. The high-level objects include Material: an abstraction for defining crystalline materials, including unit cell parameters and anisotropic elastic moduli Imageseries: an interface for series of diffraction images containing one or more frames, including provisions for metadata. Instrument: an abstraction of an X-ray diffraction instrument including support for the definition of multiple detector elements as well as powder, Laue, and mono-chromatic single/multi-grain diffraction analysis modalities (including attendant calibration routines). The hexrdgui package is a standalone PySide2/Qt5 GUI built on top of hexrd. The 0.8.1 release includes various bug fixes as well as some new functionality in the GUI, including: o Enhancements to the import tools for LLNL diffraction instruemtns (PXRDIP and TARDIS) o The ability to save 1-d LeBail fits of integrated diffraction spectra.

Barton, NathanR.↗
DOE OSTI2020

Open Chemistry, JupyterLab, REST, and quantum chemistry

Quantum chemistry must evolve if it wants to fully leverage the benefits of the internet age, where the worldwide web offers a vast tapestry of tools that enable users to communicate and interact with complex data at the speed and convenience of a button press. The Open Chemistry project has developed an open-source framework that offers an end-to-end solution for producing, sharing, and visualizing quantum chemical data interactively on the web using an array of modern tools and approaches. These tools build on some of the best open-source community projects such as Jupyter for interactive online notebooks, coupled with 3D accelerated visualization, state-of-the-art computational chemistry codes including NWChem and Psi4, and emerging machine learning and data mining tools such as ChemML and ANI. They offer flexible formats to import and export data, along with approaches to compare computational and experimental data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗