Newer
Older
`hotwheels` is a codebase for hydrodynamic N-body cosmological simulations, developed by **Antonio Ragagnin**.
As part of my commitment to advancing simulation tools, I am working on a flexible and modular implementation of a [Gadget](https://wwwmpa.mpa-garching.mpg.de/gadget/)-like code (temporarily named `hotwheels`). This new implementation incorporates lessons learned from over a decade of experience working with HPC and Gadget codes (e.g., OpenGadget3, see Dolag et al., in prep).
`hotwheels` is designed to leverage CPU and GPU parallelism, with a strong emphasis on modularity. This modular approach facilitates collaboration with HPC engineers and GPU vendors, enabling efficient utilization of HPC systems. Modularity also enhances testability, simplifies the addition of new features, and allows rethinking code structure based on past experiences.
- **Modularity:** Each module is self-contained, adhering to the [Open-Closed Principle](https://en.wikipedia.org/wiki/Open%E2%80%93closed_principle). `hotwheels` consists of C libraries wrapped in Python for high-level processing, following the successful paradigm of machine learning frameworks ([reference](https://www.mdpi.com/2078-2489/11/4/193)). Developers can focus solely on their module’s files without altering other parts of the code.
- **Mini-Apps:** From over 10 years of experience collaborating with HPC engineers and GPU vendors, I found it crucial to deploy the code as minimal, standalone applications. This approach allows engineers to focus on improving code quality and porting it to different architectures without being distracted by unrelated components.
- **Testability:** A robust testing framework is a priority in `hotwheels`. Each module is isolated and can be independently tested, leveraging the modular structure to facilitate efficient debugging and verification.
- **Parallelism:** Recognizing the dominance of **GPUs** in HPC, especially driven by AI advancements, `hotwheels` is designed from scratch to run efficiently on GPUs, multi-core CPUs, and MPI-parallel systems.
- **Documentation:** Navigation through a large codebase can be challenging. `hotwheels` prioritizes comprehensive documentation, including high-level overviews and detailed API references for each component. Documentation is generated automatically, and commits to the `main` branch are rejected if they fail the indentation and documentation pipelines.
## Modules
- [core](https://www.ict.inaf.it/gitlab/hotwheels/core): Provides essential utilities for: (1) Compiling and executing C code from Python (`core.make`); (2) Generating and managing particle data structures as structures of arrays (SoAs) (`core.soa`); (3) General C utilities (e.g., MPI patterns, MyMalloc, panic handling, and timers in `core.c_utils`).
- [IO](https://www.ict.inaf.it/gitlab/hotwheels/io): Handles parallel reading and writing of Gadget1-2 snapshots (thus including [N-GenIC](https://www.h-its.org/2014/11/05/ngenic-code/) and [MUSIC](https://www-n.oca.eu/ohahn/MUSIC/) formats), as well as HDF5 files.
- [tree](https://www.ict.inaf.it/gitlab/hotwheels/tree): A standalone octree implementation designed for GPU compatibility. It supports Hilbert or Morton ordering, multi-particle leafs, and can be used in standalone C projects.
- [domain](https://www.ict.inaf.it/gitlab/hotwheels/domain): A GPU-friendly domain decomposition module based on Hilbert particle ordering.
- [PM](https://www.ict.inaf.it/gitlab/hotwheels/pm): A standalone particle-mesh (PM) module, heavily inspired by the Gadget PM implementation.
- [integrate](https://www.ict.inaf.it/gitlab/hotwheels/integrate): Provides APIs and callbacks for cosmological and non-cosmological integration. This module also manages cosmological parameters.
- [barnes_hut](https://www.ict.inaf.it/gitlab/hotwheels/barnes_hut): **[To-Do]** A Gadget-like short-range gravity and potential computation module, based on the [Barnes & Hut (1986)](https://www.nature.com/articles/324446a0) algorithm.
- [FoF](https://www.ict.inaf.it/gitlab/hotwheels/fof): **[To-Do]** a simple stand-alone C friend-of-friend (FoF) halo identification.
- [PESPH](https://www.ict.inaf.it/gitlab/hotwheels/PESPH): **[To-Do]** A pressure-entropy smoothed particle hydrodynamics (SPH) module.
- [SH03](https://www.ict.inaf.it/gitlab/hotwheels/SH03): **[To-Do]** A star formation recipe based on [Springel & Hernquist (2003)](https://ui.adsabs.harvard.edu/abs/2003MNRAS.339..289S/abstract), with cooling processes handled by [Grackle](https://grackle.readthedocs.io/en/latest/).
Currently it is possible to run particle-mesh gravity only simulations with and without comoving integrations:
- [A refined PM DMO run with fixed timesteps](run_pm_dmo_NFW_fixed_timestep.md)
- **soon:** A cosmological PM DMO run with fixed timesteps
This code is currently unlicensed. The majority of the code has been written from scratch by Antonio Ragagnin, and redistribution of this code requires explicit authorization from the author.
Certain portions of the code are adapted from GPL-licensed software, and their respective licenses apply to those parts (see code comments). Specifically, this includes:
Gadget code ([Springel 2001](https://ui.adsabs.harvard.edu/abs/2001NewA....6...79S/abstract)), Gadget4 ([Springel et al. 2021](https://ui.adsabs.harvard.edu/abs/2021MNRAS.506.2871S/abstract)), and Blue Tides ([Rupert et al. 2015](https://ui.adsabs.harvard.edu/abs/2015CSE....17b..40C/abstract)).