Release Notes for Mesh-based Monte Carlo 2.0 (v2023)

code name: Moon Cake beta, released on September 28, 2023

Click this link to download MMC/MMCLAB v2023

Acknowledgement: This software release is made possible with the funding support from the NIH/NIGMS under grant R01-GM114365.

1. What's New
2. Introduction
3. System requirements
4. Known issues
5. Reference

1. What's New

MMC v2023 (2.0.0) includes a major feature upgrade - implicit MMC (iMMC) - to enable simulations of highly complex anatomical structures. The details of the iMMC algorithm, developed by Yaoshen Yuan, are described in his recent paper [Yuan2021]. Briefly, iMMC permits one to associate cylindrical inclusions, spherical inclusions, and thin-membrane with the edges, vertices and faces, respectively, in a tetrahedral mesh. Currently, this feature is only supported on the CPU (must be used with -G -1 or cfg.gpuid=-1).

In 2020, we have also merged the mmc-trinity branch to the main code tree. The "trinity" version of MMC simultaneously supports CPU/SSE4, OpenCL and CUDA as the compute-backend, and can be selected using -c/--compute opencl|sse|cuda command line flag. By default, we do not compile the CUDA editor in the binary. However, users can compile the "trinity" binary by make cuda or cmake -DBUILD_CUDA=on.

Finally, in the release, we upgraded the normalization formula for nodal-based MMC simulation on the mesh surface nodes to correct for an observed deviation from MCX and DMMC outputs. This update only affects outputs on the exterior surface nodes, and does not impact DMMC (-M G, default for GPU based MMC) or element based (-M S) MMC.

A detailed (long) list of updates can be found in the ChangeLog.txt or the Github commit history: <https://github.com/fangq/mmc/commits/master>;

To highlight a few most important updates:

  • Support iMMC (edge-, node- and face-based implicit structures), CPU only (by Yaoshen Yuan)
  • Added MMCLAB examples for iMMC, including a large dense vessel network (by Yaoshen Yuan)
  • About 30%-40% faster OpenCL based simulations on NVIDIA GPUs using PTX-based atomic operations
  • More accurate normalization of nodal-based MMC simulation on the exterior surface
  • Provide mmc-trinity binaries with simultaneous SSE/OpenCL/CUDA backend support
  • CMake based portable compilation system
  • Added GNU Octave mex file for MMCLAB on Linux/MacOS/Windows
  • Several critical bug fixes (by Shijie Yan)
  • Github Action based continuous integration (CI) system for automated building/packaging/testing

Please file bug reports to <https://github.com/fangq/mmc/issues>;

Reference:

  1. [Fang2019] Qianqian Fang* and Shijie Yan, "GPU-accelerated mesh-based Monte Carlo photon transport simulations," J. of Biomedical Optics, 24(11), 115002 (2019) URL: http://dx.doi.org/10.1117/1.JBO.24.11.115002

  1. [Brain2Mesh2020] Anh Phong Tran† , Shijie Yan† , Qianqian Fang*, (2020) "Improving model-based fNIRS analysis using mesh-based anatomical and light-transport models," Neurophotonics, 7(1), 015008, URL: https://doi.org/10.1117/1.NPh.7.1.015008

  1. [Yan2020] Yan S, Yao R, Intes X, and Fang Q*, "Accelerating Monte Carlo modeling
of structured-light-based diffuse optical imaging via 'photon sharing'," Opt. Lett. 45, 2842-2845 (2020) URL: https://www.biorxiv.org/content/10.1101/2020.02.16.951590v2

  1. [Yuan2021] Yaoshen Yuan, Shijie Yan, and Qianqian Fang, "Light transport modeling in highly complex tissues using the implicit mesh-based Monte Carlo algorithm," Biomed. Optics Express, 12(1), 147-161, (2021), URL: https://www.osapublishing.org/boe/fulltext.cfm?uri=boe-12-1-147

2. Introduction

Mesh-based Monte Carlo (MMC) is a 3D Monte Carlo (MC) simulation software for photon transport in complex turbid media. MMC combines the strengths of the MC-based technique and the finite-element (FE) method: on the one hand, it can handle general media, including low-scattering ones, as in the MC method; on the other hand, it can use an FE-like tetrahedral mesh to represent curved boundaries and complex structures, making it even more accurate, flexible, and memory efficient. MMC uses the state-of-the-art ray-tracing techniques to simulate photon propagation in a mesh space. It has been extensively optimized for excellent computational efficiency and portability. MMC currently supports multi-threaded parallel computing via OpenMP, Single Instruction Multiple Data (SIMD) parallelism via SSE and, starting from v2019.10, OpenCL to support a wide range of CPUs/GPUs from nearly all vendors.

To run an MMC simulation, one has to prepare an FE mesh first to discretize the problem domain. Image-based 3D mesh generation has been a very challenging task only until recently. One can now use a powerful yet easy-to-use mesh generator, iso2mesh [1], to make tetrahedral meshes directly from volumetric medical images. You should download and install the latest iso2mesh toolbox in order to run the build-in examples in MMC.

The most relevant publication describing this work is the GPU-accelerated MMC paper:

Qianqian Fang and Shijie Yan, "GPU-accelerated mesh-based Monte Carlo photon transport simulations," J. of Biomedical Optics, in press, 2019. Preprint URL: https://www.biorxiv.org/content/10.1101/815977v1

Please keep in mind that MMC is only a partial implementation of the general Mesh-based Monte Carlo Method (MMCM). The limitations and issues you observed in the current software will likely be removed in the future version of the software. If you plan to perform comparison studies with other works, please communicate with the software author to make sure you have correctly understood the details of the implementation.

The details of MMCM can be found in the following paper:

Qianqian Fang, "Mesh-based Monte Carlo method using fast ray-tracing in Plücker coordinates," Biomed. Opt. Express 1, 165-175 (2010) URL: https://www.osapublishing.org/boe/abstract.cfm?uri=boe-1-1-165

While the original MMC paper was based on the Plücker coordinates, a number of more efficient SIMD-based ray-tracers, namely, Havel SSE4 ray-tracer, Badouel SSE ray-tracer and branchless-Badouel SSE ray-tracer (fastest) have been added since 2011. These methods can be selected by the -M flag. The details of these methods can be found in the below paper

Qianqian Fang and David R. Kaeli, "Accelerating mesh-based Monte Carlo method on modern CPU architectures," Biomed. Opt. Express 3(12), 3223-3230 (2012) URL: https://www.osapublishing.org/boe/abstract.cfm?uri=boe-3-12-3223

and their key differences compared to another mesh-based MC simulator, TIM-OS, are discussed in

Qianqian Fang, "Comment on 'A study on tetrahedron-based inhomogeneous Monte-Carlo optical simulation'," Biomed. Opt. Express, vol. 2(5) 1258-1264, 2011. URL: https://www.osapublishing.org/boe/abstract.cfm?uri=boe-2-5-1258

In addition, the generalized MMC algorithm for wide-field sources and detectors are described in the following paper, and was made possible with the collaboration with Ruoyang Yao and Prof. Xavier Intes from RPI

Yao R, Intes X, Fang Q, "Generalized mesh-based Monte Carlo for wide-field illumination and detection via mesh retessellation," Biomed. Optics Express, 7(1), 171-184 (2016) URL: https://www.osapublishing.org/boe/abstract.cfm?uri=boe-7-1-171

In addition, we have been developing a fast approach to build the Jacobian matrix for solving inverse problems. The technique is called "photon replay", and is described in details in the below paper:

Yao R, Intes X, Fang Q, "A direct approach to compute Jacobians for diffuse optical tomography using perturbation Monte Carlo-based photon 'replay'," Biomed. Optics Express, in press, (2018)

In 2019, we published an improved MMC algorithm, named "dual-grid MMC", or DMMC, in the below JBO Letter. This method allows to use separate mesh for ray-tracing and fluence storage, and can be 2 to 3 fold faster than the original MMC without loss of accuracy.

Shijie Yan, Anh Phong Tran, Qianqian Fang*, "A dual-grid mesh-based Monte Carlo algorithm for efficient photon1transport simulations in complex 3-D media," J. of Biomedical Optics, 24(2), 020503 (2019).

The authors of the papers are greatly appreciated if you can cite the above papers as references if you use MMC and related software in your publication.

3. System requirements

Although MMC/MMCLAB can be executed independently if all input data (including mesh data) are provided, most pre- and post-processing scripts in both packages require the installation of Iso2Mesh Toolbox - a MATLAB based 3D mesh generator. Please browse http://iso2mesh.sf.net for download and installation information.

By default, MMC/MMCLAB uses OpenCL-based simulations to utilize all GPUs and CPUs installed on your system. If you have a GPU (NVIDIA, AMD or Inel), the OpenCL support is typically installed if you have correctly installed the latest version of the graphics driver. Please verify that the OpenCL library (libOpenCL.so* on Linux, OpenCL.dll on Windows or /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL on the Mac) must exist in your system.

Generally speaking, AMD and NVIDIA high-end dedicated GPU performs the best, about 20-60x faster than a multi-core CPU; Intel's integrated GPU is about 3-4 times faster than a multi-core CPU.

In addition, MMC has been fully tested with the open-source OpenCL runtime pocl (http://portablecl.org/) on the CPU. To install pocl on a Ubuntu/Debian system, please run 

  sudo apt-get install pocl-opencl-icd

If you happen to use a system that does not have OpenCL support, you may still use MMC/MMCLAB by calling it in the CPU-only SSE4 mode, with -G -1 or cfg.gpuid=-1. To verify your CPU supports SSE4, please run 

  grep 'sse4' /proc/cpuinfo

or using a freeware "CPU-Z" on windows. If you attempt to run the SSE4 on an unsupported computer, you will get an error when executing the binary. In that case, you should recompile MMC using "make omp" command.

4. Known issues

  • One must use savemmcmesh.m to generate all mesh input files for MMC; otherwise, the element orientation is not guaranteed.
  • Currently, this code only supports element-based optical properties; nodal-based optical properties (for continuously varying media) will be added in a future release
  • The maximum photon number per MMC session is 2^32-1=4,294,967,295, if you need to run a large number of photons, we suggest you splitting the job into smaller jobs with each one running a smaller chunk of photons (for example 1E8). Each job should initialized with a different seed (-E).

5. Reference

Powered by Habitat