P4est.jl

P4est.jl is a Julia package that wraps p4est, a C library to manage multiple connected adaptive quadtrees or octrees in parallel.

P4est.jl mainly provides low-level wrappers of functionality from p4est. The Julia package P4estTypes.jl builds on top of P4est.jl and provides a high-level interface.

Installation

If you have not yet installed Julia, please follow the instructions for your operating system. P4est.jl works with Julia v1.6 and later.

P4est.jl is a registered Julia package. Hence, you can install it by executing the following commands in the Julia REPL:

julia> import Pkg; Pkg.add(["P4est", "MPI"])

With this command, you install both P4est.jl and MPI.jl. Currently, P4est.jl supports only builds of the C library p4est with MPI support, so you need to initialize MPI appropriately as described in the Usage section below.

P4est.jl depends on the binary distribution of the p4est library, which is available in the Julia package P4est_jll.jl and which is automatically installed as a dependency. The binaries provided by P4est_jll.jl support MPI and are compiled against the default MPI binaries of MPI.jl. At the time of writing, these are the binaries provided by MicrosoftMPI_jll.jl on Windows and MPICH_jll.jl on all other platforms.

By default, P4est.jl provides pre-generated Julia bindings to all exported C functions of the underlying p4est library. If you want/need to generate new bindings, please follow the instructions in the dev folder and copy the generated files to the appropriate places in src.

Using a custom version of MPI and/or p4est

The C library p4est needs to be compiled against the same MPI implementation used by MPI.jl. Thus, if you want to configure MPI.jl to not use the default MPI binary provided by JLL wrappers, you also need to build the C library p4est locally using the same MPI implementation. This is typically the situation on HPC clusters. If you are just using a single workstation, the default installation instructions should be sufficient.

P4est.jl allows using a p4est binary different from the default one provided by P4est_jll.jl. To enable this, you first need to obtain a local binary installation of p4est. Next, you need to configure MPI.jl to use the same MPI implementation used to build your local installation of p4est, see the documentation of MPI.jl. At the time of writing, this can be done via

julia> using MPIPreferences

julia> MPIPreferences.use_system_binary()

if you use the default system MPI binary installation to build p4est.

Next, you need to set up the Preferences.jl setting containing the path to your local build of the shared library of p4est.

julia> using Preferences, UUIDs

julia> set_preferences!(
           UUID("7d669430-f675-4ae7-b43e-fab78ec5a902"), # UUID of P4est.jl
           "libp4est" => "/path/to/your/libp4est.so", force = true)

On Windows you also need to set the path to the local build of the shared library of libsc, which is a subpackage of p4est. On other systems, this is not necessary as the library is already linked by libp4est.so, but it can be used to employ also a custom build of libsc.

julia> set_preferences!(
           UUID("7d669430-f675-4ae7-b43e-fab78ec5a902"), # UUID of P4est.jl
           "libsc" => "/path/to/your/libsc.so", force = true)

Note that you should restart your Julia session after changing the preferences. To sum up, follow these steps to use P4est.jl with a custom installation of the underlying C libraries.

• Create a Julia project for your setup.

  julia> import Pkg; Pkg.activate(".")

  This uses the Julia project in your current working directory or creates a new one if there is none.
• Install the required packages.

  julia> Pkg.add(["MPIPreferences", "MPI", "UUIDs", "Preferences", "P4est"])

• Set MPI.jl preferences.

  julia> using MPIPreferences
  
  julia> MPIPreferences.use_system_binary()

• Set P4est.jl preferences.

  julia> using Preferences, UUIDs
  
  julia> set_preferences!(
             UUID("7d669430-f675-4ae7-b43e-fab78ec5a902"), # UUID of P4est.jl
             "libp4est" => "/path/to/your/libp4est.so", force = true)
  
  julia> set_preferences!(
             UUID("7d669430-f675-4ae7-b43e-fab78ec5a902"), # UUID of P4est.jl
             "libsc" => "/path/to/your/libsc.so", force = true)

• Restart the Julia REPL and load the packages.

  julia> import Pkg; Pkg.activate(".")
  
  julia> using P4est, MPI; MPI.Init()

Currently, custom builds of p4est without MPI support are not supported.

Usage

The P4est.uses_mpi() function can be used to check if the p4est binaries that P4est.jl uses were compiled with MPI enabled. This returns true for the default binaries provided by the P4est_jll.jl package. In this case P4est.jl can be used as follows.

In the Julia REPL, first load the packages P4est.jl and MPI.jl in any order and initialize MPI.

julia> using P4est, MPI; MPI.Init()

You can then access the full p4est API that is defined by the headers. For example, to create a periodic connectivity and check its validity, execute the following lines:

julia> using P4est, MPI; MPI.Init()

julia> connectivity = p4est_connectivity_new_periodic()
Ptr{p4est_connectivity} @0x0000000002412d20

julia> p4est_connectivity_is_valid(connectivity)
1

julia> p4est = p4est_new_ext(MPI.COMM_WORLD, connectivity, 0, 2, 0, 0, C_NULL, C_NULL)
Into p4est_new with min quadrants 0 level 2 uniform 0
New p4est with 1 trees on 1 processors
Initial level 2 potential global quadrants 16 per tree 16
Done p4est_new with 10 total quadrants
Ptr{p4est} @0x0000000002dd1fd0

julia> p4est_obj = unsafe_load(p4est)
P4est.LibP4est.p4est(1140850688, 1, 0, 0, 0x0000000000000000, Ptr{Nothing} @0x0000000000000000, 0, 0, 0, 10, 10, Ptr{Int64} @0x00000000021a5f70, Ptr{p4est_quadrant} @0x0000000002274330, Ptr{p4est_connectivity} @0x000000000255cdf0, Ptr{sc_array} @0x00000000023b64a0, Ptr{sc_mempool} @0x0000000000000000, Ptr{sc_mempool} @0x00000000023b1620, Ptr{p4est_inspect} @0x0000000000000000)

julia> p4est_obj.connectivity == connectivity
true

julia> connectivity_obj = unsafe_load(p4est_obj.connectivity)
p4est_connectivity(4, 1, 1, Ptr{Float64} @0x00000000021e8170, Ptr{Int32} @0x00000000020d2450, 0x0000000000000000, Cstring(0x0000000000000000), Ptr{Int32} @0x0000000002468e10, Ptr{Int8} @0x00000000022035e0, Ptr{Int32} @0x0000000002667230, Ptr{Int32} @0x000000000219eea0, Ptr{Int32} @0x000000000279ae00, Ptr{Int8} @0x00000000021ff910)

julia> connectivity_obj.num_trees
1

julia> p4est_destroy(p4est)

julia> p4est_connectivity_destroy(connectivity)

As shown here, unsafe_load allows to convert pointers to p4est C structs to the corresponding Julia wrapper type generated by Clang.jl. They follow the basic C interface of Julia.

If you start Julia on multiple MPI ranks, you can check whether everything is set up correctly by the following extended smoke test. Copy the following code to a local file:

using P4est, MPI; MPI.Init()
connectivity = p4est_connectivity_new_periodic()
p4est = p4est_new_ext(MPI.COMM_WORLD, connectivity, 0, 2, 0, 0, C_NULL, C_NULL)
p4est_obj = unsafe_load(p4est)
MPI.Barrier(MPI.COMM_WORLD)
rank = MPI.Comm_rank(MPI.COMM_WORLD)
@info "Setup" rank p4est_obj.local_num_quadrants p4est_obj.global_num_quadrants
p4est_destroy(p4est)
p4est_connectivity_destroy(connectivity)

Run Julia with the MPI implementation used by MPI.jl and execute the script shown above. You should get an output like

Into p4est_new with min quadrants 0 level 2 uniform 0
New p4est with 1 trees on 2 processors
Initial level 2 potential global quadrants 16 per tree 16
Into p4est_new with min quadrants 0 level 2 uniform 0
New p4est with 1 trees on 2 processors
Initial level 2 potential global quadrants 16 per tree 16
Done p4est_new with 13 total quadrants
Done p4est_new with 13 total quadrants
┌ Info: Setup
│   rank = 0
│   p4est_obj.local_num_quadrants = 5
└   p4est_obj.global_num_quadrants = 13
┌ Info: Setup
│   rank = 1
│   p4est_obj.local_num_quadrants = 8
└   p4est_obj.global_num_quadrants = 13

To suppress the relatively verbose output of p4est in the example above, you can call P4est.init(C_NULL, SC_LP_ERROR) before calling other functions from p4est.

Many functions and types in p4est have been documented with comments by the p4est authors; you can access this documentation as you would for any Julia-native entity through ?:

help?> p4est_memory_used
search: p4est_memory_used p4est_mesh_memory_used p4est_ghost_memory_used p4est_connectivity_memory_used

  p4est_memory_used(p4est_)

  Calculate local memory usage of a forest structure. Not collective. The memory used on the current rank is
  returned. The connectivity structure is not counted since it is not owned; use
  p4est_connectivity_memory_usage (p4est->connectivity).

  Parameters
  ––––––––––––

    •  p4est:[in] Valid forest structure.

  Returns
  –––––––––

  Memory used in bytes.

  Prototype
  –––––––––––

  size_t p4est_memory_used (p4est_t * p4est);

The same is of course true for the higher-level interface functions provided by P4est.jl, e.g.,

help?> P4est.init
  P4est.init(log_handler, log_threshold)

  Calls p4est_init if it has not already been called, otherwise do nothing. Thus, P4est.init can safely be
  called multiple times.

  To use the default log handler and suppress most output created by default by p4est, call this function as

  P4est.init(C_NULL, SC_LP_ERROR)

  before calling other functions from p4est.

For more information on how to use p4est, please refer to the documentation for p4est itself or to the header files (*.h) in the p4est repository.

For more information on how to use the Julia wrapper P4est.jl, please consult the documentation.

Authors

P4est.jl is mainly maintained by Michael Schlottke-Lakemper (RWTH Aachen University/High-Performance Computing Center Stuttgart (HLRS), Germany) and Hendrik Ranocha (University of Hamburg, Germany). The full list of contributors can be found in AUTHORS.md. The p4est library itself is written by Carsten Burstedde, Lucas C. Wilcox, and Tobin Isaac.

License and contributing

P4est.jl is licensed under the MIT license (see LICENSE.md). p4est itself is licensed under the GNU General Public License, version 2.