Checkpoints
Checkpoints.jl allows packages to register checkpoints which can serialize objects to disk
during the execution of an application program, if the application program configures them.
A minimal working example consists of the package:
module MyPackage
using Checkpoints
MODULE = "MyPackage"
__init__() = Checkpoints.register(MODULE, ["foo", ])
function foo(x)
with_checkpoint_tags(:foo1 => 1, :foo2 => 2) do
checkpoint(MODULE, "foo", :data => 2x)
end
return 2x
end
endand the application program:
using Checkpoints
Checkpoints.config("MyPackage.foo", "./path/to/checkpoints")
for i in 1:2
with_checkpoint_tags(:iteration => i) do
MyPackage.foo(1.0)
end
endwhich results in recorded checkpoints at
./path/to/checkpoints/iteration=1/foo1=1/foo2=2/MyPackage/foo.jlso
./path/to/checkpoints/iteration=2/foo1=1/foo2=2/MyPackage/foo.jlso
You can use index_checkpoint_files to get an index of the files, which is a Tables.jl table and so can e.g. be passed to DataFrame (and then you can do things like groupby etc):
julia> using DataFrames
julia> DataFrame(index_checkpoint_files("./path/to/checkpoints/"))
2×6 DataFrame
Row │ prefixes checkpoint_name iteration foo1 foo2 checkpoint_path
│ Tuple… SubString… SubString… SubString… SubString… PosixPath…
─────┼────────────────────────────────────────────────────────────────────────────────────────────────────────
1 │ ("MyPackage",) foo 1 1 2 ./path/to/checkpoints/iteration=…
2 │ ("MyPackage",) foo 2 1 2 ./path/to/checkpoints/iteration=…or worked with directly:
julia> [checkpoint_path(out) for out in index_checkpoint_files("./path/to/checkpoints/") if out.iteration=="1"]
1-element Array{FilePathsBase.PosixPath,1}:
p"./path/to/checkpoints/iteration=1/foo1=1/foo2=2/MyPackage/foo.jlso"