IncrementalPruning

This Julia package implements the incremental pruning solver for partially observable Markov decision processes.

Installation

using Pkg
# Pkg.Registry.add("https://github.com/JuliaPOMDP/Registry) # for julia 1.1+

# for julia 1.0 add the registry throught the POMDP package
# Pkg.add("POMDPs")
# using POMDPs
# POMDPs.add_registry() 
Pkg.add("IncrementalPruning")

Usage

using IncrementalPruning
using POMDPModels
pomdp = TigerPOMDP() # initialize POMDP

solver = PruneSolver() # set the solver

policy = solve(solver, pomdp) # solve the POMDP

The result of solve is a Policy that contains the alpha vectors of the solution.

IncrementalPruning.jl solves problems implemented using the POMDPs.jl interface. See the documentation for POMDPs.jl for more information.

Algorithm Details

This solver implements the incremental pruning algorithm as described in Zhang and Liu (1996) and Cassandra et al. (1997). This solution method is exact (ϵ-optimal) but is much slower than modern approximate solution techniques. As such, it is only computationally feasible for small problems.

References

Cassandra, A., Littman, M., & Zhang, N. (1997). Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes. Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI-97), 54–61.

Zhang N. L., Liu W. (1996). Planning in stochastic domains: Problem characteristics and approximation. Technical Report HKUST-CS96-31, Hong Kong University of Science and Technology.