Offline POMDP solver computing an upper bound of the value function in a discretized belief space.
Author JuliaPOMDP
3 Stars
Updated Last
2 Years Ago
Started In
September 2019


Build Status codecov Coverage Status

An offline POMDP solver from "Computationally Feasible Bounds for Partially Observed Markov Decision Processes" (1991), by W. S. Lovejoy. It computes an upper bound on the value function by performing value iteration on a discretized belief space.


Start Julia and make sure you have the JuliaPOMDP registry:

import POMDPs

Then install using the standard package manager:

using Pkg; Pkg.add("BeliefGridValueIteration")


using POMDPs
using POMDPModels # for the tiger pomdp problem
using BeliefGridValueIteration

pomdp = TigerPOMDP()

solver = BeliefGridValueIterationSolver(m = 2, verbose=true)

policy = solve(solver, pomdp)

# Evaluate the value at a given belief point
b0 = [0.5, 0.5]
value(policy, b0)


Solver Options:

  • m::Int64 = 1 Granularity of the belief grid for the triangulation
  • precision::Float64 = 0.0 The solver stops when the desired convergence precision is reached
  • max_iterations::Int64 = 100 Number of iteration of value iteration
  • verbose::Bool = false whether or not the solver prints information


This should return a list of the following functions to be implemented for your POMDP to be solved by this solver:

@requirements_info BeliefGridValueIterationSolver() YourPOMDP()


The authors thank Tim Wheeler and Mykel Kochenderfer for providing a starter implementation of this code.

Used By Packages

No packages found.