Marmote is a software library and API dedicated to the modeling with Markov Chains. It is written in C++.
It allows to specify the state space and the transitions of a Markov chain and gives access to a large variety of solution methods: Monte Carlo simulation, exact and approximate numerical solutions.
The application package MarmoteMDP is built on top of Marmote: it is dedicated to the modeling and solution of Markov Decision Processes.
Both Marmote and MarmoteMDP are available through this web site which explains how to install the software, and provides numerous examples of programming with Markov Chains and Markov Decision Processes.