Software on this page is provided in the hope that some people will find them
useful. Of course, use all code at your own risk, etc.All code here is
LGPLd.
Critics, bug-reports, and suggestions to be sent to:
RLFAPP:
Value Function Approximation methods in Reinforcement Learning 0.01 - C++
code
Code used to compare Soft State Aggregation (Singh
et. al, NIPS94 paper), Kernel-Based Reinforcement
Learning (Ormoneit and
Sen, TR version of Machine Learning paper here)
and Interpolation-based Reinforcement Learning algorithms introduced
in my
ICML'04 paper written together with Bill
Smart. Simulations run on the Mountain car
domain, other domains should not be that hard to add, though the code is not
(yet) in a shape that would make this particularly easy.. The main goal was
simply to provide efficient (well, not very inefficient) implementations of
the 3 algorithms to make the comparisions.The code is provided here for
people interested in the implementation details or who'd like to extend the
scope of experiments, etc.
Developers: Bill Smart and Csaba Szepesvári
Funspec - Octave
code.
Allows inline function specifications for octave. Imagine for example that you want to find the root of
y = -2*x^2 + 3*x + 4*sin(x) - 6;
Then you simply type
[x, info] = fsolve (funspec("x","y","y=-2*x**2+3*x+4*sin(x)-4")
What does it do? funspec defines a temporary function that can be passed to other
octave functions.
Download in tar.gz or zip
format! Due to popular demand, matlab "versions" are found here: tar.gz,
zip. The matlab 'version' uses 'end' uniformly instead of the more
expressive 'endif','endwhile', etc. Otherwise it is the same (I could not test
it, but some people did - they have also corrected the numeruous
incompatibility problems!)
Monte-Carlo
package - Octave code
The idea is to have a number of generic Monte-Carlo based sampling and
integration routines that can accept functions and return the results. Note
that this
is an experimental version (say version 0.1) with a rather limited
functionality (but it worked for me). Main functionality is provided by the
following methods:
impint - integration using importance sampling
impint_stat - tests impint
rejection_rnd - rejection sampling
rejection_rnd_test - tests the previous method on two problems; some graphics.
You will need the statistics package of octave-forge
installed on your system and the above Funspec
package in order to make this package work. Funspec is used only in
rejection_rnd_test. The plan is to extend this package to other sampling
methods. Contributions are warmly welcome!
Download in tar.gz or zip
format! Matlab versions: tar.gz,
zip. The matlab version needs the
statistics package.
My (Biased) Doxygen
Quick Reference Guide.
It is only 2 pages long! You can get it in pdf format. Versions: Single
page, 2 pages.
Last update: 05/28/2004