Installation Instructions For HPOlib

First:

git clone https://github.com/automl/HPOlib.git

Installing inside an virtualenv

  1. Get virtualenv, then load a freshly created virtualenv. (If you are not familiar with virtualenv, you might want to read more about it)

    pip install virtualenv
    virtualenv virtualHPOlib
    source virtualHPOlib/bin/activate
    
  1. Install numpy, scipy, matplotlib, as this doesn’t work through setup.py.

    pip install numpy
    pip install scipy
    pip install matplotlib
    

    This may take some time. Afterwards you can verify having those libs installed with:

    pip freeze
    
  2. run setup.py

    python setup.py install
    

    This will install HPOlib and some requirements (networkx, protobuf, pymongo). Be sure your system is connected to the internet, so setup.py can download optimizer and runsolver code. Your environment now looks like that

    pip freeze
    
        HPOlib==0.0.1
        argparse==1.2.1
        backports.ssl-match-hostname==3.4.0.2
        distribute==0.7.3
        matplotlib==1.3.1
        networkx==1.8.1
        nose==1.3.0
        numpy==1.8.0
        protobuf==2.5.0
        pymongo==2.6.3
        pyparsing==2.0.1
        python-dateutil==2.2
        scipy==0.13.3
        six==1.5.2
        tornado==3.2
        wsgiref==0.1.2
    

    and

    ls optimizers/smac
        smac_2_10_00-dev_parser.py   smac_2_10_00-dev.py   smac_2_10_00-dev_src    smac_2_10_00-devDefault.cfg
    
  3. You can now run, e.g. smac with 200 evaluations on the branin function:

    cd benchmarks/branin
    HPOlib-run -o ../../optimizers/smac/smac_2_10_00-dev -s 23
    

    This takes depending on your machine ~2 minutes. You can now plot the results of your first experiment:

    HPOlib-plot FIRSTRUN smac_2_10_00-dev_23_*/smac_*.pkl -s `pwd`/Plots/
    

    You can test the other optimizers (spearmint will take quite longer 30min):

    HPOlib-run -o ../../optimizers/tpe/h -s 23
    HPOlib-run -o ../../optimizers/spearmint/spearmint_april2013 -s 23
    

    and again:

    HPOlib-plot SMAC smac_2_10_00-dev_23_*/smac_*.pkl TPE hyperopt_august2013_mod_23_*/hyp*.pkl SPEARMINT spearmint_april2013_mod_23_*/spear*.pkl -s `pwd`/Plots/
    

    and to check the general performance on this super complex benchmark:

    HPOlib-plot BRANIN smac_2_10_00-dev_23_*/smac_*.pkl hyperopt_august2013_mod_23_*/hyp*.pkl spearmint_april2013_mod_23_*/spear*.pkl -s `pwd`/Plots/
    

Problems during installation

python setup.py crashes with ImportError: cannot import name Feature during installing pymongo. This happens due to pymongo using a deprecated feature :python:’Feature’, which is not available in the setuptools version (>2.2). This error is fixed, but not yet available on PYPI.

Solution: Downgrade setuptools with pip install setuptools==2.2 and try again or install pymongo manually.

Fork me on GitHub

Updating optimizers We also provide an updated and adjusted version of spearmint. To also install this version do:

cd optimizers
rm spearmint_gitfork_mod_src
git clone https://github.com/automl/spearmint.git
mv spearmint spearmint_gitfork_mod_src