Installing Dependencies

Dependencies

Required: g++ and bash
Filtering and Estimation: Boost, zlib, and cmake
Log-linear Interpolation (rare): Eigen3 and OpenMP support in g++
Optional: bzip2 and xz

To install dependencies, choose one of the options below.

Recommended: Package Managers

If you have root, package managers are easiest. Some distributions separately package compiled libraries and headers (often by appending -dev). Since you will be compiling code from source, both are required.

On Debian/Ubuntu

To get a working compiler, install the build-essential package. Boost is known as libboost-all-dev. The three supported compression options each have a separate dev package.


sudo apt-get install build-essential libboost-all-dev cmake zlib1g-dev libbz2-dev liblzma-dev

On Gentoo

Static libraries are faster; this script will configure static libraries and install dependencies.

cat >> /etc/portage/package.use <<EOF
dev-libs/boost static-libs
sys-libs/zlib static-libs
app-arch/bzip2 static-libs
app-arch/xz-utils static-libs
EOF
emerge boost cmake sys-libs/zlib bzip2 xz-utils

Fallback: Home Directory

Software can be installed in your home directory by compiling from source. I only recommend this method if you do not have root. Since your home directory is not a standard installation path, several environment variables are needed:

PATH: Path to executables searched by the shell (bin).
CPATH: Additional paths that gcc searches for headers at compile time (include).
LIBRARY_PATH: Additional paths that gcc searches for libraries at link time (lib or lib64).
LD_LIBRARY_PATH: On Linux and BSD, the linker searches these paths for shared libraries at runtime (lib or lib64). On OS X, this is called DYLD_LIBRARY_PATH.

Each of these is an ordered list of paths delimited by colon. Note that empty string is sometimes interpreted as the current directory, so it is not correct to do LIBRARY_PATH=$LIBDIR:$LIBRARY_PATH because it may have been empty to begin with. Bash users can add this script to .bashrc (assuming the environment is setup to read it):

#Determine where you want to install packages
PREFIX=$HOME/usr
#If your system has lib64 directories, lib64 should be used instead of lib
if [ -d /lib64 ]; then
  LIBDIR=$PREFIX/lib64
else
  LIBDIR=$PREFIX/lib
fi
#If you're installing to a non-standard path, tell programs where to find things:
export PATH=$PREFIX/bin${PATH:+:$PATH}
export CPATH=$PREFIX/include${CPATH:+:$CPATH}
export LIBRARY_PATH=$LIBDIR${LIBRARY_PATH:+:$LIBRARY_PATH}
if [ "$(uname)" == Darwin ]; then
  export DYLD_LIBRARY_PATH=$LIBDIR${DYLD_LIBRARY_PATH:+:$DYLD_LIBRARY_PATH}
else
  export LD_LIBRARY_PATH=$LIBDIR${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
fi

Remember to configure the running shell as well.

source .bashrc

Packages with `./configure`

For packages based on GNU autotools, use

./configure --prefix=$PREFIX --libdir=$LIBDIR
make -j4
make install

zlib

zlib is optional for queries but required for filtering and estimation. Install before compiling Boost. You may skip this step if you have both libz.a and libz.so in a lib directory.


wget http://zlib.net/zlib-1.2.11.tar.gz

tar xzf zlib-1.2.11.tar.gz

cd zlib-1.2.11

./configure --prefix=$PREFIX --libdir=$LIBDIR

make -j4

make install

make clean

./configure --prefix=$PREFIX --libdir=$LIBDIR --static

make -j4

make install

cd ..

bzip2

bzip2 is optional for KenLM, but required for Boost to compile correctly. It has a Makefile but no ./configure.


wget https://netix.dl.sourceforge.net/project/bzip2/bzip2-1.0.6.tar.gz

tar xzvf bzip2-1.0.6.tar.gz

cd bzip2-1.0.6/

#Compile and install libbz2.a (static library)

make

make install PREFIX=$PREFIX

mkdir -p $LIBDIR

#Note this may be the same file; you can ignore the error

mv $PREFIX/lib/libbz2.a $LIBDIR 2>/dev/null

#Compile and install libbz2.so (dynamic library)

make clean

make -f Makefile-libbz2_so

cp libbz2.so.* $LIBDIR

ln -sf libbz2.so.1.0 $LIBDIR/libbz2.so

cd ..

Boost

Boost requires zlib and bzip2. After those are installed (and most systems have these installed already), compile Boost:


wget https://dl.bintray.com/boostorg/release/1.72.0/source/boost_1_72_0.tar.bz2

tar xjf boost_1_72_0.tar.bz2

cd boost_1_72_0

./bootstrap.sh

#Note that this may fail.  A common cause is incorrectly installed gzip or bzip2.  KenLM does not use the iostreams library, so it might work anyway.  But an incomplete install will break other packages like Moses.

./b2 --prefix=$PREFIX --libdir=$LIBDIR --layout=tagged link=static,shared threading=multi,single install -j4 || echo FAILURE

cd ..