Connecting a Windows box to a remote Engine on a GNU+Linux server

What's this page about, who's it for, and the disclaimer

This page is a how-to guide for fusing a windows chess gui, such as chessbase, on a windows computer to a chess engine living on a linux server. The guide is intended for anyone who wants to do that. It's assumed the machines can ping eachother and that wine is installed on the linux server.

Now, the disclaimer:
if the tides of oceans rise, the polarity of the earth changes, if you realize Jupiter is not 6AU from the earth but 6AU from the sun, if reading this guide causes you to daydream about the many meanings of the phrase 'quarter horse', if your dog passes through a puddle and sneezes and this causes the ctrl+alt key combinations to stop working on your laptop IF day of the week is <= 2 && week number is odd, if everything turns out bad and even this does not change that then all i can say is, 'Dear reader may life be as kind to you as it has been to me,' and at once vanish while thinking, you've been warned.

Fundamental nuts and bolts: engines talk

Engines and gui's talk in plain text. The language they use is defined and explained here by its author, UCI by Stefan Meyer-Kahlen.

Ok, let's talk to a chess engine

# grab an engine
~$ wget http://u.komodochess.com/stockfish

# give it permission to execute
~$ chmod +x stockfish

# start the engine
~$ ./stockfish

# type 'uci'. hit return. engine should kick out its configuration
# type 'go depth 1 [return]' to calculate one ply. 'go ponder [return]' to calculate.
# note "nps" is nodes per second.
# stop [return]' to stop calculating.
# type 'setoption name Threads value 16 [return]' and engine will use 16 cores, default is 1 core.
# 'quit [return]' to exit

Connecting machines

OPTION 1 (prefered) -- SSH
Configure your ssh server to keep connections open, default is to close after ~120 seconds, if connection is very slow compress data (use the -C option, or edit ssh config), and set ssh to reconnect if connection is dropped. On the windows side you'll run a combination of putty/plink/pageant to make an ssh connection using private keys and InBetween.exe (ver. 1.5) by O.Gunnar Malin (download) as your 'UCI engine' in, for example, Chessbase. The following how-to covers private keys and connecting to your engine from windows using putty/pageant/pink Remote Engines: Secure shell and Putty.

Note: You can (probably) skip the "pageant" part of those instructions as we will explicitly tell plink.exe where our key file is.

Edit your InBetween.ini file to include this:

CommandLine := C:\putty\plink.exe -ssh -C -i mykey.ppk me@serverIP "./engine"

InBetween.exe (download)- for use with Chessbase GUI. Link includes description and download of InBetween.exe

usage example of InBetween.exe - immortalchess.net forum post

OPTION 2 -- RTHomeServer.exe + wine
for windows engines only

Use ~$ wine RTHomeServer.exe (download) on the linux box and RTUciClient.exe (download) on the windows box and follow the instructions at ChessOK. Basically just update port number, engine name, engine path, and ip of the server and open port on the router if accessing from outside the lan. 

OPTION 3 -- netchess
for windows engines only. netchess not working in ubuntu 13.10

use netChess and Linux and remote engine instructions by Felix King

Running *.exe engines on Linux: ~$ wine engine.exe

Install wine from the distribution's repositories and try running ~$ wine engine.exe. Hopefully that starts the engine. Some engines don't behave so well from the command line, notably Houdini 1.5, but so long as they start up, you're in business.

Performance optimization

- there are two benchmarks you can use: 1) nodes per second 2) time to solve a tricky position.
Engines cache information so, in general, it's one test per login (reboot), or one test, clear the cache (untested ~$ sudo sh -c "sync; echo 3 > /proc/sys/vm/drop_caches").
For nps tests, use stockfish's built-in benchmark tool which works as follows:
~$ ./stockfish bench [hash] [cores] [depth] default depth ...
so something like:
~$ ./stockfish bench 128 8 18 default depth

- change the kernel.

I've measured performance gains with with "low latency" kernels such as Zen kernel. Could compile your own kernel! the engines stockfish, komodo and houdini all have command line tests which benchmark engine speed. one can use these to measure performance of various kernels, aiming to get the highest nps.

- enable large pages on the server? am not sure if these are now part of the mainline kernel

- look around for Oracle and IBM documentation about improve, for example, db performance

- hash size

- stop programs / daemons wasting cpu cycles

- stockfish has compilation settings for various cpus.

Best regards,

Jesse Gersenson jesse in/on/at jesseo.com

to be added to main article

I use Debian and also Xubuntu which is ubuntu with the xfce windows manager. The benefit of xfce is it doesn't (visually) change -- it always looks like windows 98.

Here is the iso for Xubuntu 14.04:
http://ftp.uni-kl.de/pub/linux/ubuntu-dvd/xubuntu/releases/14.04/release/xubuntu-14.04.4-desktop-amd64.iso

After you install it run these commands from a terminal:
sudo apt-get install wine gufw && sudo gufw

turn the firewall 'on'.

"netchess" doesn't work with current operating systems.

InBetween.exe by Gunnar Malin https://komodochess.com/pub/InBetween-ver-1.5.zip works.
It is what I use to connect Arena, which is a windows program I run on my linux machine with the command 'wine Arena.exe', to a 24 core linux machine which runs a development version of Komodo.

InBetween has two files.. InBetween.exe and InBetween.ini. One is the executable and the *.ini file is a text file you must edit. Note, you can change the name from InBetween to something else but make sure the base filename is the same, ex. MyNewEngine.exe & MyNewEngine.ini

1. Stick both InBetween files in a folder. And in the folder also put plink.exe.
2. install an ssh server on your linux machine, in xubuntu you do this with the command:
sudo apt-get install openssh-server

3. make sure ssh is working by trying to connect to your machine with this command (change 'yourusename' to your actual username):
ssh yourusername@localhost

you should be prompted for your password. enter it and you'll get some 'thanks for connecting' text and a command prompt. type exit to close the connection and return to your non-ssh command prompt.

4. set up InBetween.ini. Here is an example:
[InBetween]
Priority := high
CommandLine := PLINK.EXE -ssh -pw SecretPassword USERNAME@localhost "/home/USERNAME/linux-engine"
[Client2Server]
[Server2Client]


The text in bold face will be changed; replace SecretPassword with your actual password and change USERNAME to your USERNAME and linux-engine with be komodo-10.1-linux or stockfish_15082821_32bit or something similar...

5. run "wine InBetween.exe" from the command line and answer 'y' when asked to cache the machine key.
6. In Aquarium, or any windows based GUI, use InBetween.exe as your engine.