Categories

Versions

You are viewing the RapidMiner Deployment documentation for version 9.8 -Check here for latest version

GPU-enabled Job Agent

Before you create your own GPU-enabled Job Agent, be aware that RapidMiner providespre-configured Docker images for Deep Learning. Nevertheless, if you want more technical detail, continue reading!

The main points are the following:

  • Your Job Agent must be installed on a computer withCUDA-compatible GPU.
  • You must install the Nvidia library CUDA and should install the library cuDNN for enhanced performance.
  • You must install the following RapidMiner extensions:

Pay careful attention to the compatibility matrix:

CUDA compatibility

Deep Learning Extension ND4J Back-End Supported CUDA Supported cuDNN
1.0.1 1.0 10.1 7.6
1.0 1.0 10.1 7.6
0.9.4 0.1.1 10.0 7.4
0.9.3 0.1.0 10.0 -
0.9.1 - 9.0 -
0.9.0 - 9.0 -
0.8.1 - 9.1 -
0.8.0 - 9.1 -

Create a GPU-enabled Job Agent

A Job Agent can take advantage of a GPU for processing images or training and scoring neural networks. Currently, one GPU can be used per Job Agent.

Take the following steps:

  1. Install the Job Agent on a computer that has aCUDA-compatible GPU.

  2. Follow the installation instructions forCUDA 10.1(and cuDNN version 7.6).

  3. Download theND4J Back Endand theDeep Learningextensions from the RapidMiner marketplace, and move them to the extensions folder belonging to the Job Agent{homeDir}/resources/extensions/.

  4. Create the settings file{homeDir}/config/rapidminer/rapidminer.propertiesas follows:

    rapidminer.backend.nd4j=GPU-CUDA rapidminer.backend.nd4j.max_bytes=32G rapidminer.backend.nd4j.max_physical_bytes=48G rapidminer.deeplearning.training_ui.ports=60080

    In the current context, the first of these settings is required (GPU-CUDA), but the last three are optional, corresponding to theRapidMiner Studio settings discussed here.

Settings

The settings are defined by aproperties file,{homeDir}/config/rapidminer/rapidminer.properties, located in the home directory of the Job Agent.

Parameter Key Possible Parameter Value Explanation
rapidminer.backend.nd4j CPU-OpenBLAS,CPU-MKL,GPU-CUDA Choose the computation back-end to use for calculation.
rapidminer.backend.nd4j.max_bytes 1024米,16G The JVM off-heap memory limit for the computation backend (~native memory limit)
rapidminer.backend.nd4j.max_physical_bytes 1024米,16G The maximum bytes for the entire process - usually set to max-bytes plus Xmx plus a bit extra, in case other libraries require some off-heap memory as well.
rapidminer.deeplearning.training_ui.ports 1-65535 Choose a port a Job Agents training UI should be listening to. The port0is also allowed, but will assign a random port.