Using GPU Capabilities in MATLAB

First Dirty Trial

We reserve one GPU node in the queue for interactive session with “salloc” command. Say, we want one whole node in “GPU” queue for 2 hours

[aa3025@zeus4 ~]$ salloc -N1 -p GPU --exclusive -t 2:00:00

TO FREE THE NODEs YOU just BOOKED WITH salloc from residual processes on them, EXECUTE ONCE "srun hostname" AFTER THE JOB IS ALLOCATED (i.e. NOW )!
salloc: Granted job allocation 4989799

[aa3025@zeus4 ~]$ srun hostname
zeus205.bullx

Note that step “srun hostname” is important!

We log in to the node zeus205ssh zeus205

and launching Matlab, by just doing “module load matlab/last” and “matlab” in command line.

[aa3025@zeus205 ~]$ module load matlab/last
[aa3025@zeus205 ~]$ matlab
MATLAB is selecting SOFTWARE OPENGL rendering.

< M A T L A B (R) >
Copyright 1984-2021 The MathWorks, Inc.
R2021a Update 5 (9.10.0.1739362) 64-bit (glnxa64)
August 9, 2021


To get started, type doc.
For product information, visit www.mathworks.com.

>>

Let’s query gpu device
```
>> g = gpuDevice
```
```
g =

CUDADevice with properties:

Name: 'Tesla K20m'
Index: 1
ComputeCapability: '3.5'
SupportsDouble: 1
DriverVersion: 11.4000
ToolkitVersion: 11
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 5.3088e+09
AvailableMemory: 5.1680e+09
MultiprocessorCount: 13
ClockRateKHz: 705500
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 0
CanMapHostMemory: 1
DeviceSupported: 1
DeviceAvailable: 1
DeviceSelected: 1

>>
```
And system has
```
>> n = gpuDeviceCount

n =

     1
```
one of these. On other GPU nodes we actually have 2 GPU devices on each.

Let’s try some silly test: we create two 10,000 x 10,000 matrices of random entries and multiply them together on both CPU and GPU. For GPUs we have to use gpuArray() command to say that we store this array in GPU memory. Matrix multiplication command will be performed on GPU automatically:
```
>> tic; A=gpuArray(rand(10000,10000)); B=gpuArray(rand(10000,10000)); A*B; toc
Elapsed time is 3.990927 seconds.
```
Now the same on CPU:
```
 tic;a=(rand(10000,10000));b=(rand(10000,10000));a*b;toc
Elapsed time is 13.500388 seconds.
```
We have about 3.4 times increase in speed. However there is also time needed to get the results back from GPU memery. For short calculation this time delay may kill or overcome the real benefit of GPU useage. So GPUs are good to huge calculations….

Single it out

E.g. if we measure how long it take to generate and just to multiply two 10,000×10,000 GPU matrices
```
>> clear all;
>> tic; for i=1:100;i=i+1;A=gpuArray.rand(10000,10000); B=gpuArray.rand(10000,10000);end; toc/(i-1), tic;(A*B);toc

ans =

   2.3355e-04

Elapsed time is 0.000432 seconds.
```
When we use specialized random number generator for GPU (gpuArray.rand) it takes 0.23ms to generate two 10,000 x 10,000 matrices in GPU memory and 0.43 ms to multiply them, so 0.63 ms all together!

The same on GPUs with data retrieval from GPU memory
will take another 2.5 seconds.

and on CPU
```
>> clear a b;
tic; a=(rand(10000,10000)); b=(rand(10000,10000)); toc, tic;(a*b);toc
Elapsed time is 3.387468 seconds.
Elapsed time is 10.253760 seconds.
```
We see enormous increase in performance (10/0.001=10,000 times???) on GPU when just multiplying matrices without retrieving them from GPU memory. Generation takes about the same time on both CPU and GPU.

.. well even with data retrieval we still have about 4 times faster result on GPU’s.