MJay
NVBLAS 논문 본문
NVBLAS 논문
tistory NVBlas
Fatman vs LittleBoy
어떻게 구조가 되어있는지
대치어 없음 Wrapper라면 Spark에서 어떻게 작동하는 건지 알아봐야겠다.
OpenBLAS 랑 NVBLAS를 둘다 쓰면 된다.
sparse matrix에서 는 성능이 안좋다.
you might need to pick the appropriate value of NVBLAS_TILE_DIM from nvblas.conf, because for older GPU the default value is too big and some operations might return zero.
CublasXT is a set of routines which accelerate Level 3 BLAS (Basic Linear Algebra Subroutine) calls by spreading work across more than one GPU. By using a streaming design, cublasXT efficiently manages transfers across the PCI-Express bus automatically, which allows input and output data to be stored on the host’s system memory. This provides out-of-core operation – the size of operand data is only limited by system memory size, not by GPU on-board memory size.
netlib-java is a wrapper for low-level BLAS, LAPACK and ARPACK that performs as fast as the C / Fortran interfaces with a pure JVM fallback. netlib-java is included with recent versions of Apache Spark.
Apache Spark makes use of a component called netlib-java, which provides a Java API for Linear Algebra routines, such as BLAS, LAPACK, etc. The netlib-java package doesn’t implement these directly, but rather delegates incoming calls to one of three implementations, in the following order:
On the other hand, we observe that, for sparse matrices with
5% of non-zero terms, hardware acceleration may not increase the performance much. Scala implementation of matrix mul- tiplication in Spark is faster or comparable to OpenBLAS, and much faster than NVBLAS, as reported in [43]. This is mainly because NVBLAS pads zero terms into the matrix and sends dense matrix blocks to the GPU, which causes extra data transfer between GPU and CPU, as well as extra computation in GPU and thus affects performance. Discussion:
sparse matrix 어떻게 나타내지
The two warnings above mean that the first two implementations were not usable, and MLlib is using the Java implementations under the covers.
If you have a modern Nvidia GPU in your system, you might be considering using Nvidia’s cuBLAS implementation. This is possible through Nvidia’s NVBLAS library, and you can read about how to configure it in netlib-java here. However, I don’t recommend going this route for Apache Spark. If you take a look at section 3 of the NVBLAS documentation, you’ll see that only a handful of BLAS3 routines get sent to the GPU: gemm, syrk, herk, syr2k, her2k, trsm, trmm, symm, and hemm. Of these, only gemm is available in Spark’s MLlib, and only if explicitly used by the end user; MLlib doesn’t make use of any of these internally. This means that all other BLAS calls will get directed to the backup implementation that you have defined in your nvblas configuration file, and won’t run on the GPU. If you’re on a RedHat system like me, this means you’ll likely be building and using the reference CBLAS library, and not using a better-performing library such as OpenBLAS. So unless you have an application that sends a large amount of data through explicit calls to Matrix.multiply(y: DenseMatrix), you’re not gaining anything for your efforts to get cuBLAS working in Spark.
Discussion: In Spark, the job of multiplication on dis-
tributed matrix is divided into a number of tasks, where each task calculates the multiplication of small block matrices. The
default size of each block is 1024 × 1024. The number of tasks generated by Spark increases as the matrix size increases. As the number of tasks increases, data shuffle overhead increases, diminishing advantages of leveraging the advan- tage of hardware accelerators for performing high throughput computations. However, even with this shuffle and framework overhead, hardware acceleration can still speed-up overall
performance up to more than 2× as shown in Section IV-A. On the flip side, we find that for matrix with small sizes, OpenBLAS performs similar to or faster than NVBLAS, but
for the 66331×66331 matrix, NVBLAS actually spends less time on computation than OpenBLAS. Thus, we expect the benefits from utilizing high-throughput accelerators such as GPUs to grow with growing matrix sizes.
Since Spark is not aware of any GPU devices, it assumes that each CPU core is performing multiplications assigned to it independently of the other ones. But in actuality, all CPU cores are delegating their matrix multiplication tasks to a single GPU device. This might lead to a queue for tasks incoming to the GPU device, and ultimately means that Spark is making decisions regarding the delegation of workload and the capabilities of its resources on false assumptions.
Finally,
OpenBLAS가 있나? ubuntu 이미지에 없다
