![\"The](\"https:\/\/tel-zur.net\/blog\/wp-content\/uploads\/2024\/03\/roofline_paper.png\" "\\"Figure")
<\/figure>\n\n\n\nAs a test case I used the famous stream<\/em> benchmark. At its core stream does the following computational kernel:<\/p>\n\n\n\n c[i] = a[i] + b[i];<\/p>\n\n\n\n Where a, b and c are large arrays. The computational intensity in this case consists of 1 floating point operation (‘+’) and 3 data movement (read a and b from memory and write back c). if a, b, and c are of type float<\/em>, it means that each element contains 4bytes and the total the data movement is 12bytes, therefore the computational intensity is 1\/12 which is about 0.083. We will test this prediction later on. The official stream benchmark can be downloaded from [3]. However, for my purpose this code seems to be too-complex and also according to [2] the roofline results that it produces may be miss-leading. Therefore, I wrote a simple stream code myself. The reference code is enclosed in the code section below.<\/p>\n\n\n The computational environment<\/strong><\/p>\n\n\n\n I use a laptop running Linux Mint 21.3 with 8GB RAM on an Intel’s Core-i7. The compiler was Intel’s OneAPI (version 2024.0.2) and Intel Advisor for measuring and visualizing the roofline. If you want to reproduce my test you need as a first step to prepare the environment as can be seen here:<\/p>\n\n\n Another, one time, preparation stage is setting ptrace_scope<\/em> otherwise Advisor won’t work:<\/p>\n\n\n The results<\/strong><\/p>\n\n\n\n First, I tested the un-optimized version that was listed above. The measured point obtained sits at 0.028FLOP\/Byte, this result is lower than the theoretical prediction and this means that we need to put more effort to improve the code. The roofline result of this un-optimized version is shown here:<\/p>\n\n\n\n One can verify that the CPU spent most of its time in the main loop:<\/p>\n\n\n\n In the recommendations section Intel Advisor state: “The performance of the loop is bounded by the private cache bandwidth. The bandwidth of the shared cache and DRAM may degrade performance. and the vectorization report says: <\/p>\n\n\n This time the roofline plot reports on a performance improvement compared to the non-optimized code. However, in both cases the performance bottleneck is still the DRAM bandwidth, as expected. The vectorized roofline plot is shown here:<\/p>\n\n\n\n This time the performance is 0.083FLOP\/Byte which is our theoretical prediction! This means that although the code hasn’t changed, the compiler managed to do the more ‘add’ instructions per unit of time, in parallel, due to the vectorization support:<\/p>\n\n\n\n Another possible optimization one could think of is adding an alignment to the arrays in memory:<\/p>\n\n\n However, adding this requirement also didn’t improve much the performance. It seems that we really reached the performance wall and the reason for that is that the bottleneck isn’t in the computation but in the DRAM bus performance.<\/p>\n\n\n\n As a last step I tried another optimization technique, which is to add multi-threading, i.e. parallelizing the code with OpenMP. Adding an OpenMP parallel-for pragma causes the computational kernel to be computed in parallel. However, once again, there wasn’t any performance improvement.<\/p>\n\n\n To conclude, the roofline mode is a strong tool for checking where are the performance bottlenecks in the code. As long that we suffer from the limitations of the DRAM (or the caches) there isn’t much we can do about improving the performance. The CPU can ingest more operations on new data but since the memory is slow the performance are poor. Unfortunately, there is nothing we can do about it. This is a challenging issue that is pending to future computer architectures.<\/p>\n\n\n\n If you enjoyed this article you are invited to leave a comment below. You can also subscribe to my YouTube channel <\/a>(@tel-zur_computing) and follow me on X<\/a> and Linkedin<\/a>.<\/p>\n\n\n\n References:<\/strong><\/p>\n\n\n\n [1] SamueL Willias, Andrew Waterman, and David Patterson, “Roofline: An insightful Visual Performance model for multicore Architectures<\/a>“, Communications of the ACM, April 2009, vol. 52, no. 4, pp 65-76. <\/p>\n\n\n\n [2] Supplementary material to [1]: https:\/\/dl.acm.org\/doi\/10.1145\/1498765.1498785#sup<\/a><\/p>\n\n\n\n#include <stdio.h>\n#include <stdlib.h> \/\/ for random numbers\n#include <omp.h> \/\/ for omp_get_wtime()\n\n#define SIZE 5000000 \/\/ size of arrays\n#define REPS 1000 \/\/ number of repetitions to make the program run longer\n\nfloat a[SIZE],b[SIZE],c[SIZE]; \ndouble t_start, t_finish, t;\nint i,j;\n\nint main() {\n\n\/\/ initialize arrays\nfor (i=0; i<SIZE; i++) {\n a[i] = (float)rand();\n b[i] = (float)rand();\n c[i] = 0.;\n}\n\n\/\/ compute c[i] = a[i] + b[i]\nt_start = omp_get_wtime();\nfor (j=0; j<REPS; j++)\n for (i=0; i<SIZE; i++)\n c[i] = a[i] + b[i];\nt_finish = omp_get_wtime();\n\nt = t_finish - t_start;\n\nprintf("Run summary\\n");\nprintf("=================\\n");\nprintf("Array size: %d\\n",SIZE);\nprintf("Total time (sec.):%f\\n",t);\n\n\/\/ That's it!\nreturn 0;\n}<\/code><\/pre>\n\n\n$ <strong>source ~\/path\/to\/opt\/intel\/oneapi\/setvars.sh<\/strong> \n # change the line above according to the path in your file system\n:: initializing oneAPI environment ...\n bash: BASH_VERSION = 5.1.16(1)-release\n args: Using "$@" for setvars.sh arguments: \n:: advisor -- latest\n:: ccl -- latest\n:: compiler -- latest\n:: dal -- latest\n:: debugger -- latest\n:: dev-utilities -- latest\n:: dnnl -- latest\n:: dpcpp-ct -- latest\n:: dpl -- latest\n:: inspector -- latest\n:: ipp -- latest\n:: ippcp -- latest\n:: itac -- latest\n:: mkl -- latest\n:: mpi -- latest\n:: tbb -- latest\n:: vtune -- latest\n:: oneAPI environment initialized ::<\/code><\/pre>\n\n\n$ cat \/proc\/sys\/kernel\/yama\/ptrace_scope\n1\n$ echo "0"|sudo tee \/proc\/sys\/kernel\/yama\/ptrace_scope\n[sudo] password for your_user_name: \n0<\/code><\/pre>\n\n\n![\"\"](\"https:\/\/tel-zur.net\/blog\/wp-content\/uploads\/2024\/03\/guy_stream_not_optimized-1024x537.png\")
<\/figure>\n\n\n\n![\"\"](\"https:\/\/tel-zur.net\/blog\/wp-content\/uploads\/2024\/03\/profiling.png\")
<\/figure>\n\n\n\n
To improve performance: “Improve caching efficiency.\u00a0The loop is also scalar. To fix: Vectorize the loop<\/strong><\/em>“. Indeed in the next step I repeat the roofline measurement but with a vectorized executable. The compilation command I used is:<\/p>\n\n\nicx -g -O3 -qopt-report-file=guy_stream.txt -qopenmp -o guy_stream_vec .\/guy_stream_vec.c<\/code><\/pre>\n\n\nGlobal optimization report for : main\n\nLOOP BEGIN at .\/guy_stream.c (15, 1)\n remark #15521: Loop was not vectorized: loop control variable was not identified. Explicitly compute the iteration count before executing the loop or try using canonical loop form from OpenMP specification\nLOOP END\n\nLOOP BEGIN at .\/guy_stream.c (23, 1)\n remark #15553: loop was not vectorized: outer loop is not an auto-vectorization candidate.\n\n <strong>LOOP BEGIN at .\/guy_stream.c (24, 5)\n remark #15300: LOOP WAS VECTORIZED\n remark #15305: vectorization support: vector length 4<\/strong>\n LOOP END\nLOOP END<\/code><\/pre>\n\n\n![\"\"](\"https:\/\/tel-zur.net\/blog\/wp-content\/uploads\/2024\/03\/vectorization_1-1024x485.png\")
<\/figure>\n<\/figure>\n\n\n\n![\"\"](\"https:\/\/tel-zur.net\/blog\/wp-content\/uploads\/2024\/03\/Screenshot-at-2024-03-19-16-37-07.png\")
<\/figure>\n\n\n\n__attribute__((aligned (64)))<\/code><\/pre>\n\n\n# pragma openmp parallel for\nfor (j=0; j<REPS; j++)\n for (i=0; i<SIZE; i++)\n c[i] = a[i] + b[i];<\/code><\/pre>\n\n\n