Small optimizations - #768

Open

rhenry-nv wants to merge 14 commits into

marian-nmt:masterfrom

rhenry-nv:small_optimizations

Commits on Dec 2, 2020

Removed memcpy when batchA == batchB for batched matrix multiplication. Reduced the memcpys from 3 to 1 for the general case.
rhenry-nv
committed
3c0cb25
Issue matrix multiplications in cublas and cusparse to the per thread default stream. This have issue each thread issue calls to their own stream instead of the global default stream when marian is…
rhenry-nv
committed
396f6cf
Reduced the number of times indices need to be copied to the GPU
rhenry-nv
committed
23d3038
Adds licenses to files changed
rhenry-nv
committed
6c11ba8
Updates licences
rhenry-nv
committed
7eaa40b
Rounds input batches to a multiple of 8 when the gpu backend is being used for better tensor core usage.
rhenry-nv
committed
c52c53b
updates changelog
rhenry-nv
committed
359430c

Commits on Dec 4, 2020

Fixes changelog
rhenry-nv
committed
342dc8f
Renames maxDims to maxWordsInBatchSentence
rhenry-nv
committed
919bcbe
Renames batchX to batchDimX
rhenry-nv
committed
160f0c8
Adds some comments to state select
rhenry-nv
committed
0caca66

Commits on Dec 15, 2020

Removes NVIDIA notices in source code and adds NVIDIA to license file
rhenry-nv
committed
c2537c1
Fixes type in license file
rhenry-nv
committed
8e1c35e

Commits on Mar 5, 2021

Merge remote-tracking branch 'public/master' into small_optimizations
rhenry-nv
committed
f501eab