[DL輪読会]Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
2 likes2,272 views
This document summarizes several Japanese deep learning papers and resources: - It introduces a Japanese website that shares deep learning papers and information. - It briefly outlines two papers on efficient deep learning methods and convolutional neural networks. - It shares links to deep learning code and frameworks. - It presents mathematical equations for a deep learning model combining multiple neural networks.
![DEEP LEARNING JP
[DL Papers]
http://deeplearning.jp/
- - - -
- - - -](https://image.slidesharecdn.com/20180209misono-180209000316/85/DL-Quantization-and-Training-of-Neural-Networks-for-Efficient-Integer-Arithmetic-Only-Inference-1-320.jpg)
![[DL輪読会]Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://image.slidesharecdn.com/20180209misono-180209000316/85/DL-Quantization-and-Training-of-Neural-Networks-for-Efficient-Integer-Arithmetic-Only-Inference-9-320.jpg)
![)
q(r; a, b, n) := round
✓
clamp(r; a, b) a
s(a, b, n)
◆
s(a, b, n) + a
s(a, b, n) :=
b a
n 1
clamp(r; a, b) := min(max(x, a), b)
l^ ec
.1,2/ l^ i
: 1:= 1= ;: 8 8 )2 = : 0) (
56=l^
1 9 : 2 91
1 = 1= ;:l^
[m .1,2/ b h ]
.1,2/ l^ 1 2 :
ag
S = s(a, b, n), Z = z(a, b, n)](https://image.slidesharecdn.com/20180209misono-180209000316/85/DL-Quantization-and-Training-of-Neural-Networks-for-Efficient-Integer-Arithmetic-Only-Inference-10-320.jpg)
![[DL輪読会]Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://image.slidesharecdn.com/20180209misono-180209000316/85/DL-Quantization-and-Training-of-Neural-Networks-for-Efficient-Integer-Arithmetic-Only-Inference-12-320.jpg)
![[DL輪読会]Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://image.slidesharecdn.com/20180209misono-180209000316/85/DL-Quantization-and-Training-of-Neural-Networks-for-Efficient-Integer-Arithmetic-Only-Inference-15-320.jpg)
[DL輪読会]Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference
- 1. DEEP LEARNING JP [DL Papers] http://deeplearning.jp/ - - - - - - - -
- 2. • LQH QHML LC O HLHLF ME :D O J :DQTMOI EMO 1EEHBHDLQ 5LQDFDO -OHQGKDQHB LJ 5LEDODLBD • .DLMHQ BMA IHOK LQ 7JHF .M /GDL 9DLFJMLF G 9 QQGDT LF -LCODT MT OC OQTHF -C K 0KHQO 7 JDLHBGDLIM MMFJD • AKHQQQDC ML ( 0DB ) • GQQN , O HS MOF A ) ( )) • HLQ don rZb a pif h lm ke • DL MO2JMT 8HQD g c b
- 3. • a eki // ORS • q X c g N M Pw • bf • o // e un • . 8/8D 3 88H8/8D 3 99 8/8D ,8 8/8D • e o h l s • yt t • zh B /8 B /8D B /02 8D 8D • - 1B • r • ) (
- 4. • 0 -0 A Ge h • ,8/4 0-0 M e h • bg sl V A MA • 042 k w Nn xs VMA • k /4 41 0 89 f M t A • i • 4 N 8 r • k Go ( )
- 5. • r = S(q Z) 00 1 2() 9 i k a 5 0(2 /2 5 0(2 e /2 S3(q (i,k) 3 Z3) = NX j=1 S1(q (i,j) 1 Z1)S2(q (j,k) 2 Z2) k r3 = r1r2 q (i,k) 3 = Z3 + M NX j=1 (q (i,j) 1 Z1)(q (j,k) 2 Z2) 8b M := S1S2 S3 M = 2 n M0 (M0 2 [0.5, 1)) M 2 (0, 1) /2 f231 M0
- 6. q (i,k) 3 = Z3 + M NX j=1 (q (i,j) 1 Z1)(q (j,k) 2 Z2) q (i,k) 3 = Z3 + M 0 @NZ1Z2 Z1a (k) 2 Z2¯a (i) 1 + NX j=1 q (i,j) 1 q (j,k) 2 1 A a (k) 2 := NX j=1 q (j,k) 2 , ¯a (i) 1 := NX j=1 q (i,j) 1
- 7. * + ) NX j=1 q (i,j) 1 q (j,k) 2 ( , 8 * 8 * IMin Sbias = S1S2, Zbias = 0 8 7 23 8 2 8 8 * a l fh e w o in a l t ru in fh F 0 v g M e 0 n L 8 *T c 1 = 9 /8 os
- 8. • s ixh wor • 9,0 /8 ( , n d k • p a • b g e c b • 9,0 /8 )0( 2 (8 tf • )( 1 ( (80 l • 9( 0 g e
- 10. ) q(r; a, b, n) := round ✓ clamp(r; a, b) a s(a, b, n) ◆ s(a, b, n) + a s(a, b, n) := b a n 1 clamp(r; a, b) := min(max(x, a), b) l^ ec .1,2/ l^ i : 1:= 1= ;: 8 8 )2 = : 0) ( 56=l^ 1 9 : 2 91 1 = 1= ;:l^ [m .1,2/ b h ] .1,2/ l^ 1 2 : ag S = s(a, b, n), Z = z(a, b, n)
- 11. • • ) 0 1 9 3: 8 C 3: 8 C 33 9/ 3 3: 8 C 1 : 0 /: 3 • 1 3/ 3. / : : . / 1 3/ 3.3 /8. / • 3: 8 C , 3 F L • : 3 3 /: 3 • ) 0 1 9 3: 8 C 3: 8 C 08 0 9/ 3 3: 8 C 1 : 0 8 3 1 2 1 1928 :3. 3 3 3:13 92 (
- 13. -
- 14. • D6 A A 6A8 A 9D9A 9 I F AF9 9D A 99C 19 D6 19FI D /4 ( • AF RN • 0 98 D9 A D6 A A A F A6 19 D6 19FI D A AF9 9D 2C9D6F A /4 ( • AF T PO -0 T LM 98 CD9 A AF9 9D FD6 A A ( )