SlideShare a Scribd company logo
Recurrent neural networks (rnn) and long short term memory networks (lstm)
Recurrent neural networks (rnn) and long short term memory networks (lstm)
Recurrent neural networks (rnn) and long short term memory networks (lstm)
Recurrent neural networks (rnn) and long short term memory networks (lstm)
𝑿𝑡
RNN
𝑾𝒙
𝑾𝒚
𝑰𝒏𝒑𝒖𝒕
𝑶𝒖𝒕𝒑𝒖𝒕
𝑌𝑡
𝑾𝒔
𝑿1
𝑾𝒙
𝑰𝒏𝒑𝒖𝒕 𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑺0
𝑾𝒔
𝑾𝒙
𝑿1
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆0+
𝑊
𝑥𝑋1)
𝑾𝒙
𝑰𝒏𝒑𝒖𝒕 𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑺1
𝑺0
𝑾𝒔
𝑾𝒙
𝑿1
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆0+
𝑊
𝑥𝑋1)
𝑾𝒙
𝑾𝒚
𝑰𝒏𝒑𝒖𝒕 𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑺1
𝑺0
𝑾𝒔
𝑾𝒙
𝒀1
𝑶𝒖𝒕𝒑𝒖𝒕
𝑿1
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆0+
𝑊
𝑥𝑋1)
𝑾𝒙
𝑾𝒚
𝑰𝒏𝒑𝒖𝒕
𝑾𝒔
𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆1+
𝑊
𝑥𝑋2)
𝑺1
𝑺0
𝑾𝒔
𝑾𝒙
𝒀1
𝑶𝒖𝒕𝒑𝒖𝒕
𝑿1
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆0+
𝑊
𝑥𝑋1)
𝑾𝒙
𝑾𝒚
𝑰𝒏𝒑𝒖𝒕
𝑾𝒔
𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆1+
𝑊
𝑥𝑋2)
𝑺1
𝑺0 𝑺2
𝑾𝒔
𝑾𝒙
𝒀1 𝒀2
𝑾𝒚
𝑶𝒖𝒕𝒑𝒖𝒕
𝑶𝒖𝒕𝒑𝒖𝒕
𝑿1
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆0+
𝑊
𝑥𝑋1)
𝑾𝒙
𝑾𝒚
𝑰𝒏𝒑𝒖𝒕
𝑾𝒔
𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆1+
𝑊
𝑥𝑋2)
𝑺1
𝑺0 𝑺2
𝑾𝒔
𝑾𝒙
𝒀1 𝒀2
𝑾𝒚
𝑶𝒖𝒕𝒑𝒖𝒕
𝑶𝒖𝒕𝒑𝒖𝒕
𝑿1
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆0+
𝑊
𝑥𝑋1)
𝑾𝒙
𝑾𝒚
𝑰𝒏𝒑𝒖𝒕
𝑾𝒔
𝑿2
𝑰𝒏𝒑𝒖𝒕
𝑡𝑎𝑛ℎ(𝑊
𝑠𝑆1+
𝑊
𝑥𝑋2)
𝑺1
𝑺0 𝑺2
𝑾𝒔
𝑾𝒙
𝒀1 𝒀2
𝑾𝒚
𝑡𝑎𝑛ℎ(𝑊
𝑠ℎ0+
𝑊′𝑦𝑌1)
𝑡𝑎𝑛ℎ(𝑊
𝑠ℎ1+
𝑊′𝑦𝑌2)
𝒉1 𝒉2
𝒉0
𝑾′𝒚 𝑾′𝒚
𝑾𝒉 𝑾𝒉
𝑿1
𝑅𝑁𝑁
0.01
𝑅𝑁𝑁
𝑿2
𝑅𝑁𝑁
𝑿𝑛
0.01
0.01 0.01 0.01
𝑌 𝐿𝑂𝑆𝑆
Update Weights
100 time steps
𝑿1
𝑅𝑁𝑁
0.01
𝑅𝑁𝑁
𝑿2
𝑅𝑁𝑁
𝑿𝑛
0.01
0.01 0.01 0.01
𝑌 𝐿𝑂𝑆𝑆
Update Weights
100 time steps
Update in weight =
(𝟎. 𝟎𝟏)𝟏𝟎𝟎 ≈ 0
Recurrent neural networks (rnn) and long short term memory networks (lstm)
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ 𝑖𝑡
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
tanh
𝑖𝑡
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
tanh Ĉ𝑡
𝑖𝑡
∗
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
σ
tanh Ĉ𝑡
𝑖𝑡
∗
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
σ
σ
tanh Ĉ𝑡
𝑜𝑡
𝑖𝑡
𝑓𝑡
∗
∗
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
σ
σ
tanh Ĉ𝑡
𝑜𝑡
𝑖𝑡
𝑓𝑡
∗
∗ + 𝐶1
𝒄0
𝑺0
𝑿2
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
σ
σ
tanh Ĉ𝑡
𝑜𝑡
𝑖𝑡
𝑓𝑡
∗
∗ + 𝐶1
𝒄0
𝑺0
𝑿2
tanh
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
σ
σ
tanh Ĉ𝑡
𝑜𝑡
𝑖𝑡
𝑓𝑡
∗
∗ + 𝐶1
𝒄0
𝑺0
𝑿2
tanh
∗
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
σ
σ
σ
tanh Ĉ𝑡
𝑜𝑡
𝑖𝑡
𝑓𝑡
∗
∗ + 𝐶1
𝒄0
𝑺0
𝑿2
tanh
∗
𝑾𝒌
𝑾𝒌𝒔
𝑾𝒌𝒙
𝑾𝒄
𝑾𝒐
𝑾𝒊
𝑾𝒇
Recurrent neural networks (rnn) and long short term memory networks (lstm)

More Related Content

PDF
Second grade schedule 2011 2012
PPTX
PDF
04.mdsd_laplace_fourier
PDF
INTEGRAL CALCULUS .pdf hahahahahahhwwhhwh
PPTX
PDF
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
PDF
05.mdsd_modelado_mecanico_electrico
Second grade schedule 2011 2012
04.mdsd_laplace_fourier
INTEGRAL CALCULUS .pdf hahahahahahhwwhhwh
Backpropagation: Understanding How to Update ANNs Weights Step-by-Step
05.mdsd_modelado_mecanico_electrico

Similar to Recurrent neural networks (rnn) and long short term memory networks (lstm) (9)

PPTX
SUEC 高中 Adv Maths (Irrational Part 3)
PPTX
SUEC 高中 Adv Maths (Irrational Part 2)
PDF
07.mdsd_modelado_termicos_liquidos
PPTX
SUEC 高中 Adv Maths (Biquadratic Equation, Method of Changing the Variable, Rec...
PPTX
SUEC 高中 Adv Maths (2 Roots Part 2)
PDF
PR 113: The Perception Distortion Tradeoff
PPTX
SUEC 高中 Adv Maths (Rational Root Theorem)
PPTX
Carbohydrate Metabolism
PDF
06.mdsd_modelado_electromecanico
SUEC 高中 Adv Maths (Irrational Part 3)
SUEC 高中 Adv Maths (Irrational Part 2)
07.mdsd_modelado_termicos_liquidos
SUEC 高中 Adv Maths (Biquadratic Equation, Method of Changing the Variable, Rec...
SUEC 高中 Adv Maths (2 Roots Part 2)
PR 113: The Perception Distortion Tradeoff
SUEC 高中 Adv Maths (Rational Root Theorem)
Carbohydrate Metabolism
06.mdsd_modelado_electromecanico
Ad

More from Minhazul Arefin (9)

PPTX
LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning ...
PPTX
pic2code: Generating HTML Code from Handwritten Picture.pptx
PPTX
Controlling Home Appliances adopting Chatbot using Machine Learning Approach
PPTX
Object Detection on Dental X-ray Images using R-CNN
PPTX
Natural Language Query to SQL conversion using Machine Learning Approach
PPTX
Efficient estimation of word representations in vector space (2013)
PPTX
Semantic scaffolds for pseudocode to-code generation (2020)
PPTX
SPoC: search-based pseudocode to code
PPTX
The rise of “Big Data” on cloud computing
LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning ...
pic2code: Generating HTML Code from Handwritten Picture.pptx
Controlling Home Appliances adopting Chatbot using Machine Learning Approach
Object Detection on Dental X-ray Images using R-CNN
Natural Language Query to SQL conversion using Machine Learning Approach
Efficient estimation of word representations in vector space (2013)
Semantic scaffolds for pseudocode to-code generation (2020)
SPoC: search-based pseudocode to code
The rise of “Big Data” on cloud computing
Ad

Recently uploaded (20)

PPTX
Geodesy 1.pptx...............................................
PDF
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
PPTX
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
PPTX
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
PDF
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
PPTX
Construction Project Organization Group 2.pptx
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
PDF
Operating System & Kernel Study Guide-1 - converted.pdf
PDF
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
PPTX
bas. eng. economics group 4 presentation 1.pptx
PPTX
UNIT 4 Total Quality Management .pptx
PDF
Embodied AI: Ushering in the Next Era of Intelligent Systems
PPTX
Lecture Notes Electrical Wiring System Components
PPTX
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
PPT
Project quality management in manufacturing
PDF
Unit I ESSENTIAL OF DIGITAL MARKETING.pdf
PPTX
Safety Seminar civil to be ensured for safe working.
PDF
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
PDF
composite construction of structures.pdf
PPT
Introduction, IoT Design Methodology, Case Study on IoT System for Weather Mo...
Geodesy 1.pptx...............................................
July 2025 - Top 10 Read Articles in International Journal of Software Enginee...
FINAL REVIEW FOR COPD DIANOSIS FOR PULMONARY DISEASE.pptx
CARTOGRAPHY AND GEOINFORMATION VISUALIZATION chapter1 NPTE (2).pptx
The CXO Playbook 2025 – Future-Ready Strategies for C-Suite Leaders Cerebrai...
Construction Project Organization Group 2.pptx
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
Operating System & Kernel Study Guide-1 - converted.pdf
SM_6th-Sem__Cse_Internet-of-Things.pdf IOT
bas. eng. economics group 4 presentation 1.pptx
UNIT 4 Total Quality Management .pptx
Embodied AI: Ushering in the Next Era of Intelligent Systems
Lecture Notes Electrical Wiring System Components
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
Project quality management in manufacturing
Unit I ESSENTIAL OF DIGITAL MARKETING.pdf
Safety Seminar civil to be ensured for safe working.
Mohammad Mahdi Farshadian CV - Prospective PhD Student 2026
composite construction of structures.pdf
Introduction, IoT Design Methodology, Case Study on IoT System for Weather Mo...

Recurrent neural networks (rnn) and long short term memory networks (lstm)