"The Perspective Transform in Embedded Vision," a Presentation from Cadence
2 likes176 views
The document discusses the mathematical model of perspective transformation, outlining the relationship between physical and projected coordinates of an image from a camera's perspective. It highlights applications such as image stitching, sensor noise filtering, and video stabilization, while addressing the challenges and computational complexities involved in efficiently applying perspective transforms. A division-free perspective transform approach is proposed to reduce computational cycles significantly in typical use cases.
"The Perspective Transform in Embedded Vision," a Presentation from Cadence
- 5. Image Capture from Perspective of the Camera
- 6. Mathematical Model of Perspective Projection - 1 • Q : Point in physical plane L1 • (x, y) – coordinates of Q in physical plane • P : Projection of Q on image plane L2 • (u, v) – coordinates of P in image plane • f – camera focal length • Projected (u,v) are related to Physical (x,y) as: • u = h11x+h12y+h13 h31x+h32y +h33 • = h21x+h22y+h23 h31x+h32y+h33 • This is the Perspective Transform between destination coordinates (u,v) and source coordinates (x,y)
- 7. Mathematical Model of Perspective Projection -2 u= 𝑥1 𝑥3 *f v = 𝑥2 𝑥3 *f hi1 = ෝ𝑒 𝑋 . ෝ𝑒𝑖 hi2 = ෝ𝑒 𝑌 . ෝ𝑒𝑖 hi3 = ഥ𝐯0 . ෝ𝑒𝑖 𝑥1 = ഥQ. ෝe1 = h11x+h12y+h13 ഥQ = ෝ𝑒 𝑋 .x + ෝ𝑒 𝑌 .y + ഥv0 𝑥2 = ഥQ. ෝe2 = h21x+h22y+h23 𝑥3 = ഥQ. ෝe3 = h31x+h32y+h33 Deriving the Perspective Transform Equations: • 𝐯 𝟎 • ෝ𝑒 𝑋 ෞ𝑒 𝑌 • ෝ𝑒1 ෝ𝑒2 ෝ𝑒3
- 8. Mathematical Model of Perspective Transform - 3 • In camera 1 perspective: – Coordinates (X, Y) of point Q related to coordinates (xs, ys) of projected point, Ps, via Perspective transform with coefficients H = {h11, h12, h13, h21, h22, h23, h31, h32, h33} • In camera 2 perspective: – Coordinates (X, Y) of point Q related to coordinates (xd, yd) of projected point, Pd, via Perspective transform with coefficients: H’ = {h′11, h′12, h′13, h′21, h′22, h′23, h′31, h′32, h′33} • It can be also shown that coordinates of Ps = (xs, ys) are related to coordinates of Pd = (xd, yd) via a Perspective Transform: A={a11, a12, a13, a21, a22, a23, a31, a32, 1.0 } 𝑥𝑠 = a11xd +a12yd+ a13 a31xd+a32yd + 1.0 , 𝑦𝑠 = a21xd +a22yd+ a23 a31xd+a32yd + 1.0 • This mapping of coordinates from one perspective (xs, ys) to another perspective (xd, yd) is central to many imaging and vision applications.
- 10. Application Example 1: Image Stitching • • • •
- 11. Image Stitching
- 12. Application Example 2: Sensor Noise Filtering • • –
- 13. Application Example 3: Video Stabilization •
- 16. Estimating the Perspective Transform • • → • • 𝑎11 𝑎12 𝑎13 𝑎21 𝑎22 𝑎23 𝑎31 𝑎32 𝑥 𝑠1 𝑦 𝑠1 . . . . 𝑥 𝑠𝑛 𝑦 𝑠𝑛 𝑥 𝑑1 𝑦 𝑑1 0 0 1 0 0 𝑥 𝑑1 0 0 𝑦 𝑑1 1 −𝑥 𝑠1 𝑥 𝑑1 − 𝑥 𝑠1 𝑦 𝑑1 −𝑦 𝑠1 𝑥 𝑑1 −𝑦 𝑠1 𝑦 𝑑1 . . . 𝑥 𝑑𝑛 𝑦 𝑑𝑛 0 0 1 0 0 𝑥 𝑑𝑛 0 0 𝑦 𝑑𝑛 1 −𝑥 𝑠𝑛 𝑥 𝑑𝑛 −𝑥 𝑠𝑛 𝑦 𝑑𝑛 −𝑦𝑠𝑛 𝑥 𝑑𝑛 −𝑦𝑠𝑛 𝑦 𝑑𝑛 =
- 17. Applying the Perspective Transform • – – ixs iys ixs iys α α ixs iys xs ,ys = a21 +a22 + a23 xd, yd x = a11 +a12 + a13 ixs=floor(xs), αx = xs – ixs, iys=floor(ys), αy = ys – iys,
- 19. Implementation challenges on DSP • – – • – – •
- 20. Compute efficient Perspective Transform -1 • • ( ) • • • ( ) –
- 21. Compute efficient Perspective Transform - 2 • – Use modest tile sizes : (8x8), (16x16), (32x32) – Use full divide based Perspective equation to compute only the source coordinates of four tile corner pixels – For all other pixels in tile use bilinear interpolation of computed tile corner source coordinates to estimate the source coordinates – Dynamically determine the tile size based on the coefficient values (a31,a32) to limit the coordinate estimation error < ErrMax – ErrMax : maximum allowed coordinate error – ErrMax range : 0.01 – 0.05 (depending on application architect specification)
- 22. Division Free Perspective Transform : Results •
- 23. Summary • Perspective transform accurately computes an image as seen from a different camera position (perspective). • Used in the imaging applications which involve joint processing of multiple pictures (frames) taken from slightly different camera positions. • Per-cycle-divide operation involved in a canonical implementation poses a large computational complexity problem. • We described a minimal division modification of Perspective transform - achieves DSP cycles reduction by a factor of 3x for typical use cases
- 24. References
- 25. © 2018 Cadence Design Systems, Inc. All rights reserved worldwide. Cadence, the Cadence logo and the other Cadence marks found at www.cadence.com/go/trademarks are trademarks or registered trademarks of Cadence Design Systems, Inc. All other trademarks are the property of their respective holders.