Robust Block-Matching Motion Estimation of Flotation Froth Using Mutual Information

Anthony Amankwah & Chris Aldrich
International Journal of Image Processing (IJIP), Volume (6) : Issue (5) : 2012 380
Robust Block-Matching Motion Estimation of Flotation Froth
Using Mutual Information
Anthony Amankwah anthony.amankwah@wits.ac.za
School of Computer Science
University of Witwatersrand
Johannesburg, 2050, South Africa
Chris Aldrich chris.aldrich@curtin.edu.au
Western Australian School of Mines
Curtin University of University, GPO Box U1987
Perth, WA, 6845, Australia
Abstract
In this paper, we propose a new method for the motion estimation of flotation froth using mutual
information with a bin size of two as the block matching similarity metric. We also use three-step
search and new-three-step-search as a search strategy. Mean sum of absolute difference (MAD)
is widely considered in blocked based motion estimation. The minimum bin size selection of the
proposed similarity metric also makes the computational cost of mutual information similar to
MAD. Experimental results show that the proposed motion estimation technique improves the
motion estimation accuracy in terms of peak signal-to-noise ratio of the reconstructed frame. The
computational cost of the proposed method is almost the same as the standard machine vision
methods used for the motion estimation of flotation froth.
Keywords: Mutual information, Mean sum of absolute difference, Motion Estimation, Froth
Flotation.
1. INTRODUCTION
Froth flotation is one of the most widely used separation methods in the mineral processing
industries. Despite being in use for more than a century, it is still not fully understood and remains
comparatively inefficient. Conventionally, the flotation process is controlled by trained operators
who visually inspect the froth and adjust the process control parameters based on their training. A
problematic situation is that different operators tend to interpret the visual characteristics of the
froth differently.
However, during the last couple of decades, froth image analysis has been studied widely as a
means to obviate these problems, as summarized in a recent review by Aldrich et al. [1]. Froth
images are a potentially rich source of information with regard to the state of the flotation process
and can be used as a basis for the development of advanced online control systems. Apart from
the appearance of the froth, the motion of flotation froth is also an important variable related to
the optimal recovery of minerals on industrial plants and has been used in industry among other
as a basis for mass pull control strategies [2].
The estimation of motion of froth using image processing techniques is not easy, since bubbles
collapse and merge, resulting in bubble deformations that may be difficult to track. Moreover,
although the froth as a whole has an average motion, it moves rapidly at the edges of the cell at
the launder overflow, while remaining almost stagnant at the centre of the cell [1].
Several approaches to froth motion estimation have been investigated, including bubble tracking
[3], cluster matching [4], and pixel tracing [5], but the most popular motion estimation technique is

block matching [6-7] using the mean absolute difference (MAD) between successive images of
parts thereof [2]. In the block matching algorithm, the image frame is typically divided into
nonoverlapping rectangular blocks [8]. The best match to a given block of pixels is subsequently
searched in the previous frame of the video sequence within a search area about the location of
the current block. In principle, the best solution can be obtained with a full search, which
exhaustively searches for the best matched block within all locations of the search window.
However, this approach is computationally expensive and several fast search techniques have
been proposed to reduce the high computational cost of full search methods. These fast search
techniques select a subgroup of possible search candidate locations and thus reduce the number
of matching calculations per block image.
Examples of such fast search techniques are three-step-search (TSS) [9], new-three-step-search
(NTSS) [10], four-step-search [11], 2-D logarithmic search [12] and the diamond search [13]. The
mean square error (MSE) or mean absolute difference (MAD) are considered to be among the
best similarity metrics for motion estimation. In this paper, a new method is proposed for
estimating the motion of flotation froth using mutual information. As will be shown, estimation of
mutual information with a bin size of 2 as the block matching criterion improves the accuracy of
motion estimation in terms of the peak signal-to-noise ratio of the reconstructed frame, while the
computational complexity of the proposed similarity metric is also similar to MAD.
2. SIMILARITY METRICS
.
2.1 Mutual Information with Bin Size of Two
If X and Y are two image blocks to be matched, the mutual information (MI) can defined by
),()()(),( YXHYHXHYXMI −+= (1)
where H(X) and H(Y) are the Shannon entropies [14] of X and Y respectively, and H (X,Y) is the
Shannon entropy of the joint distribution of X and Y. If pX(x) and pY(y) are the marginal
probabilities, and pXY(xy) is defined as the joint probability density of X and Y. Then MI is defined
as
∑∑ 





∗
=
x y YX
XY
XY
ypxp
yxp
yxpYXMI
)()(
),(
log*),(),( .. (2).
In this work, we use the histogram method to estimate MI. Let HX(x) and HY(y) be the histograms
of X and Y respectively and HXY(x,y) their joint histogram. then MI can be defined as
∑∑ 





=
x y YX
XY
yHxH
yxEH
yxH
E
YXMI
)(*)(
),(
log*),(
1
),( (3)
where E is the number of entries [15-17].
Figure 1 shows the joint grey value histograms and corresponding joint entropies of an image
block with values shifted (a) 0 pixels, (b) 5 pixels, (c) 10 pixels and (d) 100 pixels. since the image
pairs are identical, all corresponding grey values lie on the diagonal in Figure 1(a). From the joint
histograms it can be deduced that i) diagonal features diminish with increasing dissimilarity. ii) the
spatial dispersion pattern increases in areas with increasing dissimilarity, and iii) joint entropy
increases with increasing similarity.

(a) H(X,Y) = 0.53 (b) H(X,Y) = 0.98
(c) H(X,Y) = 1.05 (d) H(X,Y) = 1.20
Figure 1. Joint grey values of a block image of a flotation froth and a shifted version of the image.
(a) (b)
Figure 2. Example of (a) MAD surface of a block for a search area of 5 pixels (b) MI surface of the
same block over a search area of 5 pixels
Mutual information contains the term –H (A,B), which means that the minimization of the joint
entropy will maximize the mutual information. Since generally joint entropy increases with

increasing image block mismatching, the mutual information decreases with increased
mismatching, i.e. similarity corresponds with the maximization of the mutual information. That is, if
images are aligned the amount of information they contain about each other is maximal.
The computational complexity between two block images depends not only on the number of
pixels in each block image (M) but also the number of bins used the estimate MI. The
computational cost of the histogram is )(MO . The computational cost relative to the number of
histogram bins is )( 2
RO , where R is the number of bins. In this work only two bins were used
(R = 2), with a block size of 40 x 40 pixels, so that the computational cost of MAD and MI were
approximately the same. The principle of maximization of mutual information is applied to the
motion vector of blocks. The motion vector is obtained by equation
),(maxarg
1,
nmMImv
pnmp −≤≤−
= (4)
Figures 2(a) and 2(b) respectively show the MAD surface and MI2 surface of same block over a
search area of 5 pixels in each of the vertical and horizontal directions. The MI2 surface is
sharper than the MAD surface, suggesting that MI2 can be more robust than MAD, since both
surfaces are smooth.
2.2 Mean Absolute Difference (MAD)
The full MAD search of an image frame divided into NN × blocks, can be expressed as
∑∑
−
=
−
−=
=
++−=
1
0
1
1
0
,,(),(),(
N
j
tt
Ni
i
njmiFjiFnmMAD
pnmp ≤≤− , (5)
where MAD(m,n) is the MAD computed for the displacement of (m,n) pixels. F
t
(i,j) and F
t-1
(i,j) are
the grey level values of the current and previous frame respectively. The search area or the
maximum displacement, p, allowed in the vertical and horizontal directions. The motion vector for
a block is obtained by the equation
),(minarg
1,
nmMADmv
pnmp −≤≤−
= (6)
where mv is the motion vector of the vertical and horizontal displacements. The computational
complexity of MAD is )(MO additions, where M is the number of pixels in each frame
3. SEARCH METHODS
The three-step-search and the new-three-step-search algorithms were used in this investigation.
3.1 Three-Step-Search (TSS)
The TSS algorithm begins with search location considered as the centre and sets a step size
depending on the search window. It searches eight locations around the centre. From the nine
locations searched, the one with the best similarity metric is selected as the new centre. A new
step size is selected, which is half of the original step size and a similar search is repeated for two
more iterations. Figure 3(a) illustrates the TSS algorithm.

3.2 New-Three-Step-Search (NTSS)
The NTSS is a modified version of TSS and a centre biased search method. In the first step, 16
points are checked in addition to the search origin for the lowest or highest weight of the cost
function. If the optimal cost is at the origin, the search is stopped and the motion vector is (0,0). If
the optimal cost is at any of the 8 locations nearest to the origin, then the origin of the search is
changed to that point and points adjacent to it are searched. Depending on which point it is, 5
points or 3 points night be checked, as shown in Figure 3(b). The location that gives the optimal
cost is the best match and the motion vector is set to that location. On the other hand, if the
optimal cost is associated with one of the 8 locations further away from the origin, then the TSS
search method is followed. Thus although the NTSS method might need a minimum of 17 points
for every macroblock in the image, it has a worst case scenario of 33 points that have to be
searched.
First Step Second Step Third Step Out 1st
Step Inner 1st
Step Three Step Five Step
(a) (b)
Figure 3(a). The Three Step Search algorithm with rectangles, triangles and circles representing
first step search, second step search and third step search respectively, and (b) NTSS with
rectangles as first outer step, triangle as inner first step, circles and squares as three step and
five step respectively.
4. EXPERIMENTS AND RESULTS
The performance of the proposed similarity metric and MAD were evaluated using two video
sequences “Flot1” and “Flot2” from the Anglo Platinum Centre of Excellence laboratory in
Stellenbosch. The TSS and NTSS search strategies were used. The frame rate was 30 Hz and
the resolution of both sequences was 720 x 1280 pixels. A 4 dB Gaussian was added to Flot2 to
test the robustness of the algorithms, while a block size of 40 x 40 and a search area of 5 pixels
were used in the vertical and horizontal directions. The peak signal to noise ratio (PSNR) of the
image given in equation 6 is used to evaluate the performance of the algorithms






=
MSE
PSNR
2
10
255
log10
. (7)
Figures 4(a) and 4(b) show the motion vector estimate of frame 6 in the Flot1 sequence using
MAD with TSS and MI2 with TSS respectively. Figures 4(c) and 4(d) also show the motion vector

estimate of frame 7 in the Flot2 sequence using MAD with TSS and MI2 with TSS respectively.
Figure 5(a) and 5(b) show the comparisons of the PSNR of MI2-with-NTSS, MI2-with-TSS, MAD-
with-NTSS, and MAD-with-TSS of the Flot1 and Flot2 sequences respectively. The performance
of the algorithms in terms of the PSNR of the reconstructed frame for the Flot1 sequence is
almost the same. The performance of the MI2 algorithms (MI2-with-NTSS and MI2-with-TSS) is
better than the MAD algorithms (MAD-with-NTSS and MAD-with-TSS). The overall average
PSNR of the experiments for the MI2 algorithms and the MAD algorithms are 66.6 dB and 66.0
dB respectively.
(a) (b)
(c) (d)
Figure 4(a). Motion vector of frame 6 in the Flot1 sequence estimated by conventional MAD using
TSS, (b) Motion vector of frame 6 in the Flot1 sequence estimated by conventional MI2 using
TSS (c) Motion vector of frame 7 in the Flot2 sequence estimated by the conventional MAD using
TSS, and. (d) Motion vector of frame 2 in the Flot2 sequence estimated by the conventional MI2
using TSS.
Tabel.1 Average peak to signal noise ratio (PSNR) (dB) of the Flot1 and Flot2 seqeunces
reconstructed by various motion estimation algorithms.
Method Flot1 Flot2
MAD-with-TSS 67.34 64.67
MAD-with-NTSS 67.35 64.67
MI2-with-TSS 66.94 66.12
MI2-with-NTSS 67.00 66.24

(a)
(b)
Figure 5(a). Comparisons of the peak signal to noise ratio (PSNR) of sequences reconstructed by
various motion estimation algorithms. Using the Flot1 sequence with a resolution and frame rate
of 720 x 1280 and 30Hz respectively, and (b) using the Flot2 sequence with a resolution and
frame rate of 720 x 1280 and 30Hz respectively.

5. CONCLUSIONS
In this paper, a new motion estimation method for flotation froth is proposed by using mutual
information with a bin size of two (MI2) as a similarity criterion. The computational cost is similar
to the popular MAD approach. The three-step-search and the new-three-step-search algorithms
were used to search the images. MI2 yielded better results in terms of the PSNR of the
reconstructed image frames than the classical minimum absolute difference (MAD). It reduces the
amount of bad motion vectors especially for noisy images compared the MAD approach. In the
future, the algorithm will tested with more video sequences in combination with different search
strategies. We will also investigate optimization algorithms such as the Spall’s algorithm [18] to
reduce computational cost and the lightening systems for the experimental data.
6. REFERENCES
[1] C. Aldrich, C. Marais, B.J. Shean and J.J. Cilliers. “Online monitoring and control of froth
flotation systems with machine vision: A review”. International Journal of Mineral
Processing, 96(1-4), 1-13, 2010.
[2] C. Marais and C. Aldrich. “Estimation of platinum grades from flotation froth images”.
Minerals Engineering, 24(5), 433-441, 2010.
[3] C.P. Botha. 1999. An Online Machine Vision Froth Flotation Analysis Platform.
M.Sc.thesis, University of Stellenbosch, Stellenbosch, South Africa.
[4] D.P. Kottke and Y. Sun. “Motion estimation via cluster matching”. IEEE Transactions on
Pattern Analysis and Machine Intelligence 16(11), 1128-1132, 1994.
[5] N. Barbian, J.J. Cilliers, S.H. Morar, D.J. Bradshaw. “Froth imaging, air recovery and
bubble loading to describe flotation bank performance”. International Journal of Mineral
Processing 84(1-4), 81–88, 2007.
[6] K.K. Nguyen and A.J. Thornton. “The application of texture based image analysis
techniques in froth flotation”. In: Anthony Maeder & Brian Lovell (Eds.), Proceedings of
the DICTA-95, the 3rd Conference on Digital Imaging Computing Techniques and
Applications, 371–376, 1995, Brisbane, Australia, 6–8 December, St John's College,
University of Queensland.
[7] D.W. Moolman, C. Aldrich, J.S.J. van Deventer and W.W. Stange. “Digital image
processing as a tool for on-line monitoring of froth in flotation plants”. Minerals
Engineering 7 (9), 1149–1164, 1994.
[8] J.M. Jou, P.-Y. Chen, and J.-M. Sun, “The gray prediction search algorithm for block
motion estimation,” IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 6, pp. 843–848,
Sep. 1999.
[9] T. Koga, K. Linuma, A. Hirano, Y. Lijima, and T. Ishiguro, “Motion-compensated
interframe coding for video conferencing”, in Proc. NTC’81, 1981, pp. G5.3.1–
G5.3.5.
[10] Renxiang Li, Bing Zeng, and Ming L. Liou, “A New Three-Step Search Algorithm for Block
Motion Estimation”, IEEE Trans. Circuits And Systems For Video Technology, vol 4., no.
4, pp. 438-442, August 1994.

[11] L.M. Po and W.C. Ma, “A novel four-step search algorithm for fast block motion
estimation”, IEEE Trans. Circuits Syst. Video Technol., vol.6, no. 3, pp. 313–317, Jun.
1996.
[12] J. Jain and A. Jain, “Displacement measurement and its application in internal image
coding”, IEEE Trans. Commun., vol. 29, no. COM–12, pp. 1799–1808, Dec. 1981.
[13] S. Zhu and K.-K. Ma, “A new diamond search algorithm for fast block matching motion
estimation”, IEEE Trans. Image Process., vol. 9, no. 2,pp. 287–290,Feb.2000.
[14] C.E. Shannon “A mathematical theory of communication,” Bell System technical Journal,
vol.27, pp 379-423, 1948.
[15] F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality
image registration by maximization of mutual information”, IEEE Trans. Med. Imag., vol.
16, Apr. 1997.
[16] W.M. Wells III, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis, “Multi-modal volume
registration by maximization of mutual information”, Med. Imag. Anal., vol. 1, pp. 35–51,
1996.
[17] P. Thevenaz and M. Unser, “Optimization of mutual information for multiresolution image
registration”, IEEE Trans. Image Processing, vol. 9, pp. 2083–2099, Dec. 2000.
[18] J. C. Spall, “Multivariate stochastic approximation using a simultaneous perturbation
gradient approximation,” IEEE Trans. Automat. Contr., vol.37, no. 3, pp. 332–341, 1992.

Robust Block-Matching Motion Estimation of Flotation Froth Using Mutual Information

More Related Content

What's hot (18)

Viewers also liked (18)

Similar to Robust Block-Matching Motion Estimation of Flotation Froth Using Mutual Information (20)

Recently uploaded (20)

Robust Block-Matching Motion Estimation of Flotation Froth Using Mutual Information