SlideShare a Scribd company logo

A Fast Hadamard Transform for Signals with Sub-linear Sparsity

Robin Scheibler, profile picture
Robin Scheibler

The document discusses the development of a Sparse Fast Hadamard Transform (Sparse FHT) algorithm which efficiently computes the non-zero coefficients of k-sparse signals in a universal setting. It highlights the algorithm's improvements over traditional Hadamard transforms, achieving faster computation times and reduced sample complexity. Empirical results and analyses of failure probabilities reinforce the efficacy of this new algorithm.

Engineering
1 of 83
Downloaded 13 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion A Fast Hadamard Transform for Signals with Sub-linear Sparsity Robin Scheibler Saeid Haghighatshoar Martin Vetterli School of Computer and Communication Sciences École Polytechnique Fédérale de Lausanne, Switzerland October 28, 2013 SparseFHT 1 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Why the Hadamard transform ? I Historically, low computation approximation to DFT. I Coding, 1969 Mariner Mars probe. I Communication, orthogonal codes in WCDMA. I Compressed sensing, maximally incoherent with Dirac basis. I Spectroscopy, design of instruments with lower noise. I Recent advances in sparse FFT. 16 ⇥ 16 Hadamard matrix Mariner probe SparseFHT 2 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Contribution: Sparse fast Hadamard transform Assumptions I The signal is exaclty K-sparse in the transform domain. I Sub-linear sparsity regime K = O(N↵), 0 < ↵ < 1. I Support of the signal is uniformly random. Contribution An algorithm computing the K non-zero coefficients with: I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of failure asymptotically vanishes. SparseFHT 4 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Contribution: Sparse fast Hadamard transform Assumptions I The signal is exaclty K-sparse in the transform domain. I Sub-linear sparsity regime K = O(N↵), 0 < ↵ < 1. I Support of the signal is uniformly random. Contribution An algorithm computing the K non-zero coefficients with: I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of failure asymptotically vanishes. SparseFHT 4 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Outline 1. Sparse FHT algorithm 2. Analysis of probability of failure 3. Empirical results SparseFHT 5 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. I = {0, . . . , 23 1} SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. I = {(0, 0, 0), . . . , (1, 1, 1)} SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) Xk0,...,kn 1 = 1X m0=0 · · · 1X mn 1=0 ( 1)k0m0+···+kn 1mn 1 xm0,...,mn 1 , SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) Xk = X m2Fn 2 ( 1)hk , mi xm, k, m 2 Fn 2, hk , mi = n 1X i=0 ki mi. Treat indices as binary vectors. SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) Xk = X m2Fn 2 ( 1)hk , mi xm, k, m 2 Fn 2, hk , mi = n 1X i=0 ki mi. Treat indices as binary vectors. SparseFHT 6 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. SparseFHT 7 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. SparseFHT 7 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. (0,0,0) (0,1,0) (1,0,1) (1,1,1)(0,1,1)(0,0,1) (1,0,0) (1,1,0) (0,0,0) (0,1,0) (1,0,1) (1,1,1) (0,1,1) (0,0,1) (1,0,0) (1,1,0) WHT SparseFHT 7 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. (0,0,0) (0,1,0) (0,1,1)(0,0,1) (0,0,0) (0,1,0) (1,0,1) (1,1,1) (0,1,1) (0,0,1) (1,0,0) (1,1,0) (1,0,1) (1,1,1) (1,0,0) (1,1,0) WHT SparseFHT 7 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT 4-WHT Checks Variables I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Success Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property II: shift/modulation Theorem (shift/modulation) Given p 2 Fn 2, xm+p WHT ! Xk ( 1)hp , ki . Consequence The signal can be modulated in frequency by manipulating the time-domain samples. SparseFHT 10 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property II: shift/modulation Theorem (shift/modulation) Given p 2 Fn 2, xm+p WHT ! Xk ( 1)hp , ki . Consequence The signal can be modulated in frequency by manipulating the time-domain samples. SparseFHT 10 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Collision: if 2 variables connected to same check. I Xi ( 1)hp , ii+Xj ( 1)hp , ji Xi +Xj 6= ±1, (mild assumption on distribution of X). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Collision: if 2 variables connected to same check. I Xi ( 1)hp , ii+Xj ( 1)hp , ji Xi +Xj 6= ±1, (mild assumption on distribution of X). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Collision: if 2 variables connected to same check. I Xi ( 1)hp , ii+Xj ( 1)hp , ji Xi +Xj 6= ±1, (mild assumption on distribution of X). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Sparse fast Hadamard transform Algorithm 1. Set number of checks per downsampling B = O(K). 2. Choose C downsampling matrices H1, . . . , HC. 3. Compute C(log2 N/K + 1) size-K fast Hadamard transform, each takes O(K log2 K). 4. Decode non-zero coefficients using peeling decoder. Performance I Time complexity – O(K log2 K log2 N/K). I Sample complexity – O(K log2 N K ). I How to construct H1, . . . , HC ? Probability of success ? SparseFHT 12 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Sparse fast Hadamard transform Algorithm 1. Set number of checks per downsampling B = O(K). 2. Choose C downsampling matrices H1, . . . , HC. 3. Compute C(log2 N/K + 1) size-K fast Hadamard transform, each takes O(K log2 K). 4. Decode non-zero coefficients using peeling decoder. Performance I Time complexity – O(K log2 K log2 N/K). I Sample complexity – O(K log2 N K ). I How to construct H1, . . . , HC ? Probability of success ? SparseFHT 12 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Sparse fast Hadamard transform Algorithm 1. Set number of checks per downsampling B = O(K). 2. Choose C downsampling matrices H1, . . . , HC. 3. Compute C(log2 N/K + 1) size-K fast Hadamard transform, each takes O(K log2 K). 4. Decode non-zero coefficients using peeling decoder. Performance I Time complexity – O(K log2 K log2 N/K). I Sample complexity – O(K log2 N K ). I How to construct H1, . . . , HC ? Probability of success ? SparseFHT 12 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion SparseFHT – Probability of success Probability of success - N = 222 0 1/3 2/3 1 0 0.2 0.4 0.6 0.8 1 ↵ SparseFHT 16 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion SparseFHT vs. FHT Runtime [µs] – N = 215 0 1/3 2/3 1 0 200 400 600 800 1000 Sparse FHT FHT ↵ SparseFHT 17 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Conclusion Contribution I Sparse fast Hadamard algorithm. I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of success asymptotically equal to 1. What’s next ? I Investigate noisy case. SparseFHT 18 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Conclusion Contribution I Sparse fast Hadamard algorithm. I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of success asymptotically equal to 1. What’s next ? I Investigate noisy case. SparseFHT 18 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Thanks for your attention! Code and figures available at http://lcav.epfl.ch/page-99903.html SparseFHT 19 / 20 EPFL
Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Reference [1] M.G. Luby, M. Mitzenmacher, M.A. Shokrollahi, and D.A. Spielman, Efficient erasure correcting codes, IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 569–584, 2001. [2] S. Pawar and K. Ramchandran, Computing a k-sparse n-length discrete Fourier transform using at most 4k samples and O(k log k) complexity, arXiv.org, vol. cs.DS. 04-May-2013. SparseFHT 20 / 20 EPFL

More Related Content

PDF
Dsp 2018 foehu - lec 10 - multi-rate digital signal processing
Amr E. Mohamed
 
PDF
DSP_FOEHU - Lec 08 - The Discrete Fourier Transform
Amr E. Mohamed
 
PDF
DSP_2018_FOEHU - Lec 08 - The Discrete Fourier Transform
Amr E. Mohamed
 
PDF
DSP_FOEHU - Lec 09 - Fast Fourier Transform
Amr E. Mohamed
 
PDF
DSP_FOEHU - MATLAB 04 - The Discrete Fourier Transform (DFT)
Amr E. Mohamed
 
PDF
Unit 8
Harish kumar Lekkala
 
PDF
DSP_FOEHU - Lec 07 - Digital Filters
Amr E. Mohamed
 
PPT
Seminar 20091023 heydt_presentation
douglaslyon
 
Dsp 2018 foehu - lec 10 - multi-rate digital signal processing
Amr E. Mohamed
 
DSP_FOEHU - Lec 08 - The Discrete Fourier Transform
Amr E. Mohamed
 
DSP_2018_FOEHU - Lec 08 - The Discrete Fourier Transform
Amr E. Mohamed
 
DSP_FOEHU - Lec 09 - Fast Fourier Transform
Amr E. Mohamed
 
DSP_FOEHU - MATLAB 04 - The Discrete Fourier Transform (DFT)
Amr E. Mohamed
 
Unit 8
Harish kumar Lekkala
 
DSP_FOEHU - Lec 07 - Digital Filters
Amr E. Mohamed
 
Seminar 20091023 heydt_presentation
douglaslyon
 

What's hot (18)

PDF
Dsp U Lec08 Fir Filter Design
taha25
 
PDF
Hm2513521357
IJERA Editor
 
PDF
DSP_2018_FOEHU - Lec 06 - FIR Filter Design
Amr E. Mohamed
 
PDF
DSP_FOEHU - Lec 10 - FIR Filter Design
Amr E. Mohamed
 
PDF
Dft and its applications
Agam Goel
 
PPTX
fft using labview
kiranrockz
 
PDF
Current limitations of sequential inference in general hidden Markov models
Pierre Jacob
 
PDF
Lecture 9
Wael Sharba
 
PDF
10.1.1.2.9988
ahmedmudher
 
PDF
DPPs everywhere: repulsive point processes for Monte Carlo integration, signa...
Advanced-Concepts-Team
 
PDF
RSS discussion of Girolami and Calderhead, October 13, 2010
Christian Robert
 
PDF
Towards a stable definition of Algorithmic Randomness
Hector Zenil
 
PDF
Nyquist criterion for zero ISI
Gunasekara Reddy
 
PDF
An Intuitive Approach to Fourier Optics
jose0055
 
PPTX
Fourier transforms
Fahad B. Mostafa
 
PDF
A novel particle swarm optimization for papr reduction of ofdm systems
aliasghar1989
 
PDF
Smooth entropies a tutorial
wtyru1989
 
PPT
Optics Fourier Transform Ii
diarmseven
 
Dsp U Lec08 Fir Filter Design
taha25
 
Hm2513521357
IJERA Editor
 
DSP_2018_FOEHU - Lec 06 - FIR Filter Design
Amr E. Mohamed
 
DSP_FOEHU - Lec 10 - FIR Filter Design
Amr E. Mohamed
 
Dft and its applications
Agam Goel
 
fft using labview
kiranrockz
 
Current limitations of sequential inference in general hidden Markov models
Pierre Jacob
 
Lecture 9
Wael Sharba
 
10.1.1.2.9988
ahmedmudher
 
DPPs everywhere: repulsive point processes for Monte Carlo integration, signa...
Advanced-Concepts-Team
 
RSS discussion of Girolami and Calderhead, October 13, 2010
Christian Robert
 
Towards a stable definition of Algorithmic Randomness
Hector Zenil
 
Nyquist criterion for zero ISI
Gunasekara Reddy
 
An Intuitive Approach to Fourier Optics
jose0055
 
Fourier transforms
Fahad B. Mostafa
 
A novel particle swarm optimization for papr reduction of ofdm systems
aliasghar1989
 
Smooth entropies a tutorial
wtyru1989
 
Optics Fourier Transform Ii
diarmseven
 
Ad

Viewers also liked (11)

PDF
03 image transform
Rumah Belajar
 
PPTX
Visible Light Communication
Meherunnesha (Nishat)
 
PPTX
VISIBLE LIGHT COMMUNICATION
Maheshnagakumar Tokala
 
PPT
LiFi Visible light Communication technology
hawkisk
 
PPTX
Soil pollution, health effect of the soil
Jasmine John
 
PDF
visible light communication
Hossam Zein
 
PPT
Lifi ppt
rr140688
 
PPTX
UWB and applications
Thomas George C
 
PPTX
Visible light communication
Saginesh Mk
 
PPTX
Visible light communication
Parth Saxena
 
PPTX
Visible light communication
Naveen Sihag
 
03 image transform
Rumah Belajar
 
Visible Light Communication
Meherunnesha (Nishat)
 
VISIBLE LIGHT COMMUNICATION
Maheshnagakumar Tokala
 
LiFi Visible light Communication technology
hawkisk
 
Soil pollution, health effect of the soil
Jasmine John
 
visible light communication
Hossam Zein
 
Lifi ppt
rr140688
 
UWB and applications
Thomas George C
 
Visible light communication
Saginesh Mk
 
Visible light communication
Parth Saxena
 
Visible light communication
Naveen Sihag
 
Ad

Similar to A Fast Hadamard Transform for Signals with Sub-linear Sparsity (20)

PDF
Performance analysis of wavelet based blind detection and hop time estimation...
Saira Shahid
 
PDF
Kanal wireless dan propagasi
Mochamad Guntur Hady Putra
 
PDF
Lecture 3 sapienza 2017
Franco Bontempi Org Didattica
 
PDF
FPGA Implementation of Large Area Efficient and Low Power Geortzel Algorithm ...
IDES Editor
 
PPTX
Multi resolution & Wavelet Transform.pptx
sushreesatvatee
 
PDF
hankel_norm approximation_fir_ ijc
Vasilis Tsoulkas
 
PDF
Estimation of the score vector and observed information matrix in intractable...
Pierre Jacob
 
PDF
EC8553 Discrete time signal processing
ssuser2797e4
 
PPT
signal space analysis.ppt
PatrickMumba7
 
PDF
Pres metabief2020jmm
Mercier Jean-Marc
 
PDF
H infinity optimal_approximation_for_cau
Al Vc
 
PDF
Nc2421532161
IJERA Editor
 
PDF
"Evaluation of the Hilbert Huang transformation of transient signals for brid...
TRUSS ITN
 
PDF
conference_poster_4
Jiayi Jiang
 
PDF
Er24902905
IJERA Editor
 
PDF
Lt2419681970
IJERA Editor
 
PDF
SIMULATION OF FIR FILTER BASED ON CORDIC ALGORITHM
VLSICS Design
 
PDF
SIMULATION OF FIR FILTER BASED ON CORDIC ALGORITHM
VLSICS Design
 
PDF
DESIGN OF QUATERNARY LOGICAL CIRCUIT USING VOLTAGE AND CURRENT MODE LOGIC
VLSICS Design
 
PDF
Integration with kernel methods, Transported meshfree methods
Mercier Jean-Marc
 
Performance analysis of wavelet based blind detection and hop time estimation...
Saira Shahid
 
Kanal wireless dan propagasi
Mochamad Guntur Hady Putra
 
Lecture 3 sapienza 2017
Franco Bontempi Org Didattica
 
FPGA Implementation of Large Area Efficient and Low Power Geortzel Algorithm ...
IDES Editor
 
Multi resolution & Wavelet Transform.pptx
sushreesatvatee
 
hankel_norm approximation_fir_ ijc
Vasilis Tsoulkas
 
Estimation of the score vector and observed information matrix in intractable...
Pierre Jacob
 
EC8553 Discrete time signal processing
ssuser2797e4
 
signal space analysis.ppt
PatrickMumba7
 
Pres metabief2020jmm
Mercier Jean-Marc
 
H infinity optimal_approximation_for_cau
Al Vc
 
Nc2421532161
IJERA Editor
 
"Evaluation of the Hilbert Huang transformation of transient signals for brid...
TRUSS ITN
 
conference_poster_4
Jiayi Jiang
 
Er24902905
IJERA Editor
 
Lt2419681970
IJERA Editor
 
SIMULATION OF FIR FILTER BASED ON CORDIC ALGORITHM
VLSICS Design
 
SIMULATION OF FIR FILTER BASED ON CORDIC ALGORITHM
VLSICS Design
 
DESIGN OF QUATERNARY LOGICAL CIRCUIT USING VOLTAGE AND CURRENT MODE LOGIC
VLSICS Design
 
Integration with kernel methods, Transported meshfree methods
Mercier Jean-Marc
 

Recently uploaded (20)

PPT
Project quality management in manufacturing
BarMusaTetik
 
PPTX
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
bapushinde7218
 
PPTX
Current and future trends in Computer Vision.pptx
Subramanyam Natarajan
 
PPTX
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
ShriNRPrasad
 
PPTX
Construction Project Organization Group 2.pptx
vj5agdales
 
DOCX
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
AYUSHMANAV
 
PDF
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
IJAEMSJORNAL
 
PPTX
bas. eng. economics group 4 presentation 1.pptx
kiribakimoses1993
 
PDF
Automation-in-Manufacturing-Chapter-Introduction.pdf
pcmth867
 
PDF
R24 SURVEYING LAB MANUAL for civil enggi
MNANDITHACIVILSTAFF
 
PPTX
UNIT-1 - COAL BASED THERMAL POWER PLANTS
Chandra Kumar S
 
PPTX
additive manufacturing of ss316l using mig welding
premcdr7799
 
PPTX
Geodesy 1.pptx...............................................
abhi1361yadav
 
PDF
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
jpsjournal1
 
PPT
Mechanical Engineering MATERIALS Selection
arsalanahmad705384
 
PPT
Introduction, IoT Design Methodology, Case Study on IoT System for Weather Mo...
MariaRufina4
 
PDF
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
IJCNCJournal
 
PDF
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
AinieButt1
 
PDF
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
PRIZ Guru
 
PPTX
Safety Seminar civil to be ensured for safe working.
DILJEETKUMAR8
 
Project quality management in manufacturing
BarMusaTetik
 
Infosys Presentation by1.Riyan Bagwan 2.Samadhan Naiknavare 3.Gaurav Shinde 4...
bapushinde7218
 
Current and future trends in Computer Vision.pptx
Subramanyam Natarajan
 
M Tech Sem 1 Civil Engineering Environmental Sciences.pptx
ShriNRPrasad
 
Construction Project Organization Group 2.pptx
vj5agdales
 
ASol_English-Language-Literature-Set-1-27-02-2023-converted.docx
AYUSHMANAV
 
Mitigating Risks through Effective Management for Enhancing Organizational Pe...
IJAEMSJORNAL
 
bas. eng. economics group 4 presentation 1.pptx
kiribakimoses1993
 
Automation-in-Manufacturing-Chapter-Introduction.pdf
pcmth867
 
R24 SURVEYING LAB MANUAL for civil enggi
MNANDITHACIVILSTAFF
 
UNIT-1 - COAL BASED THERMAL POWER PLANTS
Chandra Kumar S
 
additive manufacturing of ss316l using mig welding
premcdr7799
 
Geodesy 1.pptx...............................................
abhi1361yadav
 
Evaluating the Democratization of the Turkish Armed Forces from a Normative P...
jpsjournal1
 
Mechanical Engineering MATERIALS Selection
arsalanahmad705384
 
Introduction, IoT Design Methodology, Case Study on IoT System for Weather Mo...
MariaRufina4
 
Enhancing Cyber Defense Against Zero-Day Attacks using Ensemble Neural Networks
IJCNCJournal
 
TFEC-4-2020-Design-Guide-for-Timber-Roof-Trusses.pdf
AinieButt1
 
PRIZ Academy - 9 Windows Thinking Where to Invest Today to Win Tomorrow.pdf
PRIZ Guru
 
Safety Seminar civil to be ensured for safe working.
DILJEETKUMAR8
 

A Fast Hadamard Transform for Signals with Sub-linear Sparsity

  • 1. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion A Fast Hadamard Transform for Signals with Sub-linear Sparsity Robin Scheibler Saeid Haghighatshoar Martin Vetterli School of Computer and Communication Sciences École Polytechnique Fédérale de Lausanne, Switzerland October 28, 2013 SparseFHT 1 / 20 EPFL
  • 2. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Why the Hadamard transform ? I Historically, low computation approximation to DFT. I Coding, 1969 Mariner Mars probe. I Communication, orthogonal codes in WCDMA. I Compressed sensing, maximally incoherent with Dirac basis. I Spectroscopy, design of instruments with lower noise. I Recent advances in sparse FFT. 16 ⇥ 16 Hadamard matrix Mariner probe SparseFHT 2 / 20 EPFL
  • 3. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
  • 4. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
  • 5. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
  • 6. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Fast Hadamard transform I Butterfly structure similar to FFT. I Time complexity O(N log2 N). I Sample complexity N. + Universal, i.e. works for all signals. Does not exploit signal structure (e.g. sparsity). Can we do better ? SparseFHT 3 / 20 EPFL
  • 7. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Contribution: Sparse fast Hadamard transform Assumptions I The signal is exaclty K-sparse in the transform domain. I Sub-linear sparsity regime K = O(N↵), 0 < ↵ < 1. I Support of the signal is uniformly random. Contribution An algorithm computing the K non-zero coefficients with: I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of failure asymptotically vanishes. SparseFHT 4 / 20 EPFL
  • 8. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Contribution: Sparse fast Hadamard transform Assumptions I The signal is exaclty K-sparse in the transform domain. I Sub-linear sparsity regime K = O(N↵), 0 < ↵ < 1. I Support of the signal is uniformly random. Contribution An algorithm computing the K non-zero coefficients with: I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of failure asymptotically vanishes. SparseFHT 4 / 20 EPFL
  • 9. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Outline 1. Sparse FHT algorithm 2. Analysis of probability of failure 3. Empirical results SparseFHT 5 / 20 EPFL
  • 10. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. I = {0, . . . , 23 1} SparseFHT 6 / 20 EPFL
  • 11. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. I = {(0, 0, 0), . . . , (1, 1, 1)} SparseFHT 6 / 20 EPFL
  • 12. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
  • 13. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
  • 14. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
  • 15. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) SparseFHT 6 / 20 EPFL
  • 16. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) Xk0,...,kn 1 = 1X m0=0 · · · 1X mn 1=0 ( 1)k0m0+···+kn 1mn 1 xm0,...,mn 1 , SparseFHT 6 / 20 EPFL
  • 17. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) Xk = X m2Fn 2 ( 1)hk , mi xm, k, m 2 Fn 2, hk , mi = n 1X i=0 ki mi. Treat indices as binary vectors. SparseFHT 6 / 20 EPFL
  • 18. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Another look at the Hadamard transform I Consider indices of x 2 RN, N = 2n. I Take the binary expansion of indices. I Represent signal on hypercube. I Take DFT in every direction. (0,0,0) (0,1,0) (1,0,1) (1,1,1) (1,1,0)(1,0,0) (0,0,1) (0,1,1) Xk = X m2Fn 2 ( 1)hk , mi xm, k, m 2 Fn 2, hk , mi = n 1X i=0 ki mi. Treat indices as binary vectors. SparseFHT 6 / 20 EPFL
  • 19. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. SparseFHT 7 / 20 EPFL
  • 20. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. SparseFHT 7 / 20 EPFL
  • 21. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. (0,0,0) (0,1,0) (1,0,1) (1,1,1)(0,1,1)(0,0,1) (1,0,0) (1,1,0) (0,0,0) (0,1,0) (1,0,1) (1,1,1) (0,1,1) (0,0,1) (1,0,0) (1,1,0) WHT SparseFHT 7 / 20 EPFL
  • 22. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property I: downsampling/aliasing Given B = 2b, a divider of N = 2n, and H 2 Fb⇥n 2 , where rows of H are a subset of rows of identity matrix, xHT m WHT ! X i2N(H) XHT k+i, m, k 2 Fb 2. e.g. H = ⇥ 0b⇥(n b) Ib ⇤ selects the b high order bits. (0,0,0) (0,1,0) (0,1,1)(0,0,1) (0,0,0) (0,1,0) (1,0,1) (1,1,1) (0,1,1) (0,0,1) (1,0,0) (1,1,0) (1,0,1) (1,1,1) (1,0,0) (1,1,0) WHT SparseFHT 7 / 20 EPFL
  • 23. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
  • 24. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
  • 25. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
  • 26. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
  • 27. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT 4-WHT I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
  • 28. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Aliasing induced bipartite graph time-domain Hadamard-domain 4-WHT 4-WHT Checks Variables I Downsampling induces an aliasing pattern. I Different downsamplings produce different patterns. SparseFHT 8 / 20 EPFL
  • 29. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 30. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 31. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 32. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 33. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 34. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 35. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Genie-aided peeling decoder A genie indicates us I if a check is connected to only one variable (singleton), I in that case, the genie also gives the index of that variable. Success Peeling decoder algorithm: 1. Find a singleton check: {X1, X8, X11} 2. Peel it off. 3. Repeat until nothing left. SparseFHT 9 / 20 EPFL
  • 36. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property II: shift/modulation Theorem (shift/modulation) Given p 2 Fn 2, xm+p WHT ! Xk ( 1)hp , ki . Consequence The signal can be modulated in frequency by manipulating the time-domain samples. SparseFHT 10 / 20 EPFL
  • 37. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Hadamard property II: shift/modulation Theorem (shift/modulation) Given p 2 Fn 2, xm+p WHT ! Xk ( 1)hp , ki . Consequence The signal can be modulated in frequency by manipulating the time-domain samples. SparseFHT 10 / 20 EPFL
  • 38. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Collision: if 2 variables connected to same check. I Xi ( 1)hp , ii+Xj ( 1)hp , ji Xi +Xj 6= ±1, (mild assumption on distribution of X). SparseFHT 11 / 20 EPFL
  • 39. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Collision: if 2 variables connected to same check. I Xi ( 1)hp , ii+Xj ( 1)hp , ji Xi +Xj 6= ±1, (mild assumption on distribution of X). SparseFHT 11 / 20 EPFL
  • 40. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Collision: if 2 variables connected to same check. I Xi ( 1)hp , ii+Xj ( 1)hp , ji Xi +Xj 6= ±1, (mild assumption on distribution of X). SparseFHT 11 / 20 EPFL
  • 41. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
  • 42. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
  • 43. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
  • 44. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion How to construct the Genie Non-modulated Modulated I Singleton: only one variable connected to check. I Xi ( 1)hp , ii Xi = ( 1)hp , ii = ±1. We can know hp , ii! I O(log2 N K ) measurements sufficient to recover index i, (dimension of null-space of downsampling matrix H). SparseFHT 11 / 20 EPFL
  • 45. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Sparse fast Hadamard transform Algorithm 1. Set number of checks per downsampling B = O(K). 2. Choose C downsampling matrices H1, . . . , HC. 3. Compute C(log2 N/K + 1) size-K fast Hadamard transform, each takes O(K log2 K). 4. Decode non-zero coefficients using peeling decoder. Performance I Time complexity – O(K log2 K log2 N/K). I Sample complexity – O(K log2 N K ). I How to construct H1, . . . , HC ? Probability of success ? SparseFHT 12 / 20 EPFL
  • 46. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Sparse fast Hadamard transform Algorithm 1. Set number of checks per downsampling B = O(K). 2. Choose C downsampling matrices H1, . . . , HC. 3. Compute C(log2 N/K + 1) size-K fast Hadamard transform, each takes O(K log2 K). 4. Decode non-zero coefficients using peeling decoder. Performance I Time complexity – O(K log2 K log2 N/K). I Sample complexity – O(K log2 N K ). I How to construct H1, . . . , HC ? Probability of success ? SparseFHT 12 / 20 EPFL
  • 47. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Sparse fast Hadamard transform Algorithm 1. Set number of checks per downsampling B = O(K). 2. Choose C downsampling matrices H1, . . . , HC. 3. Compute C(log2 N/K + 1) size-K fast Hadamard transform, each takes O(K log2 K). 4. Decode non-zero coefficients using peeling decoder. Performance I Time complexity – O(K log2 K log2 N/K). I Sample complexity – O(K log2 N K ). I How to construct H1, . . . , HC ? Probability of success ? SparseFHT 12 / 20 EPFL
  • 48. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
  • 49. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
  • 50. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
  • 51. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
  • 52. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
  • 53. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Very sparse regime Setting I K = O(N↵), 0 < ↵ < 1/3. I Uniformly random support. I Study asymptotic probability of failure as n ! 1. Downsampling matrices construction I Achieves values ↵ = 1 C , i.e. b = n C . I Deterministic downsampling matrices H1, . . . , HC, SparseFHT 13 / 20 EPFL
  • 54. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 55. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 56. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 57. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 58. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Uniformly random support model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 59. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 60. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 61. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 62. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 63. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 64. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 65. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 66. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 67. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Balls-and-bins model Balls-and-bins model I Theorem: Both constructions are equivalent. Proof: By construction, all rows of Hi are linearly independent. I Reduces to LDPC decoding analysis. I Error correcting code design (Luby et al. 2001). I FFAST (Sparse FFT algorithm) (Pawar & Ramchandran 2013). SparseFHT 14 / 20 EPFL
  • 68. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 69. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 70. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 71. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 72. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 73. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 74. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 75. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 76. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 77. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Extension to less-sparse regime I K = O(N↵), 2/3  ↵ < 1. I Balls-and-bins model not equivalent anymore. I Let ↵ = 1 1 C . Construct H1, . . . , HC, I By construction: N(Hi) T N(Hj) = 0. SparseFHT 15 / 20 EPFL
  • 78. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion SparseFHT – Probability of success Probability of success - N = 222 0 1/3 2/3 1 0 0.2 0.4 0.6 0.8 1 ↵ SparseFHT 16 / 20 EPFL
  • 79. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion SparseFHT vs. FHT Runtime [µs] – N = 215 0 1/3 2/3 1 0 200 400 600 800 1000 Sparse FHT FHT ↵ SparseFHT 17 / 20 EPFL
  • 80. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Conclusion Contribution I Sparse fast Hadamard algorithm. I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of success asymptotically equal to 1. What’s next ? I Investigate noisy case. SparseFHT 18 / 20 EPFL
  • 81. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Conclusion Contribution I Sparse fast Hadamard algorithm. I Time complexity O(K log2 K log2 N K ). I Sample complexity O(K log2 N K ). I Probability of success asymptotically equal to 1. What’s next ? I Investigate noisy case. SparseFHT 18 / 20 EPFL
  • 82. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Thanks for your attention! Code and figures available at http://lcav.epfl.ch/page-99903.html SparseFHT 19 / 20 EPFL
  • 83. Introduction Sparse FHT algorithm Analysis of probability of failure Empirical results Conclusion Reference [1] M.G. Luby, M. Mitzenmacher, M.A. Shokrollahi, and D.A. Spielman, Efficient erasure correcting codes, IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 569–584, 2001. [2] S. Pawar and K. Ramchandran, Computing a k-sparse n-length discrete Fourier transform using at most 4k samples and O(k log k) complexity, arXiv.org, vol. cs.DS. 04-May-2013. SparseFHT 20 / 20 EPFL