Reshaping Core Genomics Software Tools for the Manycore Era

Ben Langmead
Assistant Professor
Johns Hopkins University, Department of Computer Science
November 2016
Reshaping core genomics software
tools for the many-core era

3
The Intel Parallel Computing Center at

4
Outline
§  Introduction to sequencing data analysis & Bowtie
§  Thread scaling improvements using TBB
–  Choice of mutex
–  Two-stage parsing
§  AVX2, AVX512-KNC & AVX512-KNC improvements
§  Impact on the field

9
Read alignment
§  Needle in a haystack
§  Billions of reads from
a single week-long
sequencing run
§  Human reference
genome is ~3B bases
(letters) long

10
Bowtie and Bowtie 2
§  Together cited by
>12K other scientific
studies since 2009
§  Bundled with dozens
of other tools & many
Linux distros

11
HISAT
§  Based on Bowtie 2 and a leading spliced aligner for RNA sequencing data
§  Cited in >75 scientific studies since 2015

12
Design of Bowtie & Bowtie 2
Bowtie 1
Bowtie 2

13
Bowtie 1
Bowtie 2
Random access to large index
data structure and minimal ILP

14
Bowtie 1
Bowtie 2
Dynamic programming, lots of ILP

0
250
500
750
1000
0 25 50 75 100 125
# threads (unpaired)
Normalizedrunningtime
lock
TBB spin_mutex
tinythreads fast_mutex
15
Thread scaling
§  Switching to analogous TBB lock could bring big improvement
Ivy Bridge, 4 NUMA nodes,
120 threads
Vertical axis is per-
thread running time;
lower is better
Bowtie 1 unpaired

100
150
200
250
300
350
0 25 50 75 100 125
lock
None (stubbed I/O)
TBB spin_mutex
version
Original parsing
16
Thread scaling
§  Removing synchronization by “stubbing” input lock gives further improvement
Bowtie 2 unpairedIvy Bridge, 4 NUMA nodes,
120 threads
Vertical axis is per-thread
running time; lower is better

17
Thread scaling
§  Vtune investigation indicates synchronization itself (e.g. see __TBB_LockByte)
is taking the time

18
Thread scaling
Bowtie 2 unpaired
How to close the
gap between
actual and ideal
performance?

19
Thread scaling
Bowtie 2 unpaired
Why does mutex
choice have outsize
effect?

CMU 15-418/618, Spring 2015
Test-and-set lock performance
Benchmark&executes:&
lock(L);&
critical>section(c)&
unlock(L);
Time(us)
Number of processors
Benchmark: total of N lock/unlock sequences (in aggregate) by P processors
Critical section time removed so graph plots only time acquiring/releasing the lock
Bus contention increases amount of
time to transfer lock (lock holder must
wait to acquire bus to release)
Not shown: bus contention also slows
down execution of critical section
Figure credit: Culler, Singh, and Gupta
20
Thread scaling
§  Mutex spinning on atomic op
(compare-and-swap, test-and-
set), spurs exchange of cache
coherence messages
§  Image by Kayvon Fatahalian,
Copyright 2015 Carnegie
Mellon University

21
Thread scaling
§  Even a standard pthreads mutex was outperforming the
spin lock when running one thread per available core
–  More evidence that cache coherence traffic is culprit
§  Queue locks are known to have better cache properties
–  Waiting thread spins on normal (non-atomic) read
–  Cache line read belongs exclusively to that thread
and can live in L1

22
Thread scaling
§  We hypothesized a NUMA-aware “cohort lock” could help further
Dice, David, Virendra J. Marathe, and Nir Shavit. "Lock cohorting: a general technique for
designing NUMA locks." ACM SIGPLAN Notices. Vol. 47. No. 8. ACM, 2012.

23
Cohort locking
class
CohortLock
{

public:

CohortLock()
:
lockers_numa_idx(-‐1)
{

starvation_counters
=
new
int[MAX_NODES]();

own_global
=
new
bool[MAX_NODES]();

local_locks
=
new
TKTLock[MAX_NODES];

}

~CohortLock()
{

delete[]
starvation_counters;

delete[]
own_global;

delete[]
local_locks;

}

void
lock();

void
unlock();

private:

static
const
int
STARVATION_LIMIT
=
100;

static
const
int
MAX_NODES
=
128;

volatile
int*

starvation_counters;
//
1
per
node

volatile
bool*
own_global;

//
1
per
node

volatile
int

lockers_numa_idx;

//
1
per
node

TKTLock*
local_locks;

//
1
per
node

PTLLock
global_lock;

};

§  Each NUMA node has per-node ticket lock
§  Other per-node information tracks when to
pass lock to other threads on same node
§  Single global partitioned ticket lock

24
Cohort locking
void
CohortLock::lock()
{

const
int
numa_idx
=
determine_numa_idx();

local_locks[numa_idx].lock();

if(!own_global[numa_idx])
{

global_lock.lock();

}

starvation_counters[numa_idx]++;

own_global[numa_idx]
=
true;

lockers_numa_idx
=
numa_idx;

}

void
CohortLock::unlock()
{

assert(lockers_numa_idx
!=
-‐1);

int
numa_idx
=
lockers_numa_idx;

lockers_numa_idx
=
-‐1;

if(local_locks[numa_idx].q_length()
==
1
||

starvation_counters[numa_idx]
>
STARVATION_LIMIT)

{

global_lock.unlock();

starvation_counters[numa_idx]
=
0;

own_global[numa_idx]
=
false;

}

local_locks[numa_idx].unlock();

}

§  When locking:
–  Grab local lock
–  Once grabbed, grab global lock if not
already owned by this node
§  When unlocking:
–  Is another thread on same node queued?
If so, hand lock to next in queue
–  Otherwise release global & local locks
–  Override hand-off if others are starving

25
Cohort locking
§  Another implementation of cohort locking available in ConcurrencyKit:
http://concurrencykit.org
–  https://github.com/concurrencykit/ck/blob/master/include/ck_cohort.h

26
Thread scaling
§  Chris Wilks added TBB queue locks, JHU/TBB
Cohort locks (2 flavors) to Bowtie 2, Bowtie & HISAT
§  Available in public branches, with all but cohort locks
available in master branch and in recent releases

27
Thread scaling
§  Novel strategy splits input parsing into two “phases”
§  First (“light parsing”) rapidly detects record
boundaries, requiring synchronization but with very
brief critical section
§  Second (“full parsing”) fully parses each record
(pictured, right) with no synchronization
§  Minimizes time spent in crucial critical section
@ABC_123_1
GCTATTATGCTAT
+
JJSYEGGU8233^
@ABC_424_1
GTGATATGCAT
+
SYEG!U8@233
@ABCD_9_1
GCTATTATGCTATAAAC
+
JJSYEGGU8233^32FR
@D_91231_1
GCTATTATGCTAT
+
JJSYEGGU8233^
…

100
200
300
0 25 50 75 100 125
lock
None (stubbed I/O)
TBB mutex
TBB queuing_mutex
TBB spin_mutex
TBB/JHU CohortLock
28
Thread scaling: Bowtie 2 unpaired
120 threads
§  TBB queuing_mutex and TBB/JHU cohort lock perform best

100
200
300
0 25 50 75 100 125
None (stubbed I/O)
TBB mutex
TBB queuing_mutex
TBB spin_mutex
TBB/JHU CohortLock
version
Optimized parsing
Original parsing
29
Thread scaling: Bowtie 2 unpaired
120 threads
§  Two-phase parsing yields substantial thread-scaling boost; close to perfect up
to 120 threads, regardless of mutex

100
200
300
0 25 50 75 100 125
lock
None (stubbed I/O)
TBB mutex
TBB queuing_mutex
TBB spin_mutex
TBB/JHU CohortLock
30
Thread scaling: Bowtie 2 paired-end
120 threads
§  queuing_mutex and cohort lock again perform the best, near ideal

31
Thread scaling: Bowtie 2 paired-end
120 threads
§  Two-phase parsing yields substantial thread-scaling boost; close to perfect up
to 120 threads, with mutex having smaller impact

0
100
200
300
0 25 50 75 100 125
# threads (paired−end)
lock
None (stubbed I/O)
TBB mutex
TBB queuing_mutex
TBB spin_mutex
TBB/JHU CohortLock
32
Thread scaling: Bowtie
120 threads
§  As with Bowtie 2, near-ideal scaling with queuing and cohort locks

0
100
200
300
400
0 25 50 75 100 125
# threads
version
Optimized parsing
Original parsing
lock
TBB queuing_mutex
33
Thread scaling: HISAT unpaired
120 threads
§  Huge improvements with queuing_lock and two-phase parsing

34
Thread scaling
§  Further gains possible with batch parsing, where the first phase “lightly” parses
several reads at once, reducing # critical section entrances
● ● ● ● ●
● ● ● ● ●
● ● ● ● ● ● ● ●
● ●
● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
20 30 40
bowtie # threads
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●
●
● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
100
200
300
0 25 50 75 100 125
bowtie # threads
●
● ●
●
●
●
● ● ●
●
● ● ● ● ●
● ●
●
●
●
● ● ● ● ● ● ●
● ●
●
● ● ●
● ● ● ●
●
●
●
20 30 40
bowtie2 # threads
● ●
● ● ● ● ● ● ● ● ● ●
● ● ●
●
● ●
● ● ● ● ● ● ● ● ● ● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
● ●
● ● ●
●
●
● ● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
● ● ●
● ● ●
● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
30
40
50
60
70
0 25 50 75 100 125
bowtie2 # threads
●
●
●
●
●
●
●
●
● ● ●
●
● ● ● ● ● ● ● ● ●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
100
150
200
250
lizedrunningtime
20 30 40
bowtie # threads
0 25 50 75 100 125
bowtie # threads
●
● ●
●
●
●
● ● ●
●
● ● ● ● ●
● ●
●
●
●
● ● ● ● ● ● ●
● ●
●
● ● ●
● ● ● ●
●
●
●
20 30 40
bowtie2 # threads
● ●
● ● ● ● ● ● ● ● ● ●
● ● ●
●
● ●
● ● ● ● ● ● ● ● ● ● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
● ●
● ● ●
●
●
● ● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
● ● ●
● ● ●
● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
30
40
50
60
70
0 25 50 75 100 125
bowtie2 # threads
● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●
● ●
● ● ● ● ● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
20 30 40
hisat # threads
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ●
●
●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
50
100
150
200
250
0 25 50 75 100 125
hisat # threads
lock MP tinythreads fast_mutex TBB queuing_mutex
version ● ● ●Batch parsing Original parsing Two−phase parsing

35
Thread scaling: Bowtie 2 on Broadwell
§  Experiment conducted by John Oneill at Intel
§  TBB + optimized parsing yields speedups of 1.1x - 1.8x on 88 threads on
Broadwell E5-2699 v4 part. TBB/JHU Cohort lock outperforms other mutexes.

36
Thread scaling: Bowtie 2 on Knight’s Landing
§  Experiment conducted by John Oneill at Intel
§  TBB + optimized parsing yields speedups of 2x - 2.7x on 192 threads on KNL
B0 bin3 part. TBB/JHU Cohort lock outperforms other mutexes.

37
Thread scaling: summary
§  Using a queue mutex / cohort lock can yield big improvement over spin /
normal lock
§  Achieved near-ideal scaling up to 120 threads with (a) queue/cohort locks and
(b) cleaner parsing for Bowtie, Bowtie 2.
§  Promising scaling results on KNC & KNL; more to do
§  Cohort locks were best option in Broadwell & KNL experiments
§  Cohort locks seem to put KNL in a better position to outperform Xeon on
genomics workloads

38
Vectorization of Bowtie 2 inner loop
§  Dynamic programming alignment not unique to Bowtie 2
§  Common to many sequence alignment problems

39

40
The wider the vector word, the more times the fixup loop iterates
§  Mitigates the benefit of
having wider words

41
…but in some situations, the fixup loop can be skipped with little or no downside
§  Important future work is to
determine whether selective
suppression of fixup loop
can remove most or all of
the downside of having
wider words

42
Impact on the field
§  As of Bowtie 1.0.1 release / Bowtie 2 2.2.0 release, Intel improvements are “in
the wild,” assisting life science researchers

43
Impact on the field
§  Added TBB to Bowtie 1.1.2, Bowtie 2 2.2.6. Also added to public branch of
HISAT. Plan to make TBB the default threading library in upcoming release.

44
Impact on the field
§  Daehwan Kim of JHU IPCC team parallelized the index building process in
Bowtie 2; TBB version of parallel index building available as of 2.2.7

45
Impact on the field
§  With changes fully reflected in
Bowtie 1.2.0 and Bowtie 2 2.3.0,
JHU team drafting manuscript
describing improvements and
lessons learned

46
Future directions
§  Where and why does the cohort lock help?
§  Does cohort lock have a future in TBB?
§  Can selective suppression of Bowtie 2 fixup loop
unlock power of wider vector words?
§  Can all of the above yield a big Knight’s Landing
throughput win?

47
Other resources
§  http://www.langmead-lab.org
§  https://www.coursera.org/learn/dna-sequencing
–  YouTube videos for above: http://bit.ly/ADS1_videos

48
Thank you
§  John Oneill, Ram Ramanujam, Kevin O’leary, and many other great Intel
engineers we spoke to and worked with
§  Lisa Smith, Brian Napier and others in IPCC program
§  Langmead lab team: Chris Wilks, Valentin Antonescu
§  Salzberg lab team: Steven Salzberg, Daehwan Kim
§  Intel

Thank you for your time
Ben Langmead
langmea@cs.jhu.edu
www.intel.com/hpcdevcon

Reshaping Core Genomics Software Tools for the Manycore Era

More Related Content

What's hot (20)

Similar to Reshaping Core Genomics Software Tools for the Manycore Era (20)

More from Intel® Software (20)

Recently uploaded (20)

Reshaping Core Genomics Software Tools for the Manycore Era