SlideShare a Scribd company logo

Structure for scale: Dialing in your apps for optimal performance

Atlassian, profile picture
Atlassian

The document discusses upgrading a JIRA instance to improve performance by upgrading hardware including adding more CPUs, memory, and disk space as well as upgrading software including Java, the operating system, and database. It provides details on the specific hardware and software upgrades made and shows significant performance improvements such as reducing reindex times by 59% and full garbage collections by over 80%.

TechnologyNews & Politics
1 of 147
Downloaded 46 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
Structure for scale: Dialing in your apps for optimal performance
Structure for scale: Dialing in your apps for optimal performance
Performance tuning for Java applications Andrew Prentice Atlassian 2
Topics • How to make your apps run faster • Tuning ideas 3
A note • X vs Y will always depend on context • Test with your application! 4
You are here • How I did the testing • Hardware & software • Must have improvements • Jave, Hardware, Virtualization and Garbage Collection • Worth doing • Gigabit vs 100 mbit, Heap size tuning, use an SSD 5
The contenders • Dell PE 860 - “WALL-E” • Xeon X3220 2.4 Ghz Quad core • Quad Core, 2 x 4Mb L3 • 8 Gb DDR2 RAM • 2 x 7.2K SAS Disks • 1 x Intel X25-E 32 GB SSD 6
The contenders • Dell PE 2950 - “Johnny 5” • 2 x Xeon E5 405 2.0 Ghz • Quad Core, 2 x 6M L3 • 32 Gb RAM • 2 x 72 Gb 15 K Drives • ~$4000 7
The contenders • Dell PE R610 - “EVE” • 2 x Xeon E5520 2.2Ghz • Quad Core, 8M “SmartCache” • 32 Gb RAM • 2 x 146G 15K drives • ~$4000 8
The Workload • JIRA 4, MySQL 5, RHEL 5 • Java 1.6, 3G heap, ParNew, ParOld GC • ~70,000 Issues, ~80 projects • ~30 reqs/sec, JMeter 2.3.4 (patched, JSR-223) • Traffic modeled on http://jira.atlassian.com with higher writes 9
About the graphs • Most show Average Response Time in milliseconds • Includes time to generate AND deliver the response • Does not include browser time • Arrows show if lower or higher is better 10
You are here • How I did the testing • Hardware & software • Must have improvements • Jave, Hardware, Virtualization and Garbage Collection • Worth doing • Gigabit vs 100 mbit, Heap size tuning, use an SSD 11
Must. Have. 12
Upgrade Java 1.5 to 1.6 • Advances in complier/VM technology • Produces better runtime code • Able to use modern instructions • Able to use modern hardware 13
Java 1.5 vs 1.6 (Johnny 5) Java 1.5 Java 1.6.0 1500 1125 750 375 6.3X 0 Average Response Time (ms) 14
Java 1.5 vs 1.6 (Johnny 5) Java 1.5.0 Java 1.6.0 380 285 190 95 26% 0 Average CPU (%) 15
Upgrade Java to 1.6 • HotSpot VM updates in 1.6 • Advancing technology • Compressed OOPS on 64bit • Escape Analysis • NUMA • http://java.sun.com/performance/reference/ whitepapers/6_performance.html 16
Java 1.6 vs 1.6 (Eve) Java 1.6.0_7 Java 1.6.0_16 148 111 74 37 7% 0 Average Response Time (ms) 17
Hardware Upgrades • Hyper-Threading • QuickPath Interconnect • Nehalem (Eve) have on die DDR3 Memory Controllers • Memory throuput improvements - good for Java! Technology Speed DDR2-800 (dual channel) 6.4 GB/sec DDR3-1333 (dual channel) 21.3 GB/sec 18
Hyper Threading Thread 1 CPU Thread 2 Thread 1 runs until it hits a cache miss. Thread 2 runs on the same CPU while T1 is awaiting data 19
Hardware Upgrades 2.4Ghz Xeon (Wall-E) 2.0Ghz Xeon (Johnny 5) 2.2Ghz Xeon (Eve) 2000 1500 1000 500 8-12X 0 Average Response Time (ms) 20
Improved GC Throughput 2.4Ghz Xeon (Wall-E) 2.0Ghz Xeon (Johnny 5) 2.2Ghz Xeon (Eve) 30000 22500 15000 2-3X 7500 0 GC Throughput (MB/sec) 21
Garbage Collection • Java’s Virtual Machine manages memory • New objects are continually created during runtime • GC is the process of collecting dead objects to reclaim memory 22
Tune Garbage Collection • Getting your GC tuning wrong is bad • How bad? • Are the defaults sane? 23
Tune Garbage Collection CMS ParNew ParOld 4000 3000 2000 1000 2X 0 Wall-E (ms) Johnny 5 (ms) Eve (ms) Average Response Time 24
Tune Garbage Collection CMS ParNew ParOld 500 375 250 125 2X 0 Johnny 5 (ms) Eve (ms) Average Response Time 25
Tune Garbage Collection • Defaults are sane • Parallel New + Parallel Old is fastest • CMS prioritizes low pauses over CPU usage • CMS does not compact - can lead to heap fragmentation 26
Virtualisation • Does being in a VM hurt? • What if the risks are managed? 27
Virtualisation Real VMWare ESX 3.5 VMWare ESX 4i 170.0 127.5 85.0 42.5 43% 0 Average Response Time (ms) 28
Virtualisation • On average ~30% slower • Under extreme resource starvation, up to 100% slower • Avoid virtualization for high traffic instances • See the Atlassian docs for recommendations • http://confluence.atlassian.com/display/DOC/Running +Confluence+in+a+Virtualised+Environment 29 • http://confluence.atlassian.com/display/JIRA/Running+JIRA+in
You are here • How I did the testing • Hardware & software • Must have improvements • Java, Hardware, Virtualization and Garbage Collection • Worth doing • Gigabit vs 100 mbit, Heap size tuning, use an SSD 30
Maybe. 31
Gigabit Network vs 100Mbit • Gigabit is faster, sure • But 100Mb is OK if you’re not doing big transfers, RIGHT?! 32
Use Gigabit 100mb/sec 1000mb/sec 1900 1425 950 475 10% 0 Average Response Time (ms) 33
Use Gigabit • Lower RTT on Gigabit • If you have enough CPU power, keep the DB on localhost • Check your development environment 34
Use Gigabit 100Mbit 1000Mbit 200 150 100 50 26% 0 ReIndex Time (seconds) 35
Heap size tuning • Memory starvation is bad, but can we “drown” Java with too much memory? 36
Heap size - Wall-E 1.5G 3G 6G 2000 1500 1000 500 23% 0 Average Response Time 37
Heap size - Johnny 5 1.5G 3G 6G 15G 290.0 217.5 145.0 72.5 49% 0 Average Response Time 38
Heap size - Eve 1.5G 3G 6G 15G 170.0 127.5 85.0 42.5 28% 0 Average Response Time 39
Heap size tuning • Heap too small - bad • Heap too big - not bad! • Bigger heaps give the JVM more room to work • Diminishing returns from really big heaps 40
Buy an SSD • SSDs are FAST! • SSDs are EXPENSIVE! • SSDs are all the rage! • DBAs are finally happy (almost) 41
Buy an SSD SATA SSD 800 600 400 200 33% 0 Browse Issue Create Issue 42
Buy an SSD • Fast writes • 0.1ms Avg Service Time • Most improvement comes from SSD on the DB • Writing to the Lucene index means re-opening IndexReaders • Much faster on SSD 43 • Lucene Search Term Cache fits in OS buffers
Remember! • Upgrade! • Java to 1.6 • Your server • Use Parallel Garbage Collection with a big heap • Remove virtualization if you can • Use 1000mbit network 44
...Part Two: Environment and Joanna Thurmann JIRA Systems Administrator 2
Don’t let this happen to you! 3
“Step1 - We admitted we were powerless - our JIRA had become unmanageable.” JIRAcholics Anonymous 4
Contents • Performance Considerations • Hardware & Software Upgrade • JIRA and Plugin Upgrades • Structure for Scale 5
Our JIRA 6
Our JIRA • 210,000 issues 6
Our JIRA • 210,000 issues • 142 projects 6
Our JIRA • 210,000 issues • 142 projects • 66 workflows 6
Our JIRA • 210,000 issues • 142 projects • 66 workflows • 320 custom fields 6
Our JIRA • 210,000 issues • 142 projects • 66 workflows • 320 custom fields • 6,000 user directory (~400 concurrent) 6
Our JIRA • 210,000 issues • 142 projects • 66 workflows • 320 custom fields • 6,000 user directory (~400 concurrent) • 24x7 usage 6
Considerations for Performance and Usability • Total Issue Count • User Concurrency • # and Complexity of Configurations • Integrations • Plugins 7
In General… • JIRA uses the disk much more than db (due to Lucene indexing) • JIRA needs considerable memory 8
Our Upgrade Summary Before After  HP ProLiant DL580 G4  HP ProLiant DL380 G7  4 x Dual-Core processors - Intel Xeon  2 x 6-CORE processors - Intel Xeon X5680 7140M 3.4GHz 3.33GHz  RAID5 array  5 x HP 300GB 10K HDD  RAID10 array  11 x HP 300GB 10K HDD  512 MB cache (25%R-75%W)  1GB cache (50%R-50%W)  3.5 GB for JIRA (8 GB total)  6 GB for JIRA (32 GB total)  Windows Server 2003 Std Edition 64-bit  Windows Server 2008 R2 Std Edition 64-bit  SQL Server 2005 Std Edition  SQL Server 2008 R2 Std Edition 9
Our Upgrade – Disk and CPU • Chose Raid10 over Raid5 for better performance • Upgraded Controller from 512 MB cache to 1GB cache • Chose 12 CPUs for concurrency 10
Our Upgrade – Memory 11
Our Upgrade – Memory • 3.5GB  6GB 11
Our Upgrade – Memory • 3.5GB  6GB • Full garbage collections times lower and less frequent 11
Our Upgrade – Memory • 3.5GB  6GB • Full garbage collections times lower and less frequent • Free up more memory 11
Our Upgrade – Memory • 3.5GB  6GB • Full garbage collections times lower and less frequent • Free up more memory • Eliminate out-of-memory errors 11
Our Upgrade - Memory Before (3.5 GB JIRA, 8 GB Total) After (6 GB JIRA, 32 GB Total) Full GC: 11-50s, 1x/hour Full GC: 3-7s, <5x/day 2.5 – 3.25 GB avg. usage 3-5 GB avg. usage 20-30% free 50-75% free Out-of-memory: 1x/month Out-of-memory: 0 12
Our Upgrade - Memory Before (3.5 GB JIRA, 8 GB Total) After (6 GB JIRA, 32 GB Total) Full GC: 11-50s, 1x/hour Full GC: 3-7s, <5x/day 2.5 – 3.25 GB avg. usage 3-5 GB avg. usage 20-30% free 50-75% free Out-of-memory: 1x/month Out-of-memory: 0 12
Our Upgrade - Software • SQL Server 2008 R2 • OK to run JIRA and DB on the same server • Allocate sufficient memory to the db • Use the latest JDBC driver & JDK 13
What’s the bottom line? 14
What’s the bottom line? 14
Results – backend Then Now Percentage faster Total
Reindex 60
mins 25
mins 59% XML
Backup 30
mins 4
½
mins 85% SQL
Server
Backup 7
mins 2
mins
 71% Metrics
service 20
mins 6
mins 70%
 Index
opNmizaNon 7+
mins 25
secs 94% Index
excepNons 3‐4
per
day 0 15
Results – front end Then Now Percentage faster Search
all
issues 3
secs 2
secs 34% Create
/
Edit
issue 4
secs 2
secs 50% 16
Plugin challenges we faced • Memory leak • End of support of *key* workflow plugin • Odd behavior in JIRA 4.1.1 upgrade • Performance issue with plugin 17
Plugin Install & Upgrades 18
Plugin Install & Upgrades • Scrutinize Plugins 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? • Does it perform well? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? • Does it perform well? • Do you really need it? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? • Does it perform well? • Do you really need it? • Does it introduce other bugs? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Get the latest release • Will it be supported? • How widely used is it? • Does it perform well? • Do you really need it? • Does it introduce other bugs? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Get the latest release • Will it be supported? • How widely used is it? • Test in QA environment • Does it perform well? • Do you really need it? • Does it introduce other bugs? 18
Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Get the latest release • Will it be supported? • How widely used is it? • Test in QA environment • Does it perform well? • Do you really need it? • Test! Test! Test! • Does it introduce other bugs? 18
Why Upgrade JIRA Regularly? 19
Why Upgrade JIRA Regularly? • Performance improvements & key fixes in each release for BOTH end-users AND admins 19
Why Upgrade JIRA Regularly? • Performance improvements & key fixes in each release for BOTH end-users AND admins • Example: in-place DB upgrades in 4.3 19
My Weekend Upgrade 20
My Weekend Upgrade Before 20
My Weekend Upgrade Before 20
My Weekend Upgrade Before After 20
My Weekend Upgrade Before After 20
Why else? 21
Why else? • Fewer internal support calls 21
Why else? • Fewer internal support calls • Less time to resolve them 21
Why else? • Fewer internal support calls • Less time to resolve them • Easier & faster for Atlassian to support you 21
Why else? • Fewer internal support calls • Less time to resolve them • Easier & faster for Atlassian to support you • Early bird gets the worm 21
Our last JIRA upgrade rocked! 22
Our last JIRA upgrade rocked! • Resolved 4 bugs found internally 22
Our last JIRA upgrade rocked! • Resolved 4 bugs found internally • Delivered 5 must-have features 22
Our last JIRA upgrade rocked! • Resolved 4 bugs found internally • Delivered 5 must-have features • Resulted in ~25% fewer support calls 22
Our last JIRA upgrade rocked! • Resolved 4 bugs found internally • Delivered 5 must-have features • Resulted in ~25% fewer support calls • Saved ~3hrs/week with JQL instead of SQL 22
Structure – For Startup JIRAs 23
Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them 23
Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them • Use fewer issue types, resolutions, statuses, custom fields 23
Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them • Use fewer issue types, resolutions, statuses, custom fields • Use custom field contexts vs. new fields 23
Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them • Use fewer issue types, resolutions, statuses, custom fields • Use custom field contexts vs. new fields • Use a simple workflow and apply to many projects 23
Structure – Project Roles 24
Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes 24
Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before Permission Scheme for Project 1 Administer Projects Group (“”Project 1 Admins”) Edit Issues Group(“Project 1 Team”) 24
Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before After Global Permission Scheme Permission Scheme for Project 1 Administer Projects Project Role (Administrators) Administer Projects Group (“”Project 1 Admins”) Edit Issues Project Role (Developers) Edit Issues Group(“Project 1 Team”) 24
Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before After Global Permission Scheme Permission Scheme for Project 1 Administer Projects Project Role (Administrators) Administer Projects Group (“”Project 1 Admins”) Edit Issues Project Role (Developers) Edit Issues Group(“Project 1 Team”) Project Roles for Project 1 Administrators “Project 1 Admins” Developers “Project 1 Team” 24
Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before After Global Permission Scheme Permission Scheme for Project 1 Administer Projects Project Role (Administrators) Administer Projects Group (“”Project 1 Admins”) Edit Issues Project Role (Developers) Edit Issues Group(“Project 1 Team”) Project Roles for Project 1 Administrators “Project 1 Admins” Developers “Project 1 Team” 24
Structure – For Epic JIRAs 25
Structure – For Epic JIRAs • Consolidate any and all schemes 25
Structure – For Epic JIRAs • Consolidate any and all schemes • Use Scheme Tools: e.g. Scheme Comparison Tool, Group to Project Role Mapping Tool 25
Structure – For Epic JIRAs • Consolidate any and all schemes • Use Scheme Tools: e.g. Scheme Comparison Tool, Group to Project Role Mapping Tool • ‘Archive’ projects - hide them from view of users, so that they don’t clutter the menus. Create ‘Archived’ config schemes 25
We Declutter 26
We Declutter • Created project naming conventions 26
We Declutter • Created project naming conventions • Eliminated & consolidated custom fields (25 out of 345) 26
We Declutter • Created project naming conventions • Eliminated & consolidated custom fields (25 out of 345) • Took custom fields out of global context 26
We Declutter • Created project naming conventions • Eliminated & consolidated custom fields (25 out of 345) • Took custom fields out of global context • Eliminated issue types, status, resolution choices (15-30) 26
We Tidy 27
We Tidy • Super User (SU) plugin 27
We Tidy • Super User (SU) plugin • Easier to troubleshoot user issues 27
We Tidy • Super User (SU) plugin • Easier to troubleshoot user issues • Helps us “terminate” JIRA accounts •Delete their dashboards •Delete their personal filters 27
We Tidy • Super User (SU) plugin • Easier to troubleshoot user issues • Helps us “terminate” JIRA accounts •Delete their dashboards •Delete their personal filters •Delete subscriptions (the “silent killer”) 27
We Train 28
We Train • Limit General Permissions & train the users on how/when to use 28
We Train • Limit General Permissions & train the users on how/when to use • Bulk Change 28
We Train • Limit General Permissions & train the users on how/when to use • Bulk Change • Create Shared Objects (filters + dashboards) 28
We Train • Limit General Permissions & train the users on how/when to use • Bulk Change • Create Shared Objects (filters + dashboards) • Manage Group Filter Subscriptions 28
We Train • Limit General Permissions & train the users on how/when to use • Bulk Change • Create Shared Objects (filters + dashboards) • Manage Group Filter Subscriptions • Train project administrators / super-users 28
We Do More 29
We Do More • Use db-specific vs. XML-backup 29
We Do More • Use db-specific vs. XML-backup • Schedule regular maintenance 29
We Do More • Use db-specific vs. XML-backup • Schedule regular maintenance • Regularly check logs for errors 29
We Do More • Use db-specific vs. XML-backup • Schedule regular maintenance • Regularly check logs for errors • Read the JIRA performance doc 29
30
We Keep Going 30
“Step12 - we promised to help other JIRAcholics, and to always practice what we preach JIRAcholics Anonymous 31
A word from my sponsors….
A word from my sponsors….
A word from my sponsors…. Polycom, Inc. (Nasdaq: PLCM) is a global leader in unified communications solutions with industry-leading telepresence, video, voice, and infrastructure solutions built on open standards. Polycom powers smarter conversations, transforming lives and businesses worldwide. Please visit www.polycom.com for more information.
blogs.atlassian.com/developer Flickr Attributions: Day 11 - http://www.flickr.com/photos/ivanomak/4057632458/ Blue Streak - http://www.flickr.com/photos/jettajet/2061715292/
blogs.atlassian.com/developer Flickr Attributions: Day 11 - http://www.flickr.com/photos/ivanomak/4057632458/ Blue Streak - http://www.flickr.com/photos/jettajet/2061715292/

More Related Content

PDF
Best Practices of HA and Replication of PostgreSQL in Virtualized Environments
Jignesh Shah
 
PDF
2010 ilmf asprs hot topics session
raj.m.rao
 
PDF
VMworld 2013: Extreme Performance Series: Monster Virtual Machines
VMworld
 
PDF
VMworld 2013: DRS: New Features, Best Practices and Future Directions
VMworld
 
PDF
Virtualization Primer for Java Developers
Richard McDougall
 
PDF
VMworld 2013: Performance and Capacity Management of DRS Clusters
VMworld
 
ZIP
Operational Efficiency Hacks Web20 Expo2009
John Allspaw
 
PDF
5 Steps to PostgreSQL Performance
Command Prompt., Inc
 
Best Practices of HA and Replication of PostgreSQL in Virtualized Environments
Jignesh Shah
 
2010 ilmf asprs hot topics session
raj.m.rao
 
VMworld 2013: Extreme Performance Series: Monster Virtual Machines
VMworld
 
VMworld 2013: DRS: New Features, Best Practices and Future Directions
VMworld
 
Virtualization Primer for Java Developers
Richard McDougall
 
VMworld 2013: Performance and Capacity Management of DRS Clusters
VMworld
 
Operational Efficiency Hacks Web20 Expo2009
John Allspaw
 
5 Steps to PostgreSQL Performance
Command Prompt., Inc
 

What's hot (18)

PPTX
INF7827 DRS Best Practices
Brian Graf
 
PPTX
Azug - successfully breeding rabits
Yves Goeleven
 
PPTX
24 Hours of PASS, Summit Preview Session: Virtual SQL Server CPUs
David Klee
 
PDF
Inside the Hadoop Machine @ VMworld
Richard McDougall
 
PDF
VMworld 2013: How SRP Delivers More Than Power to Their Customers
VMworld
 
PDF
Scaling Your Cache And Caching At Scale
Alex Miller
 
PPTX
iMinds The Conference: Jan Lemeire
imec
 
PDF
My experience with embedding PostgreSQL
Jignesh Shah
 
PDF
Inside the Atlassian OnDemand Private Cloud
Atlassian
 
PPT
saurabh soni rac
saurabh soni
 
PDF
PostgreSQL and Benchmarks
Jignesh Shah
 
PDF
SSD based storage tuning for databases
Angelo Rajadurai
 
PPT
ESX performance problems 10 steps
Concentrated Technology
 
PDF
Ahorro energético archivado de backups
Omega Peripherals
 
PDF
VMworld 2013: Storage DRS: Deep Dive and Best Practices to Suit Your Storage ...
VMworld
 
PPT
Training netbackup6x2
M Shariff
 
PDF
Distributed DNN training: Infrastructure, challenges, and lessons learned
Wee Hyong Tok
 
PDF
Transcendent memoryupdate xensummit2010-final
The Linux Foundation
 
INF7827 DRS Best Practices
Brian Graf
 
Azug - successfully breeding rabits
Yves Goeleven
 
24 Hours of PASS, Summit Preview Session: Virtual SQL Server CPUs
David Klee
 
Inside the Hadoop Machine @ VMworld
Richard McDougall
 
VMworld 2013: How SRP Delivers More Than Power to Their Customers
VMworld
 
Scaling Your Cache And Caching At Scale
Alex Miller
 
iMinds The Conference: Jan Lemeire
imec
 
My experience with embedding PostgreSQL
Jignesh Shah
 
Inside the Atlassian OnDemand Private Cloud
Atlassian
 
saurabh soni rac
saurabh soni
 
PostgreSQL and Benchmarks
Jignesh Shah
 
SSD based storage tuning for databases
Angelo Rajadurai
 
ESX performance problems 10 steps
Concentrated Technology
 
Ahorro energético archivado de backups
Omega Peripherals
 
VMworld 2013: Storage DRS: Deep Dive and Best Practices to Suit Your Storage ...
VMworld
 
Training netbackup6x2
M Shariff
 
Distributed DNN training: Infrastructure, challenges, and lessons learned
Wee Hyong Tok
 
Transcendent memoryupdate xensummit2010-final
The Linux Foundation
 
Ad

Similar to Structure for scale: Dialing in your apps for optimal performance (20)

PDF
Performance Tuning: Pulling a Rabbit From a Hat - Atlassian Summit 2010
Atlassian
 
PDF
[Outdated] Secrets of Performance Tuning Java on Kubernetes
Bruno Borges
 
PDF
Secrets of Performance Tuning Java on Kubernetes
Bruno Borges
 
PDF
Basics of JVM Tuning
Vladislav Gangan
 
PDF
Memory, Big Data, NoSQL and Virtualization
Bigstep
 
PDF
Hadoop - Disk Fail In Place (DFIP)
mundlapudi
 
PPTX
Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, Cloudera
Cloudera, Inc.
 
PDF
Nick Fisk - low latency Ceph
ShapeBlue
 
PPT
Mysql talk
LogicMonitor
 
PDF
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
Chester Chen
 
PDF
VMworld 2013: Just Because You Could, Doesn't Mean You Should: Lessons Learne...
VMworld
 
PDF
Fixing twitter
Roger Xia
 
PDF
Fixing_Twitter
liujianrong
 
PDF
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
smallerror
 
PDF
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
xlight
 
PDF
淺談 Java GC 原理、調教和 新發展
Leon Chen
 
PPTX
Loom promises: be there!
Jean-Francois James
 
PDF
Introduction of Java GC Tuning and Java Java Mission Control
Leon Chen
 
PPT
14 scaleabilty wics
ashish61_scs
 
PDF
OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan
OpenStorageSummit
 
Performance Tuning: Pulling a Rabbit From a Hat - Atlassian Summit 2010
Atlassian
 
[Outdated] Secrets of Performance Tuning Java on Kubernetes
Bruno Borges
 
Secrets of Performance Tuning Java on Kubernetes
Bruno Borges
 
Basics of JVM Tuning
Vladislav Gangan
 
Memory, Big Data, NoSQL and Virtualization
Bigstep
 
Hadoop - Disk Fail In Place (DFIP)
mundlapudi
 
Hadoop World 2011: Hadoop and Performance - Todd Lipcon & Yanpei Chen, Cloudera
Cloudera, Inc.
 
Nick Fisk - low latency Ceph
ShapeBlue
 
Mysql talk
LogicMonitor
 
SF Big Analytics & SF Machine Learning Meetup: Machine Learning at the Limit ...
Chester Chen
 
VMworld 2013: Just Because You Could, Doesn't Mean You Should: Lessons Learne...
VMworld
 
Fixing twitter
Roger Xia
 
Fixing_Twitter
liujianrong
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
smallerror
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
xlight
 
淺談 Java GC 原理、調教和 新發展
Leon Chen
 
Loom promises: be there!
Jean-Francois James
 
Introduction of Java GC Tuning and Java Java Mission Control
Leon Chen
 
14 scaleabilty wics
ashish61_scs
 
OSS Presentation VMWorld 2011 by Andy Bennett & Craig Morgan
OpenStorageSummit
 
Ad

More from Atlassian (20)

PPTX
International Women's Day 2020
Atlassian
 
PDF
10 emerging trends that will unbreak your workplace in 2020
Atlassian
 
PDF
Forge App Showcase
Atlassian
 
PDF
Let's Build an Editor Macro with Forge UI
Atlassian
 
PDF
Meet the Forge Runtime
Atlassian
 
PDF
Forge UI: A New Way to Customize the Atlassian User Experience
Atlassian
 
PDF
Take Action with Forge Triggers
Atlassian
 
PDF
Observability and Troubleshooting in Forge
Atlassian
 
PDF
Trusted by Default: The Forge Security & Privacy Model
Atlassian
 
PDF
Designing Forge UI: A Story of Designing an App UI System
Atlassian
 
PDF
Forge: Under the Hood
Atlassian
 
PDF
Access to User Activities - Activity Platform APIs
Atlassian
 
PDF
Design Your Next App with the Atlassian Vendor Sketch Plugin
Atlassian
 
PDF
Tear Up Your Roadmap and Get Out of the Building
Atlassian
 
PDF
Nailing Measurement: a Framework for Measuring Metrics that Matter
Atlassian
 
PDF
Building Apps With Color Blind Users in Mind
Atlassian
 
PDF
Creating Inclusive Experiences: Balancing Personality and Accessibility in UX...
Atlassian
 
PDF
Beyond Diversity: A Guide to Building Balanced Teams
Atlassian
 
PDF
The Road(map) to Las Vegas - The Story of an Emerging Self-Managed Team
Atlassian
 
PDF
Building Apps With Enterprise in Mind
Atlassian
 
International Women's Day 2020
Atlassian
 
10 emerging trends that will unbreak your workplace in 2020
Atlassian
 
Forge App Showcase
Atlassian
 
Let's Build an Editor Macro with Forge UI
Atlassian
 
Meet the Forge Runtime
Atlassian
 
Forge UI: A New Way to Customize the Atlassian User Experience
Atlassian
 
Take Action with Forge Triggers
Atlassian
 
Observability and Troubleshooting in Forge
Atlassian
 
Trusted by Default: The Forge Security & Privacy Model
Atlassian
 
Designing Forge UI: A Story of Designing an App UI System
Atlassian
 
Forge: Under the Hood
Atlassian
 
Access to User Activities - Activity Platform APIs
Atlassian
 
Design Your Next App with the Atlassian Vendor Sketch Plugin
Atlassian
 
Tear Up Your Roadmap and Get Out of the Building
Atlassian
 
Nailing Measurement: a Framework for Measuring Metrics that Matter
Atlassian
 
Building Apps With Color Blind Users in Mind
Atlassian
 
Creating Inclusive Experiences: Balancing Personality and Accessibility in UX...
Atlassian
 
Beyond Diversity: A Guide to Building Balanced Teams
Atlassian
 
The Road(map) to Las Vegas - The Story of an Emerging Self-Managed Team
Atlassian
 
Building Apps With Enterprise in Mind
Atlassian
 

Recently uploaded (20)

PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
SHREYAS PHANSE
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
PDF
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
CIFDAQ
 
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
PPTX
MYSQL Presentation for SQL database connectivity
Swati270511
 
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
SHREYAS PHANSE
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Cloud computing and distributed systems.
kkkkkhan
 
Spectral efficient network and resource selection model in 5G networks
IAESIJAI
 
Advanced methodologies resolving dimensionality complications for autism neur...
IAESIJAI
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Unlocking AI with Model Context Protocol (MCP)
Brian McKeiver
 
Agricultural_Statistics_at_a_Glance_2022_0.pdf
SandeepSingh286037
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Peter Tan
 
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
Jian-Hong Pan
 
Review of recent advances in non-invasive hemoglobin estimation
IAESIJAI
 
CIFDAQ's Market Insight: SEC Turns Pro Crypto
CIFDAQ
 
Dropbox Q2 2025 Financial Results & Investor Presentation
Dropbox
 
MYSQL Presentation for SQL database connectivity
Swati270511
 
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
ReZa AdineH
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
blackmambaettijean
 

Structure for scale: Dialing in your apps for optimal performance

  • 3. Performance tuning for Java applications Andrew Prentice Atlassian 2
  • 4. Topics • How to make your apps run faster • Tuning ideas 3
  • 5. A note • X vs Y will always depend on context • Test with your application! 4
  • 6. You are here • How I did the testing • Hardware & software • Must have improvements • Jave, Hardware, Virtualization and Garbage Collection • Worth doing • Gigabit vs 100 mbit, Heap size tuning, use an SSD 5
  • 7. The contenders • Dell PE 860 - “WALL-E” • Xeon X3220 2.4 Ghz Quad core • Quad Core, 2 x 4Mb L3 • 8 Gb DDR2 RAM • 2 x 7.2K SAS Disks • 1 x Intel X25-E 32 GB SSD 6
  • 8. The contenders • Dell PE 2950 - “Johnny 5” • 2 x Xeon E5 405 2.0 Ghz • Quad Core, 2 x 6M L3 • 32 Gb RAM • 2 x 72 Gb 15 K Drives • ~$4000 7
  • 9. The contenders • Dell PE R610 - “EVE” • 2 x Xeon E5520 2.2Ghz • Quad Core, 8M “SmartCache” • 32 Gb RAM • 2 x 146G 15K drives • ~$4000 8
  • 10. The Workload • JIRA 4, MySQL 5, RHEL 5 • Java 1.6, 3G heap, ParNew, ParOld GC • ~70,000 Issues, ~80 projects • ~30 reqs/sec, JMeter 2.3.4 (patched, JSR-223) • Traffic modeled on http://jira.atlassian.com with higher writes 9
  • 11. About the graphs • Most show Average Response Time in milliseconds • Includes time to generate AND deliver the response • Does not include browser time • Arrows show if lower or higher is better 10
  • 12. You are here • How I did the testing • Hardware & software • Must have improvements • Jave, Hardware, Virtualization and Garbage Collection • Worth doing • Gigabit vs 100 mbit, Heap size tuning, use an SSD 11
  • 14. Upgrade Java 1.5 to 1.6 • Advances in complier/VM technology • Produces better runtime code • Able to use modern instructions • Able to use modern hardware 13
  • 15. Java 1.5 vs 1.6 (Johnny 5) Java 1.5 Java 1.6.0 1500 1125 750 375 6.3X 0 Average Response Time (ms) 14
  • 16. Java 1.5 vs 1.6 (Johnny 5) Java 1.5.0 Java 1.6.0 380 285 190 95 26% 0 Average CPU (%) 15
  • 17. Upgrade Java to 1.6 • HotSpot VM updates in 1.6 • Advancing technology • Compressed OOPS on 64bit • Escape Analysis • NUMA • http://java.sun.com/performance/reference/ whitepapers/6_performance.html 16
  • 18. Java 1.6 vs 1.6 (Eve) Java 1.6.0_7 Java 1.6.0_16 148 111 74 37 7% 0 Average Response Time (ms) 17
  • 19. Hardware Upgrades • Hyper-Threading • QuickPath Interconnect • Nehalem (Eve) have on die DDR3 Memory Controllers • Memory throuput improvements - good for Java! Technology Speed DDR2-800 (dual channel) 6.4 GB/sec DDR3-1333 (dual channel) 21.3 GB/sec 18
  • 20. Hyper Threading Thread 1 CPU Thread 2 Thread 1 runs until it hits a cache miss. Thread 2 runs on the same CPU while T1 is awaiting data 19
  • 21. Hardware Upgrades 2.4Ghz Xeon (Wall-E) 2.0Ghz Xeon (Johnny 5) 2.2Ghz Xeon (Eve) 2000 1500 1000 500 8-12X 0 Average Response Time (ms) 20
  • 22. Improved GC Throughput 2.4Ghz Xeon (Wall-E) 2.0Ghz Xeon (Johnny 5) 2.2Ghz Xeon (Eve) 30000 22500 15000 2-3X 7500 0 GC Throughput (MB/sec) 21
  • 23. Garbage Collection • Java’s Virtual Machine manages memory • New objects are continually created during runtime • GC is the process of collecting dead objects to reclaim memory 22
  • 24. Tune Garbage Collection • Getting your GC tuning wrong is bad • How bad? • Are the defaults sane? 23
  • 25. Tune Garbage Collection CMS ParNew ParOld 4000 3000 2000 1000 2X 0 Wall-E (ms) Johnny 5 (ms) Eve (ms) Average Response Time 24
  • 26. Tune Garbage Collection CMS ParNew ParOld 500 375 250 125 2X 0 Johnny 5 (ms) Eve (ms) Average Response Time 25
  • 27. Tune Garbage Collection • Defaults are sane • Parallel New + Parallel Old is fastest • CMS prioritizes low pauses over CPU usage • CMS does not compact - can lead to heap fragmentation 26
  • 28. Virtualisation • Does being in a VM hurt? • What if the risks are managed? 27
  • 29. Virtualisation Real VMWare ESX 3.5 VMWare ESX 4i 170.0 127.5 85.0 42.5 43% 0 Average Response Time (ms) 28
  • 30. Virtualisation • On average ~30% slower • Under extreme resource starvation, up to 100% slower • Avoid virtualization for high traffic instances • See the Atlassian docs for recommendations • http://confluence.atlassian.com/display/DOC/Running +Confluence+in+a+Virtualised+Environment 29 • http://confluence.atlassian.com/display/JIRA/Running+JIRA+in
  • 31. You are here • How I did the testing • Hardware & software • Must have improvements • Java, Hardware, Virtualization and Garbage Collection • Worth doing • Gigabit vs 100 mbit, Heap size tuning, use an SSD 30
  • 32. Maybe. 31
  • 33. Gigabit Network vs 100Mbit • Gigabit is faster, sure • But 100Mb is OK if you’re not doing big transfers, RIGHT?! 32
  • 34. Use Gigabit 100mb/sec 1000mb/sec 1900 1425 950 475 10% 0 Average Response Time (ms) 33
  • 35. Use Gigabit • Lower RTT on Gigabit • If you have enough CPU power, keep the DB on localhost • Check your development environment 34
  • 36. Use Gigabit 100Mbit 1000Mbit 200 150 100 50 26% 0 ReIndex Time (seconds) 35
  • 37. Heap size tuning • Memory starvation is bad, but can we “drown” Java with too much memory? 36
  • 38. Heap size - Wall-E 1.5G 3G 6G 2000 1500 1000 500 23% 0 Average Response Time 37
  • 39. Heap size - Johnny 5 1.5G 3G 6G 15G 290.0 217.5 145.0 72.5 49% 0 Average Response Time 38
  • 40. Heap size - Eve 1.5G 3G 6G 15G 170.0 127.5 85.0 42.5 28% 0 Average Response Time 39
  • 41. Heap size tuning • Heap too small - bad • Heap too big - not bad! • Bigger heaps give the JVM more room to work • Diminishing returns from really big heaps 40
  • 42. Buy an SSD • SSDs are FAST! • SSDs are EXPENSIVE! • SSDs are all the rage! • DBAs are finally happy (almost) 41
  • 43. Buy an SSD SATA SSD 800 600 400 200 33% 0 Browse Issue Create Issue 42
  • 44. Buy an SSD • Fast writes • 0.1ms Avg Service Time • Most improvement comes from SSD on the DB • Writing to the Lucene index means re-opening IndexReaders • Much faster on SSD 43 • Lucene Search Term Cache fits in OS buffers
  • 45. Remember! • Upgrade! • Java to 1.6 • Your server • Use Parallel Garbage Collection with a big heap • Remove virtualization if you can • Use 1000mbit network 44
  • 46. ...Part Two: Environment and Joanna Thurmann JIRA Systems Administrator 2
  • 47. Don’t let this happen to you! 3
  • 48. “Step1 - We admitted we were powerless - our JIRA had become unmanageable.” JIRAcholics Anonymous 4
  • 49. Contents • Performance Considerations • Hardware & Software Upgrade • JIRA and Plugin Upgrades • Structure for Scale 5
  • 50. Our JIRA 6
  • 52. Our JIRA • 210,000 issues • 142 projects 6
  • 53. Our JIRA • 210,000 issues • 142 projects • 66 workflows 6
  • 54. Our JIRA • 210,000 issues • 142 projects • 66 workflows • 320 custom fields 6
  • 55. Our JIRA • 210,000 issues • 142 projects • 66 workflows • 320 custom fields • 6,000 user directory (~400 concurrent) 6
  • 56. Our JIRA • 210,000 issues • 142 projects • 66 workflows • 320 custom fields • 6,000 user directory (~400 concurrent) • 24x7 usage 6
  • 57. Considerations for Performance and Usability • Total Issue Count • User Concurrency • # and Complexity of Configurations • Integrations • Plugins 7
  • 58. In General… • JIRA uses the disk much more than db (due to Lucene indexing) • JIRA needs considerable memory 8
  • 59. Our Upgrade Summary Before After  HP ProLiant DL580 G4  HP ProLiant DL380 G7  4 x Dual-Core processors - Intel Xeon  2 x 6-CORE processors - Intel Xeon X5680 7140M 3.4GHz 3.33GHz  RAID5 array  5 x HP 300GB 10K HDD  RAID10 array  11 x HP 300GB 10K HDD  512 MB cache (25%R-75%W)  1GB cache (50%R-50%W)  3.5 GB for JIRA (8 GB total)  6 GB for JIRA (32 GB total)  Windows Server 2003 Std Edition 64-bit  Windows Server 2008 R2 Std Edition 64-bit  SQL Server 2005 Std Edition  SQL Server 2008 R2 Std Edition 9
  • 60. Our Upgrade – Disk and CPU • Chose Raid10 over Raid5 for better performance • Upgraded Controller from 512 MB cache to 1GB cache • Chose 12 CPUs for concurrency 10
  • 61. Our Upgrade – Memory 11
  • 62. Our Upgrade – Memory • 3.5GB  6GB 11
  • 63. Our Upgrade – Memory • 3.5GB  6GB • Full garbage collections times lower and less frequent 11
  • 64. Our Upgrade – Memory • 3.5GB  6GB • Full garbage collections times lower and less frequent • Free up more memory 11
  • 65. Our Upgrade – Memory • 3.5GB  6GB • Full garbage collections times lower and less frequent • Free up more memory • Eliminate out-of-memory errors 11
  • 66. Our Upgrade - Memory Before (3.5 GB JIRA, 8 GB Total) After (6 GB JIRA, 32 GB Total) Full GC: 11-50s, 1x/hour Full GC: 3-7s, <5x/day 2.5 – 3.25 GB avg. usage 3-5 GB avg. usage 20-30% free 50-75% free Out-of-memory: 1x/month Out-of-memory: 0 12
  • 67. Our Upgrade - Memory Before (3.5 GB JIRA, 8 GB Total) After (6 GB JIRA, 32 GB Total) Full GC: 11-50s, 1x/hour Full GC: 3-7s, <5x/day 2.5 – 3.25 GB avg. usage 3-5 GB avg. usage 20-30% free 50-75% free Out-of-memory: 1x/month Out-of-memory: 0 12
  • 68. Our Upgrade - Software • SQL Server 2008 R2 • OK to run JIRA and DB on the same server • Allocate sufficient memory to the db • Use the latest JDBC driver & JDK 13
  • 71. Results – backend Then Now Percentage faster Total
Reindex 60
mins 25
mins 59% XML
Backup 30
mins 4
½
mins 85% SQL
Server
Backup 7
mins 2
mins
 71% Metrics
service 20
mins 6
mins 70%
 Index
opNmizaNon 7+
mins 25
secs 94% Index
excepNons 3‐4
per
day 0 15
  • 72. Results – front end Then Now Percentage faster Search
all
issues 3
secs 2
secs 34% Create
/
Edit
issue 4
secs 2
secs 50% 16
  • 73. Plugin challenges we faced • Memory leak • End of support of *key* workflow plugin • Odd behavior in JIRA 4.1.1 upgrade • Performance issue with plugin 17
  • 74. Plugin Install & Upgrades 18
  • 75. Plugin Install & Upgrades • Scrutinize Plugins 18
  • 76. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… 18
  • 77. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? 18
  • 78. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? 18
  • 79. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? 18
  • 80. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? • Does it perform well? 18
  • 81. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? • Does it perform well? • Do you really need it? 18
  • 82. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Will it be supported? • How widely used is it? • Does it perform well? • Do you really need it? • Does it introduce other bugs? 18
  • 83. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Get the latest release • Will it be supported? • How widely used is it? • Does it perform well? • Do you really need it? • Does it introduce other bugs? 18
  • 84. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Get the latest release • Will it be supported? • How widely used is it? • Test in QA environment • Does it perform well? • Do you really need it? • Does it introduce other bugs? 18
  • 85. Plugin Install & Upgrades • Scrutinize Plugins Ask yourself… • Is the plugin reputable? • Get the latest release • Will it be supported? • How widely used is it? • Test in QA environment • Does it perform well? • Do you really need it? • Test! Test! Test! • Does it introduce other bugs? 18
  • 86. Why Upgrade JIRA Regularly? 19
  • 87. Why Upgrade JIRA Regularly? • Performance improvements & key fixes in each release for BOTH end-users AND admins 19
  • 88. Why Upgrade JIRA Regularly? • Performance improvements & key fixes in each release for BOTH end-users AND admins • Example: in-place DB upgrades in 4.3 19
  • 90. My Weekend Upgrade Before 20
  • 91. My Weekend Upgrade Before 20
  • 92. My Weekend Upgrade Before After 20
  • 93. My Weekend Upgrade Before After 20
  • 94. Why else? 21
  • 95. Why else? • Fewer internal support calls 21
  • 96. Why else? • Fewer internal support calls • Less time to resolve them 21
  • 97. Why else? • Fewer internal support calls • Less time to resolve them • Easier & faster for Atlassian to support you 21
  • 98. Why else? • Fewer internal support calls • Less time to resolve them • Easier & faster for Atlassian to support you • Early bird gets the worm 21
  • 99. Our last JIRA upgrade rocked! 22
  • 100. Our last JIRA upgrade rocked! • Resolved 4 bugs found internally 22
  • 101. Our last JIRA upgrade rocked! • Resolved 4 bugs found internally • Delivered 5 must-have features 22
  • 102. Our last JIRA upgrade rocked! • Resolved 4 bugs found internally • Delivered 5 must-have features • Resulted in ~25% fewer support calls 22
  • 103. Our last JIRA upgrade rocked! • Resolved 4 bugs found internally • Delivered 5 must-have features • Resulted in ~25% fewer support calls • Saved ~3hrs/week with JQL instead of SQL 22
  • 104. Structure – For Startup JIRAs 23
  • 105. Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them 23
  • 106. Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them • Use fewer issue types, resolutions, statuses, custom fields 23
  • 107. Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them • Use fewer issue types, resolutions, statuses, custom fields • Use custom field contexts vs. new fields 23
  • 108. Structure – For Startup JIRAs • Configurations  Less is more since JIRA caches them • Use fewer issue types, resolutions, statuses, custom fields • Use custom field contexts vs. new fields • Use a simple workflow and apply to many projects 23
  • 110. Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes 24
  • 111. Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before Permission Scheme for Project 1 Administer Projects Group (“”Project 1 Admins”) Edit Issues Group(“Project 1 Team”) 24
  • 112. Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before After Global Permission Scheme Permission Scheme for Project 1 Administer Projects Project Role (Administrators) Administer Projects Group (“”Project 1 Admins”) Edit Issues Project Role (Developers) Edit Issues Group(“Project 1 Team”) 24
  • 113. Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before After Global Permission Scheme Permission Scheme for Project 1 Administer Projects Project Role (Administrators) Administer Projects Group (“”Project 1 Admins”) Edit Issues Project Role (Developers) Edit Issues Group(“Project 1 Team”) Project Roles for Project 1 Administrators “Project 1 Admins” Developers “Project 1 Team” 24
  • 114. Structure – Project Roles • Use Project Roles vs individual names & groups in permission schemes Before After Global Permission Scheme Permission Scheme for Project 1 Administer Projects Project Role (Administrators) Administer Projects Group (“”Project 1 Admins”) Edit Issues Project Role (Developers) Edit Issues Group(“Project 1 Team”) Project Roles for Project 1 Administrators “Project 1 Admins” Developers “Project 1 Team” 24
  • 115. Structure – For Epic JIRAs 25
  • 116. Structure – For Epic JIRAs • Consolidate any and all schemes 25
  • 117. Structure – For Epic JIRAs • Consolidate any and all schemes • Use Scheme Tools: e.g. Scheme Comparison Tool, Group to Project Role Mapping Tool 25
  • 118. Structure – For Epic JIRAs • Consolidate any and all schemes • Use Scheme Tools: e.g. Scheme Comparison Tool, Group to Project Role Mapping Tool • ‘Archive’ projects - hide them from view of users, so that they don’t clutter the menus. Create ‘Archived’ config schemes 25
  • 120. We Declutter • Created project naming conventions 26
  • 121. We Declutter • Created project naming conventions • Eliminated & consolidated custom fields (25 out of 345) 26
  • 122. We Declutter • Created project naming conventions • Eliminated & consolidated custom fields (25 out of 345) • Took custom fields out of global context 26
  • 123. We Declutter • Created project naming conventions • Eliminated & consolidated custom fields (25 out of 345) • Took custom fields out of global context • Eliminated issue types, status, resolution choices (15-30) 26
  • 124. We Tidy 27
  • 125. We Tidy • Super User (SU) plugin 27
  • 126. We Tidy • Super User (SU) plugin • Easier to troubleshoot user issues 27
  • 127. We Tidy • Super User (SU) plugin • Easier to troubleshoot user issues • Helps us “terminate” JIRA accounts •Delete their dashboards •Delete their personal filters 27
  • 128. We Tidy • Super User (SU) plugin • Easier to troubleshoot user issues • Helps us “terminate” JIRA accounts •Delete their dashboards •Delete their personal filters •Delete subscriptions (the “silent killer”) 27
  • 129. We Train 28
  • 130. We Train • Limit General Permissions & train the users on how/when to use 28
  • 131. We Train • Limit General Permissions & train the users on how/when to use • Bulk Change 28
  • 132. We Train • Limit General Permissions & train the users on how/when to use • Bulk Change • Create Shared Objects (filters + dashboards) 28
  • 133. We Train • Limit General Permissions & train the users on how/when to use • Bulk Change • Create Shared Objects (filters + dashboards) • Manage Group Filter Subscriptions 28
  • 134. We Train • Limit General Permissions & train the users on how/when to use • Bulk Change • Create Shared Objects (filters + dashboards) • Manage Group Filter Subscriptions • Train project administrators / super-users 28
  • 135. We Do More 29
  • 136. We Do More • Use db-specific vs. XML-backup 29
  • 137. We Do More • Use db-specific vs. XML-backup • Schedule regular maintenance 29
  • 138. We Do More • Use db-specific vs. XML-backup • Schedule regular maintenance • Regularly check logs for errors 29
  • 139. We Do More • Use db-specific vs. XML-backup • Schedule regular maintenance • Regularly check logs for errors • Read the JIRA performance doc 29
  • 140. 30
  • 141. We Keep Going 30
  • 142. “Step12 - we promised to help other JIRAcholics, and to always practice what we preach JIRAcholics Anonymous 31
  • 143. A word from my sponsors….
  • 144. A word from my sponsors….
  • 145. A word from my sponsors…. Polycom, Inc. (Nasdaq: PLCM) is a global leader in unified communications solutions with industry-leading telepresence, video, voice, and infrastructure solutions built on open standards. Polycom powers smarter conversations, transforming lives and businesses worldwide. Please visit www.polycom.com for more information.
  • 146. blogs.atlassian.com/developer Flickr Attributions: Day 11 - http://www.flickr.com/photos/ivanomak/4057632458/ Blue Streak - http://www.flickr.com/photos/jettajet/2061715292/
  • 147. blogs.atlassian.com/developer Flickr Attributions: Day 11 - http://www.flickr.com/photos/ivanomak/4057632458/ Blue Streak - http://www.flickr.com/photos/jettajet/2061715292/

Editor's Notes