Full stack development best practice and toolset

FULL STACK DEVELOPMENT
BEST PRACTICES AND
TOOLSETS
By Reid Lai

WHAT IS FULL-STACK
DEVELOPER?
PART 1

WHAT IS FULL-STACK DEVELOPER?
STACK OVERFLOW 2016 DEVELOPER SURVE

“A Full-Stack Web Developer is someone who is able to
work on both the front-end and back-end portions of an
application.”

FULL-STACK DEVELOPER SKILLS
Full-stack die-hards would consider a full-stack developer to have specialised knowledge in all stages of
software development. Thus, a full-stack developer would be proficient, if not fluent, in:
▸Server, network, and hosting environment
▸Relational and non-relational databases
▸How to interact with APIs and the external world
▸User interface and user experience
▸Quality assurance
▸Security concerns throughout the program
▸Understanding customer and business needs

FULL-STACK DEVELOPER IN
SCRUM TEAM
PART 2

FULL-STACK DEVELOPER IN SCRUM TEAM
SCRUM PRINCIPLE
▸Individuals and interactions over processes and tools
▸Working software over comprehensive documentation
▸Customer collaboration over contract negotiation
▸Responding to change over following a plan

Full stack development best practice and toolset

SMALL SCRUM TEAM EXAMPLE
The Small Scrum Team could looks like that at the very beginning:
▸3 x Full-stack Developers
▸Scrum Master
▸Product Owner

BIG SCRUM TEAM EXAMPLE
When the team grows, it will become a large team like that:
▸3 x Backend Developers
▸2 x Frontend Developers
▸1 x Tester
▸1 x DevOps/Admin
▸1 x Technical/Business Analyst
▸Scrum Master
▸Product Owner

FULL-STACK DEVELOPER SHOULD BE CROSS-
FUNCTIONAL
▸In The Scrum Guide you may find that the Scrum Team should be Cross-
Functional.
▸Cross-functionality means that the team (as a team) has all the competences
skills needed to build an increment of the product.
▸At least two developers finished one feature (user story) together - XP or Pair
Programming
▸Full-stack developer can handle backend/frontend/DevOps development either in
small or large team.

AGILE DEVELOPMENT AND
DEVOPS
PART 3

AGILE DEVELOPMENT AND DEVOPS
WHY DEVOPS RELATES TO AGILE?
▸DevOps is a new term emerging from the collision of two major related trends.
▸The first was also called “agile infrastructure” or “agile operations”; it sprang from
applying Agile and Lean approaches to operations work.
▸The second is a much expanded understanding of the value of collaboration
between development and operations staff throughout all stages of the
development lifecycle when creating and operating a service, and how important
operations has become in our increasingly service-oriented world.

SDLC IN OLD DAYS
How long will it takes?

DEVOPS IN SCRUM SPRINT

DEVOPS TOOLCHAIN
CONTINUOUS
INTEGRATION
CONTINUOUS
DELIVERY
CONTINUOUS
DEPLOYMENT

CONTINUOUS INTEGRATION (CI)
▸Continuous Integration (CI) was intended to be used in combination with
automated unit tests written through the practices of test-driven development.
Initially this was conceived of as running all unit tests in the developer's local
environment and verifying they all passed before committing to the mainline.

CONTINUOUS DELIVERY VS CONTINUOUS
DEPLOYMENT

CONTINUOUS DELIVERY VS CONTINUOUS
DEPLOYMENT (CONT’D)
▸There are business cases in which IT must wait for a feature to go live, making
continuous deployment impractical.
▸While continuous deployment may not be right for every company, continuous
delivery is an absolute requirement of DevOps practices.

CONTINUOUS DELIVERY (CD)
▸Continuous delivery (CD) is a series of practices designed to ensure that code
can be rapidly and safely deployed to production by delivering every change to a
production-like environment and ensuring business applications and services
function as expected through rigorous automated testing.

CD PIPELINE
▸The pipeline breaks down the software delivery process into stages. Each stage is
aimed at verifying the quality of new features from a different angle to validate the
new functionality and prevent errors from affecting your users. The pipeline should
provide feedback to the team and visibility into the flow of changes to everyone
involved in delivering the new feature(s)
▸SIT, UAT, Penetration Test, Stress Test, Pre-launch, and Production environment
will be implemented as CD Pipeline.
▸No standard for the number of pipelines and dependencies.

DEVOPS SKILLS

DEVOPS AND CONTAINER
ORCHESTRATION
PART 4

DEVOPS AND CONTAINER ORCHESTRATION
WHY USE CONTAINER AND DOCKER?
▸Today 44% of enterprises are looking to adopt a DevOps initiative within their organisation.
▸This cultural shift is geared towards tearing down the traditional barrier that has existed between Developer teams and
IT operations teams.
▸Docker is the world’s leading container platform enables developers to build applications in a self service manner and
select from image repository that the IT operations team has deemed okay (passed QA process) for developer use.
▸Developers use Docker to eliminate “works on my machine”
▸Developers can then use these images to create new applications, quickly and securely.
▸Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density.
▸Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with
confidence for both Linux, Windows Server, and Linux-on-mainframe apps.

LXC AND DOCKER
▸Docker is a container system making use of LXC containers
▸LXC (Linux Containers) is an operating-system-level virtualisation method for
running multiple isolated Linux systems (containers) on a control host using a
single Linux kernel.

DOCKERFILE EXAMPLE
FROM reactioncommerce/base:v2.0.2
# Default environment variables
ENV ROOT_URL "http://localhost"
ENV MONGO_URL "mongodb://127.0.0.1:27017/reaction"

DOCKER-COMPOSE EXAMPLE# Usage:
# docker-compose up -d
reaction:
image: reactioncommerce/reaction:latest
links:
- mongo
ports:
- "80:3000"
environment:
ROOT_URL: "http://localhost"
MONGO_URL: "mongodb://mongo:27017/reaction"
mongo:
image: mongo:latest
command: mongod --storageEngine=wiredTiger

EXAMPLE TO START ALL REQUIRED SERVICES
FOR APPLICATION
Reids-Pactera-MacBook-Pro:reaction reidlai$ docker-compose up
Starting reaction_mongo_1 ...
Starting reaction_mongo_1 ... done
Starting reaction_reaction_1 ...
Starting reaction_reaction_1 ... done
Attaching to reaction_mongo_1, reaction_reaction_1
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=4ed98475f631
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] db version v3.4.9
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] git version: 876ebee8c7dd0e2d992f36a848ff4dc50ee6603e
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1t 3 May 2016
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] allocator: tcmalloc
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] modules: none
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] build environment:
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] distmod: debian81
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] distarch: x86_64
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] target_arch: x86_64
mongo_1 | 2017-09-21T05:34:34.189+0000 I CONTROL [initandlisten] options: { storage: { engine: "wiredTiger" } }
mongo_1 | 2017-09-21T05:34:34.189+0000 W - [initandlisten] Detected unclean shutdown - /data/db/mongod.lock is not empty.
reaction_1 | => Starting app on port 3000...
mongo_1 | 2017-09-21T05:34:34.192+0000 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
mongo_1 | 2017-09-21T05:34:34.192+0000 I STORAGE [initandlisten]
mongo_1 | 2017-09-21T05:34:34.193+0000 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
mongo_1 | 2017-09-21T05:34:34.193+0000 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
mongo_1 | 2017-09-21T05:34:34.193+0000 I STORAGE [initandlisten] wiredtiger_open config:
create,cache_size=487M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
mongo_1 | 2017-09-21T05:34:34.932+0000 I CONTROL [initandlisten]
mongo_1 | 2017-09-21T05:34:34.932+0000 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
mongo_1 | 2017-09-21T05:34:34.932+0000 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
mongo_1 | 2017-09-21T05:34:34.932+0000 I CONTROL [initandlisten]
mongo_1 | 2017-09-21T05:34:34.967+0000 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
mongo_1 | 2017-09-21T05:34:34.969+0000 I NETWORK [thread1] waiting for connections on port 27017
mongo_1 | 2017-09-21T05:34:35.004+0000 I FTDC [ftdc] Unclean full-time diagnostic data capture shutdown detected, found interim file, some metrics may have been lost. OK
mongo_1 | 2017-09-21T05:34:35.535+0000 I NETWORK [thread1] connection accepted from 172.17.0.3:41980 #1 (1 connection now open)
mongo_1 | 2017-09-21T05:34:35.542+0000 I NETWORK [conn1] received client metadata from 172.17.0.3:41980 conn1: { driver: { name: "nodejs", version: "2.2.30" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.9.41-moby" }, platform: "Node.js v4.8.4, LE, mongodb-core: 2.1.14" }
reaction_1 | Note: you are using a pure-JavaScript implementation of bcrypt.
reaction_1 | While this implementation will work correctly, it is known to be
reaction_1 | approximately three times slower than the native implementation.
reaction_1 | In order to use the native implementation instead, run
reaction_1 |
reaction_1 | meteor npm install --save bcrypt
reaction_1 |
reaction_1 | in the root directory of your application.
mongo_1 | 2017-09-21T05:34:36.446+0000 I NETWORK [thread1] connection accepted from 172.17.0.3:41982 #2 (2 connections now open)
reaction_1 | 05:34:41.238Z INFO Reaction: Load default data from /private/data/
reaction_1 | 05:34:41.554Z INFO Reaction: JobServer started
reaction_1 | 05:34:41.568Z WARN Reaction: Skipped loading settings from reaction.json.
reaction_1 | 05:34:45.813Z INFO Reaction: Reaction Version: 1.4.2
reaction_1 | 05:34:45.947Z WARN Reaction: OpenExchangeRates API not configured. Not adding fetchRates job
reaction_1 | 05:34:45.953Z WARN Reaction: OpenExchangeRates API not configured. Not adding flushRates job
mongo_1 | 2017-09-21T05:34:46.038+0000 I COMMAND [conn1] CMD: dropIndexes reaction.ProductSearch
mongo_1 | 2017-09-21T05:34:46.139+0000 I INDEX [conn7] build index on: reaction.ProductSearch properties: { v: 2, key: { _fts: "text", _ftsx: 1 }, name: "title_text_description_text_metafields_text_vendor_text", ns: "reaction.ProductSearch", default_language: "en", weights: { description: 1, metafields: 1, title: 1, vendor: 1 },
language_override: "language", textIndexVersion: 3 }
mongo_1 | 2017-09-21T05:34:46.139+0000 I INDEX [conn7] building index using bulk method; build may temporarily use up to 500 megabytes of RAM
mongo_1 | 2017-09-21T05:34:46.161+0000 I INDEX [conn7] build index done. scanned 2 total records. 0 secs
reaction_1 | 05:34:46.329Z INFO Reaction: Reaction initialization finished.

SAMPLE APP UP AND RUNNING

CONTAINER ORCHESTRATION

KUBERNETES OVERVIEW

KUBERNETES ARCHITECTURE

KUBERNETES UI

CI AND CODE QUALITY
CONCEPTUAL DEVOPS DIAGRAM FROM
DEVELOPER LAPTOP TO DEPLOYMENT

CI AND CODE QUALITY
CONTINUOUS INTEGRATION WITH KUBERNETES
‣ This is one of example to use
Kubernetes with Openstack
‣ Focus on CI
‣ Artifactory is optional
‣ Build step should include linting,
unit test and code analysis

CI AND CODE QUALITY
DETAIL TASKS OF CONTINUOUS INTEGRATION

CI AND CODE QUALITY
CHOICE OF AUTOMATION SERVER
▸Open Source
▸Jenkins (Java based)
▸GoCD (GoLang based)
▸Commercial
▸Atlassian Bamboo
▸IBM UrbanCode
▸Jetbrains TeamCity
▸Cloud Based
▸Travis CI, Codeship, Shippable, Buildkite

CI AND CODE QUALITY
CI TOOLS
Process Purpose Tools
Linting
Crosscheck all code style compiling with specific language common
standard, e.g. indentation, camel-case, etc
ESLint (Javascript), JLint (java), Checkstyle
(java), etc.
Unit Tests
Unit tests target to check most of core functions or class objects compile
with design and handle all exceptional parameters. All tests should be
server independent.
Mocha (javascript), Chai (javascript), JUnit
(javascript), nUnit (C#), Mockito, AsserJ, etc.
Code Quality Check
(SecOps)
Report all bugs, vulnerability, reliability, and maintainability. Can detect
most of problems in PR not until code merge.
SonarQube
Result Notification
Immediately notify developers and security team to have immediate
action
Email, Slack, Messenger, etc

CI AND CODE QUALITY
CI TOOLS (CONT’D)
▸Developer can gain trust from Ops and Sec team by using objective unit tests and code quality reporting
mechanism.
▸Can know most bugs and vulnerability immediately before build.
▸All team members know test coverage percentage and see if additional effort is required.
▸Docker image will be pushed if all CI tasks passed
▸Coding vs Test Development Ratio = 6:4 in single sprint
▸Test Coverage % of Mature Team = 80%

FROM USER STORY TO BDD
TDD/ATDD PROBLEM
▸Test-Driven Development (TDD) was more of telling to make sure the code works fine
but it did not say that if the code that is written was even required at first place
▸Acceptance Test-Driven Development (ATDD) leans towards the developer-focused
side of things like TDD does
▸ATDD is a collaborative exercise that involves product owners, business analysts,
testers, and developers. ATDD helps to ensure that all project members understand
precisely what needs to be done and implemented - without describing how the
deliverables works
▸TDD and ATDD can only be applied for “build for unknown”

BDD VS TDD/ATDD
▸BDD is largely facilitated through the use of a simple domain-specific language (DSL) using
natural language constructs (e.g., English-like sentences) that can express the behavior
and the expected outcomes.
▸Behaviour-driven development specifies that tests of any unit of software should be
specified in terms of the desired behaviour of the unit.
▸Borrowing from agile software development the "desired behaviour" in this case consists of
the requirements set by the business — that is, the desired behaviour that has business
value for whatever entity commissioned the software unit under construction.
▸Within BDD practice, this is referred to as BDD being an "outside-in" activity.

USER WRITES GHERKIN LANGUAGE
▸Gherkin is the language that Cucumber understands. It is a Business Readable, Domain Specific
Language that lets you describe software’s behaviour without detailing how that behaviour is
implemented.
▸Gherkin serves two purposes — documentation and automated tests. The third is a bonus feature
telling you what code you should write.
▸Gherkin’s grammar is defined in the Treetop grammar that is part of the Cucumber codebase. The
grammar exists in different flavours for many spoken languages (60 at the time of writing), so that
your team can use the keywords in your own language.
▸Single Gherkin source file contains a description of a single feature (user story).
▸Source files have .feature extension.

GIVEN-WHEN-THEN CONSTRUCT
▸The Given-When-Then formula is a
template intended to guide the writing
of acceptance tests for a User Story:
▸(Given) some context
▸(When) some action is carried out
▸(Then) a particular set of
observable consequences should
obtain
An example:
Given my bank account is in credit, and I made no withdrawals recently,
When I attempt to withdraw an amount less than my card's limit,
Then the withdrawal should complete without errors or warnings

GHERKIN LANGUAGE EXAMPLE

DEVELOPER IMPLEMENT CUCUMBER STEPS FOR
GHERKIN FEATURE FILE
▸Use Gherkin language, human-readable
▸For Behaviour-Driven Development
▸Developers, testers and business folks explore the problem domain, and collaborate to produce concrete examples that describe the
behaviour they want
▸Use RegExp to match pattern and locate steps in step definition file
▸Accept testing data feed by Gherkin Example data table
▸Used in Continuous Delivery
▸Automatic Acceptance Test or Smoke Test before actual acceptance testing done by users
▸Automatic UI Test run in Test Cloud (Amazon Test Farm or Microsoft Xamarin Test Cloud) - UI Test run on devices
▸Offload developer effort
▸Ruby version is recommended to support Calaba.sh and other plugins’ predefined steps

STEP DEFINITION FILE EXAMPLE (RUBY, JAVA,
NODEJS, C#, ETC.)

CUCUMBER-SELENIUM PREDEFINED STEPS
▸Installation Step: gem install selenium-cucumber
▸https://github.com/selenium-cucumber/selenium-cucumber-
ruby/blob/master/doc/canned_steps.md

CUCUMBER-API PREDEFINED STEPS
▸Installation Step: gem install cucumber-api
▸https://github.com/hidroh/cucumber-api

CALABASH-IOS PREDEFINED STEPS (FROM
CALABA.SH)
▸Please visit Calaba.sh website to follow Calaba.sh sandbox installation
▸https://github.com/calabash/calabash-ios/wiki/02-Predefined-steps

CALABASH-ANDROID PREDEFINED STEPS (FROM
CALABA.SH)
▸Please visit Calaba.sh website to follow Calaba.sh sandbox installation
▸https://github.com/calabash/calabash-android/blob/master/ruby-gem/lib/calabash-
android/canned_steps.md

RUN ACCEPTANCE TEST USING
CUCUMBER/CALABA.SH
▸Just run “cucumber” in your project repository except mobile
▸Need to use calabash-sandbox to run mobile acceptance (smoke) test

SUPPORTING PLATFORM
▸Cloud Platform
▸Amazon Test Farm
▸Xamarin Test Cloud (acquired by Microsoft)
▸PaaS
▸SourceLab, TravisCI, CircleCI, etc.
▸On-premise
▸Jenkins

EXAMPLE OF USER STORY
▸Syntax
• As a < type of user >, I want/need < some goal > so that < business beneift/value >.
‣ Example
• As a potential customer, I want to collect books in a shopping cart so that I can order several books
at once.

CONVERT USER STORY TO DRAFT GHERKIN
FEATURE FILEFeatures: As a potential customer, I want to collect books in a shopping cart so that I can order several books at once.

Acceptance Criteria:
- Books can be placed into shopping cart.
- Books can be removed from shopping cart.
- Shopping cart should be empty when entering the shop.
- Adding same book again to shopping cart should increase quantity.

Acceptance Criteria:
- Books can be placed into shopping cart.
- Books can be removed from shopping cart.
- Shopping cart should be empty when entering the shop.
- Adding same book again to shopping cart should increase quantity.
Scenario: Place book into shopping cart successfully
Given my shopping basket is empty
When I add the book “Harry Potter” to my shopping basket
Then my shopping basket should contain 1 copy of “Harry Potter”

SOURCE CODE MANAGEMENT
GIT OVERVIEW

GIT OVERVIEW (CONT’D)
▸Having a distributed architecture, Git is an example of a DVCS (hence Distributed Version Control
System).
▸Rather than have only one single place for the full version history, every developer's working copy of the
code is also a repository that can contain the full history of all changes.
▸All of objects in the Git repository are secured with a cryptographically secure hashing algorithm called
SHA1.
▸Recommended GUI tool - Atlassian SourceTree
https://www.sourcetreeapp.com/

GIT FLOW ‣ Git-flow use rebase - cleaner history
‣ Have to follow the Golden Rule of Rebasing -
https://www.atlassian.com/git/tutorials/merging-vs-
rebasing#the-golden-rule-of-rebasing
‣ Automated rebase when finishing feature, release
or hot fix
‣ Git-flow is plugin of git. Please visit git-flow GitHub
repo and follow installation steps for specific OS
(https://github.com/nvie/gitflow/wiki/Installation)
‣ Many developers made mistake to create
feature and hot fixes manually
‣ Create feature per user story

GIT-FLOW USAGE USING COMMAND LINE

UNIT TEST
HOW TO PREPARE UNIT TEST LOCALLY
▸Unit test should be component or class based
▸Server-independent (can use docker to simulate server locally if server is
required)

UNIT TEST
SAMPLE MOCHA UNIT TEST
describe('User', function() {
describe('#save()', function() {
it('should save without error');
});
});
COMPONENT/CLASS
METHOD
UNIT TEST
‣ Create component/class skeleton first (two developer
works together to design)
‣ Day 1: Developer A implement the method
‣ Day 2: Developer A continues to code and Developer B
implements unit test
‣ If Developer B find error, both developers should work
together to see if bug or test defect
‣ Work until all unit test passed
‣ Pair-programming can identify problem immediately
through collaboration between two developers
PAIR PROGRAMMING

UNIT TEST
SAMPLE MOCHA UNIT TEST (CONT’D)
describe('User', function() {
describe('#save()', function() {
it('should save without error', function(done) {
var user = new User('Luna');
user.save(done);
});
});
});

CODE QUALITY ANALYSIS
CONTINUOUS INTEGRATION AND CODE QUALITY
CHECK
▸CI steps should execute code quality check (SecOps).
▸Code quality check is usually done by using SonarQube scanner. So security consultant and
developers can receive alert or report through SonarQube server.
▸Can config through sonar-project.properties file in SCM repository.
▸Set minimum Test Coverage % to 40% at the very beginning of project. And then set to a high value in
the middle of project, say 60% or 80% depending on team size.

CODE QUALITY ANALYSIS
SONARQUBE MAIN SCREEN

CODE QUALITY CHECK
RUNNING SONARQUBE SCANNER

CODE QUALITY CHECK
SONARQUBE PROJECT API

CODE QUALITY CHECK
SONARQUBE - METRICS DEFINITION
▸Quality Gate
▸Can I deliver my project to production today or not?
▸Reliability
▸Number of Bugs
▸Security
▸Number of Vulnerabilities
▸Maintainability
▸Number of Code Smell - any symptom in the source code of a program that possibly indicates a deeper problem.
▸Coverage
▸How much of the source code has been covered by the unit tests?
▸Duplication
▸Number of duplicated blocks of lines.

CODE QUALITY CHECK
SONARQUBE - QUALITY GATE

Full stack development best practice and toolset

More Related Content

What's hot (20)

Similar to Full stack development best practice and toolset (20)

Recently uploaded (20)

Full stack development best practice and toolset