Refactoring to GO modules

@JFrog | jfrog.com | #GoCenter
Refactoring to modules
Why, How and Everything Else

Who am I?
•DevOps Consultant
•Full-stack software engineer
• Tackling everything from
DevOps and Backend challenges
to the latest Frontend frameworks
Elad Hirsch , DevOps Consultant

FROGS - THE LIQUID SOFTWARE COMPANY

GoCenter
Xray
JFrog CLI
Metadata server
Replicator
E+ Platform Installer
We absolutely love Go and we use it a lot

Let’s travel back in time…

When did you start with Go?
Let’s turn to the audience for a poll…
1.0
2012
1.2
2013
1.5
2015
1.8
2017
1.11
2018

A quick history of go
Go 1.0
First major
milestone as a
long term
stable release
2012 2015
Go 1.5
First release to
no longer use C
(except for cgo)
2017
Go 1.8
Introduction of
Go plugins
2018
Go 1.11
This is where
the magic is! (at
least for this
talk ☺)

Dependency Management…
So what is that magic?

“Tis impossible to be sure of anything
but Death and Taxes”
- Christopher Bullock

Let’s start with the why…

Why do we have
this problem?

GO origins

Dependency management @ Google
•One Huge Makeﬁle
•Refactor into multi Makeﬁle per module
•Pain of dependency management is huge this days
Dependency Hell term
is born

Let’s do something very simple

One easy solution…?
Dependencies are sources!
Remote imports are in VCS
Dump everything into a single folder
Compile everything together
Proﬁt!!

•Which dependencies I use?
•Which dependencies you used?
•Which dependencies I should use?
•Which code I’m editing right now?
•Is there any dependency duplications?
•What is going on?!
Wait ! But… how do I know…

The official response - Let’s duplicate dependencies
“Check your dependencies to your own VCS.”
Andrew Gerrand

Vendoring – the worst kind of forking
“Copyall of the ﬁlesat some version from one version control repository and
pastethem into a differentversion control repository”

• History, branch, and tag information is lost
(metadata is lost)
• Pulling updates is impossible
• It invites modiﬁcation, divergence, and bad
fork
• It wastes space
• Good luck ﬁnding which version of the code
you using
But what is wrong with vendoring?

Build your own dependency manager…
“It’s not the role of the tooling provided by
the language to dictate how you manage
your code (...)”
Andrew Gerrand

The community steps in

AND THE NEXT THING YOU KNOW…
THERE ARE 19 DEPENDENCY MANAGERS

•Working in project directories
•Local cache for dependencies
•Version declarations
•Conﬂict resolution
Go dep – Proper dependency management?

•Started as an experiment
•Lesson learned from - java / npm …
•Main challenge - how to solve conﬂict resolution
• Dep disallow multiple major versions
• GO Tech lead Russ Cox took another approach
• Go -> Simple solution -> SMV
Go dep status

Let’s actually go ﬁx it!

Who is using Go modules?
Let’s turn to the audience for another poll…
Yes No

Module
A module is a collection of
related Go packages that
are versioned together as a
single unit.
Let’s go over some definitions

Sources
A module is a tree
(directory) of Go source
ﬁles with a go.mod ﬁle in
the root directory.

Version Control
Most often, a single
version-control repository
corresponds exactly to a
single module

Set the environment variable
GO111MODULE to ON
you do not need to set
GO111MODULE
Using go modules
1
2
Work inside $GOPATH Work outside $GOPATH

mkdir mymodule
cd mymodule
go mod init github.com/retgits/mymodule
Creating a module

•Import dependencies from Gopkg.lock
go mod init <module path>
•Remove unnecessary imports and add indirect imports
go mod tidy
•Delete the vendor folder
rm –rf vendor/
•Delete Gopkg ﬁles
rm Gopkg.*
Moving from DEP to modules

Upgrade from dep to go modules
git clone https://github.com/eladh/sample-app-go-dep.git

module github.com/retgits/mymodule
go 1.12
require (
github.com/naoina/go-stringutil v0.1.0
github.com/some/dependency v1.2.3
github.com/google/go-github/v25 v25.0.1
)
replace github.com/some/dependency github.com/retgits/dependency
Versioning

go 1.12
require (
)
replace github.com/some/dependency github.com/retgits/dependency
Replacing packages

go 1.12
require (
)
replace github.com/some/dependency ../../Downloads/dependency
It’s great when you got dependencies
that still under local development

•You can create the vendor folder
go mod vendor
•You can use the vendor folder during builds
go build –mod=vendor
This does mean you need to have all dependencies
listed in your go.mod ﬁle
But I kinda like the vendor folder?
It’s provide us dependencies beforehand

Before modules
go get https://github.com/sirupsen/logrus
git clone https://github.com/sirupsen/logrus
<build module locally>
With modules
go get github.com/sirupsen/logrus
GET github.com/sirupsen/logrus/@v/list
GET github.com/sirupsen/logrus/@v/v1.0.0.mod
GET github.com/sirupsen/logrus/@v/v1.0.0.zip
Go get my a new super duper module

Houston… I think I lost my module?

The <module>@v<version> construct should be immutable
That means that
github.com/retgits/checkiday/@v/v1.0.0
Should forever be the same…
Modules are immutable

But are they really?
”Friends don’t let friends do git push -f”
- Aaron Schlesinger

Using the GOPROXY variable
export GOPROXY=https://myawesomeproxy.com
go get github.com/retgits/checkiday

A Go module proxy is any web server that can
respond to GET requests for URLs of a specified form.
The requests have no query parameters, so even a
site serving from a fixed file system (including a
file:/// URL) can be a module proxy.

Keeping modules
Highly available, CDN, no infra, free
Public cache (public proxy)
Immediate access, not shared, can be wiped…
Local cache ($GOPATH/pkg/mod)
Fast access, requires infra, shared across devs
Organizational cache (private proxy)

And also faster builds…

Let’s check public proxy performance

What is a 5* module for you?
How can we ensure module quality ?

•If you haven’t, start with Go modules:
GO111MODULE=on if you must stay on $GOPATH
•Start working outside of your $GOPATH
•Use a public proxy server for immutable Go modules
(like GoCenter.io)
•Think about running your own proxy server or repository
manager such as Artifactory or Athens
Final thoughts

Go modules are in beta stage ?
Modules are supported from Go version 1.11
but it will be ﬁnalized in Go version 1.13

There’s one more thing…
But before I go…

Promo - 100NIS_less

Thank you!

▪
JFrog CLI
// Configure Artifactory:
jfrog rt c
//Configure the project's repositories::
jfrog rt go-config
//Build the project with go and resolve the project dependencies from
Artifactory.
jfrog rt go build --build-name=my-build --build-number=1
//Publish the package we build to Artifactory.
jfrog rt gp go v1.0.0 --build-name=my-build --build-number=1
//Collect environment variables and add them to the build info.
jfrog rt bce my-build 1
//Publish the build info to Artifactory.
jfrog rt bp my-build 1
JFrog CLI - GO support

Refactoring to GO modules

More Related Content

What's hot (20)

Similar to Refactoring to GO modules (20)

More from Elad Hirsch (10)

Recently uploaded (20)

Refactoring to GO modules