Engineering Efficiency in LINE
0 likes436 views
The document describes Lineflow, an internal tool developed by LINE corp to improve engineering efficiency. It aims to automate LINE's software development process through a command line interface. Key points: - Lineflow was built with Golang to define commands as a domain-specific language that reflects LINE's development workflow. It tracks command history to suggest next steps. - This helped save engineers time by automating repetitive manual tasks like copying code between branches. It is estimated to save 41 engineer-hours per day. - Developing internal tools requires dedicated resources and evangelism to gain adoption, but it lays the foundation for further improvements and integration with other tools.
![type Command struct {
Execute func(cmd *Command, args *Args)
Flag flag.FlagSet
...
Name string
parent *Command
childs map[string]*Command
}
type Args struct {
Executable string
Command string
ProgramPath string
Params []string
...
}
type Executor struct {
commands map[string]*Command
execute func(cmds []*cmd.Cmd) error
}
We wrote simple wrapper
for os/exec](https://image.slidesharecdn.com/gophercon-171104132317/85/Engineering-Efficiency-in-LINE-29-320.jpg)
![func (r *Executor) Execute() ExecError {
....
cmd := Find(args.Command)
if cmd != nil {
return r.Call(cmd, args)
}
printUsage()
return newExecError(errors.New("command not exist"))
}
func (r *Executor) Find(name string) *Command {
return r.commands[name]
}](https://image.slidesharecdn.com/gophercon-171104132317/85/Engineering-Efficiency-in-LINE-30-320.jpg)
![type History struct {
fileLocation string
commands []*c.Command
}
func (h *History) Register(cs ...*c.Command)
func (h *History) Save(cmd *c.Command, meta string) error
func (h *History) traverseFromTop(
f func(*c.Command, string) bool) error
func (h *History) Last(parent *c.Command, meta string)
*c.Command
func (h *History) toHistoryString(cmd *c.Command) string
func (h *History) historyStringToCommand(s string)
(*c.Command, string, error)](https://image.slidesharecdn.com/gophercon-171104132317/85/Engineering-Efficiency-in-LINE-41-320.jpg)
Engineering Efficiency in LINE
- 1. Engineering Efficiency powered by Go Huy Do Software Engineering LINE corp
- 2. reflect.TypeOf(Who) me := &Who { Name: "Huy Do", Location: "Tokyo", Org: "LINE corp", Misc: "admin member of Ruby Vietnam group, and co-founded kipalog.com", }
- 6. “We massively underinvest in this kind of work” Peter Seibel, Twitter
- 10. We do anything, except raising your salary!
- 11. • Building in-house deploy system • Building scale in-house monitoring system • Building / maintaining OSS CI/Repository system Building many tools help our colleague engineers focus in write code efficiently https://linecorp.com/ja/career/position/786
- 14. https://engineering.linecorp.com/ja/blog/detail/38 1. Make JIRA ticket 2. Create local branch follow TICKET-ID-feature 3. Make a Pull Request to upstream/develop 4. Release to Beta server 5. Request QA team to confirm feature 6. Cherry-pick changes to local TICKET-ID-release branch 7. Make a Pull Request to upstream/release 8. Increase artifact version 9. Create RC branch and release to staging server and confirm integration 10. Release to Canary server and confirm no big problem 11. Release to all server 12. Close JIRA ticket
- 15. We need to follow many conventions as well
- 16. 12 steps needed to release your code production!!!!
- 17. it's just too much!!!!!
- 18. But Process is needed • Procedures protect us from stepping on another's toes • Conventions help us inspect problem
- 19. Problem 1: Manual copy paste is just painful
- 21. design CLI tool for process automation
- 22. Lineflow goals • Extensible • Safe • Reflect the domain • Zero dependency
- 26. Some concepts • flow • Set of commands belong to same domain • subcommand • Represent single execute step
- 28. Implementation
- 29. type Command struct { Execute func(cmd *Command, args *Args) Flag flag.FlagSet ... Name string parent *Command childs map[string]*Command } type Args struct { Executable string Command string ProgramPath string Params []string ... } type Executor struct { commands map[string]*Command execute func(cmds []*cmd.Cmd) error } We wrote simple wrapper for os/exec
- 30. func (r *Executor) Execute() ExecError { .... cmd := Find(args.Command) if cmd != nil { return r.Call(cmd, args) } printUsage() return newExecError(errors.New("command not exist")) } func (r *Executor) Find(name string) *Command { return r.commands[name] }
- 31. Problem 2: I just forgot what I should do next
- 34. Implementation
- 37. type FlowSpec struct { root *CmdNode last *CmdNode } func (f *FlowSpec) NextOf(cmd *c.Command) (*c.Command, error) func (f *FlowSpec) Next(cmd *c.Command) *FlowSpec struct called FlowSpec to implement the DSL
- 38. type CmdNode struct { value *c.Command next *CmdNode prev *CmdNode } func (n *CmdNode) Prev(cn *CmdNode) { n.prev = cn cn.next = n } func (n *CmdNode) Next(cn *CmdNode) { n.next = cn cn.prev = n } func (n *CmdNode) GetNext() *CmdNode { return n.next } func (n *CmdNode) GetPrev() *CmdNode { return n.prev } Simple double linked list
- 39. What is the last thing that you've done ?
- 40. • Branch name as unique identity • Just store every executed command in ~/.lineflow_history (same idea with .bash_history)
- 41. type History struct { fileLocation string commands []*c.Command } func (h *History) Register(cs ...*c.Command) func (h *History) Save(cmd *c.Command, meta string) error func (h *History) traverseFromTop( f func(*c.Command, string) bool) error func (h *History) Last(parent *c.Command, meta string) *c.Command func (h *History) toHistoryString(cmd *c.Command) string func (h *History) historyStringToCommand(s string) (*c.Command, string, error)
- 42. So basically that's how lineflow is built
- 44. Lineflow impact • Saved time • 500 engineer * 5 minutes ~ 41 hours (per day) • Make foundation for further internal tool intergration • Help new comer get used to the process
- 45. What we learned • Golang is bad at writing DSL • Hard to develop plugin-style due static code loader (compare to ruby)
- 46. How golang survive in Java shop • IntelliJ Golang plugin is great (we can work with both java/golang in same editor) • go tools ecosystem is great (debugger, benchmark, test tool..) • Distribute golang code is great (we love binary)
- 47. How to build good internal product • MUST invest resource (there are no good short-term solution) • API, and good API • You need to be an evangelist for your own product, or no body will use yours :(
- 48. Thank you