It’s been a while since the first post on this subject, and looking back it reads like it was written by someone who had not really used Go, just read “Effective Go” and then recited it back.
How much does it suck to see that I wrote like that. Oh, well this post will be different. You see I have used Go now for several programs that run in production. In doing so I have uncovered some of the things in go that I find really useful.
I did mention that the defer keyword and channels and goroutines would feature in
this post when I got around to writing it and indeed they will. They are still awesome.
Let’s get them out of the way first.
This is a lovely little language addition that is a bit like a finally block, but not
quite. It is a function call that will be deferred until the function we are in returns.
Here’s a quick contrived example based on the fact that sometimes you need to open a resource (or acquire a lock), then some logic, then return a value however the logic meant that there are maybe now N different logic forks that you could return from. Do you remember to release the resource (or lock)? So in my example we protect access to an integer which we are incrementing. But we can’t let it hit 10.
type ProtectedInt struct {
theInt int
theLock sync.Mutex
}
func (*p ProtectedInt) Inc (x int) int {
p.theLock.Lock()
defer p.theLock.Unlock() // <-- this call means whatever happens next, we will unlock
if p.theInt + x == 10 {
panic()
}
p.theInt = p.theInt + x
return p.theInt
}
The code above panics if the Int gets to 10, but we can catch that if we want. More importantly, even in the case of panic, the lock is unlocked.
I find defer to very very useful for housekeeping tasks like this and use it often.
I might seem strange that these are almost the last thing I talk about here, given what an intgeral part of the language they are. They are easily the most useful tool Go gives us and they seem to me to be less of a language feature than the language itself. When I think about how I will solve a problem in Go: I will solve it using both these tools and I will use patterns that only exist because of their properties.
Let’s put that into context.
My programming experience started with PHP. In PHP (unless you are very clever and we will get on to threading) all code is executed sequentially and you cannot do 2 things at once. This makes it easy to write - everything is sequential and predictable.
Most developers soon experience Javascript and in so doing you exeprience a language with an event loop. This means that although we can only do one thing at a time, when you are waiting for something outside of the code, IO or user interaction, the code can get on with something else. The freedom is liberating - but as they say in the Node.JS circles
In Node.JS nothing blocks but your code.
That is to say, most of the libraries and functions return immediately, using callbacks for continued execution once data is available, allowing your code to continue and not block the event loop. However any work you do will still block everything else.
In any language you have the option to spawn new threads, which have their own seperate blocking environment and can be heavy to set up, but allow you to do 2 things at once. Traditionally threading was the solution for any situation when you needed to block.
Now comes Go, with it’s goroutines (and yes similiar features exist in other languages, but not as integrated into the langauge) which behave like threads - seperate context which can block without blocking the main thread - but without the weight of traditional threads. For example, you can safely spawn 1000’s in a single process. Go achieves this through it’s runtime, which multiplexes many goroutines into a each real thread and does the scheduling of which one gets to execute code when.
With goroutines, we get the best of both worlds, we can spawn asynchronous tasks that don’t block our main thread and have the simplicity of writing code that does block.
Channels allow us to safely transfer data around between goroutines. The two are intrinsically linked and one would be no use without the other. In my own exeprience I found that goroutines and channels are excellently suited to pipeline jobs.
I was writing a daemon that got a job package, donwloaded an image file, did some analysis on the image, produced a feww different sized versions, uploaded the new versions to S3, write some data to the output.
Once I had decided what these steps were I designed a type which could mirror what the
state of the job is and the raw data to work with. Then I create a type for a function
to process this type of data, and and method on the function type to Stream data along
the pipeline:
//A work fn accepts a piece of work and a channel to send it once finished.
type WorkFn func(*Work) *Work
//the magic is in the stream method which produces a loop that consumes a channel of Work,
//and creates a response channel to return
func (wf WorkFn) Stream(cw chan *Work) chan *Work {
ch := make(chan *Work)
go func() {
for w := range cw {
ch <- wf(w)
}
close(ch)
}()
return ch
}
Which then make it really easy to write each step in turn, cast the step functions
to WorkFn and create a pipeline like:
//build the chain
workStart := make(chan *Work)
//this is Inside out, so we do img fetcher then processimage, then upload
workEnd := UploadImages.Stream(ProcessImage.Stream(ImgFetcher.Stream(workStart)))
OK here’s one of the big wins and the big fails. Deployment is great in Go because it all compiles statically into a single binary.
`scp app server:~/bin/app && echo "Boom!"`
Even for me with an OSX dev environment, I can still cross-compile for linux and deploy in the same way!
Except when I can’t.
It sooo almost works, but then you get code with a dependency on some C. Go makes it easy to bind C libraries to you code and many people have leveraged this to write bindings for complex libraries. However, at least in my experience, it is not possible to cross compile Go code using Cgo. This has tripped me up a couple of times and I’ll have to fire up my trusty VM to compile in a linux environment.
That’s not the end of the world of course and I do try to avoid libs using Cgo if I can
for that reason. For example, I basically stopped using YAML for a config file language
because no good YAML parsers exist for go, that don’t leverage libyaml. I switched
to TOML which is similarly easy to write, and has a pure-Go implementation (and for
many other languages).
Ok, this post has gotten far too big, you get the point though - Go has it’s ugly bits but all in all I have found it to be a simple, accessible, fast, intuitive, rewarding language to program in and I can’t say that about many others.
I’ll definitely be using in more in the future.