Why are 'Cool Kids' at Github Moving to GO Language?

These days, when we check for most starred/forked codes at github in various tags: [] categories, we often see them being written in GO language. Python used to have the same status 2-3 years back. What changed?

GO is a new programming language released by Google in 2010. The language has more going on in addition to the backing by Google. In the programming world, the creators of the language are more famous than Google.

The syntax of the code is very similar to C without strong type-checking, but GO has one advantage over C and python. It is very easy to write code and run them concurrently into multiple-processor. In that way, it is similar to Haskell or Erlang (check “Bye bye Python; Enter Haskell”), but is easier to learn. Writing concurrent code is not easy in traditional programming languages due to their history of growing in single- processor era.

Where does GO Stand in the spectrum of languages and tools? The following benchmarking done by the backers of yet another concurrent language (Julia) explains. Please note that the benchmarks are biased toward numerical computing, where Julia excels. Moreover, the C, Fortran and Julia benchmarks were done using OpenBLAS, whereas GO was not. Still you can see the differences with python, Matlab and R. For a qualitative comparison of Julia and Go, please check the link at the bottom of the commentary.

Will the python world switch to GO? Here are a few blog posts arguing for and against such idea.

Why I went from Python to Go (and not node.js)

People often ask me why I have decided that Id be writing the bulk of my new code in Go, which I started programming in November of 2011 while attending Hacker School. At that time, concurrency was a very hot topic in Hacker School, and we were all trying out different ways of writing concurrent code. A bunch of us pitched in and helped out with Brubeck, a framework for doing concurrency programming with Python, which is probably the least awkward way of doing concurrency in a Python web application that Ive found. But lets rewind a little bit, because the context here is exceptionally important to understand why I chose Go, as it explains why it appeals to me. Ultimately, your experiences may be very, very different.

Why Im not leaving Python for Go

First of all, Go seems like a great language. It has an excellent tutorial which I joyfully went through and found:

Go is Fast.

Concurrent by design.

Typed (important for JIT and IDEs) but not cumbersome and ugly like C or C++s spirals.

Duck-type-esque interfaces.

The defer mechanism is really nifty.

But theres one problem I cant live with. Which is a shame as I was eager to make the leap of faith in the name of concurrency. That problem is errors are handled in return values. 70?s style.

What Python developers need to know before migrating to Go(lang)

This is a (long) blog post about our experience at Repustate in migrating a big chunk of code from Python/Cython to Go. If you want to read the whole story, background and all, read on. If youre interested in just what Python developers need to know before taking the plunge, click the link below.

Tips & tricks in migrating from Python to Go.

The Background

One of the best technological feats that weve done here at Repustate was implementing Arabic sentiment analysis. Arabic is one tough nut to crack because of the complex morphological forms Arabic words can take. Tokenization (splitting a sentence up into individual words) is also tougher in Arabic than in say, English, because Arabic words can contain whitespace within the word itself (e.g. the position of aleph within a word). Without giving away our secret recipe, Repustate uses support vector machines (SVM) to come up with the most likely meaning behind a sentence and then apply sentiment to that. In total, we use 22 models (i.e 22 SVMs) and each word in a document is analyzed. So if you have 500 words in a document, thats more than 10,000 comparisons against the SVMs.

Python

Repustate is almost entirely a Python shop; we use Django for the API and website. So it only made sense (at the time) to keep the code base homogenous and implement all of the Arabic sentiment engine in Python as well. As far as prototyping and implementing goes, Python is hard to beat. Very expressive, awesome 3rd party libraries etc. If youre serving up web pages, its perfect. But when youre doing low level computations, doing lots of comparisons against hash tables (dictionaries in Python), things get slow. We were able to process about 2-3 Arabic documents per second, which is too slow. By comparison, our English language sentiment engine can do about 500 per second.

The Bottleneck

So we fired up the Python profiler and began investigating what was taking so long. Remember above how I said we have 22 SVMs and each word passes through it? Well that was all done in serial, not in parallel. OK, our first thought was to change to this to a map/reduce like operator. TL;DR: The map/reduce idiom stinks in Python. When you need concurrency, Python is just not your friend. At PyCon 2013, Guido spoke about Tulip, his new project that was hoping to remedy this, but thats not due out for a while, and why wait theres already something better out there.

Golang or go home

My friend at Mozilla told me that Mozilla Services was switching over to Go for much of their logging infrastructure, in part because of the awesomeness of goroutines. Go was designed by the folks at Google and it was designed with concurrency as a first-class notion, not an afterthought, as Pythons various solutions are. So we went about making the change from Python to Go.

Regarding Julia vs GO:

Programming Languages: How do Go, Scala and Julia compare with each other?

Go is a systems programming language. Google created it to safely solve three specific problems that C++ was biting them with: concurrency, memory management and compilation time on large systems. It’s concise and readable, like Scala and Julia, but a bit fussy and low-level, unlike Julia. You would use it to write safe, well-performing services that your main application might call to. It would certainly be possible to write scientific software in Go, much as you would with C or C++, but it doesn’t have the same flexibility and interactivity as, say, Python or Julia, or even Scala. If you’re trying to use Go for data analysis, the fussiness and lack of interactivity will disappoint you.

Julia is a scientific applications programming language, designed to be a good replacement for Matlab and Python+SciPy. It’s very new, so I won’t attempt to predict which specific features it will or won’t pick up, but the underlying motivations should drive it to be a very good programming language for scientific software – not syntactically optimized for statistical operations on data arrays like R, but good to write a computationally intensive program that uses multiple CPUs and generates pretty graphics.