Looks like Go is succeeding more in the space where scripting languages like Python and Ruby are dominant and less in it's original intended space (systems programming). Rob Pike has written about it too. I wonder if they see this as an opportunity missed or unexpected boon.
We definitely made a mistake at launch using the term "systems programming", because people automatically think of operating systems. What we meant were the kind of systems we build at google. Some of the examples in the blog post are "systems programming" in the same sense.
So we are pleased to see Go being used for the job it was designed for, but we are also happy to see that Go has a broader appeal.
> Go was clearly initially targeted at replacing C++ and it's not being successful in that area at all.
This is a strong overstatement. It's true that we thought more C++ programmers would be into Go, but it's also true that there more than a few teams at Google that have moved from C++ to Go, or chosen to use Go instead of C++ for new projects. It's not a black and white thing; there is no archetypal C++ programmer that does or does not like Go.
To reiterate: Go was designed for systems programming. People are using Go for systems programming. On top of that, there are many others using Go for other purposes. We're pretty thrilled about this.
I see that as an obvious mistake. C++ is so different than C, and I imagine most C purists despite it. Go created a smarter C, and presented it as the solution. The problem though is that C++ people likely aren't C purists to begin with, or that C purists are too into C. Compounded by those who only drop to C when needed, you begin to see why it failed initially in this segment. I do think it will continue to gain adoption, but linux is such a firm C membership club that it will either be painfully slow, or else require Go to play nicer with C.
> C++ is so different than C, and I imagine most C purists despite it. Go created a smarter C, and presented it as the solution.
I think it has nothing to do with the difference between C and C++ and everything to do with the reasons why people use C and C++: extreme performance, zero-cost abstractions, and integration with native libraries.
I think you are right, and I tried to convey that in my last two sentences but in retrospect worded it poorly. I guess what I was trying to say there is that people who "drop to C"(or for that matter, start in C) for performance reasons or compatibility reasons aren't going to consider a slower and less integrated language. This is probably most people, I'm not sure many people write everything in C regardless of task anymore(such as say, webapps).
I still think the initial part stands. C and C++ are not visually or mentally similar languages. Go is very similar to C in logic, so I'm not sure why it was expected that C++ people would flock to it(unless of course they weren't happy with C++, which brings us back to the first condition in that they were probably only using it for performance).
The Go team rightly point out that C++ is needlessly complicated. That is why I despise C++. (I guess I was spoiled by doing Borland ObjectPascal first, as well)
Go provides most of the benefits of the JVM without being as much of a memory hog. Thus, it makes it a nice replacement for somebody who would rather use Pascal/Modula than C++. (Java originally was seen by me as a Pascal in C++ clothing for Unix)
Go is more like a Modula (Algol???) variant with curly braces.
Mapping interfaces onto a funky function/routine call syntax (an optional parameter in a second set of parens for the "receiver"), and maintaining stack frames around a procedural type reference (closure) after the enclosing scope returns, are about the only new tricks.
I like Go, but it's too bad they didn't simply enhance FreePascal, if their main concern was compilation speed. Was it just a "Bell Labs" ego concern?
Or, was adding garbage collection that big a need? GC of course makes Go quite different from C++, so they should not be surprised if some "speed demons" refuse to convert over. (note that I hate C++, but can see the point of this aspect)
Well, I would argue that there's a level between "db replication infrastructure" type projects and OS kernels. That is the space occupied by things like database engines (everything from BigTable and descendants to Oracle), distributed storage engines (GFS and descendants) etc. I don't think I've seen a project in Go that aims to deal with that level of "system-ness" yet. And I suspect I know the reasons (why, say, I wouldn't write the packetzoom protocol stack in Go) but I'd like to know more about whether to Go team thinks it made the right choices in language and runtime design for a systems language.
There are a few Go databases mentioned in the blog post, and several more besides. Also there's weed-fs a distributed storage system (and at least one other whose name escapes me right now). I also think Go would be a great language for implementing a BigTable server.
Hmm... I did miss the line with groupcache and kv. Sorry about that. Though I still think you'd agree with me that go is not exactly killing it at that level of the stack (as opposed to the numerous examples of rather more successful usage at a level higher than that).
I think it's mostly a case of storage systems being a very core, important part of any software stack. If shit goes wrong at that level you're really screwed, so it takes longer to develop that kind of software and also longer for people to trust it enough to use it for serious things.
But there is plenty of action at that level in the Go community, and as distributed consensus libraries like go-raft mature it will only become easier to write and trust such software in Go.
I developed software system with hundreds of millions to a few billions data entries items in the core object store (C++) and need to very fast access time. There are needs to open/load/read those objects in database in very fast speed (< 0.01 seconds). When I profiled those operations in early version of the code, the biggest bottleneck is always the malloc/free when the system reached > 10 millions records.
To get around those limitation, I end up design the data structures / datastore that eliminated the malloc/free, new/delete and do my own memory manager to map all the info directly to the structure in the file via mmap and do my own sub allocations. With those, I can get everything in < 0.01 seconds constant access time regardless the size of the database. (from startup of the program to return the value to client.)
I think all modern day database or BIG DATA type system app depend on its own custom memory manager. My theory, a language that depend on garbage collection can not be used in those big data app design.
I love to be proven wrong by someone re-implement the mysql, (or even just sqlite) in "pure golang" (not sqlite driver in C) that can provide similar level of performance with golang's garbage collector compare to sqlite's C counter part.
The use case I care about is DB size of 0.1GB - 50 GB with hundreds of millions of records in the DB.
Go has mmap and casting via the unsafe package, so the go solution to your performance problem looks exactly the same as the c++ solution.
I'd say go is actually an ideal language for building a database because:
1) syscall's and low level io (including mmap) are all pretty much first class.
2) networking (duh)
Some final ramblings:
I realize "systems programming language" has lots of different definitions, some of which go doesn't meet due to it's stop the world gc. I'll concede that point, but as a parting shot let me remind you all that free'ing a tree of objects can cause similar pauses, although at a more convenient time. For me, I would assume that there are no guarantees in a database type system, you just want to go for something like: "99% of requests complete in less than a millisecond under certain controlled conditions". You never actually know if:
1) You have disk contention for that data you're reading causing pauses.
2) You've been swapped out and that pure in-memory operation slows down by an order of magniture
3) A billion other things.
For me personally, I don't really mind adding
4) The golang gc ran
To that list of things because benefits are worth the tradeoff, but that decision depends on the project.
I knew in theory one can use mmap and unsafe pointer in go.
One main different between C and go is this:
C can type cast any mmap pointer and de-reference it got the values (pointer, offset to other locations) extremely fast - < 0.01 micro-seconds (assuming page is swap in.) BTW, I do track C line code execute time nano-seconds resolution all the time. It is not difficult to do with RDTSC instruction. It gave you timing resolution in term of CPU clock cycles - 1 2GHz CPU clock cycle = 500 picoseconds, One can easily see the system paging and context switching info from those resolution.
In C code, that is only a few assembly instructions. In go (or other GC type language) that one operations is translate to multiple C functions calls. That's probably tens if not at lease hundred times different in pure CPU cycles.
When one has to do this hundreds of millions times, that usually makes huge different in overall execution time. I personally know a lot of test cases that means minutes to finished certain operations v.s. days (+GBs of RAM).
No, I found the term systems programming to be brilliant to describe the kind of things exactly between applications and operating systems, and I think the Go team did a good job communicating this from day 1. It just seems it was misunderstood (maybe even intentionally so) by detractors that focussed too much on their very own interpretation of the term.
Well, we have Android and ChromeOS, which are both Linux. While we do have a lot of people doing Linux kernel work, I'm not aware of any "green field" operating systems being built at Google. (Although I wouldn't be surprised to be wrong here.)
Most of the software we build at Google are server processes that talk to other server processes. Go was designed primarily for this role, but it is obviously useful for a lot more.
