/dev/dump

(Sorry for the political/religious slant this post takes... I've been trying to stay focused on technology, but sometimes events are simply too large to ignore...) The execution of John Foley is just the latest.  But for me, its the straw that broke the camel's back.  Over the past weeks, I've become extremely frustrated and angry.  The "radical Islamists" have become the single biggest threat to world peace since Hitler's Nazi's.  And they are worse than the Nazi's.  Which takes some doing.  (Nazi's "merely" exterminated Jews.  The Islamists want to exterminate everyone who does't believe exactly their own particular version of extreme religion.) I'm not a Muslim.  I'm probably not even a Christian when you get down to it.  I do believe in God, I suppose.  And I do believe that God certainly didn't intend for one group of believes to exterminate another simply because they have different beliefs. Parts of ...

A year in the making... and finally the code is pushed.  Hooray! I've just pushed 2964  into illumos , which adds support for a bunch of new libc calls for thread safe and thread-specific locales, as well as explicit locale_t objects.   Some of the interfaces added fall under the BSD/MacOS X " xlocale " class of functions. Note that not all of the xlocale functions supplied by BSD/MacOS are present.  However, all of the routines that were added by POSIX 2008 for this class are present, and should conform to the POSIX 2008 / XPG Issue 7 standards.  (Note that we are not yet compliant with POSIX 2008, this is just a first step -- albeit a rather major one.) The webrev is also available for folks who want to look at the code. The new APIs are documented in newlocale(3c), uselocale(3c), etc.   (Sadly, man pages are not indexed yet so I can't put links here.) Also, documentation for some APIs that was missing (e.g. strfmon(3c)) are now added. This ...

Recently, there has been a bit of a debate on the illumos mailing lists, beginning I suppose with an initial proposal I made concerning the output from the uname (1) command on illumos, which today, when presented with "-s -r" as options, emits "SunOS 5.11". A lot of the debate centers on various ways that software can or cannot identify the platform it is running on, and use that information to make programmatic decisions about how to operate. Most often these are decisions made at compile time, by tools such as autoconf and automake . Clearly, it would be better for software not to rely on uname for programmatic uses, and detractors of my original proposal are correct that even in the Linux community, the value from uname cannot be relied upon for such use.  There are indeed better mechanisms to use, such as sysconf (3c), or the ZFS feature flags, to determine individual capabilities.  Indeed, the GNU autotools contains many individual tests for such thin...

Introduction As a result of some investigations performed in response to my first performance tests for my SP implementation , I've made a bunch of changes to my code. First off, I discovered that my code was rather racy.  When I started bumping up GOMAXPROCS, and and used the -race flag to go test, I found lots of issues.  Second, there were failure scenarios where the performance fell off a cliff, as the code dropped messages, needed to retry, etc.  I've made a lot of changes to fix the errors.  But, I've also made a major set of changes which enable a vastly better level of performance, particularly for throughput sensitive workloads. Note that to get these numbers, the application should "recycle" the Messages it uses (using a new Free() API... there is also a NewMessage() API to allocate from the cache), which will cache and recycle used buffers, greatly reducing the garbage collector workload. Throughput So, here are the new numbers ...

So I've been thinking about naming for my pure Go implementation of nanomsg 's SP protocols. nanomsg is trademarked by the inventor of the protocols.  (He does seem to take a fairly loose stance with enforcement though -- since he advocates using names that are derived from nanomsg, as long as its clear that there is only one "nanomsg".) Right now my implementation is known as "bitbucket.org/gdamore/sp".  While this works for code, it doesn't exactly roll off the tongue.  Its also a problem for folks wanting to write about this.  So the name can actually become a barrier to adoption.  Not good. I suck at names.  After spending a day online with people, we came up with " illumos " for the other open source project I founded.  illumos has traction now, but even that name has problems.  (People want to spell it "illumOS", and they often mispronounce it as "illuminos"  (note there are no "n"'s in illumos).  ...

I've added a benchmark tool to my Go implementation of nanomsg 's SP protocols, along with the inproc transport, and I'll be pushing those changes rather shortly. In the meantime, here's some interesting results: Latency Comparision (usec/op) transport nanomsg 0.3beta gdamore/sp inproc 6.23 8.47 ipc 15.7 22.6 tcp 24.8 50.5 The numbers aren’t all that surprising.  Using go, I’m using non-native interfaces, and my use of several goroutines to manage concurrency probably creates a higher number of context switches per exchange.  I suspect I might find my stuff does a little better with lots and lots of servers hitting it, where I can make better use of multiple CPUs (though one could write a C program that used threads to achieve the same effect). The story for throughput is a little less heartening though: Throughput Comparision (Mb/s) transport message size nanomsg 0.3beta gdamore/sp inproc 4k 4322 5551 ipc 4k 9470 2379 tcp 4...

I'm pleased to announce that this past weekend I released the first version of my implementation of the SP (scalability protocols, sometimes known by their reference implementation, nanomsg ) implemented in pure Go . This allows them to be used even on platforms where cgo is not present.  It may be possible to use them in playground (I've not tried yet!) This is released under an Apache 2.0 license .  (It would be even more liberal BSD or MIT, except I want to offer -- and demand -- patent protection to and from my users.) I've been super excited about Go lately.  And having spent some time with ØMQ in a previous project, I was keen to try doing some things in the successor nanomsg project.   (nanomsg is a single library message queue and communications library.) Martin (creator of ØMQ ) has written rather extensively about how he wishes he had written it in C instead of C++.  And with nanomsg, that is exactly what he is done. And C is a great choice f...