/dev/dump

If you follow illumos very closely, you may already know some of this. A New Fork Several months ago, I forked illumos-gate (the primary source code repository for the kernel and system components of illumos) into illumos-core . I had started upstreaming my work from illumos-core into illumos-gate.  I've since ceased that effort, largely because I simply have no time for the various arguments that my work often generates.  I think this is largely because my vision for illumos is somewhat different from that of other folks, and sadly illumos proper lacks anything resembling a guiding vision now, which means that only entirely non-contentious changes can get integrated into illumos. However, I still want to proceed apace with illumos-core, because I believe that work has real value, and I firmly believe that my vision for illumos is the one that will lead to greater adoption by users, and by distributors as well, since much of what I'm trying to achieve in illumos-ga

As a result of work I've been doing for illumos , I've recently gotten re-engaged with internationalization, and the support for this in libc and localedef (I am the original author for our localedef.) I've decided that human languages suck.  Some suck worse than others though, so I thought I'd write up a guide.  You can take this as "your language sucks if...", or perhaps a better view might be "your program sucks if you make assumptions this breaks..." (Full disclosure, I'm spoiled.  I am a native speaker of English.  English is pretty awesome for data-processing, at least at the written level.  I'm not going to concern myself with questions about deeper issues like grammar, natural language recognition, speech synthesis, or recognition, automatic translation, etc.  Instead this is focused strictly on the most basic display and simple operations like collation (sorting), case conversion, and character classification.) 1. Too many cod

I'm going to say some things here that may be controversial.  Certainly that headline is.  But please, bear with me, and read this before you judge too harshly. As another writer said, 2014 has been a terrible year for women in tech.  (Whether in the industry, or in gaming.)  Arguably, this is not a new  thing, but rather events are reaching a head.  Women (some at any rate) are being more vocal, and awareness of women's issues is up.  On the face of it, this should be a good thing. And yet, we have incredible conflict between women and men.  And this is at the heart of my problem with "Feminism". The F-Word Don't get me wrong.  I strongly believe that women should be treated fairly and with respect; in the professional place they should receive the same level of professional respect -- and compensation! -- as their male counterparts can expect.  I believe this passionately -- as a nerd, I prefer to judge people on the merits of their work, rather than

Please have a look at Sage Weil's blog post on supporting the Ada Initiative, which supports women in open source development. Sage is sponsoring an $8192 matching grant, to support women in open source development of open storage technology. You may have heard my talk recently, where I expressed that there have been no female contributions to illumos (that includes ZFS by the way!)  This is kind of a tragedy; intelligence and creativity of at least half the population are simply not represented here, and we are worser for it. If you want to try to do something about it, heres a small thing.  There's a week remaining to do so, so I encourage folks to step up.  ($3392 has already been granted.) I'm making a donation myself, if you think supporting more women in open source is a worthwhile cause, please join me!

I have just spent an all-nighter doing something I didn't expect to do. I've " modernized " less (1).  (That link is to the changeset.) First off, let me explain the motivation.  We need a pager for illumos that can meet the requirements for POSIX IEEE 2003.1-2008 more (1).  We have a suitable pager (barely), in closed source form only, delivered into /usr/xpg4/bin/more .  We have an open source /usr/bin/more , but it is incredibly limited, hearkening back to the days of printed hard copy I think.  (It even has Microsoft copyrights in it!) So closed source is kind of a no go for me. less (1) looks attractive.  It's widely used, and has been used to fill in for more(1) to achieve POSIX compliance on other systems (such as MacOS X.) So I started by unpacking it into our tree, and trying to get it to work with an illumos build system. That's when I discovered the crazy contortions autoconf was doing that basically wound up leaving it with just legacy

(Sorry for the political/religious slant this post takes... I've been trying to stay focused on technology, but sometimes events are simply too large to ignore...) The execution of John Foley is just the latest.  But for me, its the straw that broke the camel's back.  Over the past weeks, I've become extremely frustrated and angry.  The "radical Islamists" have become the single biggest threat to world peace since Hitler's Nazi's.  And they are worse than the Nazi's.  Which takes some doing.  (Nazi's "merely" exterminated Jews.  The Islamists want to exterminate everyone who does't believe exactly their own particular version of extreme religion.) I'm not a Muslim.  I'm probably not even a Christian when you get down to it.  I do believe in God, I suppose.  And I do believe that God certainly didn't intend for one group of believes to exterminate another simply because they have different beliefs. Parts of

A year in the making... and finally the code is pushed.  Hooray! I've just pushed 2964  into illumos , which adds support for a bunch of new libc calls for thread safe and thread-specific locales, as well as explicit locale_t objects.   Some of the interfaces added fall under the BSD/MacOS X " xlocale " class of functions. Note that not all of the xlocale functions supplied by BSD/MacOS are present.  However, all of the routines that were added by POSIX 2008 for this class are present, and should conform to the POSIX 2008 / XPG Issue 7 standards.  (Note that we are not yet compliant with POSIX 2008, this is just a first step -- albeit a rather major one.) The webrev is also available for folks who want to look at the code. The new APIs are documented in newlocale(3c), uselocale(3c), etc.   (Sadly, man pages are not indexed yet so I can't put links here.) Also, documentation for some APIs that was missing (e.g. strfmon(3c)) are now added. This project has

Recently, there has been a bit of a debate on the illumos mailing lists, beginning I suppose with an initial proposal I made concerning the output from the uname (1) command on illumos, which today, when presented with "-s -r" as options, emits "SunOS 5.11". A lot of the debate centers on various ways that software can or cannot identify the platform it is running on, and use that information to make programmatic decisions about how to operate. Most often these are decisions made at compile time, by tools such as autoconf and automake . Clearly, it would be better for software not to rely on uname for programmatic uses, and detractors of my original proposal are correct that even in the Linux community, the value from uname cannot be relied upon for such use.  There are indeed better mechanisms to use, such as sysconf (3c), or the ZFS feature flags, to determine individual capabilities.  Indeed, the GNU autotools contains many individual tests for such thin

Introduction As a result of some investigations performed in response to my first performance tests for my SP implementation , I've made a bunch of changes to my code. First off, I discovered that my code was rather racy.  When I started bumping up GOMAXPROCS, and and used the -race flag to go test, I found lots of issues.  Second, there were failure scenarios where the performance fell off a cliff, as the code dropped messages, needed to retry, etc.  I've made a lot of changes to fix the errors.  But, I've also made a major set of changes which enable a vastly better level of performance, particularly for throughput sensitive workloads. Note that to get these numbers, the application should "recycle" the Messages it uses (using a new Free() API... there is also a NewMessage() API to allocate from the cache), which will cache and recycle used buffers, greatly reducing the garbage collector workload. Throughput So, here are the new numbers

So I've been thinking about naming for my pure Go implementation of nanomsg 's SP protocols. nanomsg is trademarked by the inventor of the protocols.  (He does seem to take a fairly loose stance with enforcement though -- since he advocates using names that are derived from nanomsg, as long as its clear that there is only one "nanomsg".) Right now my implementation is known as "bitbucket.org/gdamore/sp".  While this works for code, it doesn't exactly roll off the tongue.  Its also a problem for folks wanting to write about this.  So the name can actually become a barrier to adoption.  Not good. I suck at names.  After spending a day online with people, we came up with " illumos " for the other open source project I founded.  illumos has traction now, but even that name has problems.  (People want to spell it "illumOS", and they often mispronounce it as "illuminos"  (note there are no "n"'s in illumos).  

I've added a benchmark tool to my Go implementation of nanomsg 's SP protocols, along with the inproc transport, and I'll be pushing those changes rather shortly. In the meantime, here's some interesting results: Latency Comparision (usec/op) transport nanomsg 0.3beta gdamore/sp inproc 6.23 8.47 ipc 15.7 22.6 tcp 24.8 50.5 The numbers aren’t all that surprising.  Using go, I’m using non-native interfaces, and my use of several goroutines to manage concurrency probably creates a higher number of context switches per exchange.  I suspect I might find my stuff does a little better with lots and lots of servers hitting it, where I can make better use of multiple CPUs (though one could write a C program that used threads to achieve the same effect). The story for throughput is a little less heartening though: Throughput Comparision (Mb/s) transport message size nanomsg 0.3beta gdamore/sp inproc 4k 4322 5551 ipc 4k 9470 2379 tcp 4

I'm pleased to announce that this past weekend I released the first version of my implementation of the SP (scalability protocols, sometimes known by their reference implementation, nanomsg ) implemented in pure Go . This allows them to be used even on platforms where cgo is not present.  It may be possible to use them in playground (I've not tried yet!) This is released under an Apache 2.0 license .  (It would be even more liberal BSD or MIT, except I want to offer -- and demand -- patent protection to and from my users.) I've been super excited about Go lately.  And having spent some time with ØMQ in a previous project, I was keen to try doing some things in the successor nanomsg project.   (nanomsg is a single library message queue and communications library.) Martin (creator of ØMQ ) has written rather extensively about how he wishes he had written it in C instead of C++.  And with nanomsg, that is exactly what he is done. And C is a great choice for implement

My dislike for C++ is well-known by those who know me.  As is my lack of fondness for Perl. I have a new one though.  Java.  Oracle and Apple have conspired to kill it. (I should say this much -- its been a long time, about a decade, since I developed any Java code.  Its entirely possible that I remember the language with more fondness than it truly warrants.  C++ was once beautiful too -- before ANSI went and mutated it beyond all hope back in 1990 or thereabouts.) Which is a shame, because Java started with such  promise.  Write-once, run-anywhere.  Strongly typed, and a paradigm for OO that was so far superior to C++.  (Multiple inheritance is the bane of semi-competent C++ engineers, who often can't even properly cope with pointers to memory.) For years, I bemoaned the fact that the single biggest weakness of Java was the fact that it was seen as a way to make more dynamic web pages.  (I remember HotJava -- what a revolutionary thing it was indeed.)  But even sta

Some of you may have noticed that my email, or my blog, or website, was down. I discontinued my hosting service with BlueHost, and when I did, I mistakenly thought that by keeping my domain registration with them, that I'd still have DNS services.  It was both a foolish mistake, and yet an easy one to make.  (I should have known better...) Anyway, things are back to normal now, once the interweb's negative caches have timed out... It does seem unfortunate that BlueHost: Does not include DNS with domain registration, nor do they have a service for it unless you actually host content with them. Did not backup my DNS zone data.  (So even if I paid them to reactivate hosting, I was going to have to re-enter my zone records by hand.)  This is just stupid. So, I've moved my DNS, and when my registration expires, I'll be moving that to another provider as well.  (Dyn, in case you wondered.)

So I've not blogged in a while, but my recent experience with a major mobile provider was so astonishingly bad that I feel compelled to write about my experiences.  (Executive summary: never do business with T-mobile again!) I had some friends out from Russia back in November, and they wanted to purchase an iPhone 5s for their son as a birthday gift.  Sadly, we couldn't get unlocked ones from the Apple Store at the time, because they simply lacked inventory. So we went to the local T-mobile store here on San Marcos Blvd (San Marcos, CA.)  I knew that they were offering phones for full-price (not subsidized), and it seemed like a possible way for them to get a phone.  Tiffany was the customer service agent that "assisted" us.  She looked like she was probably barely out of high school,  but anyway she seemed pleased to help us make a purchase.  I asked her no fewer than three times whether the phone was unlocked, and was assured that, yes, the phone was unlocked an