zfs should not depend on python... and doesn't anymore

As of the recent integration of a colleague of mine, illumos now has a zfs command that does not depend on python at all.

changeset: 13246:fe5d6e0b0bce
tag: tip
user: Alexander Stetsenko
date: Wed Dec 01 02:30:25 2010 +0300
description:
278 get rid zfs of python and pyzfs dependencies
Reviewed by: gordon.w.ross@gmail.com
Reviewed by: trisk@opensolaris.org
Reviewed by: alexander.r.eremin@gmail.com
Reviewed by: jerry.jelinek@joyent.com
Approved by: garrett@nexenta.com

The zfs command is now entirely a C program. This may make it more friendly for use in other environments or platforms. FreeBSD folks, you might want to incorporate this into your tree. If you do, I'd sure like to know.

Job Opp @ Nexenta: Director of Sustaining/Certifications

We're looking for a Director of Engineering, to own sustaining (aka bug fixing) and hardware platform certifications (where partners provide a hardware platform and ask us to certify it for use with NexentaStor.)

The job qualifications:

1. Must be local to the SF bay area (because the hardware lab is here)
2. Must have strong communication skills
3. Must be able to deal with stressful situations, and able to "manage" strong personalities
4. Must have Solaris/OpenSolaris expertise... hands-on kernel work (crash dump analysis, coding, etc.)
5. Desire experience with Storage protocols and products
6. Desire expertise with x86 hardware
7. Desire perl and/or python skills

To be clear, this will start out as a hands-on position, with a fast-paced startup environment. But the growth opportunities here are enormous. If you think you're up to this, please let me know.

New desktop image

Here's a sample of the new logo as a desktop image. I've not made this available publicly yet (mostly because I don't know how to capture this in a form that will include the gradient and post it where people can find it.) If someone with some gnome expertise on how to share this for others contacts me, I can work to make it available.

New illumos logo

Today at the OpenStorage Summit 2010, I unveiled the new illumos logo. We will be updating our branding, which also includes a new font, and other elements, over the next few weeks.

There were other updates on illumos that I covered in this talk. I think this was recorded, but I'm not sure right now where it was recorded and how to acces it. I'll be sure to share that when I find out.

CFV: web/HTML/graphics people

I have an urgent need to rennovate the illumos website. If you'd like to help the project out, and you have got both time and talent, please let me know. A major overhaul of the site is in order, and we need someone willing to dedicate some time on it. There may be some funds available for the right person, but to be clear, illumos can't afford the services of a professional design bureau.

New implementation of printf

So I finally got tired of waiting for someone else to do a printf(1) replacement in illumos for the closed binary from Oracle. I had thought this would be a trivial thing to do via ksh93/libcmd using a symbolic link ala /usr/bin/alias.

Lo and behold, it wasn't! Why? Because ksh93 printf insists (like all ksh93 builtins) on having -- and - getopt style processing. This is fundamentally incompatible with legacy printf. (Why does it do this? So it can dump its builtin man page, e.g. printf --man, to the console. A feature I've railed against in the past.)

Here's what should happen:

% printf -v
-v% 

Here's what ksh93 does:

garrett@thinkpad:~$ printf -v                                                   
ksh93: printf: -v: unknown option
Usage: printf [ options ] format [string ...]

Now there is an argument to be made that a script which relies on the legacy behavior is fundamentally broken. But it doesn't matter -- the scripts are in the field (there are real examples of them), and the legacy behavior must be preserved. Breaking these legacy scripts just so that we can dump printf --version output is... silly. This is case where pragmatism wins over purity.

Rather than try to rip this out and fight with the ksh93 about "deviation from the upstream" (apparently the ksh93 folks view any changes we make in illumos or OpenSolaris as automatically toxic unless they originate from David Korn or Glenn Fowler), I've just gone ahead and implemented my own printf(1) on top of FreeBSD's. This will be the implementation in illumos.

I've added significantly to FreeBSD's code though. Specifically, I added handling of %n$ processing to get parameterized position handling. This is needed for internationalization -- it allows you to change the order of output as part of the output from something like gettext(1). (This is needed when you have to change word order to accommodate different natural language grammars.)

So my implementation is superior to FreeBSD's, and its superior to the legacy closed binary version. Why? Because rather than a half-hearted attempt at processing positional parameters, my version really handles these, including full support for the usual format specifiers. For example:

New open code:

garrett@thinkpad{4}> printf '%2$1d %1$s\n' one 2 three 4
2 one
4 three

Old closed code:

garrett@master{22}> printf '%2$1d %1$s\n' one 2 three 4
134511600 one

Clearly the old behavior is just plain wrong. For the record, ksh93 does the right thing here too. (Although somewhat older versions of ksh93 would dump core on this command line.)

My diffs (which also include style and lint fixes required for illumos) relative to FreeBSD are online. You can also review a webrev of the changes that I hope to integrate into illumos. The license remains BSD, so the various BSD operating systems (or even Oracle) are free to incorporate these improvements if they like.

illumos gets global

I just pushed a major set of changes:

8 libc locale work needs updated license files
223 libc needs multibyte locale support for collation
225 libc locale binary files should be in native byte order
309 populate initial locales for illumos

As a result, illumos has gained base support for some 157 different locales, spanning 67 languages and 116 different territories. This includes nearly all the major languages of the world -- missing are Serbian, Javanese, Farsi, Malaysian, Burmese, and some languages spoken in central and west Africa. (Some of these will be very easy for someone else to add... let me know if you want one of these and are willing to do the work.)

The support for these locales includes full POSIX compliant collating support, which was completely absent in illumos before this integration.

Also, included, is a new open source implementation of localedef(1). To my knowledge, this new implementation is the only non-GNU version of localedef that is fully open, and this version is more fully functional than the GNU version. (The GNU localedef lacks full support for collation data.)

Other notes: this is only the base support for these locales. This will for example give localized output from "date". There is quite a lot of additional effort required to fully localize an illumos system, including support for input methods, fonts, and message catalogs for all the various applications. However, with this base support, it makes doing that other work much more practical.

This integration adds nearly 2 million lines to illumos, although far and away the vast majority of it is in the form of data from Unicode and the CLDR (common locale data repository). The ability to import data directly from these sources is the new code that I've written, including a major overhaul of the underlying ctype and collation support in libc to properly support multibyte locales.

Its my belief that with this integration, one of the biggest feature gaps between illumos and Solaris is closed.