illumos hackathon a resounding success

Yesterday we had our first ever illumos hack-a-thon. About a dozen people from the community showed up, and worked on some very very cool things. At the end of the day, about a half-dozen different demos were done, to show what was worked on.

We'll be posting photos shortly. But here are some of the projects that got attention:

• tab completion for dcmds and data types in mdb
• ::print for DTrace
• expanded truss support for ZFS ioctls (nvlist expansion)
• time-ordered output for DTrace
• a comment field in the vdev label for ZFS pool devices
• the ability to change the GUID of a ZFS pool

Several other projects were in progress.

We selected these projects out of a much larger list of project proposals (which I'll post soon), based on the what people thought was most useful, and the ability to achieve results in a single day hack-a-thon. (And what people we willing to either work on, or mentor.)

Most importantly, people got to work on areas that they weren't intimately familiar with, and work with mentors who were much more familiar. For my own part, I had a blast working with George Wilson to implement the ability to alter the GUID of a pool (which is going to have some further uses we can talk about later). My webrev of these changes is now on-line, so I welcome review feedback.

We had domain experts like Adam Leventhal, Matt Ahrens, Eric Schrock, George Wilson, Gordon Ross, and Dan McDonald on hand, to help mentor projects if they needed it, and the amount of cross fertilization was amazing.

While none of the actual code changes are absolutely earth-shattering, they are still very very cool, useful things, that many of us will be happy to have available. I'm already imagining several useful use cases for the ability to reguid a pool, for example. And I can't wait until I have ::print in DTrace and tab completion in mdb. These will make my life significantly better.

Ultimately, this was the most enjoyable coding experience I've had in a long time. I can't wait until we do it again!

NexentaStor 3.1 available now

After a long, and arduous, release cycle, I am pleased to report that NexentaStor 3.1 is available now.

Customers running existing 3.0 installations may upgrade at no cost.

This release includes a number of key features, including some significant improvements for performance and manageability.

Folks using SCSI target mode will probably see the biggest performance boost relative to earlier editions of NexentaStor, especially those folks using NexentaStor to serve up storage to VMware guests -- thanks to the VAAI offload support that is part of this release. And for the record, yes, this release includes the fix the long standing problem with iSCSI timeouts.

For ZFS fans, this release also includes the updates for ZFS version 28. (This does mean that folks upgrading need to be cautious -- their pools will not be automatically updated to ZFS version 28, but if they are manually updated then there will be no way to move those pools back to an older release. That also means that pools should not be updated in HA cluster configurations unless both cluster partners are updated to NexentaStor 3.1 first.) This also means faster snapshot creation and deletion, and the ability to import pools in read only mode or that have encountered a failure of their SLOG device, making some disaster recovery scenarios much less challenging.

Nexenta Core Platform (our general purpose Debian based open core) will be updated shortly to 3.1 as well -- probably sometime in the next week or so. This will be the same core software as supplied with NexentaStor. (We do strongly encourage folks using Nexenta Core Platform for serving up storage to use our commercial NexentaStor product. There is even a free edition for users with up to 18TB of data.)

We are already (and have been for a while actually) hard at work on the next release, which will be based upon illumos, and include a number of other innovative features. Stay tuned for an update.

Bombproof taskqs

As part of fixing some recent bugs, I integrated the following into illumos:

734 taskq_dispatch_prealloc() desired

943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP

The interesting one is the first of these. The interface is actually called taskq_dispatch_ent(), and is private to the "consolidation" (i.e. for use within bundled code only).

What this interface provides for, however, is a way to bomb-proof your taskq dispatches, if you can arrange for the dispatching state structure (taskq_ent_t) to be allocated in advance. This means you never have to worry about the possibility of a dispatch failing due to insufficient resources.

What's even cooler, is that the cost of the dispatching is much cheaper; taskq_dispatch() was the hottest piece of code on a very busy storage server. Now it goes much much faster, because it is just twiddling some linked list pointers and sending a signal to wake up the thread processing the taskq.

More importantly, the fact that we don't ever have to sleep, or have an expensive call into the kernel memory subsystem, has solved some frustrating "stalls" in the storage subsystem.

I encourage developers working with taskqs in illumos to have a look at this interface. If you can use it, you will simplify your code, and shorten your run-paths. Both of which are good things. The only limitation: this interface is not available for dynamic taskqs. (Which makes sense since dynamic taskqs might need to allocate whole threads.)

illumos podcast

Constantin Gonzalez recently interviewed me for his "HELDENFfunk" podcast series, while I was in Amsterdam for our European User's Conference. Also interviewed were OpenIndiana founders Alasdair Lumsden and Andrzej Szeszo.

illumos Panel Discussion in SF Bay Area

All:

There will be a panel discussion about illumos as part of the San Fransisco and Silicon Valley OpenSolaris User's Group meeting tomorrow, June 15. I will be joining Bryan Cantrill and Adam Leventhal to answer your questions about illumos.

The doors open at 6:45pm, and we will continue until about 8pm.

The location is 275 Middlefield #50, Menlo Park, California. Hope to see you there!

NexentaStor 3.0.5 available now

NexentaStor 3.0.5 is now available.

Apart from fixing some key bugs, the main thing that this release includes is a significant update to the CIFS stack, which addresses both performance concerns, and AD failover concerns.

Note that NS 3.1 is due out *any day now*, and will include all these changes, plus a boat load of others. I'll have a lot more to say about the 3.1 release soon.

GSoC Candidates Selected

You may be aware that we have selected two candidates for the slots allocated by Google to illumos -- the first is to replace some system utilities from code in perl to native C. The second of which is to bring GRUB2 to illumos.

What you may not know, is that Nexenta will be sponsoring three additional candidates to pursue projects of their own to benefit illumos. These candidates have been selected already, and we will have more to say about them and their work in the future. Stay tuned!