/dev/dump

Yesterday we had our first ever illumos hack-a-thon. About a dozen people from the community showed up, and worked on some very very cool things. At the end of the day, about a half-dozen different demos were done, to show what was worked on. We'll be posting photos shortly. But here are some of the projects that got attention: tab completion for dcmds and data types in mdb ::print for DTrace expanded truss support for ZFS ioctls (nvlist expansion) time-ordered output for DTrace a comment field in the vdev label for ZFS pool devices the ability to change the GUID of a ZFS pool Several other projects were in progress. We selected these projects out of a much larger list of project proposals (which I'll post soon), based on the what people thought was most useful, and the ability to achieve results in a single day hack-a-thon. (And what people we willing to either work on, or mentor.) Most importantly, people got to work on areas that they weren't intimately familiar wi...

After a lon g, and arduous, release cycle, I am pleased to report that NexentaStor 3.1 is available now . Customers running existing 3.0 installations may upgrade at no cost. This release includes a number of key features, including some significant improvements for performance and manageability. Folks using SCSI target mode will probably see the biggest performance boost relative to earlier editions of NexentaStor, especially those folks using NexentaStor to serve up storage to VMware guests -- thanks to the VAAI offload support that is part of this release. And for the record, yes, this release includes the fix the long standing problem with iSCSI timeouts. For ZFS fans, this release also includes the updates for ZFS version 28. (This does mean that folks upgrading need to be cautious -- their pools will not be automatically updated to ZFS version 28, but if they are manually updated then there will be no way to move those pools back to an older release. That also means that poo...

As part of fixing some recent bugs, I integrated the following into illumos : 734 taskq_dispatch_prealloc() desired 943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP The interesting one is the first of these. The interface is actually called taskq_dispatch_ent (), and is private to the "consolidation" (i.e. for use within bundled code only). What this interface provides for, however, is a way to bomb-proof your taskq dispatches, if you can arrange for the dispatching state structure ( taskq_ent_t ) to be allocated in advance. This means you never have to worry about the possibility of a dispatch failing due to insufficient resources. What's even cooler, is that the cost of the dispatching is much cheaper; taskq_dispatch () was the hottest piece of code on a very busy storage server. Now it goes much much faster, because it is just twiddling some linked list pointers and sending a signal to wake up the thread processing the taskq. More impor...

Constantin Gonzalez recently interviewed me for his "HELDENFfunk" podcast series, while I was in Amsterdam for our European User's Conference. Also interviewed were OpenIndiana founders Alasdair Lumsden and Andrzej Szeszo.

All: There will be a panel discussion about illumos as part of the San Fransisco and Silicon Valley OpenSolaris User's Group meeting tomorrow, June 15. I will be joining Bryan Cantrill and Adam Leventhal to answer your questions about illumos. The doors open at 6:45pm, and we will continue until about 8pm. The location is 275 Middlefield #50, Menlo Park, California. Hope to see you there!

NexentaStor 3.0.5 is now available. Apart from fixing some key bugs, the main thing that this release includes is a significant update to the CIFS stack, which addresses both performance concerns, and AD failover concerns. Note that NS 3.1 is due out *any day now*, and will include all these changes, plus a boat load of others. I'll have a lot more to say about the 3.1 release soon.

You may be aware that we have selected two candidates for the slots allocated by Google to illumos -- the first is to replace some system utilities from code in perl to native C. The second of which is to bring GRUB2 to illumos. What you may not know, is that Nexenta will be sponsoring three additional candidates to pursue projects of their own to benefit illumos. These candidates have been selected already, and we will have more to say about them and their work in the future. Stay tuned!

Joyent have continued to demonstrate their commitment to and support of illumos . In addition to a string of recent source code integrations, they are now hosting some of our infrastructure in their cloud, with more to follow. After moving the stuff there, we're now enjoying significantly better performance, and enhanced functionality. Try out the new OpenGrok instance yourself to see! I'd also like to give a special thank you to Circonus , who are providing active monitoring services for our site now, as a gratuity to illumos. Apparently, they're going to be hosting their stuff on illumos based systems as well, so there's additional synergy here.

So there has been some things brewing in a sub-sect of the illumos community about a project to fork illumos, because of alleged problems with my leadership. You can read the thread here if you want. I want to address this head on. First the claim is that I've got omnipotent control over illumos. This is absolutely false. While I created the project, and serve as technical lead, I've offered to step down if the developer-council and admin-council would like to me to do so. Notably my employer (Nexenta) has minority representation on both councils, and I've tried to keep the groups as neutral as possible. I said when I created the illumos project, and I still maintain, illumos is a community project, not a Nexenta one. I'm working on the process to make this more formal through non-profit governance. I should have more to say here before the end of week. (I've got a meeting about this today.) I've also handed over determination of the Advocate list (th...

Joyent posted an update -- they've released a branch of illumos , on github, containing much of their illumos contributions. Some of the stuff is probably fairly Joyent specific, but some of it is highly useful to almost everyone using illumos! From their mail: ZFS I/O fair-share scheduling for zones the Joyent brand, which can be used as a template for other non-SysVR4 or IPS zone brands Reintroduction of sparse zone images Crossbow vnics on demand for zones & non-unique vnic naming (unique per zone, not per system) svcs enhancements ( svcs -Z/-z for interrogating zone services, -L for outputting log files directly (no more ls /var/svc/log | grep... )) vfsstat and iostat tweaks and ziostat, iostat(1M) for ZFS I/O more per-zone IO kstats the zonemon utility for zone kernel state troubleshooting DTrace enhancements such as llquantize I just want to say again, thank you very much Joyent! Now, how quickly can we merge this stuff into illumos mainline?

I'm looking for people interested in contributing content to the illumos website. Right now we have a test website but it needs help with producing content. First and foremost we need English content, but the new framework will support other localizations as well. If you're interested in contributing here, drop me an email. I'll be setting up a mailing list for this soon.

A big thank you goes out to Areca . Areca have provided the source code for their Solaris driver, including support for the newer 6G RAID adapters. As a result, I've integrated a (somewhat cleaned up) copy this code as an update to arcmsr(7d) in illumos , under generous open source licensing: changeset: 13305:fb26af81b9b2 tag: tip user: Garrett D'Amore date: Fri Mar 25 22:14:56 2011 -0700 description: 834 need support for Areca 1880 6Gbps Reviewed by: Dan McDonald <danmcd@nexenta.com> Reviewed by: Albert Lee <trisk@nexenta.com> Reviewed by: Richard Lowe <richlowe@richlowe.net> This will make another HBA option available to folks. (Note these cards do support a JBOD mode, so you don't have to use hardware RAID -- indeed I would recommend that you don't when you have ZFS on the disks.

So, at the recommendations of others, I'm on twitter now.. Don't know how often I'll keep it updated, but I'll try.

I just integrated: changeset: 13312:537259ad27f6 tag: tip user: Garrett D'Amore date: Wed Mar 23 08:35:14 2011 -0700 description: 324 need serbian locale support Reviewed by: Rich Lowe Approved by: Garrett D'Amore This is a bit unusual relative to most of the locales, because Serbo-Croatian is a language fraught with some unique political considerations: There is a common root language, that everyone speaks and understands. But speakers of it rarely agree on what to call it. In Serbia its Serbian. In Bosnia its Bosnian. And so on for Croatian and Montenegrin. In illumos , we have followed the Unicode CLDR example, and we now have these locales: hr_HR.UTF-8 - Croatian in Croatia sr_BA.UTF-8 - Serbian in Bosnia and Herzegovina sr_ME.UTF-8 - Serbian in Montenegro sr_RS.UTF-8 - Serbian in Serbia I want to apologize to anyone offended by this decision, but rather than make a contentious decision on our own, I decided it was best to simply follo...

It would appear that the old Planet OpenSolaris is no longer a community site. At least the only blog posts that seem to be there anymore are those that are hosted on blogs.sun.com. Certainly my posts, which used to show up there until quite recently, no longer do so. Its possible that this is just a technical snafu, but the recent burst of posts there from Oracle employees suggest a shuffling of things internally in how Oracle handles blogs, and I suspect that eradication of community posts is just one more step along the way. Of course, if I'm wrong, this post will show up there, and I'll have egg all over my face. :-)

Great news! We ( illumos ) have been accepted as a Google Summer of Code mentoring organization. If you are a student and want to get paid this summer to work on an enterprise grade operating system, please have a gander at our ideas page , and then go ahead and start an application. You can find our application template on our organization information page on the GSoC 2011 site. Good luck to all the applicants!

With this integration: changeset: 13304:b54231762cfa tag: tip user: Richard Lowe date: Mon Mar 14 14:05:30 2011 -0400 description: 243 system manual pages should live with the software Reviewed by: garrett@nexenta.com Reviewed by: gwr@nexenta.com Reviewed by: trisk@opensolaris.org Approved by: gwr@nexenta.com We now have manual pages in illumos . (Only the English pages -- POSIX locale -- are kept in the illumos code repository.) This is key because it means that code and documentation can be maintained together, which is how some other projects (but not Solaris) manage it. So, got a problem with the man(1) pages on illumos? File a bug! There is a category called "manpage"... please let us know, or even better, contribute a fix!

Got a pet project for illumos you would like someone to take up, and would do yourself if you had time? Like working with bright up and coming stars? Are you a student looking to get involved with a nascent community of world class engineers, and have some free time on your hands this summer? Maybe you can participate in Google's Summer of Code . We hope illumos will be selected to participate this. Some ideas are posted on our wiki already, but I'd love to hear other proposals. We have a very short window of time before we have to submit our mentoring org application, so let us know!

I just pushed this for Dan McDonald into illumos : changeset: 13297:4b9dc4ca8e9f tag: tip user: Dan McDonald <danmcd@nexenta.com> date: Fri Mar 04 13:57:09 2011 -0800 description: 701 UNMAP support for COMSTAR Reviewed by: Garrett D'Amore <garrett@nexenta.com> Reviewed by: Eric Schrock <eric.schrock@delphix.com> Reviewed by: George Wilson <gwilson@zfsmail.com> Approved by: Garrett D'Amore <garrett@nexenta.com> This change represents a significant new feature in COMSTAR and ZFS, which will greatly benefit people use SCSI target mode functionality in situations involving over-provisioning. More on that in a minute... The feature itself was developed by Sumit Gupta for Nexenta , and is part of our upcoming 3.1 release of NexentaStor . Subsequently, Dan took ownership of that code, and working with Eric and George (who are well established ZFS gurus and had significant and useful feedback) improved it still further, and got the...

As promised, I'd send illumos photos from SCALE. Here's the illumos booth staff, from left to right there is Roland, Garrett (your humble author), Delya, Albert, and Rocky. Rocky was there representing Area Data Systems , who are both Nexenta partners and illumos sponsors. We also have a facebook photo gallery up, which seems somehow apropos since we were right next to the facebook both at SCALE. I'm also pleased to report that a number of other Nexenta partners were present as well, showing off Nexenta based products. Next year, we hope they'll be back show casing technology based on illumos and NexentaStor 4.0.

Its now fairly locked into stone that we will be opening an engineering office somewhere not far from Boston (probably just north of it) sometime during 1H2011. As a result, we've started an aggressive recruiting campaign in the area. I am interested in talking to people with backgrounds in kernel software or device drivers -- especially if the background is on Solaris, but Linux and BSD backgrounds are fine too. The culture here is startup, and while I'd love to find a few more architect-level candidates, I'm also keen to find folks just beginning their career with a high level of enthusiasm who are talented and driven to become the industry's next generation of storage and networking gurus. Nexenta itself is still a small company, and so its still a great opportunity to get into the ground floor of what may well prove to be the fastest growing storage company ever. And even better, you can feel good knowing that your contributions will contribute to the greater goo...

So, I've spent the past day or so here at the Hilton LAX with Albert, Roland, and Delya manning the illumos booth at SCALE . We're at booth #72. We've been handing out disks with OpenIndiana , Nexenta Core Platform (4.0 alpha release based on illumos), and NexentaStor 3.1 (alpha release) as well as the first ever illumos T-shirts for those folks who install one of these on their system (either physical or virtual.) I'll have photos to post soon -- sorry they're not ready yet. The show has been amazing, and we've made a lot of new connections, both with other members of the open source community, and with new potential users and contributors as well as other potential collaborators. If you're around tomorrow, please drop by and say hello! (Oh yeah, and come watch Richard Elling and I mix it up with the BTRFS guys at the open source filesystems panel tomorrow!) And if you're looking for an open source job, drop by and chat. We are hiring in a number...

I've just pushed changeset: 13278:dabee83e3bb7 tag: tip user: Garrett D'Amore date: Mon Jan 31 17:40:15 2011 -0800 description: 519 RFE audiocmihd Reviewed by: gwr@nexenta.com Reviewed by: dev@opensound.com Reviewed by: trisk@nexenta.com Reviewed by: ams@nexenta.com Approved by: trisk@nexenta.com This represents the first significant contribution to illumos by a third party other than Nexenta, and is also the first hardware driver illumos has support for which Solaris does not. This is for ASUS Xonar cards. You can tell you have a card that could benefit from this if prtconf -vp | grep pci13f6,8788 shows a result. I expect this will be introduced soon into a forthcoming OpenIndiana build. Enjoy, and a big thanks again to 4Front Technologies for making this contribution!

I'd like to take a minute to publicly welcome the following engineers, who are well known in the OpenSolaris community, to my team within Nexenta . Dan McDonald - Dan just started today, and joins us from Oracle (and previously Sun), where he was one of the lead engineers on IPsec and networking security in general. He's also famous as the creator of the internal "punchin" tool used by Sun engineers for many years. At Nexenta he'll be doing some different things, but also will be our go-to man for issues involving the TCP/IP stack within NexentaStor and we expect to see contributions coming from him back into illumos . Andrew Stormont - Andy started on Jan 1, and is our first software engineer in the UK. He's known for creating the StormOS desktop distribution based on XFCE on top of Nexenta Core Platform . We're looking forward to capitalizing on the synergy of StormOS and Nexenta Core Platform forward. Roland Mainz - Roland is known throughou...

First off, let me state that the following changes are aimed at both easing the challenging of contributing changes to illumos, while increasing our level of "confidence" in what changes are being integrated into our source code tree. Up until now, illumos has used a contribution model that is primarily derived from the model used within Sun and Oracle for Solaris development. This development model is based on the notion that all contributors have (or had at least) the direct ability to "push" code to the repository, after a certain number of review steps had been followed. This model works well with a small team, or where all contributors are reasonably well trusted. This is also not typical at all of the way most FOSS projects work. (Indeed, with OpenSolaris, this model was not used for external contribution.) Going forward, we want to enable a much wider group of developers, some of whom may not hang around long enough in our community to get a high level of ...

I have an illumos partner that is interested in finding a strong system administrator/system integrator candidate for work on an illumos-based product in New York. Candidates need to be strong with Solaris, security, scripting (perl, python and/or ruby, etc.) and should have cross platform experience. If this sounds interesting to you, please contact me directly.

I'm looking to hire a few really smart folks who can work in southwest Riverside County, CA (near Temecula, CA). I want people who are sharp C programmers, want to work on open source kernel software, and are eager to stretch themselves. This is a great learning opportunity. If you know anyone like that, let me know!

I'll be giving an update on illumos , as part of my talk in LA this evening . If you're around, please join us!