Thursday, September 20, 2012

Infographic - Programming

I was forwarded the following infographic from  While I'm not normally one to just regurgitate someone else's posting (especially advertising), I think this one is worth looking at -- especially if you're at a point where you're contemplating future career or learning plans.

Programming Infographic

illumos and ZFS day!

I'm pleased to announce that DEY Storage Systems, Inc., along with Joyent and other community members, is sponsoring the illumos and ZFS Day taking place in October 1st and 2nd at the Children's Creativity Museum in San Francisco.  My partners and I will be in attendance, along with a few other of my colleagues, and we'll be speaking on related topics.

Additionally, Joyent is hosting the 2nd illumos hackathon at their facilities on October 3, and we will definitely be participating there.  We had a blast doing this last year (some excellent work came out of it), and we're looking forward to doing it again.

If you can make, I hope you'll register.  The event itself is free, and while there will be online recordings of the material, its more fun to get together in person -- especially with a beer or three.  (Did I mention there will be a Solaris Family Reunion party on Monday night?

In many ways, I view this event as a belated 2nd birthday party for illumos, so I hope you'll join me in celebration.

Sunday, September 9, 2012

The Case for a new Reference Distro for illumos

With the recent announcement and discussion around OpenIndiana, I think its time to bring to the light of day an idea I've been toying with for a while now -- and have discussed with a few others in the community.

illumos needs a reference distribution.

Now just hold on there... before you get all wound up, let's discuss what I mean by a reference distribution.

  • A reference distribution should be a base for additional development, and testing. 
  • It should support automated installation for mass deployment in test lab/QA/build farm scenarios.
  • It must support "self-hosting" -- i.e. illumos-gate itself should be buildable on this distribution -- with as few additional steps as possible.
  • It must be valid as a QA platform -- for example, it needs to support the packaging tools we use in order to validate the packaging metadata (manifests).  (Yes, this means it needs to support - and be packaged with - IPS -- in spite of my loathing of IPS itself.)
  • It should be a reasonable (but not the only reasonable possible) base to use for other distributions looking to build on top of it.
  • Ideally it should be pretty darn minimal.
  • It may have graphics support, e.g. an X server, but only in so far as having that would support validation of technologies in illumos-gate.  (Sorry, no big desktops, Gnome, web browsers, or any of that crap.)
  • Community driven - it contains the things that the entire community agrees upon as our common core.
  • It must require only a modicum of effort around release engineering - none of these multi-week release cycles.
  • It should be released biweekly with a numbered build, and probably also have nightlies.  These should be automated.
  • Multi-platform - i.e. it needs to support both SPARC and x86.

As a common reference, it must use illumos-gate verbatim, i.e. no local changes.

Here's what it is not:

  • A general purpose desktop.
  • A general purpose server.
  • A hypervisor.
  • An appliance.
  • Subject to the whims of any corporate entity.
  • An experimental version (it isn't Fedora!)
Why do we need this?  OpenIndiana has been filling the role - but OpenIndiana is insufficient for several reasons:

  • OpenIndiana is too big and unwieldy -- both to build, and to install.
  • OpenIndiana maintains their own fork of illumos-gate.
  • OpenIndiana needs the freedom to follow their own course - without being too strictly tied to illumos-gate.
  • OpenIndiana lacks any kind of formal QA.  (The recent debacles around 151 are good examples of this.)
  • OpenIndiana is not built for SPARC.
(Some of the above items could be addressed with OI, but I think addressing all of them is nigh impossible without totally changing the role of OI.)

So, here's what I propose we do instead.
  • I propose that we start with OpenIndiana or OmniOS, but using core illumos-gate (no fork), and we remove things that are not relevant to the reference.  E.g. Gnome, etc.
  • We will need lots of assistance with release engineering - I want a formal RE plan in place for this thing, with regular scheduled builds and releases following the "train" model rather than the "station wagon" model.
  • We will need to set up some central build and test servers.  (I hope to get some funding for this through the Foundation).
  • We will need to have formalized QA.  (Imagine a Jenkins instance that ran a build and regression test group every night.
  • The bug database should add fields to reference formal build numbers.  (Introduced-In, Fixed-In, Reported-In, etc.)
  • We do not need "an installer".  So the image can just be a basic ISO or tarball, probably.  Think minimalist.  I don't want any customization in the installer, excepting possibly the identification of the initial disk to install to.

Ultimately, if we're very successful, then this will help the entire illumos community.  This reference may ultimately be useful to become a base for OpenIndiana or OmniOS to build upon.  Put another way, I want to make OI's job easier, but I don't intend that this should replace OI.

So, what do I want from the community?

I want to hear from people who think this is a terrible idea.  A descriptive criticism of concerns is far more useful than just another +1.  (If you want to +1 it, do it using Google or FB to like it -- don't add a +1 comment.)  Post your concerns directly to this thread, so that we can keep them for posterity.

I want to hear from people who want - and are able - to actively contribute to this.  Most of this effort will be volunteer based initially, but I think we'll eventually need to staff some work from the Foundation.  Corporate sponsors may help here as well, either by loaning of resources, or gifting of earmarked funds.

Btw, for folks that want to debate names -- let me kill that now.  I want to name this "IRD" -- illumos Reference Distribution.   Its about as unsexy as possible - and that is the point -- this thing should be viewed as something the community uses internally, and not something that is ever given marketing attention.  (Those spot lights belong to illumos itself, and to the other end-user oriented distributions.)

To be clear, I'm not 100% set that this is all the way to go -- but I'm heavily leaning in that direction.  While I can't force the rest of the community to follow along, I can allocate resources to it, and I want to make sure that we can address any major concerns before I start investing resources here.