Friday, August 13, 2010

The Hand May Be Forced

Well, as you may have read, Oracle has decided that at some point very soon, we're going to lose normal regular access to the source code for OS/Net. (I.e. the Solaris kernel and supporting programs.)

While I would have vastly preferred for Illumos to have a cooperative and collaborative relationship with Oracle, it appears that Oracle doesn't value this. In fact, the exact words were from the management at Oracle were as follows:

Solaris is not something we outsource to others, it is not the assembly of someone else’s technology, and it is not a sustaining-only product.

While I understand the need to own the technology, there are few things that could be stated that show a stronger NIH attitude than this. Its unlikely that there will ever be a way for Oracle and the greater community to have a collaborative relationship.

This is a dark day for OpenSolaris -- its effectively dead now. (Its parent, Solaris, lives on however.)

How unfortunate.

For Oracle that is.

Because from the fertile ashes of the dead springs forth new life bringing hope and light in the form of Illumos.

Illumos has garnered the support of some of the top minds in the industry; already the list of names of Solaris contributors and potential contributors that have already publicly committed to supporting this project is extensive. Many of the names are famous, people like Bryan Cantrill. Oracle's actions and inaction have actually made this possible.

I can also say, the list goes even further -- considerably so. I have had private conversations with quite a few other people who have quietly committed to involvement. Some of the names are very surprising, and I hope that they will soon be in a position to announce their involvement for themselves. These are people that are big name contributors; folks who have made very large numbers of code commits to Solaris -- some of the deepest and most "challenging" parts of Solaris, too.

The upshot of this is that the future for Illumos is surprisingly bright. Rather than a dependency on the good will of one corporate sponsor with dubious intentions, the project will have the diverse backing of some of the most well-known innovators (and their employers) from the OpenSolaris -- nay, Open Source -- community.

So, by their actions here, Oracle may be forcing Illumos to "fork", which was always a prospect, even if not one I cherished. But with the backing of the innovators I know who are with us, I think we have a chance to actually be the premiere foundation for SunOS derived technology. Oracle may be investing more into Solaris, but if the best and brightest have left for greener pastures and are contributing to Illumos, then I think we'll have the "best" investments in the base. Following Oracle's lead when the brightest minds have already left looks less and less desirable by the moment. (And to be fair, there are still many bright folks within the Solaris organization at Oracle. But the balance is changing, and changing in favor of Illumos and the open development community.)

Oracle Solaris will not be the only source for this technology, and now it appears it may not even be the best source for this technology.

I once said I never intended for Illumos to compete with Solaris. That was true, but if Oracle forces the issue, then even despite their vast economic resources, I say, "Bring it!"

29 comments:

Prudhvi Krishna said...

I like your attitude Garret. Yes, Oracle "Bring it ON!".

Dave D said...

Bring it, Oracle! Can't wait to see what the Illumos project and community brings!

Ivan Nudzik said...

So... Oracle made fork from Illumos. Now it is right time to find valuable uniqueness for Illumos to not to be only an echo of Oracle code releases. To be against Oracle business model is not enough for success. ;-)

Brandon Barker said...

As a long time user of OpenSolaris and previously Solaris Express (if not Solaris) - I wish you all the best of luck and hope I can help out somehow, even though I do not have the time for kernel hacking. I really hope some form of Solaris continues to be a great workstation and mobile workstation system. Certainly, thanks to you and others, this is already true today for many systems.

Simon Huerlimann said...

Being faster than Oracle in publishing bug-fix releases for security problems and publicly talking about it might be a sensible start to show SunOS users the value of your project. And it's something community driven forks with good technical leaders have a good track record in.

Be the force with you!

Jason Catena said...

You shouldn't have a problem being faster about bugfixes: Oracle specifically plans to hide them (for most customers, without partner plan$) between major releases, as a supposed competitive advantage.

Emiel said...

What's to keep Oracle from following Illumos and back-porting back into Oracle Solaris? Would the current license prevent this?

dmlawrence said...

Hell yes!

Shmerl said...

This sounds good, that eventually the development of open source Solaris will become really open - out of corporate clutches. But will the fork lead to a binary incompatibility with Solaris [11]?

Gregg said...

Does this mean that the regular OpenSolaris projects are closing down? The regular mail traffic hasn't said so.

Garrett D'Amore said...

So a few things:

A) we will probably have a fairly open policy with respect to security bugs. There is some reasonable balance to be found here between immediate notification (which helps script kiddies, etc.) and timely notification with a fix. We will probably follow the lead of other systems like OpenBSD here.

B) Binary compatibility is a critical goal for Illumos. We should be compatible with Solaris 10, and therefore also Solaris 11. Of course, if the code is closed, and Oracle decides to break binary compatibility between Solaris 10, OpenSolaris, and Solaris 11, then all bets are off. I doubt this will happen.

c) Oracle is free to take code from Illumos -- this is in fact a goal, although I'm less concerned about it now that it appears we will be forking instead of spooning. They will have at least as much opportunity to do this as anyone else. Who knows, maybe some day in the future Illumos will become Oracle's upstream? :-)

kebabbert said...

Cool man! I wish you luck. I will try Illumos based dists. You sound convincing when you say you have some good developers with you.

Dominik said...

2 questions:
1/ When Nexenta - especially NexentaStor (CE) will adopt the Illumos kernel?

2/ How about driver development / fixing existing bugs in drivers (whether it will be responsible for this the community of Illumos)?

Garrett D'Amore said...

We have plans to build a future version of Nexenta's product on illumos. Hopefully you understand that this may take a while -- illumos itself is barely off the runway.

Driver development and bug fixing will be done by those who have the energy and desire to do them. In some cases this will be funded via commercial development. (For example, I see Nexenta investing in sustaining for HBA and storage drivers.) In other cases, it will take an unpaid volunteer to do the work (for example, sustaining SDcard drivers).

We will also endeavor to integrate fixes from Oracle when we can access them. The details of that are not worked out yet.

jonathan said...

I hope this project succeeds! Even though I'm a big Linux fan, I can see how having a great free Solaris implementation can be useful for many, many huge companies currently using Solaris and related technologies!

fche said...

"c) Oracle is free to take code from Illumos -- this is in fact a goal, although I'm less concerned about it now that it appears we will be forking instead of spooning. They will have at least as much opportunity to do this as anyone else."

To what extent are you folks contemplating changing the copyright-assignment rules (if any? -- where on illumos.org would one find them?) to prevent Oracle from having a privileged position in ownership of community contributions?

Garrett D'Amore said...

I am contemplating the contributor agreement rules, for sure. For now, the rules remain the same, because even in spite of the memo, the hg gates remain open and no decisive action has been taken by Oracle.

When/if Oracle does take decisive action, then it will probably be time to revisit that decision.

Senne said...

If you are going to be forced to fork, is there any reason you could not combine the fantastic ideas of Plan 9 with the concepts and implementation of OpenSolaris?

It just seems to me, if we exclude familiarity, the most talked about features of OpenSolaris are DTrace and ZFS. Plan 9 takes the good features of unix and extends them to an even greater revolutionary extent.

Both projects appear to suffer from just not quite enough developer and user critical mass. Drawing Plan 9 into your more mainstream development would be beyond cool and innovate Oracle out of existence , hopefully a serious wake-up call to Oracle and re-invigorate both projects' development process. It would hark back to the old days of unix hacking.

(As I say this, I feel like I am understating OpenSolaris / Nexenta development. That was not the intention.

I was astounded at the development of OpenSolaris, and was planning to move to Nexenta anyway as I would like to continue learning server administration etc.)

Rather than simply rehash a quite good product, would it not be more advantages to attempt to distinguish this project to extend two great projects to one fantastic one?

I know I am suggesting a lot of work when I am clearly not capable of even developing a text editor. I do think I am able to see the value of aiming even further when you need to 'get a message as far as possible'.
(ugh, that is a terrible extension of a metaphor!)

Zooko said...

Dear Garret:

I'm very happy to see this. I've been calling for it on identi.ca/twitter since even before the recent leaked Oracle memo and the Oracle->Google lawsuit. :-)

However, due to those two recent events there is now an elephant in the room and you need to tackle it head-on in order to reassure your users that it isn't going to crush you while your back is turned.

That is: how can contributors to or users of IllumOS know that Oracle isn't going to sue them for patent infringement for using or contributing to IllumOS?

I think you might have a good answer, but the details look a tad subtle:

http://hub.opensolaris.org/bin/view/Main/licensing_faq#HWhatdoestheCDDLsayaboutpatents

http://lwn.net/Articles/114839/

http://www.groklaw.net/articlebasic.php?story=20050121014650517

Garrett D'Amore said...

First, IANAL.

However, the lwn article I think mis-analyzes the license. The CDDL clearly separates Original Developer Grant from Contributor Grants, and the Original Software refers to the bits that came from the Initial Developer. The patent licenses for the Original Software apply regardless whether it is in Solaris or not.

Additionally, section 2.2 makes this clear, by including derived products in the same coverage as "Contributor Grants" with the same patent clauses.

The situation groklaw uncovers is a possible concern about patents held by a third party (e.g. Microsoft); there is no license granted for that (there can't be, since they are held by an entity not a party to the license!) Groklaw complains that Sun might know about such patents and wasn't forced to disclose them, even if it held a license for them.

The somewhat unnerving prospect here is that we might be in a similar situation with ZFS. If NTAP and ORCL were to reach a licensing arrangement where ORCL acquires a license to use NTAPs patents, it could leave the rest of us vulnerable to NTAP.

However, this situation is not unique to CDDL. The GPL, for example, offers no such protections, and even fails to protect against patents held by one of the parties to the license! So Linux is actually *worse* off in this regard.

So, the situation is what it always has been.

If you know or suspect your use may be infringing, you have to do your own research and figure out what licenses are required. This was true of *any* users of any software (including Solaris!)

At the end of the day, the situation is that software patents are almost universally an evil. Unfortunately, even the "good actors" who don't like software patents are forced to get them to use as defensive weapons.

Garrett D'Amore said...

On the topic of Plan 9...

From what I've seen and understand of Plan 9, I think its far far too much of an academic exercise, without enough really useful functionality, and with a very different development mindset from traditional UNIX.

I think it would be very hard to have much of a "meeting of the minds".

Additionally, we're unwilling to throw away compatibility with Solaris, even if it would give us other advancements.

That said, if there are some concrete proposals for functionality that we could add from Plan 9, without negatively impacting compatibility, performance, or other goals, then I'd like to hear about it.

fve said...

A not unsurprising move from Oracle - and if that tips IllumOs into flight, that's a good thing. I'll try to contribute.

Gabriele Bulfon said...

The IllumOS project is amazing, thanks Garret ;)
Will we have an easy path from Solaris/OpenSolaris to Illumos?
Do you have any idea when first isos will be available?
Will we have the ability to make custom installation isos of IllumOS containint custom addon softwares?
Can't wait to start building next releases of my softwares on it!

Itchy said...

My thoughs (OpenSolaris at the end):

http://koroshiyaitchy.wordpress.com/2010/08/22/open-source-and-a-kingdom-of-free-men/

openbabel said...

Make sure any of the forthcomming forks get 100% opensource documented code.Oracle will open a legal claim for infringement if you don't run a tight ship.

Again you need experts in Business, marketing,ISV programe,and a product strategy.IOS is not likely to succeed as a technical project,taking on business skills,marketing at an early stage is key.Clearly both sponsors and individuals will be put off by lack of structure.

Lets hope something will come along shortly.

jbarker said...

It's going to be hard to put food on the table and work on Illumos. Would it be a prudent endeavor to join an already well established open source endeavor that is battle tested in court. FreeBSD has incorporated dtrace and zfs into it and may be a useful place for talented contributors that still need to put food on the table. I would imagine Oracle will change enough of Solaris over time that it will be incompatible with Illumos. In any case, whatever the direction, I am rooting for you all the way. Oracle has really thrown a monkey wrench into what I have been trying to create at work. In the past I have been able to build a storage solution for 13k and now Oracle wants me to pay them 87k for something very similar...frustrating to say the least.

Garrett D'Amore said...

I would say FreeBSD is not the same as the legacy OpenSolaris code base, and that while some may prefer to collaborate there, I'm unwilling to abandon this particular code base. I think there are too many technological advantages here. I believe that others that have bet their business on Solaris technologies feel much the same way.

illumos can and will provide a competitive alternative here. Just watch!

Rob said...

>> Oracle may be forcing Illumos to "fork", which was always a prospect, even if not one I cherished.

...

>> Emiel said...
>> What's to keep Oracle from following Illumos and back-porting back into Oracle Solaris? Would the current license prevent this?


We should follow "Solaris 11" once the source gets released and 'un-fork' ourselves at regular intervals.

An alternate is to CLEARLY explain why we want to run "OI" instead of Nexenta, Schillix, or even "Solaris 11 Community Edition".


If we dilute the choice of Operating System with yet another 'OS-Fork' who will stand with us?

When OpenSolaris was started few came to the project, with a fork that goes off on it's own path we will have even less of a following than ever.

Lets make this ("OI or Illumos") the FREE and OPEN choice for "Solaris Users" not a new choice for someone who decides which Operating System to adopt.

Nexenta is WONDERFUL and seeks to be all things for all people (has Solaris Kernel and GNU Tools) yet very few adopt it; let _US_ NOT be an "Orphan Project".

Thanks,
Rob_C

Garrett D'Amore said...

Rob:

A few key salient points.

a) Its not clear that there will ever *be* any meaningful amount of Oracle source that we would be able to use to "unfork".

b) Oracle's following on a track that we don't want to follow. For example, they're ditching 32-bit support. Are we ready to make the same move? Sure, they are, because they only care about hardware sold by ... Oracle. But we have a broader community to play with.

c) Significant innovation can now happen outside of Oracle, especially as so many talented former Oracle engineers are now outside of Oracle and working at places like Joyent, Delphix, and .. oh yeah, Nexenta.

Ultimately this means I see the day where we become the leaders for innovation on this code base, rather than followers. We still have work to do here, but already several interesting changes have come into illumos which Oracle doesn't have -- including actual bug fixes.

Slavishly following the lead of a company that has such a narrow focus for Solaris 11, and that may not have the technical superiority it once had, doesn't seem like the best plan to me.

If you want a pure Solaris 11, then you might as well pay $$ to Oracle, and use Solaris 11. Illumos can do better than that.