Sunday, June 3, 2012

Why I *hate* external dependencies!

So, I've been trying to setup a system that can build illumos-gate.  Because my old system had a memory DIMM fault, and isn't usable anymore.

I thought, hmm... let me try something other than OpenIndiana.

Big mistake.

Only OpenIndiana seems able to build vanilla illumos-gate right now.

Why?  Because of external dependencies!  I tried building on OmniOS.  The stopper there -- Python 2.4!  On SmartOS, the dependency is IPS itself.  It seems only OpenIndiana is suitable for building stock illumos-gate.

The problem is that our build is inherently very sensitive to the build environment.  It makes life incredibly unpleasant if you try to build on any system that is not configured exactly as we specified.

This, IMO, is an unacceptable situation.

I will speak loudly against any attempt to make changes that introduce further external dependencies.  The fact that some already exist is no excuse.  Every external dependency makes working on the tree more painful.  If you're thinking of a change that would move something out of tree, but add a build-server dependency, think again!  

These dependencies are a serious barrier for contribution.

We (illumos) should take a page from NetBSD, who have successfully arranged that their environment is self contained to the point that they don't care about their build server at all -- all it needs to be able to do is bootstrap gcc and run automake/autoconf.  There is something very elegant about this.

Expect to see me on yet another crusade to kill dependencies on external runtimes, be it Python, Perl, or whatever.   Our tree should, IMO, be as "stand-alone" as possible.

If that means you have to write C, so be it.  Grow up and get a real compiler dude.  I know languages like Python and Perl are attractive because they have a lower barrier to learn -- but depending on them -- and far worse -- depending on *specific* versions of them -- is toxic for anyone else who wants to work on the code in a different environment.   (This has nothing to do with your language of your choice, but everything to do with the pain and suffering your language choice creates for anyone who just wants to do "make install".)


zdzichuBG said...

Python 2.4? This version is abandoned for something like million years, now. Even 2.7 is on life support right now.
Dependency on 2.4 is really crazy.

Brian Utterback said...

Here, Here, Garrett! I cannot tell you how often I have been bitten by this.

zdzichuBG: You're missing the point. It wouldn't matter of it was the latest version of Python. The problem is the dependency, not the version.

James Mansion said...

I would have thought that reverting to 'C or sh' is just going to be retrograde. Its worth having something that can do intermidate level stuff. If having a core of Python or Perl in the (just to support the build, right?) is painful, then at least consider lua, and build it early and keep it away from actual deployment - just use it in build.

Garrett D'Amore said...

For building, you can do "intermediate" level stuff in shell, awk, even *version agnostic* perl. I don't want runtime dependencies, nor do I want version sensitive stuff. Compilation should never require much more than modest text processing, apart from the compilers themselves.

Nick said...

Garrett, I couldn't agree more.

The fact that only _one_ distro is capable of building Illumos is very unfortunate.

The fact that the package manager is part of the build process at all, makes the Illumos kernel have enmity toward other distros that may use other or custom package managers.

However, while a rewrite of perl and python utilities in C and ksh is ideal, why not temporarily include python and perl in the Illumos source tree, so that users only have to fetch the illumos-gate repo?

As for IPS, that is the larger problem, IMHO.

Aside from fetching build dependencies for Illumos, what else is IPS needed for? If it is needed for installing Illumos to the system (looks like it), why not replace that with a shell script or make-file option?

Anyway, this comment has gotten too long already. What can enthusiasts like me do to help with this situation? I don't like it, and want to make it go away (I'm bitten by external dependencies all the time).

Nick said...

The dependency on runtimes like python or perl is unfortunate.

Clearly, it would be ideal to rewrite all of these utils in C. But in the meantime, why not add python, perl, etc to the Illumos source tree, so that users don't have to chase dependencies? I am sure there are great reasons not to, I would just like to know what they are :)

Also, IPS is the biggest blemish as far building Illumos goes. Is it needed for anything except fetching build dependencies? Or is it itself a build dependency.

If it _is_ a build dependency, what do we need it for, and how can we ween ourselves off of it?

I am not a fan of IPS, but a dependency on _any_ package manager for building the kernel, makes the kernel implicitly hostile to other package managers (and distros that use those other package managers).

What can enthusiasts like me do to rid Illumos of this blight?