Saturday, October 27, 2012

hackathon project: pfiles - postmortem analysis

During the first days of October 2012, we had an illumos day, and a one day illumos hackathon that was very well attended by some of the best and brightest in our community (and I daresay, in any open source community.)

The purpose of the hackathon, besides just doing some cool projects, is to give folks a chance to interact with other domain experts, and code, particularly in areas outside their particular area of expertise.

This year a number of interesting projects were worked on, and I think some of these are at the point of starting to bear fruit.  The project I undertook was the addition of post-mortem analysis support for pfiles(1).  That is, making it possible to use pfiles against a core(4) file.  Adam Leventhal suggested the project, and provided the initial suggestion on the method to use for doing the work.

This is a particularly interesting project, because the information that pfiles reports is located in kernel state (in the uarea) and has not traditionally been available in a core file.

To complete this project, I had to work in a few areas that were unfamiliar to me.  Especially I had never looked at the innards of ELF files, nor at the process by which a core file is generated.  To add the necessary information, I had to create a new type of ELF note, and then add code to generate it to libproc (used by gcore) and elfexec (for kernel driven core dumps).  I also had to add support to parse this to libproc (for pfiles), and elfdump.  The project turned out to be bigger than anticipated, consisting of almost a thousand lines of code.  So it took me a little more than a day to complete it.

A webrev with the associated changes is presented for your enjoyment.  I'd appreciate any review feedback, and especially any further testing.

Here's a sample run:

garrett@openindiana{250}> cat < /dev/zero &
[2] 144241
garrett@openindiana{251}> pfiles 144241
144241: cat
  Current rlimit: 256 file descriptors
   0: S_IFCHR mode:0666 dev:526,0 ino:35127324 uid:0 gid:3 rdev:67,12
   1: S_IFCHR mode:0620 dev:527,0 ino:893072809 uid:101 gid:7 rdev:27,2
   2: S_IFCHR mode:0620 dev:527,0 ino:893072809 uid:101 gid:7 rdev:27,2
garrett@openindiana{252}> gcore 144241
gcore: core.144241 dumped
garrett@openindiana{253}> pfiles core.144241 
core 'core.144241' of 144241: cat
   0: S_IFCHR mode:0666 dev:526,0 ino:35127324 uid:0 gid:3 rdev:67,12
   1: S_IFCHR mode:0620 dev:527,0 ino:893072809 uid:101 gid:7 rdev:27,2
   2: S_IFCHR mode:0620 dev:527,0 ino:893072809 uid:101 gid:7 rdev:27,2
garrett@openindiana{254}> kill -11 144241
garrett@openindiana{255}> pfiles core
core 'core' of 144241: cat
   0: S_IFCHR mode:0666 dev:526,0 ino:35127324 uid:0 gid:3 rdev:67,12
   1: S_IFCHR mode:0620 dev:527,0 ino:893072809 uid:101 gid:7 rdev:27,2
   2: S_IFCHR mode:0620 dev:527,0 ino:893072809 uid:101 gid:7 rdev:27,2
[2]  - Segmentation fault            cat < /dev/zero (core dumped)

garrett@openindiana{256}> elfdump core

  entry [10]
    namesz: 0x5
    descsz: 0x440
    type:   [ NT_FDINFO ]
    desc: (prfdinfo_t)
        pr_fd:        0
        pr_uid:       0                  pr_gid:      3
        pr_major:     526                pr_minor:    0
        pr_rmajor:    67                 pr_rminor:   12
        pr_ino:       35127324
        pr_size:      0                  pr_offset:   419414016
        pr_fileflags: [ O_RDONLY O_LARGEFILE ]
        pr_fdflags:   0
        pr_path:      /devices/pseudo/mm@0:zero

  entry [11]
    namesz: 0x5
    descsz: 0x440
    type:   [ NT_FDINFO ]
    desc: (prfdinfo_t)
        pr_fd:        1
        pr_mode:      [ S_IFCHR S_IRUSR S_IWUSR S_IWGRP ]
        pr_uid:       101                pr_gid:      7
        pr_major:     527                pr_minor:    0
        pr_rmajor:    27                 pr_rminor:   2
        pr_ino:       893072809
        pr_size:      0                  pr_offset:   296960430
        pr_fileflags: [ O_RDONLY O_NOCTTY O_LARGEFILE ]
        pr_fdflags:   0
        pr_path:      /dev/pts/2

  entry [12]
    namesz: 0x5
    descsz: 0x440
    type:   [ NT_FDINFO ]
    desc: (prfdinfo_t)
        pr_fd:        2
        pr_mode:      [ S_IFCHR S_IRUSR S_IWUSR S_IWGRP ]
        pr_uid:       101                pr_gid:      7
        pr_major:     527                pr_minor:    0
        pr_rmajor:    27                 pr_rminor:   2
        pr_ino:       893072809
        pr_size:      0                  pr_offset:   296960430
        pr_fileflags: [ O_RDONLY O_NOCTTY O_LARGEFILE ]
        pr_fdflags:   0
        pr_path:      /dev/pts/2

Monday, October 22, 2012

Latest Apple EarPods - Success!

A couple of months ago I purchased the rather expensive pair of Apple in-ear earphones, to replace an older pair of standard earbuds I'd given to someone else.  They never fit my ear correctly, and I hated them.

Recently I've also taken up getting more physically fit -- so going to the gym more often, including spending a lot of time running on the treadmill.

But my earphones kept falling out.  And my iPhone 4 is kind of heavy and inconvenient.

So I picked up the iPod nano (7th gen) today at Best Buy.  And it comes with a new style of earbud.

I've been wearing them now for about 7 hours, including a fairly intense 40 minutes of running on the treadmill.  They haven't budged once.  And while I'm no audiophile, the audio quality seems to be better than any of the other earbuds I've used.

And the nano -- really light, really nice.  It doesn't do everything, and I've not tried out all the functions yet, but for playing music, it works great.  And, unlike my iPhone, it doesn't have a tendency to change songs or activate other functions while it is in my pocket.   So, +1 on the iPod nano too.

Just be aware that the earpods that come with the nano do not include the remote (and I think they lack a mic as well).  It looks like the $29 separate ones include the remote, and a mic.  I'll be purchasing a pair for other use... I wish they'd package them the same way the in-ear headphones are packaged though.  (Much easier to keep them stored tangle-free.)

Tuesday, October 16, 2012

GNU grep - A Cautionary Tale About GPLv3

My company, DEY Storage Systems, is in the process of creating a new product around the illumos operating system.  As you might imagine, this product includes a variety of open and proprietary source code.  The product itself is not delivered as a separate executable, but as a complete product.  We don't permit our customers to crack it open, both from the sense of protecting our IP, but also to protect our support and release engineering organizations -- our software releases consist only of a single file and we don't supply tools or source for other parties to modify that file.

One of the pieces that we wanted to integrate into the tree is an excellent little piece of software called Zookeeper, produced by the Apache organization.  Like illumos, Zookeeper has a nice non-viral copyleft license, which makes it nice for integration into our product.

However, I discovered that as part of our integration, one of my engineers had decided to integrate GNU grep.  Why? Because the startup script for Zookeeper apparently makes use of some GNU grep extensions that are not illumos' grep.  (The need for us to enhance illumos' grep utility, and the need to teach folks not to rely on non-portable GNU extensions, are both topics for another time.)

Fortunately, I caught this in time.  Because that one little "worm" - used simply to support a startup script, could have created a situation where we would have been required to open source our entire product.

Now if you're Richard Stallman -- this is your goal -- all source code is "liberated".

But if you're a business trying to create value, this is actively harmful.  Its almost impossible to build a product around GPLv3 code unless the only way you create value is through selling support.  (There are a variety of business reasons this is a bad model... with open code, 3rd parties can start "selling support" for your product, possibly giving your product a bad name, and generally leaching off your engineering effort.   In the end, without proprietary code, there is vastly reduced economic incentive to innovate -- you can't use innovation in software to gain a competitive advantage when all software is open.  I would argue that even the innovation that occurs in Linux exists largely due to economic pressures arising from attempts to compete against proprietary systems. )

So, while the correction here is not big for us -- just a small change to a startup script for now -- the hazard of these viral licenses cannot be understated.  If you make your living, as I do, writing software, then you should be very wary of the lure of code carrying the GPLv3 -- its a potential license bomb that may have nasty consequences for you later.

And if you're a producer of open source software -- as I am -- be wary of your dependencies as well.  While your code may be licensed under reasonably generous terms that support commercial uses, if your dependencies include things like GPLv3 GNU grep, you may find that your product has less wide acceptance than you intended.  Inclusion (by dependency or otherwise) of a GPLv3 product in your own product means that you are willing to abdicate any decisions about the licensing of your own product.

I'll get off my soapbox now...

Thursday, September 20, 2012

Infographic - Programming

I was forwarded the following infographic from  While I'm not normally one to just regurgitate someone else's posting (especially advertising), I think this one is worth looking at -- especially if you're at a point where you're contemplating future career or learning plans.

Programming Infographic

illumos and ZFS day!

I'm pleased to announce that DEY Storage Systems, Inc., along with Joyent and other community members, is sponsoring the illumos and ZFS Day taking place in October 1st and 2nd at the Children's Creativity Museum in San Francisco.  My partners and I will be in attendance, along with a few other of my colleagues, and we'll be speaking on related topics.

Additionally, Joyent is hosting the 2nd illumos hackathon at their facilities on October 3, and we will definitely be participating there.  We had a blast doing this last year (some excellent work came out of it), and we're looking forward to doing it again.

If you can make, I hope you'll register.  The event itself is free, and while there will be online recordings of the material, its more fun to get together in person -- especially with a beer or three.  (Did I mention there will be a Solaris Family Reunion party on Monday night?

In many ways, I view this event as a belated 2nd birthday party for illumos, so I hope you'll join me in celebration.

Sunday, September 9, 2012

The Case for a new Reference Distro for illumos

With the recent announcement and discussion around OpenIndiana, I think its time to bring to the light of day an idea I've been toying with for a while now -- and have discussed with a few others in the community.

illumos needs a reference distribution.

Now just hold on there... before you get all wound up, let's discuss what I mean by a reference distribution.

  • A reference distribution should be a base for additional development, and testing. 
  • It should support automated installation for mass deployment in test lab/QA/build farm scenarios.
  • It must support "self-hosting" -- i.e. illumos-gate itself should be buildable on this distribution -- with as few additional steps as possible.
  • It must be valid as a QA platform -- for example, it needs to support the packaging tools we use in order to validate the packaging metadata (manifests).  (Yes, this means it needs to support - and be packaged with - IPS -- in spite of my loathing of IPS itself.)
  • It should be a reasonable (but not the only reasonable possible) base to use for other distributions looking to build on top of it.
  • Ideally it should be pretty darn minimal.
  • It may have graphics support, e.g. an X server, but only in so far as having that would support validation of technologies in illumos-gate.  (Sorry, no big desktops, Gnome, web browsers, or any of that crap.)
  • Community driven - it contains the things that the entire community agrees upon as our common core.
  • It must require only a modicum of effort around release engineering - none of these multi-week release cycles.
  • It should be released biweekly with a numbered build, and probably also have nightlies.  These should be automated.
  • Multi-platform - i.e. it needs to support both SPARC and x86.

As a common reference, it must use illumos-gate verbatim, i.e. no local changes.

Here's what it is not:

  • A general purpose desktop.
  • A general purpose server.
  • A hypervisor.
  • An appliance.
  • Subject to the whims of any corporate entity.
  • An experimental version (it isn't Fedora!)
Why do we need this?  OpenIndiana has been filling the role - but OpenIndiana is insufficient for several reasons:

  • OpenIndiana is too big and unwieldy -- both to build, and to install.
  • OpenIndiana maintains their own fork of illumos-gate.
  • OpenIndiana needs the freedom to follow their own course - without being too strictly tied to illumos-gate.
  • OpenIndiana lacks any kind of formal QA.  (The recent debacles around 151 are good examples of this.)
  • OpenIndiana is not built for SPARC.
(Some of the above items could be addressed with OI, but I think addressing all of them is nigh impossible without totally changing the role of OI.)

So, here's what I propose we do instead.
  • I propose that we start with OpenIndiana or OmniOS, but using core illumos-gate (no fork), and we remove things that are not relevant to the reference.  E.g. Gnome, etc.
  • We will need lots of assistance with release engineering - I want a formal RE plan in place for this thing, with regular scheduled builds and releases following the "train" model rather than the "station wagon" model.
  • We will need to set up some central build and test servers.  (I hope to get some funding for this through the Foundation).
  • We will need to have formalized QA.  (Imagine a Jenkins instance that ran a build and regression test group every night.
  • The bug database should add fields to reference formal build numbers.  (Introduced-In, Fixed-In, Reported-In, etc.)
  • We do not need "an installer".  So the image can just be a basic ISO or tarball, probably.  Think minimalist.  I don't want any customization in the installer, excepting possibly the identification of the initial disk to install to.

Ultimately, if we're very successful, then this will help the entire illumos community.  This reference may ultimately be useful to become a base for OpenIndiana or OmniOS to build upon.  Put another way, I want to make OI's job easier, but I don't intend that this should replace OI.

So, what do I want from the community?

I want to hear from people who think this is a terrible idea.  A descriptive criticism of concerns is far more useful than just another +1.  (If you want to +1 it, do it using Google or FB to like it -- don't add a +1 comment.)  Post your concerns directly to this thread, so that we can keep them for posterity.

I want to hear from people who want - and are able - to actively contribute to this.  Most of this effort will be volunteer based initially, but I think we'll eventually need to staff some work from the Foundation.  Corporate sponsors may help here as well, either by loaning of resources, or gifting of earmarked funds.

Btw, for folks that want to debate names -- let me kill that now.  I want to name this "IRD" -- illumos Reference Distribution.   Its about as unsexy as possible - and that is the point -- this thing should be viewed as something the community uses internally, and not something that is ever given marketing attention.  (Those spot lights belong to illumos itself, and to the other end-user oriented distributions.)

To be clear, I'm not 100% set that this is all the way to go -- but I'm heavily leaning in that direction.  While I can't force the rest of the community to follow along, I can allocate resources to it, and I want to make sure that we can address any major concerns before I start investing resources here.


Wednesday, August 29, 2012

Response to Alasdair's Resignation

[ Recently, Alasdair Lumsden announced his retirement as leader of the OpenIndiana community.  This is a copy of my response to that. ]

Dear Alasdair,

Here at the illumos Foundation, we are very sad to hear of your resignation as lead of the OpenIndiana distribution. You have demonstrated great leadership and have been a wonderful friend of the community. We understand that such projects can be very time consuming, and we wish you all the best in your other endeavors. 

As a community, many of us share some of your frustrations. At times, progress does appear to be slow. At other times, there may appear to be a lack of interest from many of the former Sun employees. However, although you may feel that there is a lack of support or interest, many of us active in the community feel differently. While OpenIndiana served to carry on the banner of the OpenSolaris distribution, it was obviously a dead end because the desktop wars are over, and even the Linux community could not win. Meanwhile, the flourishing embedded and special purpose markets for illumos technology continue to grow. 

For example, use of illumos technology is growing in the areas of internet and cloud infrastructure, database and storage appliances, and embedded products. Many successful companies are driving the technology forward and the future has never looked brighter for open source developers looking to contribute to the community and find interesting jobs.

Although you may think we don’t see that the expanding free and open source software marketplace is not our total playground, what we do see is a talented, dedicated group of volunteers committed to seeing illumos become an important part of specialized, niche technology solutions, especially for solving critical problems facing the Internet. 

Honestly, we do not expect the illumos core itself to be economically viable. We do expect--and see--distributions built upon the illumos core becoming commercially successful and growing. This model is similar to the successful Linux model, where the distributions can achieve different results based on the needs of the constituents or the community; some commercial, some less so. 

The model of OpenSolaris is broken. The model of OpenIndiana following OpenSolaris is broken. The illumos model is following the successful Linux model. This is exemplified by distributions such as the commercially supported, general purpose OmniOS, Joyent’s SmartOS, Delphix OS, and others. 

In the future, we expect illumos to become a bigger part of the technological community, as many companies now emerging from stealth mode are incorporating our core platform functions. As our community grows, and word continues to spread of our stability, dependability and scalability, we hope to take illumos to even more specialized markets. 

Among the illumos community of volunteers are some of the smartest, most talented, most dedicated and hard-working people I’ve ever had the pleasure of being associated with. Their years of experience, combined with their insight into the serious problems facing the Internet, assure a rock-solid future for illumos. As a community, we have a shared set of values, and while sometimes we’ve had our disagreements, it’s always been those shared values that have come through and helped the community - and the technology we are building - grow and improve. We will continue to build an enterprise-grade, highly dependable operating system. 

Again, we are saddened by your leaving. On a personal note, I have enjoyed the collaboration that we shared in the formative days of illumos and OpenIndiana, and I hope we will be able to do so in the future.  We thank you for all your efforts on behalf of OpenIndiana and illumos, and know that your efforts have not been in vain; we will take all that we have learned under your leadership and build on it. As a natural evolution of the community, distributions come and go; we take the lessons and move on. Our community has never been one to back down from a challenge; we will handle this in stride and continue to move forward with our mission and our future goals. 


Garrett D’Amore
illumos Founder

Wednesday, June 13, 2012

Women in Tech Adverstising - a rebuttal

So, there has been some that have argued vehemently against the advertising policies of GoDaddy, and its ilk.  (See for example of the complaints.)

I have a different point of view.  What about sites that always picture a beautiful woman on a headset for their representation of tech support?  Should we also boycott them?  Would you prefer a site with an ugly person answering the phone?  After all, it's a phone or text chat, so it shouldn't matter at all what the person looks like, right?

Conversely, what about all the sites that use men in their advertising?

My hosting provider,, uses both.  Take a look.  Does the pretty blonde in glasses have anything to do with domain names or web hosting?  Of course not.  The guy wearing the headset for the tech support line isn't exactly hideous either, if you swing that way.  Are you likely to talk to this guy if you call?  Probably not -- he's probably just another model.

GoDaddy uses Danica Patrick as their spokesperson.  She's admittedly beautiful, a professional athlete (she drives race cars), and probably makes a fair bit of money endorsing GoDaddy.  Some of the ads she has been in have admittedly been a little tongue in cheek in their appeal to a mostly male clientele, but I think it would be a stretch to say that GoDaddy is using sex to sell domain services.

It would be an even further stretch to say Danica is being exploited by GoDaddy.

(Note that there may be other reasons to avoid doing business with GoDaddy or its CEO, but I am only referring to the use of Danica Patrick, a female spokesperson, and women in advertising in general.)

Would the people who claim that this is somehow evil objectification of women complain as loudly if the spokesperson were a good looking baseball star?  Is this objectification of men, or of male sports stars?  What if the ad somehow appeals more strongly to women, or to gay men?

What if the site featured an animal spokesperson prominently?  Say an orangutan?  (Perhaps with computer animation. :-)  Would this be considered animal exploitation?

I submit to you, dear reader, that each one of these cases are basically the same.  We use humans, animals, or whatever, to provide a human element and appeal in a product being sold.  Sure, it would be nice if the site instead featured some kind of graphic that demonstrated its technical superiority, but come on -- we're talking about a domain registrar, which is about as boring as tech gets.  I can hardly blame GoDaddy for trying to liven up their site and generate some enthusiasm.

In fact, even in the more extreme cases, such as the use of bikini-clad booth babes, I submit that folks who think objectification like this is somehow exploitative are misunderstanding.  These women are very rarely without many choices and options.  Usually by the time we see them, its when they are in a job that they have worked hard to get -- maintaining a perfect figure can be a full-time job, after all.

If folks like author of the aforementioned tweet had their way, they would deny these women the opportunity to earn a living showing off their own accomplishments.  Isn't that even more exploitative?  In fact, in the extreme case, you could consider the more repressive societies of the middle east?  Is this the social ideal to which we should aspire?  Where the only part of woman we see in advertising is a burqa?

(Note that in the case of the site using bikini-clad booth-babes, my gut response is to question what the company is not showing -- their product.  So while the babes may catch my eye, they have yet to draw me to a booth I would not otherwise have visited for the products featured.  Actually, at one former employer a female colleague -- also quite beautiful, but quite knowledgeable -- is aware that she is often mistaken as a booth babe, and regularly uses this to her advantage; customers get surprised somehow when the beautiful woman knows more about the technology than most other men do.)

As for me, I welcome the diversity of our society, and the appreciation of the human form.  If some people are able to use this to humanize their marketing approach, good for them!  Let's face it, I'd rather look at beautiful faces online than ugly ones.

Far better, IMO, for those who would fight against this spend their energies worrying about the women who are truly exploited or repressed -- in the fundamentalist muslim societies of the middle east, in the brothels, and human slave trade.  Don't feel sorry for the woman who is able to make a nice living for herself displaying the body she has worked hard to get and to maintain, and that she is proud of.

And yes, for the record, I usually like Carl's Jr. commercials too.

Monday, June 4, 2012

Ivy Bridge Motherboards -- Don't *Really* Exist!

So, if you're like me, and you've been shopping for a new system lately because you're last one is dying or dead, you might see some fancy looking Intel E3-1200v2 Xeon systems.  (Why Xeon?  Because ECC memory is a good thing.  I attribute some of my difficulty with my last system to using non-ECC memory.  Because its not ECC, I don't really know whether the problem was in RAM or CPU, but I digress...)

Great, so you pick out a E3-1240v2 Ivy Bridge CPU, and then you look for a System Board.

I found the Supermicro X9SCA-F-O.  Looks like a sweet board.  The spec claims support for the E3-1200v2 cpus, and 1600MHz RAM.  It has a separate IPMI LAN support, as well.  Sweet!  (Look closer though....)

So, package arrives from NewEgg, and you're ready to go.

Assemble the kit.... if you're a software guy like me this might take a couple of hours.

Power it on.

Beep!  Beep!  Beep!  Beep!

WTF does that mean!?  Google "4 beeps supermicro".

Oh, "Unsupported processor."  Further reading....

The system needs a BIOS update (to v2.0) in this case.  Seems reasonable.

Download the instructions....

"Step 1. First, boot the motherboard with an E3-1200 series (not E3-1200 V2)

WTF!?!  A new E3-1200 (that's Sandy Bridge mind you, still fairly recent stuff) processor is going to cost me like 200 bucks, minimum.  And I can't return it after I buy it except to exchange it with a different CPU.  But I already have the E3-1240v2.

Anyone want to buy a very lightly used E3-1220 from me?  It looks like I'll be purchasing one tomorrow. Ultimately, I can't afford not to burn the $200, because I simply don't have the time to keep screwing around this stuff.  I've burned the better part of today trying to wrangle hardware.  (Hmmm.... where's a Systems Engineer when you need one?)  Conversely, anyone got one lying around they want to give, loan, or sell really cheap?

Shame on you Supermicro.  There are at least several things you could have done better here:

1. Make it much clearer that if buying a unit, it should include a legacy CPU first.  (I.e. that Ivy Bridge support is only for people upgrading CPUs and not building new systems.)

2. Have a separate SKU for systems that have the requisite BIOS changes.  Even if the silk-screen on the board is the same (and it shouldn't be, because the silk screening indicates only 1066 and 1333 MHz RAM supported, although the BIOS enables 1600 MHz), having the separate SKU lets buyers make sure they have the bits needed.

3. Work with retailers to ensure that existing stock is returned and upgraded as soon as possible.

4. Offer an expedited RMA process for people like me who are screwed in the meantime.  (To be fair, I've not called supermicro yet, but Googling tells me people have not had good luck with this option yet.  The only solution that has worked for people is to get their own 2nd processor.)

5. Technically, this board has an IPMI unit on it.  I think that means it has a separate microprocessor. Why can't I update the BIOS using that?  That would eliminate the need to buy a second CPU, at least for those of us with IPMI enabled gear.

Lessons for me:

1. Its probably worth a couple hundred bucks to let someone else put together the system for me.  I'm a driver guy -- futzing around with actual bits of silicon and cabling, its not my forte.

2. There's a reason it's called the bleeding edge.  I think I got cut with it today.

Sunday, June 3, 2012

Why I *hate* external dependencies!

So, I've been trying to setup a system that can build illumos-gate.  Because my old system had a memory DIMM fault, and isn't usable anymore.

I thought, hmm... let me try something other than OpenIndiana.

Big mistake.

Only OpenIndiana seems able to build vanilla illumos-gate right now.

Why?  Because of external dependencies!  I tried building on OmniOS.  The stopper there -- Python 2.4!  On SmartOS, the dependency is IPS itself.  It seems only OpenIndiana is suitable for building stock illumos-gate.

The problem is that our build is inherently very sensitive to the build environment.  It makes life incredibly unpleasant if you try to build on any system that is not configured exactly as we specified.

This, IMO, is an unacceptable situation.

I will speak loudly against any attempt to make changes that introduce further external dependencies.  The fact that some already exist is no excuse.  Every external dependency makes working on the tree more painful.  If you're thinking of a change that would move something out of tree, but add a build-server dependency, think again!  

These dependencies are a serious barrier for contribution.

We (illumos) should take a page from NetBSD, who have successfully arranged that their environment is self contained to the point that they don't care about their build server at all -- all it needs to be able to do is bootstrap gcc and run automake/autoconf.  There is something very elegant about this.

Expect to see me on yet another crusade to kill dependencies on external runtimes, be it Python, Perl, or whatever.   Our tree should, IMO, be as "stand-alone" as possible.

If that means you have to write C, so be it.  Grow up and get a real compiler dude.  I know languages like Python and Perl are attractive because they have a lower barrier to learn -- but depending on them -- and far worse -- depending on *specific* versions of them -- is toxic for anyone else who wants to work on the code in a different environment.   (This has nothing to do with your language of your choice, but everything to do with the pain and suffering your language choice creates for anyone who just wants to do "make install".)