Wednesday, January 27, 2010

Its Official

I'm no longer a Sun Microsystems Employee, since Sun no longer exists. Hopefully I'll get to keep my job at Oracle, but I've not seen any e-mail yet. I expect I will before day's end.

Monday, January 25, 2010

Auto Install Finally Working For Me!

Some of you may know, I've been struggling (and failing) to make auto install work for me. I've had challenges, because my network is not routable, and due to other issues (bugs!) in OpenSolaris Auto Install.

However, it seems that I've finally hit on a successful recipe. I want to record this here for others.

First, in order to use AI, you will need your installation server to be running a recent build of OpenSolaris. The release notes indicate b128 is sufficient. I just ran "pkg image-update" to update to build 131. If you fail to do this, there won't be a warning at all, its just your clients will simply not boot.

The next thing you'll need to do is download a full-repository and the AI image.

Unfortunately, there are not public versions of the full repo ISO file available that are "current". (No, I can't get you a copy, and no I don't know why they haven't posted a more recent update.) Hopefully this problem will be corrected soon.

Setting up the local repository can be done following these directions. (Note that you will have to change the paths to reflect your system. I stash ISO images in /data/isos, and install images under /data/install. These are separate ZFS filesystems.

# where do ISOs live, without leading /
ISODIR=data/isos
# where does the repo live, without leading /
REPO=data/install/os131_repo_full
# AI service name to use
NAME=os131_x86
# parent directory for installations, without leading /
INSTDIR=data/install
# port to use for install server
PORT=8181

mount -F hsfs -r /${ISODIR}/osol-repo-131-full.iso /mnt
zfs create -o compression=on ${REPO}
svccfg -s application/pkg/server setprop pkg/inst_root=/${REPO}
svccfg -s application/pkg/server setprop pkg/readonly=true
svccfg -s application/pkg/server setprop pkg/port=${PORT}


You'll need to edit the ${REPO}/cfg_cache file, changing the origins entry to match your system. I used a value like this:

origins = http://192.168.128.11:8181

Then you'll want to use installadm to setup an initial boot service. Here's the recipe I used:

zfs create -o compression=on ${INSTDIR}/${NAME}
installadm create-service -n ${NAME} -s /${ISODIR}/osol-dev-131-ai-x86.iso /${INSTDIR}/${NAME}

Now you need to change the default manifest file. This is the tricky part, that IMO was not terribly well explained anywhere else.

cp /${NSTDIR}/${NAME}/auto_install/default.xml /tmp

Then edit default.xml file in /tmp, changing the value of "main_url" to point to your server. I used a value like this:

<main url="http://192.168.128.11:8181" publisher="opensolaris.org"/>
Then apply this manifest to the default manifest:

installadm add -m /tmp/default.xml -n ${NAME}

Finally, I did some tweaking in my DHCP configuration. I have a macro for each service name, that provides the defaults. For example, my "os131_x86" macro looks likes this:

Include pepper
BootFile os131_x86
GrubMenu menu.lst.os131_x86


My "pepper" macro (pepper is the name of my server) sets some shared defaults, but most especially it sets BootSrvA to the IP address of the server (192.168.128.11 in my case.)

Then I just configure individual addresses for which ever version of OpenSolaris (or SXCE) I want to install using the the correct configuration macro. (For SXCE there are very different DHCP options to use. Also the SPARC version of OpenSolaris uses different options as well.)

Tuesday, January 19, 2010

Six Years & Counting

Its hard for me to believe that six years ago today at this hour in the morning I was getting myself ready to meet my bride. We had a wonderful wedding on the beach in front of the Del Mar powerhouse in San Diego, with our friends and family in attendance.

Looking back, its been the best six years of my life. I've truly been blessed. I'm looking forward to spending the next sixty together with my beautiful bride Deborah.

Thursday, January 14, 2010

Interesting device driver work

So there are a couple of "closed" drivers that are not part of OpenSolaris, and might never be because of redistribution restrictions. However, this represents an opportunity for an enterprising software engineer to contribute. The drivers are

glm - Symbios 53x810 and similiar devices
qus - QLogic ISP 10160 and similar devices
adp - Adaptec AIC 7870P and similar devices
cadp - Adaptec AIC 7896 and similar devices

There are open source drivers for these from FreeBSD and NetBSD, which could be used as a starting point for a port. I'd probably be interested in trying one of these out myself, if time allowed -- but alas it does not, my plate is already quite full.

The best part of these drivers is that there are few, if any, "political" or "business" restrictions on integrating replacement drivers. Indeed, at one point recently each of these was considered for an EOF simply because they weren't considered strategic anymore. (The EOFs were rejected, but these will only be delivered via an extras repository or somesuch.)

So, what are you waiting for? This is a good opportunity to learn about SCSA, and provide us with superior replacement drivers. (The glm replacement looks like it could be done in as few as 2 or 3 KLOC; that is all the NetBSD version of the driver uses.)

Wednesday, January 13, 2010

A "modern" elxl driver

I undertook this past weekend an effort to "port" the NetBSD "ex" driver to GLDv3, as an open source replacement for "elxl".

It took a bit over 2 days.

The new sources are on line in a webrev format. I'd really appreciate feedback. I'm hoping to integrate these changes soon (and I could use help with testing!)

This new version, apart from being Open Source, also has with it:

1. Full support for VLANs (including full-MTU frames)
2. Full support for link notification on twisted pair (and hopefully also fiber)
3. Full integration with Brussels for MII and media selection.
4. Full support for Suspend/Resume (S3)
5. Full support for Quiesce (fast reboot)
6. Support for additional devices

It can also be extended fairly easily to support Cardbus and MiniPCI variants, and hardware checksum offload. (The checksum offload part is basically written and #ifdef'd out until I find a newer card to test with personally.)

What is missing is "automatic" media selection based on active probing. The old Solaris driver would "autoselect" which port (BNC, AUI, twisted pair) to use based on some active probes to look for link. These were rather complex, and not something I could take from the closed driver. These days everyone just uses twisted pair anyway, right? (The fiber and TX4 cards don't offer any choices, so there is no probing needed for them.)

If you have such a COMBO card, you can force the media using a new "driver private" property called "_media", which you can set to various values. See the driver sources in the webrev for more information.

I've done enough work on this driver that there is probably as much of my own code in it (at least) as was in the original NetBSD code. Nonetheless, I'd like to than the NetBSD Foundation for making these sources available.

I'll be posting binaries soon, stay tune. (Sun internal users can grab the binaries from /net/temecula.sfbay/data/work/gdamore/yge/yge/usr/src/uts/intel/elxl/ -- at least for now. That path is likely to work until I get the code integrated.)

Friday, January 1, 2010

Out with the Old, In with the New

Happy New Year (2010) everyone!

I thought I'd take a second to reflect on the accomplishments of the past year, and look forward to what I think is in store for my contributions to OpenSolaris this year.

It's hard to believe that I've been a member of the OpenSolaris community for over 5 years now. (I was a pilot member.)

Undoubtedly this past year my biggest contribution to OpenSolaris was the new audio framework (Boomer) and many new audio drivers.

I did a lot of other work besides, including a bunch of work on NIC drivers (including a new common MII framework and the yge driver which supports Marvell Yukon 2 parts), and various changes to the SDcard framework.

I've also developed a device driver for a very interesting (and very high performance) hybrid storage device (which won't be integrating for non-technical reasons), and a new storage framework for block oriented storage devices. (This framework, blkdev, will be integrating once we're out past the build restrictions.)

I also did a lot of cleanup of legacy and stale code, which hopefully reduces the install foot-print and shrinks compile times somewhat.

2009 was also the year I became a full voting member of PSARC, and I was privileged to serve 3 months as PSARC chair. (The chair rotates amongst all active PSARC members on a roughly quarterly schedule.)

This past year is also the year that I became the top contributor to ON in separate integrations, since the OpenSolaris project started, at least as reported by ohloh.net. (Note that the statistics only cover the open portion of the ON consolidation.)

So what's coming up in the next year? Here's what I expect to be working on:

  • 10GbE ethernet for Mellanox ConnectX devices (hermon). This is probably my top priority at the moment. The work is largely being done by Mellanox, but I'm the Sun engineer ultimately responsible.
  • Sun Ray audio. This is one of my biggest priorities. We want a Boomer driver for Sun Ray audio, bringing in-kernel mixing to Sun Ray appliances.
  • Interrupt-less audio. This is a major rethink of the way we process audio in the kernel, and reduces a lot of complexity in device drivers and is an enabler for several other new features. The code for this is done, so expect an integration in b135 or 136.
  • Better audio hotplug support. (Especially for USB audio.)
  • Better audio virtualization support (especially with Trusted Extensions.)
  • Additional audio device support. (Asus Xonar, via audiocmihd, for example.)
  • Integration of blkdev, and hopefully faster, simpler, better kernel support for simple block oriented flash-storage devices. (I.e. those devices that don't natively understand the SCSI command set.)
  • Fixes for a variety of network device driver bugs. I've already got a few of these changes queued up.
  • Broader support for the MII framework in other NIC drivers. (I have changes queued up for rtls, already, for example.)
  • Further cleanup of not-needed legacy code.
  • Continued contributions and participation at PSARC.
  • Support for SDXC card media (and possibly also development of exFAT filesystem code, dependent on licensing concerns.)
  • Possibly work on various track pad bits, depending on time and resource. (Synaptics support finally?)
There are probably other things that will happen, and I'm sure that our new owners at Oracle (assuming that the deal gets finalized, which looks almost unstoppable at this point) will have some ideas about how my priorities will be spent.

One thing is that I feel very privileged to have been able to work on the OpenSolaris code base, and to continue to be able to do so. I often reflect that its amazing that I get paid to do this work -- I think I'd be far less productive if I didn't enjoy my job so much. When my management asks me to take a break and do something fun for a change, I think he has a hard time understanding that for me hacking on OpenSolaris code is fun. I genuinely hope that I continue to be so privileged for the foreseeable future.