New Stuff in OpenSolaris 2008.11

OpenSolaris 2008.11 is coming soon. Build 101 is the stabilization build before it, and as usual, new features are excluded from this build. So we can say pretty certainly what features are in, and what are out.

While there are a lot of new features coming in this release, there are some in particular that I've been more involved with.

1. SDcard support. Numerous laptops can now use SDcard and MultiMediaCard media directly. A good way to know if you're laptop is one of them is to search for pciclass,0805 in the output of prtconf -vp. While it isn't completely conclusive (in particular some models from Texas Instruments are schizophrenic here), if you see this, there's an excellent chance it will work. I'd always like to hear feedback about this feature -- if your unit works, or doesn't work, let me know. (Also, of course, the Tadpole SPARCLE models are supported. )
2. AudioHD improvements. Notably, many more laptops will now have working audio. The audiohd driver is also updated to support Suspend/Resume.
3. Fast Reboot. I participated as a consultant. I'll also be updating (post OpenSolaris 2008.11) additional drivers to support this feature. The upshot of this project is that a healthy system can reboot much more quickly now.
4. Brussels (NIC Administration). I've participated in converting several drivers to Brussels, and in generally improving Brussels (I'm also the ARC sponsor for this work.) The upshot of this project is greatly improved manageability for network interfaces.
5. Suspend (S3) Support. I helped review the conversions of several drivers, and provided fixes for several NIC devices.
6. Bug Fixes. I've worked on a number of them, and of course, there are huge numbers of bugs that have been fixed in this release.

All said, I think OpenSolaris 2008.11 is going to be great -- I confess that I was skeptical about the earlier releases, but this release is shaping up to be really awesome.

Ancient History Exhumed

Okay, maybe not so ancient (circa 2000), but I recently got an e-mail from the Sun IT group notifying me about changes that impacted web pages I set up for Alternate Pathing, which was the very first project I did that involved work within the Solaris kernel. Apparently they didn't notice that I'd left the company and returned.

Of course Alternate Pathing was canceled a long time ago, although someone may still be using it on older E10K Solaris 8 systems.

Here's the Sun internal URL to which the e-mail referred. The bug tracking pages are broken, since the scripts behind them were developed (by me) to talk to the BugTraq+ Sybase server, which Sun hasn't used in ~forever.

IOMMU comes to Solaris x86

This weekend, the code for the IOMMU for Solaris on Intel (PSARC 2008/560) was pushed. This has potentially profound ramifications for folks working on Solaris device drivers, and I thought I'd take some time to talk about them.

First off, it needs to be noted that we've had an IOMMU on Solaris SPARC pretty much for as long as we've had Solaris on SPARC. (In fact, most SPARC platforms have to use an IOMMU -- they have no choice.) But on x86 this technology is new.

The benefits that IOMMU brings are many fold.

1. It virtually guarantees that all DMA requests can be set up with a single DMA cookie, reducing complexity and eliminating the use of bounce buffers by the DMA framework -- even for old devices with unfortunate restrictions (such as an inability to perform dual address cycles on PCI -- i.e. no 64-bit support.) Such restrictions are actually fairly common place.
2. It allows for strong isolation to be given for devices that can be accessed via other virtual machines or domains. This can prevent one misbehaving xVM domain from crashing others by misprogramming the DMA engine on a physical device.
3. It allows for isolation of faulty devices, so that they cannot scribble into arbitrary PCI spaces, preventing a misbehaving device from accessing regions which it should not. This has major benefits for fault resilience, as well as diagnosability. (To be fair, I'm not 100% sure the code is in place yet to leverage all of this benefit, or integrate it with FMA.)
4. It facilitates debugging of faulty device drivers. To give a concrete example, when I was working on an audio driver recently it took me a long time to figure why the device was emitting white noise. It turns out that I had not initialized the DMA address register properly. With IOMMU, instead of the device just getting random data, I'd have gotten a bus fault that would have contained information that I could have used to see that the device was trying to access some weird place memory to which it had no right.

Now, these features don't come without some cost.

1. Setup and tear-down of DMA operations is potentially significantly more expensive than with the simple translation layer previously used. Device drivers that assume such operations are inherently cheap may be in for a surprise.
2. Drivers still have to retain the ability to operate in an environment without an IOMMU. Effectively, this means that they need to be prepared to see more than one DMA cookie or window per DMA region. Generally speaking, well written drivers should make no assumptions about the number of cookies used beyond the limits expressed in the DDI DMA attributes. (Namely, the framework is free to use any number of ddi_dma_cookie(9s) >= 1 and <= dma_attr_sgllen.)
3. Drivers that require physical rather than virtual memory be used (i.e. that need to bypass the IOMMU) can request mappings using DDI_DMA_FORCE_PHYSICAL in the ddi_dma_attributes(9s) dma_attr_flags field, but such requests are not guaranteed succeed. A portable driver must retry such a request without the flag set, if the first attempt with it set fails.
4. Generally, correctly written drivers will Just Work with the integration, without any changes to them. I would discourage driver authors from disabling the use of DDI_DMA_FORCE_PHYSICAL unless they have specific performance requirements. (And, normally, there are better solutions such as reuse of mappings, so that DDI_DMA_FORCE_PHYSICAL remains unnecessary.)
5. The IOMMU imposes (or should impose) a new test requirement -- namely that device drivers are tested on systems both with and without an IOMMU. While code that works on systems without an IOMMU is unlikely to notice the introduction of the IOMMU, the reverse is not true. If drivers were developed in the presence of an IOMMU, it is not unusual for them to fail on systems without an IOMMU, as the lack of an IOMMU often requires mappings to be made with multiple DMA cookies, especially for resources that page boundaries.
   

I've not had a chance to play with the new framework myself yet, but I look forward to doing so. Also, be aware that same feature set is coming soon for AMD platforms, see PSARC 2008/561.

audiohd pushed

The latest & greatest audiohd driver, which includes vastly improved support for a large number of codecs, suspend/resume support, and a generic "codec parser" has now been "pushed" into build 98 of ON. Many thanks to the Beijing audio team, who worked long and hard to bring this project to fruition.

New audiohd driver posted

We've gotten a lot of good feedback from previous posting of the audiohd driver, and the Beijing engineering team has come up with a new version (Aug 12, 2008.) Note that the latest audiohd driver will always be posted here.

One note: please use the obj32/ or obj64/ versions of the driver unless you are running a debug kernel. There are some binary dependencies where the debug drivers won't work on a production kernel.

SDcard pushed...

Finally, after months of delay, the SDcard bits have been putback^Wpushed.

This was interesting, because the SDcard bits were also some of the first bits to have been pushed into the new Mercurial tree -- and they're already in the clone. Adventurous people can start trying the code out now. Or just wait until b97.

Of course, this also means that this will be a new feature in the OpenSolaris 2008.11 release. Yay!

If you do try the bits out, let me know what works, and what doesn't. There seems to be some evidence from the Linux community that not all SDHCI compliant controllers are created equal, although the code I have works for the few variants that I've had access to.

The bugster category for bugs is solaris/driver/sdcard

SDcard RTI submitted... webrev posted

Finally got legal approval!

A webrev is available, as well, in case anyone wants to look at it. I hope to have this integrated into b97.