I undertook this past weekend an effort to "port" the NetBSD "ex" driver to GLDv3, as an open source replacement for "elxl".
It took a bit over 2 days.
The new sources are on line in a webrev format. I'd really appreciate feedback. I'm hoping to integrate these changes soon (and I could use help with testing!)
This new version, apart from being Open Source, also has with it:
1. Full support for VLANs (including full-MTU frames)
2. Full support for link notification on twisted pair (and hopefully also fiber)
3. Full integration with Brussels for MII and media selection.
4. Full support for Suspend/Resume (S3)
5. Full support for Quiesce (fast reboot)
6. Support for additional devices
It can also be extended fairly easily to support Cardbus and MiniPCI variants, and hardware checksum offload. (The checksum offload part is basically written and #ifdef'd out until I find a newer card to test with personally.)
What is missing is "automatic" media selection based on active probing. The old Solaris driver would "autoselect" which port (BNC, AUI, twisted pair) to use based on some active probes to look for link. These were rather complex, and not something I could take from the closed driver. These days everyone just uses twisted pair anyway, right? (The fiber and TX4 cards don't offer any choices, so there is no probing needed for them.)
If you have such a COMBO card, you can force the media using a new "driver private" property called "_media", which you can set to various values. See the driver sources in the webrev for more information.
I've done enough work on this driver that there is probably as much of my own code in it (at least) as was in the original NetBSD code. Nonetheless, I'd like to than the NetBSD Foundation for making these sources available.
I'll be posting binaries soon, stay tune. (Sun internal users can grab the binaries from /net/temecula.sfbay/data/work/gdamore/yge/yge/usr/src/uts/intel/elxl/ -- at least for now. That path is likely to work until I get the code integrated.)
Wednesday, January 13, 2010
Friday, January 1, 2010
Out with the Old, In with the New
Happy New Year (2010) everyone!
I thought I'd take a second to reflect on the accomplishments of the past year, and look forward to what I think is in store for my contributions to OpenSolaris this year.
It's hard to believe that I've been a member of the OpenSolaris community for over 5 years now. (I was a pilot member.)
Undoubtedly this past year my biggest contribution to OpenSolaris was the new audio framework (Boomer) and many new audio drivers.
I did a lot of other work besides, including a bunch of work on NIC drivers (including a new common MII framework and the yge driver which supports Marvell Yukon 2 parts), and various changes to the SDcard framework.
I've also developed a device driver for a very interesting (and very high performance) hybrid storage device (which won't be integrating for non-technical reasons), and a new storage framework for block oriented storage devices. (This framework, blkdev, will be integrating once we're out past the build restrictions.)
I also did a lot of cleanup of legacy and stale code, which hopefully reduces the install foot-print and shrinks compile times somewhat.
2009 was also the year I became a full voting member of PSARC, and I was privileged to serve 3 months as PSARC chair. (The chair rotates amongst all active PSARC members on a roughly quarterly schedule.)
This past year is also the year that I became the top contributor to ON in separate integrations, since the OpenSolaris project started, at least as reported by ohloh.net. (Note that the statistics only cover the open portion of the ON consolidation.)
So what's coming up in the next year? Here's what I expect to be working on:
One thing is that I feel very privileged to have been able to work on the OpenSolaris code base, and to continue to be able to do so. I often reflect that its amazing that I get paid to do this work -- I think I'd be far less productive if I didn't enjoy my job so much. When my management asks me to take a break and do something fun for a change, I think he has a hard time understanding that for me hacking on OpenSolaris code is fun. I genuinely hope that I continue to be so privileged for the foreseeable future.
I thought I'd take a second to reflect on the accomplishments of the past year, and look forward to what I think is in store for my contributions to OpenSolaris this year.
It's hard to believe that I've been a member of the OpenSolaris community for over 5 years now. (I was a pilot member.)
Undoubtedly this past year my biggest contribution to OpenSolaris was the new audio framework (Boomer) and many new audio drivers.
I did a lot of other work besides, including a bunch of work on NIC drivers (including a new common MII framework and the yge driver which supports Marvell Yukon 2 parts), and various changes to the SDcard framework.
I've also developed a device driver for a very interesting (and very high performance) hybrid storage device (which won't be integrating for non-technical reasons), and a new storage framework for block oriented storage devices. (This framework, blkdev, will be integrating once we're out past the build restrictions.)
I also did a lot of cleanup of legacy and stale code, which hopefully reduces the install foot-print and shrinks compile times somewhat.
2009 was also the year I became a full voting member of PSARC, and I was privileged to serve 3 months as PSARC chair. (The chair rotates amongst all active PSARC members on a roughly quarterly schedule.)
This past year is also the year that I became the top contributor to ON in separate integrations, since the OpenSolaris project started, at least as reported by ohloh.net. (Note that the statistics only cover the open portion of the ON consolidation.)
So what's coming up in the next year? Here's what I expect to be working on:
- 10GbE ethernet for Mellanox ConnectX devices (hermon). This is probably my top priority at the moment. The work is largely being done by Mellanox, but I'm the Sun engineer ultimately responsible.
- Sun Ray audio. This is one of my biggest priorities. We want a Boomer driver for Sun Ray audio, bringing in-kernel mixing to Sun Ray appliances.
- Interrupt-less audio. This is a major rethink of the way we process audio in the kernel, and reduces a lot of complexity in device drivers and is an enabler for several other new features. The code for this is done, so expect an integration in b135 or 136.
- Better audio hotplug support. (Especially for USB audio.)
- Better audio virtualization support (especially with Trusted Extensions.)
- Additional audio device support. (Asus Xonar, via audiocmihd, for example.)
- Integration of blkdev, and hopefully faster, simpler, better kernel support for simple block oriented flash-storage devices. (I.e. those devices that don't natively understand the SCSI command set.)
- Fixes for a variety of network device driver bugs. I've already got a few of these changes queued up.
- Broader support for the MII framework in other NIC drivers. (I have changes queued up for rtls, already, for example.)
- Further cleanup of not-needed legacy code.
- Continued contributions and participation at PSARC.
- Support for SDXC card media (and possibly also development of exFAT filesystem code, dependent on licensing concerns.)
- Possibly work on various track pad bits, depending on time and resource. (Synaptics support finally?)
One thing is that I feel very privileged to have been able to work on the OpenSolaris code base, and to continue to be able to do so. I often reflect that its amazing that I get paid to do this work -- I think I'd be far less productive if I didn't enjoy my job so much. When my management asks me to take a break and do something fun for a change, I think he has a hard time understanding that for me hacking on OpenSolaris code is fun. I genuinely hope that I continue to be so privileged for the foreseeable future.
Saturday, December 26, 2009
Shortest Audio Driver?
Using my new framework changes (which have not integrated yet), along with changes in the framework designed to simplify drivers by providing certain ordering guarantees, I've been able to shrink a lot of the audio drivers (and in the process have also found a number of small bugs that I think ultimately were responsible for jitter and corruption). Ths simplest driver so far -- audiovia97, which is about as simple an AC'97 driver that can be, is only 686 lines. Several others are now less than 800 lines, and no longer need any locking primitives. As far as device drivers go, this is minuscule. In fact, I challenge anyone to identify a driver that fully services any "real" piece of hardware, fully supporting said hardware, in fewer lines of space.
(And in case you are wondering, a significant fraction of those 686 lines of audiovia97 is still comments and whitespace and even the 30+ lines of boilerplate -- the driver is written to be as readable as reasonably possible, at least to other device driver or kernel engineers.)
The forthcoming audio DDI will be about the easiest DDI for any kind of device driver in Solaris. Contrast that with the pre-Boomer SADA framework, which one could argue had the most cumbersome and painful DDI for non-nexus devices.
Hopefully the simpler framework will make it easier for others to provide new drivers, and result in far fewer "bugs" in the drivers. (And of course, the smaller drivers will result in more efficient use of kernel memory as well, although you are unlikely to notice that particular difference.)
(However, the complexity of the audiohd driver remains a serious thorn in our side -- and unfortunately the problem is made worse by a far-too-complex specification for the Intel HD audio system That said, the complexity in audiohd stems largely from offering too many codec configuration choices, and not from the part of the driver responsible for actually transporting audio data.)
(And in case you are wondering, a significant fraction of those 686 lines of audiovia97 is still comments and whitespace and even the 30+ lines of boilerplate -- the driver is written to be as readable as reasonably possible, at least to other device driver or kernel engineers.)
The forthcoming audio DDI will be about the easiest DDI for any kind of device driver in Solaris. Contrast that with the pre-Boomer SADA framework, which one could argue had the most cumbersome and painful DDI for non-nexus devices.
Hopefully the simpler framework will make it easier for others to provide new drivers, and result in far fewer "bugs" in the drivers. (And of course, the smaller drivers will result in more efficient use of kernel memory as well, although you are unlikely to notice that particular difference.)
(However, the complexity of the audiohd driver remains a serious thorn in our side -- and unfortunately the problem is made worse by a far-too-complex specification for the Intel HD audio system That said, the complexity in audiohd stems largely from offering too many codec configuration choices, and not from the part of the driver responsible for actually transporting audio data.)
Tuesday, December 22, 2009
Do you use IDN? (E10K inter-domain networking)
I was looking, and one of the legacy device drivers we still support is the Enterprise 10000 Inter-Domain Network. This is a memory based interconnect network that emulates an ethernet.
It occurs to me that we could update this driver to use the GLDv3, which would potentially have a significant performance boost for it. (Really, a better solution would be to create a new low level MAC type for it Nemo, but clearly that's probably too much pain for a product that is no longer shipping).
Anyway, I'd be interested to hear of any sites which use IDN on E10K class systems, and would like to see this (already high performance) network get another boost.
The other legacy SPARC drivers which are not GLDv3 compliant are:
It occurs to me that we could update this driver to use the GLDv3, which would potentially have a significant performance boost for it. (Really, a better solution would be to create a new low level MAC type for it Nemo, but clearly that's probably too much pain for a product that is no longer shipping).
Anyway, I'd be interested to hear of any sites which use IDN on E10K class systems, and would like to see this (already high performance) network get another boost.
The other legacy SPARC drivers which are not GLDv3 compliant are:
- cassini (ce) (probably never going to be GLDv3, but that's another matter)
- gem (ge) -- this one could be easily converted *if* I had fiber cards to test it with, although there are political concerns as well. (It uses the same MAC controller as "eri", which I converted a while back.)
- scman -- This is used for the Starcat (E15K/25K) connection between the SC and domains. It is also based on the eri driver, but has some special hacks. I'm a bit paranoid about touching this one, but if someone really wants it to be GLDv3, please let me know. (This isn't supposed to be a high performance network!)
Wednesday, December 16, 2009
New audioctl coming soon
I integrated a replacement for mixerctl (and also removed mixerctl) back in build 130. Well, build 130 is coming out really soon (I believe it is available internally in SXCE form now.)
I think therefore that it is time to provide more detail here about the new audioctl syntax:
I expect there may be some questions or concerns about the new syntax. If they are not answered by the manual page above, please don't hesitate to ask me.
I think therefore that it is time to provide more detail here about the new audioctl syntax:
NAME
audioctl - audio mixer control command line application
SYNOPSIS
audioctl list-devices
audioctl show-device [-v] [-d device ]
audioctl show-control [-v] [-d device] [control ...]
audioctl set-control [-v] [-d device] control value
audioctl save-controls [-d device] [-f] file
audioctl load-controls [-d device] file
DESCRIPTION
The audioctl command is used to control various features of
the audio mixer and to get information about the audio mixer
and the audio device.
The audioctl command operates on the following data types:
device
An audio device, such as "audiohd#0". The subcommands
that accept this do so as an argument to an option -d.
If not supplied, then the default audio device is assumed.
Any device node associated with an audio device will work
as well, such as /dev/sound/0, /dev/dsp1, or /dev/audio.
control
A mixer control name, such as "volume".
value
The value of a control. The specific format depends on
the type of control. Monophonic values usually use a single
whole number between 0 and 100, inclusive. Stereo values
use a pair of such numbers (representing right and left
channels.) Boolean values indicate either "on" or "off".
Enumerations take a single value of one or more names.
file
An ASCII text file of control settings.
Options:
Each subcommand has its own set of options that it takes.
However, some subcommands support the special flag -v, which
indicates a request for more verbose output.
SUBCOMMANDS
The following subcommands are supported:
audioctl list-devices
List all the audio devices on the system.
audioctl show-device [-v] [-d device]
Display general information about a device.
audioctl show-control [-v] [-d device] [control ...]
Display the control setting values for the device. The named
controls are displayed. If no control names are provided, then
all control values are displayed.
audioctl set-control [-v] [-d device] control value
Changes the value of a control to the supplied value.
audioctl save-controls [-f] [-d device] file
Saves the current state of all mixer control values to the named
file. The command will abort safely if the file already exists,
unless -f is supplied.
audioctl load-controls [-d device] file
Restores previously saved state in the named file for all mixer
controls.
ENVIRONMENT VARIABLES
AUDIODEV If the -d and -a options are not specified, the
AUDIODEV environment variable is consulted. If
set, AUDIODEV contains the full path name of the
user's default audio device.
I expect there may be some questions or concerns about the new syntax. If they are not answered by the manual page above, please don't hesitate to ask me.
Monday, December 14, 2009
quick system identification
I am frequently needing to know "which system am I using". (I have trouble keeping track of which ones are which) -- i.e. is this the laptop, the Ultra 20, the whitebox, etc.
I had been using smbios a lot, but that's a bit awkward and non-portable to SPARC or systems that don't support smbios. Here's a quick script that can quickly tell me what model I'm working on. It works on SPARC and x86 alike. I have it stored in ~/bin/system -- it only works on Solaris of course. Feel free to borrow, reuse, whatever -- I hereby donate it to the public domain:
Sample outputs from a Toshiba laptop, a Sun V890, and a non-name Intel whitebox system:
While the format isn't terribly parseable, its great for giving a quick assessment about what type of machine I'm on.
I had been using smbios a lot, but that's a bit awkward and non-portable to SPARC or systems that don't support smbios. Here's a quick script that can quickly tell me what model I'm working on. It works on SPARC and x86 alike. I have it stored in ~/bin/system -- it only works on Solaris of course. Feel free to borrow, reuse, whatever -- I hereby donate it to the public domain:
#!/usr/bin/ksh
sysconf=`/usr/sbin/prtdiag | head -1`
biosconf=`/usr/sbin/prtdiag | head -2 | tail -1`
sysconf=${sysconf#*:}
biosconf=${biosconf#*:}
sysconf=`echo ${sysconf} | /usr/bin/sed -e 's/^ //g'`
biosconf=`echo ${biosconf} | /usr/bin/sed -e 's/^ //g'`
if [ -z "$sysconf" ]
then
printf "BIOS: %s\n" "$biosconf"
else
printf "%s\n" "$sysconf"
fi
Sample outputs from a Toshiba laptop, a Sun V890, and a non-name Intel whitebox system:
TOSHIBA TECRA M9
Sun Microsystems sun4u Sun Fire V890
BIOS: Intel Corp. BX97520J.86A.2777.2007.0805.1747 08/05/2007
While the format isn't terribly parseable, its great for giving a quick assessment about what type of machine I'm on.
Thursday, December 10, 2009
new aquarium lights
Some of you may know that I keep a salt water reef aquarium (210 gallon). Well, one of the challenges with such aquariums is lighting -- most corals need a large amount of high quality light to grow.
Most systems use metal halide lighting, which is extremely bright, and extremely hot. On my aquarium I've used three 250 watt MH lights, supplemented by a pair of 150 watt MH lights. (The 150s have been out for a while due to a bad ballast.) Yes, that's over a kilowatt in lighting. And MH lights have historically been considered some of the most efficient lighting -- i.e. most lumens or PAR per watt.
One alternative that has been up and coming has been LED based lighting. High power LED systems offer a lot of potential capabilities (although due to a patent problem that has put at least one company out of business and probably set the industry back at least five years, software based control of LEDs is only available on extremely expensive lighting systems -- systems costing thousands of dollars.
And until recently, inexpensive LED lighting simply did not exist.
I recently had cause to investigate this, as the bulbs in two of my 250s have died recently -- and I'm particularly unhappy with 250 watt MH lighting for several reasons: heat, problems with bulb fit (whoever designed the mechanical interface used for 250 and 400 watt double-ended MH lighting needs to quit mechanical engineering -- I've had numerous problems getting the bulbs to stay put in their "friction" based holding interface, and broken a couple of bulbs trying to increase tension so that they wouldn't fall out of the fixture), and power consumption (power in California is very expensive.)
Anyway, I'm happy to report that I found a good alternative. LED Wholesalers.com offers a 120 watt LED fixture intended for this application for just over $300. (They have an eBay store where the fixture can be purchased for under that, but with shipping and our ridiculous 9.25% CA sales tax, you'll still be over $300.) Compared to other options, this price is ridiculously low.
I put it on my aquarium yesterday, and comparing side by side with the remaining 250 watt MH fixture, I think the fixture "appears" to be about equivalent to a 250 watt MH. I've not measured it precisely, but the light appears about as bright as the 250 MH (the wife says brighter, but I'm not sure), although the spectrum is a bit bluer. If I had to guess I'd guess the spectrum to be somewhere around 18000K.
Anyway, I thought I'd post this here, since I think I'm one of the very early adopters of this. I'm going to monitor the health of my corals, and if it goes the way I hope it will, I'll be purchasing two more of these units to get full coverage over the entire aquarium. I might even purchase one for my 56 gallon tank. (I might supplement with another 100 - 200 watts of compact fluorescent at ~6500K though. That's still only about half as much power as my system previously consumed.)
I'll return to this post in the coming months to report my results as I get more experience with these lights.
Most systems use metal halide lighting, which is extremely bright, and extremely hot. On my aquarium I've used three 250 watt MH lights, supplemented by a pair of 150 watt MH lights. (The 150s have been out for a while due to a bad ballast.) Yes, that's over a kilowatt in lighting. And MH lights have historically been considered some of the most efficient lighting -- i.e. most lumens or PAR per watt.
One alternative that has been up and coming has been LED based lighting. High power LED systems offer a lot of potential capabilities (although due to a patent problem that has put at least one company out of business and probably set the industry back at least five years, software based control of LEDs is only available on extremely expensive lighting systems -- systems costing thousands of dollars.
And until recently, inexpensive LED lighting simply did not exist.
I recently had cause to investigate this, as the bulbs in two of my 250s have died recently -- and I'm particularly unhappy with 250 watt MH lighting for several reasons: heat, problems with bulb fit (whoever designed the mechanical interface used for 250 and 400 watt double-ended MH lighting needs to quit mechanical engineering -- I've had numerous problems getting the bulbs to stay put in their "friction" based holding interface, and broken a couple of bulbs trying to increase tension so that they wouldn't fall out of the fixture), and power consumption (power in California is very expensive.)
Anyway, I'm happy to report that I found a good alternative. LED Wholesalers.com offers a 120 watt LED fixture intended for this application for just over $300. (They have an eBay store where the fixture can be purchased for under that, but with shipping and our ridiculous 9.25% CA sales tax, you'll still be over $300.) Compared to other options, this price is ridiculously low.
I put it on my aquarium yesterday, and comparing side by side with the remaining 250 watt MH fixture, I think the fixture "appears" to be about equivalent to a 250 watt MH. I've not measured it precisely, but the light appears about as bright as the 250 MH (the wife says brighter, but I'm not sure), although the spectrum is a bit bluer. If I had to guess I'd guess the spectrum to be somewhere around 18000K.
Anyway, I thought I'd post this here, since I think I'm one of the very early adopters of this. I'm going to monitor the health of my corals, and if it goes the way I hope it will, I'll be purchasing two more of these units to get full coverage over the entire aquarium. I might even purchase one for my 56 gallon tank. (I might supplement with another 100 - 200 watts of compact fluorescent at ~6500K though. That's still only about half as much power as my system previously consumed.)
I'll return to this post in the coming months to report my results as I get more experience with these lights.
Subscribe to:
Posts (Atom)