Great Falcon-9 Launch!

SpaceX, one of our greatest hopes for a commercial manned space program, has achieved a huge milestone with the successful maiden launch of Falcon-9 with a Dragon capsule today. This is the craft that may one day soon be used for ISS resupply, and perhaps even crew transport.

Even as Obama shuts down the US governments manned space program, the commercial sector is picking it up. This is a momentous day.

Congratulations to Elon Musk and the rest of the team at SpaceX!

audioens in VMware...

So, we have not had audio in OpenSolaris within VMware since... well, ever.

I've been doing some investigation. I'm seeing a situation where the VMware emulated audioens device behaves rather differently from the real hardware.

For one, it seems to insist on using real interrupts. In particular, the sample count registers do not appear to be updated unless one receives and acknowledges an interrupt. (By toggling the interrupt enable bit.) This means that this virtualized device will never be able to run "interrupt free" like the other audio devices (or real audio hardware).

For another, it appears that the audio device has some weird dependency on the relationship between the size of the audio buffer, and the interrupt rate (the number of samples at which to interrupt). Using different values gives, strange results.

Finally, I cannot, for the life of me, figure out how to cause the device to actually trigger an interrupt. I've been able to make some progress by simulating a soft interrupt at 100Hz, which is how the interrupt free framework works anyway, but from what I can tell, nothing is causing a real interrupt to be delivered. This is really strange. (Without this functionality, I am able to process audio at a reasonable rate, but it still stutters, and is not really suitable for real-world use.)

My guess is that the virtual device has some weird dependencies that we don't know about. For example, while the hardware spec identifies registers as being 8, 16, or 32 bits wide, and we use those at the right bit widths, other FOSS drivers all seem to just blithely use 32-bit wide accesses. Is there a hidden dependency here?

If any reader from VMware is seeing this, and can help me understand the behavior of the simulated device, I'd appreciate it. I'd like to make audio work in this environment, if possible. I'm pretty close, I think.

Actually, it seems kind of crazy that these environments emulate such complex audio hardware. (For example sophisticated sample rate conversion hardware.) Much better, I think, would be a simple paravirtualization driver that just exposes a simple buffer and some control functions. If someone at VMware wants to work on that as a solution, I'd be happy to help with Solaris support for it. Since these things run isochronously, and chew up a fair bit of cpu when they run, such a solution would probably be quite useful. (For example, its silly to perform multiple sample rate conversions in software... instead we could express native sample rates via a PV driver to the guest, ensuring only a single SRC operation is performed appropriately in the guest operating system.)

Well *That* Didn't Work Out So Well

You may recall my recent blog post about Windows 7 being surprisingly usable.

Well, I have to recant here.

I used Windows 7 for about a week and half. While it *worked*, it was a pleasure to use. But after three BSODs in just that week and half, I have abandoned it. I'm now running Ubuntu. (Why not OpenSolaris? Because I need the ability to host VMware and Skype, and I can't do that natively on OpenSolaris -- yet.)

Sure, I could have called up support -- but Microsoft support is provided by my computer manufacturer, and I didn't feel like spending 3 hours on the phone dealing with tech support while they tried to triage my problem. In the end, it was simply faster and easier for me to reinstall with Linux, even allowing for the time it took to download the media.

Sure, the problem might have been my virtualization software, or maybe it was a shoddy audio driver, or maybe it was brokenness in my graphics driver, or maybe it was the 3rd party antivirus software (which begs the question-- why doesn't Microsoft ship with builtin malware protection -- you'd think given all the heat that they've taken over this that they would have figured out that they *need* a solution here that doesn't involve 3rd parties...)

The "automatic solution finder" that Windows 7 ships was completely unhelpful, it didn't find any links. Google was not much help either... with everything to buggy hardware, drivers, and even overtemp problems being cited as root causes.

I'm sure that tech support would have had me running around in circles trying to solve the IRQL_NOT_LESS_OR_EQUAL blue screen. (I'm guessing, from my kernel expertise, that this is probably an assertion fault somewhere that an IRQ level is set unexpectedly high or low -- exactly the kind of problem I know how to fix in Solaris.) Probably plugging and unplugging hardware, unloading and reinstalling drivers and maybe other software, and generally burning an unmentionable amount of my precious time. Especially given that the hardware tech support I'd have been routed to was unlikely to have any real software clue (which is where I think this problem was most likely located.)

Again, faster and simpler to just dump the busted OS, and load something else.

And, with Linux (or any other FOSS), I have at least a fighting chance of trying to debug the problem myself. Sure, my kernel-fu is substantially higher than average joe home user, so my leanings are more towards something I can troubleshoot myself. But, I will say this, so far I've not had a panic ("oops" in Linux parlance) yet in the past four days that I've been running Ubuntu LTS 10.04 (even though I'm running the "not recommended for general desktop use" 64-bit edition.)

Microsoft, if you see this post, I hope you'll learn something from this.

Last Day

So, today's my last day as a Sun^WOracle employee.

While I'm excited to be starting at Nexenta, I want to reiterate what I've already said, which is that I've really enjoyed working with the great folks at S'noracle, and that they made this decision to leave quite difficult. Its been quite a ride over the years, and its been fun and exciting. Thanks everyone!

Of course, my old e-mail address(es) at Oracle won't reach me after about noon today.

To reach me for matters pertaining to OpenSolaris, gdamore@opensolaris.org will continue to work. For matters pertaining to my new employer, Nexenta, you can use garrett@nexenta.com. My personal e-mail address of garrett@damore.org remains unchanged. Now please standby while I go reinforce the spam filters...

New Computer

As part of the process of changing employers, I needed to get a new computer for the new job (and return the old desktop to Oracle.)

I wound up picking this one... I didn't seem to be able to build it any cheaper (as of the date of this post) myself. And guess what... someone goofed! Instead of the 3 GHz Core i7 950, it came with a 3.2 GHz Core i7 960. Bonus! (Other goofs relative to the ad: the system has 9 GB -- but that's spelled out in the details, comes with a black aluminum chassis, and ships with a cheap logitech keyboard.)

I'm still using the stock load of Windows 7, and I'm both surprised (and maybe a bit embarrassed) to admit that the Windows environment (especially when replacing IE with Chrome) is actually quite nice -- fast and usable. Maybe running this environment (and running OpenSolaris in a VM) might not be so bad after all! (Ok, I'll go find some soap to wash my mouth out for blaspheming....) If I do this, besides being able to use Skype for work, I'll be able to use my Phoenix RC flight simulator without having to resort to borrowing the wife's computer...

Engines arrived for Squat yesterday

The Squat is a 4" diameter short high power rocket with a 54mm engine mount. My engines, 54mm hardware (including the higher end Aeropack retainer), and 38 mm adapter arrived from What's Up Hobbies yesterday. Timothy's going to fly it at the LDRS mass launch on a G67 redline -- this will be his first reloadable engine. Later that day I'll fly it on an I140 skidmark, which represents both my first 54mm engine, and my first Caeseroni engine.

Timothy and I put the rocket together last night; I must say, the higher end metal hardware and thicker fins on this rocket are definitely a step up even from the LOC IV I flew previously on my Level 1 flight (go to about 1:30 in the video link -- I haven't figured out how to edit the video file yet).

I also received the propellant for the J350W, which I'll be flying in my LOC IV as part of my Level 2 certification attempt. OpenRocket says the LOC IV will be approaching 700 mph with this particular engine! Guess I will be glassing the fins on it to help strengthen them for transonic speeds. (I'm open to alternative suggestions from the experts, as well.)

LDRS is going to be fun, indeed!

Greener pastures

I've recently made a major decision -- I'll be leaving Oracle. My last day as an Oracle employee will be on May 21, 2010. Leaving such a great group of people at Sun will be difficult indeed.

However, I hope to be able to continue as a significant contributor to the OpenSolaris community, as I'll be joining the team at Nexenta. At Nexenta, my responsibility will be to lead a group of engineers working on the OpenSolaris kernel. As such, I'm excited that I'll be able to continue to work on finest operating system kernel on the planet, and I look forward to further collaboration with some of the best software engineers on the planet.

My first day at Nexenta will be on May 24, 2010.

Going up to SF bay area

Its been a while since I've been to the Bay Area. I'm going up for two days, which is a shade longer than I usually go for. Part of the reason is to make sure I meet with folks in the Bay Area that I otherwise don't see. I'll be up Thursday and Friday April 29 and 30 -- and I expect I'll be at MPK most of that time. Anyone who wants to chat, please let me know.

audiocmihd driver (Asus Xonar cards)

Some people have been asking me about this driver. (Asus Xonar cards are fairly high-end high definition cards using the CMI 8788 chip.)

I've finally gotten the code reasonably cleaned up, and converted to my interrupt free audio framework.

I'll probably start a case to get this integrated into late b137, or b138. Mostly its just running a bunch of tests at this point.

One problem I have is that I only have Xonar DX1 cards. (PCI.)

If someone is able to help me qualify the driver with build 137 (or a nightly build) of ON, please let me know. The more I can get this driver tested, the sooner I can get it integrated into OpenSolaris.

Interrupt Free Audio

Today I integrated "interrupt free audio". This set of changes, including some other changes, represents a substantial simplification in the DDI for audio drivers.

The typical audio driver no longer needs to worry about interrupt handlers. On average, about 300 lines of code (or about 10-20% of complexity for typical drivers) was removed from each audio driver.

Furthermore, many audio drivers (for example audio810) are able to run completely lock free, since the audio framework provides synchronization for certain operations. (Operations against each audio engine are synchronized, operations against audio controls are synchronized as a whole, and everything is synchronized against suspend/resume functions.)

Even better, these changes enable some new advanced features that will be used for Sun Ray, virtualization, and hotplug support in the future.

Oh yeah, and since the asynchronous processing now happens as part of the regular timer interrupt, it means that system CPUs can remain in deeper C states for longer, even while playing audio. So, we should have an improvement on system power consumption (admittedly I've not measured this.)

There will be more stuff related to audio in the future, stay tuned.

"Legislative Sleight of Hand"

I normally have avoided using my blog as a soapbox for my political beliefs. However, I simply cannot remain silent on recent events in the House of Representatives (United States for foreign readers.

No matter what your position is on the health care reforms under consideration, everyone should agree that the reforms are sweeping; perhaps some of the most significant legislation that will affect nearly every American we've seen in quite some time.

House Democratic leadership, knowing that the measure is unpopular with many voters (and hence House Democrats may be unlikely to "vote the party line" to avoid a backlash in their constituencies) are planning a move that is even more offensive than "reconciliation".

While I'm a Republican, and generally opposed to nationalization of 1/6th of our economy, I find far more offensive that the House leadership (particularly Ms. Pelosi) would consider a move that so boldly disenfranchises the people of this nation.

This is a crime, if not against the law, then certainly against the spirit of democracy upon which our country is founded. If health care reform is to be passed, then it should be done with a regular vote where the politicians who vote for it are required to be accountable for those votes (and vice versa, by the way).

If it passes without such a vote, then it will go down as one of the greatest failures of "representative democracy" in history.

Why We Need a Human Spaceflight

Space aficionados may be aware that President Obama has canceled the previous administration's "Vision for Space Exploration", which consisted of the Constellation program including Ares I, Ares V, and Orion. This has been fairly well covered in the mainstream media.

Critics of the Constellation program raise some significant and relevant objections to the Constellation program.

However, I strongly believe that as a nation, we need a national space program that includes human spaceflight beyond low Earth orbit. The cancellation of Constellation, while perhaps with good cause, has left our national space program with a vacuum -- the lack of a heavy lift vehicle, and lack of any vision, would effectively constrain human exploration to LEO for a generation. Furthermore, it significantly constrains the kinds of activities that we can perform in LEO.

Its my belief that this is short-sighted in the extreme.

We need a space program that includes vehicles with the ability to loft large payloads into orbit. Projects like the International Space Station, and further commercialization of space, are only possible with the ability to loft a significant payload into orbit.

We also need to plan for human exploration beyond our front porch. While many people argue that sending robotic explorers is less risky, and far cheaper, the idea that we can or should abdicate all future space endeavors to robotic missions is actually offensive.

Robots won't inspire a generation of students to continue to excel at math and science. Robots can't stand in as national heroes. And robots alone won't help develop the enthusiasm required for the general public to continue to want to invest in space and space technologies. Robotic exploration is mostly a solved problem -- many new technologies that are necessary for human space travel will simply not be invented or invested in, without the "problems" to solve that are involved in human space exploration.

I'm from a generation of kids who viewed astronauts as near personal heroes; I dreamed, and still dream, of being able to see our planet from space itself one day. I dream of the days when human kind steps beyond just Earth, and has outposts on the Moon, Mars, the asteroids, and perhaps other interesting places in the solar system. And someday beyond.

My own son dreams someday of being an astronaut and visiting Mars. Unfunded as it was, at least VSE allowed a glimmer of such a hope. Obama has killed that hope, and maybe the dreams and hopes of thousands or millions of other like minded kids.

Fortunately, there is a proposal that would revive these dreams, and allow us to retain a national heavy lift capability, retain a lot of the knowledge and expertise that we acquired with the successful STS (space shuttle) program (even reusing a significant amount of the materials and technology), and allow for a "way forward" that would allow us to get beyond LEO and go to interesting places elsewhere in the solar system. The DIRECT v3.0 proposal is IMO the best way forward; it allows us to have our cake and eat it too -- giving us all the heavy lift capabilities that we need, minimizing the significant impact on our economy that both the Constellation program, and the cancellation of the STS and Constellation programs, create.

I firmly believe that we are on the cusp of a major economic shift, where commercialization of space may play as important a role in the coming decade or two as the Internet has played in the previous two. The question is, will we as a nation continue to develop that potential, or will we let it slip away, to be picked up by India, China, or Russia?

Yes, I'm an American. And I believe that it is important for America to be a leader in the exploration and utilization of space. Ultimately, I believe that "planting flags" is much more important than the proponents of solely robotic exploration would have us believe. Someday people will visit Mars. Will America be there, or will we just be an observer while one of the Asian nations celebrates a major achievement?

ON IPS surprisingly easy

So I have an EOF RTI that was in queue when the ON IPS integration happened last night.

Of course, this totally whacked my packaging changes, and I had to modify them. Making the changes was quite easy. Here's the old, and the new version of the changes. Its actually less files to update under IPS.

I was dreading retesting. Dealing with distro construction sounded "painful".

I needn't have worried. In the tools directory there is this neat tool called "onu" (on-update I guess?)

I had to load a machine with b133 to set up a baseline, but we have a nice way to do that internally via our internal infrastructure and AI. It boils down to running one command on an install server than doing "boot net:dhcp - install" at the OBP prompt. (Yes, this is a SPARC system.)

It took a little bit for it to install, but less than an hour.

Then, after rebooting and getting the initial settings on the system, it was just a simple matter of "onu -d ${ws}/packages/sparc/nightly-nd" to update it. This took a while (20-30 minutes, I wasn't counting). Eventually the system was up and ready for business. Easier than bfu. Amazing.

Thanks to the IPS team! I can't wait for bfu to finally go away.

Funny Ancient Software

I just found out that Ubuntu has been shipping (since version 6.06 -- Dapper Drake I think it was called?) and apparently included all the way into forthcoming 10.x LTS version) a program I wrote nearly two decades ago as a student -- vtprint -- and yes, that link points to manual text I wrote way back when.

(This program, "vtprint", was for use with printing from a UNIX shell prompt, when you don't have a better way to move files around. Back then we used commands like "kermit" to connect to a UNIX server from our PC over a 2400 or 9600 baud modem -- and well before PPP or even SLIP.)

I haven't used vtprint since about 1995, but its funny to still see it kicking around. Too bad the docs still have an old e-mail address for me at SDSU.... I guess nobody has needed a bug fix for it for some time.

Congratulations to BMW Oracle Racing

If you're involved in the sailing community, you'll already know that Larry Ellison, who's now ultimately my boss, had put together a team to challenge the America's Cup. They won this weekend, bringing the America's Cup back home to America, and I'm enormously proud of Ellison and his team, both as an American, as a sailor, and -- now -- as an Oracle employee.

Open Development

Note: I'm posting this on my personal blog, which as always is a reflection of my own thoughts and in no way represent any official policy from my employer (whoever that may be).

Now that we former Sun employees (for the most part) are now part of a larger company, there have been some questions about how much of the trend Sun had made towards open development will continue (particularly where Solaris/OpenSolaris is concerned.)

(I want to separate the concern of Open Source -- where source code is made available for products after they are released -- from Open Development -- where the product is developed in the open.)

Many of us who were part of that acquisition are wondering the same things. Officially, the word is "no changes in what we're doing", but unofficially there's an atmosphere that our new employer places a greater emphasis on commercial profitability and a lesser emphasis on things like "including the community."

Speaking abstractly, there are risks to any open development effort, particularly when the effort is intended to be supportive of a commercial endeavor. The risks range from enabling competitors with early information, to forestalling customer purchases of bits today as customers wait for the new feature that's being developed in the open, to simply diluting the impact that "surprise" delivery of a new product or feature can make.

Certainly there seems to be some evidence that Oracle may have a greater concern about the costs and risks associated with "early disclosure" than Sun did. No matter how passionately one may believe in Open Development, nobody can deny that there are real costs and risks associated with open development.

So the challenge for Oracle is to figure out what the balance is that makes commercial sense.

Ultimately profit is the primary responsibility of any publicly traded company.

The challenge for the community is to figure how to provide commercial justification to Oracle for the open development that the community likes to see.

If you want to retain truly Open Development (which includes public posting of webrevs for stuff developed internally, open ARC reviews, and public design discussions on mailing lists and similar fora) of Solaris and OpenSolaris, this is call out to you.

Have you or your company made a significant Sun purchase? Has open development (as opposed to open source) influenced your decision? How and why? Will open development influence future purchasing decisions? If you can, put a number to the value of open development, and provide that information to your sales reps or post it publicly.

The decision makers need to see value in the practice of open development if they're going to continue to support it.

Again, I'm only talking about open development, not about open source.

As an aside, I don't think statements coming from community contributors without the support of purchasing dollars are likely to carry much weight with Oracle decision makers. I believe that if you look at the contributions from the community-at-large in OpenSolaris, you'll find that the meaningful contributions have been fairly small and generally of little commercial interest, and have always required additional engineering expense from Sun. So "leveraging" the community for development has not, IMO, been a gamble that has yielded dividends -- at least not from the perspective of either Sun or Oracle.

Scalability FUD

Yesterday I saw yet another argument about the Linux vs. Solaris scalability debate. The Linux fans were loudly proclaiming that the claim of Solaris' superior scalability is FUD in the presence of evidence like the Cray XT class of systems which utilize thousands of processors in a system, running Linux.

The problem with comparing (or even considering!) the systems in the Top500 supercomputers when talking about "scalability" is simply that those systems are irrelevant for the typical "scalability" debate -- at least as it pertains to operating system kernels.

Irrelevant?! Yes. Irrelevant. Let me explain.

First, one must consider the typical environment and problems that are dealt with in the HPC arena. In HPC (High Performance Computing), scientific problems are considered that are usually fully compute bound. That is to say, they spend a huge majority of their time in "user" and only a minuscule tiny amount of time in "sys". I'd expect to find very, very few calls to inter-thread synchronization (like mutex locking) in such applications.

Second, these systems are used by users who are willing, expect, and often need, to write custom software to deal with highly parallel architectures. The software deployed into these environments is tuned for use in situations where the synchronization cost between processors is expected to be "relatively" high. Granted the architectures still attempt to minimize such costs, using very highly optimized message passing busses and the like.

Third many of these systems (most? all?) are based on systems that don't actually run a single system image. There is not a single universal addressable memory space visible to all processors -- at least not without high NUMA costs requiring special programming to deliver good performance, and frequently not at all. In many ways, these systems can be considered "clusters" of compute nodes around a highly optimized network. Certainly, programming systems like the XT5 is likely to be similar in many respects to programming software for clusters using more traditional network interconnects. An extreme example of this kind of software is SETI@home, where the interconnect (the global Internet) can be extremely slow compared to compute power.

So why does any of this matter?

It matters because most traditional software is designed without NUMA-specific optimizations, or even cluster-specific optimizations. More traditional software used in commercial applications like databases, web servers, business logic systems, or even servers for MMORPGs spend a much larger percentage of their time in the kernel, either performing some fashion of I/O or inter-thread communication (including synchronization like mutex locks and such.)

Consider a massive non-clustered database. (Note that these days many databases are designed for clustered operation.) In this situation, there will be some kind of central coordinator for locking and table access, and such, plus a vast number of I/O operations to storage, and a vast number of hits against common memory. These kinds of systems spend a lot more time doing work in the operating system kernel. This situation is going to exercise the kernel a lot more fully, and give a much truer picture of "kernel scalability" -- at least as the arguments are made by the folks arguing for or against Solaris or Linux superiority.

Solaris aficionados claim it is more scalable in handling workloads of this nature -- that a single SMP system image supporting traditional programming approaches (e.g. a single monolithic process made up of many threads for example) will experience better scalability on a Solaris system than on a Linux system.

I've not measured it, so I can't say for sure. But having been in both kernels (and many others), I can say that the visual evidence from reading the code is that Solaris seems like it ought to scale better in this respect than any of the other commonly available free operating systems. If you don't believe me, measure it -- and post your results online. It would be wonderful to have some quantitative data here.

Linux supporters, please, please stop pointing at the Top500 as evidence for Linux claims of superior scalability though. If there are some more traditional commercial kinds of single-system deployments that can support your claim, then lets hear about them!

Missing audio packages

I have learned that at least two packages, SUNWaudioemu10k and SUNWaudiosolo, are not part of the "standard" ("entire?") install of OpenSolaris b131. If you're looking for either of these, you should do "pfexec pkg install SUNWaudiosolo" or "pfexec pkg install SUNWaudiosolo".

Hopefully we'll get this sorted out before the next official release.

Update: Apparently (according to the expert I talked to) this problem only affects systems updating with pkg image-update. If you install a fresh system, the audio packages should be installed.

System board for ZFS NAS

I'm thinking about creating a home storage server, like many, and I want it to be performant enough to host work spaces for compilation over NFS, and efficient enough to reduce my current power consumption somewhat.

I'm thinking of a new Intel D510 system board, and looking at several, I found a board from Supermicro that looks ideally suited to the task. Does anyone else have experience with this board? It looks like its all stock Intel parts, so it should Just Work.

I'm thinking that with 4 or more SATA drives combined with RAIDZ, and dual Intel 82574 gigabit Ethernet (which I could use in an Ethernet link aggregation), I should be able to get excellent performance. (I might even set up jumbo frames, to further bump NFS performance -- if they really are 82574's then they support up to 9K MTU).

Kindle Converts a Skeptic

Recently I bought my wife an Amazon Kindle (the new international unit), at her request. Personally I was rather skeptical -- trying to read book material on computers, even laptops or netbooks, has always felt very awkward to me. I always believed that there was something about holding a paperback (or even a hardback) which would never be replaceable by technology -- maybe for others, but at least not for me.

I have to recant. Debbie has read something like a dozen novels already on her unit. I decided to try it out... and I have to say, I was surprised. I was reading H.G. Wells' War of the Worlds (not for the first time of course), which was a free download, and wow, was I surprised. After 10 or 15 minutes of reading, I almost forgot I was holding something in my hand that isn't printed paper. (The form-factor, which is quite similar to a book, works quite well here. I don't think I'd like the larger DX, as it would destroy the "illusion" of reading a paper back book.)

Not only did the technology not "get in the way", the reading experience was actually more pleasurable, largely because I was able to bump up the font size up to a more comfortable reading level. Last night I read about 1/3 of the book, before I got too tired, but I'm sold on the concept -- and I was a die hard skeptic before.

I don't think I'd like to use it for other things ... but for the primary purpose of reading novels, it settles in quite nicely bringing some technological advantages without letting the technology get in the way of reading.

Will Apple's newer iPad compete here? I'm skeptical. The Apple product is a fancier device, with its backlit screen, and probably will feel more like a hybrid between a laptop and an iPhone (of course I still have an ancient model of phone that is used pretty exclusively for making phone calls -- call me a Luddite.) I suspect that the combination of screen glare, snazz, and lower battery life (iPad users will need to be lot more cognizant of their current battery status), means that its going to be a poor replacement for a Kindle, and an even poorer replacement for the printed materials that the Kindle is meant to replace.

When I go on my round-the-world sailing trip (not any time soon!), would I want a Kindle with me? Absolutely (or something similar) -- along with a solar or wind based charging system. (Product idea... a case for the kindle that integrates photovoltaic solar charging system, so your Kindle is always charging when its closed.)

An iPad? Not likely -- if I'm going to be working or sending e-mails, sure, but then one of the netbooks is probably a better option.. with a "real" keyboard.

Reprehensible behavior from a monopoly

Misbehavior stemming from lack of competition is apparently not unique to the IT industry.

I saw this post today, and couldn't believe it. And then a bit of additional research shows this is not unique -- a number of people complained about actions on the part of Greyhound that would never be tolerated in market where there is true competition.

Forcing a grandmother to wait out in cold, while there's still snow on the ground, may not be in violation of the letter of the law, but it is certainly in violation of the basic tenets of human decency, and the management at the Memphis location showed they have none.

Its been over ten years since I've ridden a Greyhound (or any other long-haul bus for that matter), and after reading this, I am unlikely to ride another Greyhound again. Instead I'll stick to air transport where lively competition means that even the worst airlines understand that they have to at least pretend to care about their customers.

If you're reading this and thinking about taking a Greyhound somewhere, don't.

While there may not be much competition for Greyhound for long-haul ground travel, there is at least some. And there is always air transport for those able to use it.

I'll be interested to hear if Greyhound corporate does anything to fix the problems they obviously have. A good start would be firing most or all of the staff at their Memphis location (especially the management and security guard in question) and refunding the tickets of each of the passengers who were stuck there.

Its Official

I'm no longer a Sun Microsystems Employee, since Sun no longer exists. Hopefully I'll get to keep my job at Oracle, but I've not seen any e-mail yet. I expect I will before day's end.

Auto Install Finally Working For Me!

Some of you may know, I've been struggling (and failing) to make auto install work for me. I've had challenges, because my network is not routable, and due to other issues (bugs!) in OpenSolaris Auto Install.

However, it seems that I've finally hit on a successful recipe. I want to record this here for others.

First, in order to use AI, you will need your installation server to be running a recent build of OpenSolaris. The release notes indicate b128 is sufficient. I just ran "pkg image-update" to update to build 131. If you fail to do this, there won't be a warning at all, its just your clients will simply not boot.

The next thing you'll need to do is download a full-repository and the AI image.

Unfortunately, there are not public versions of the full repo ISO file available that are "current". (No, I can't get you a copy, and no I don't know why they haven't posted a more recent update.) Hopefully this problem will be corrected soon.

Setting up the local repository can be done following these directions. (Note that you will have to change the paths to reflect your system. I stash ISO images in /data/isos, and install images under /data/install. These are separate ZFS filesystems.

# where do ISOs live, without leading /
ISODIR=data/isos
# where does the repo live, without leading /
REPO=data/install/os131_repo_full
# AI service name to use
NAME=os131_x86
# parent directory for installations, without leading /
INSTDIR=data/install
# port to use for install server
PORT=8181

mount -F hsfs -r /${ISODIR}/osol-repo-131-full.iso /mnt
zfs create -o compression=on ${REPO}
svccfg -s application/pkg/server setprop pkg/inst_root=/${REPO}
svccfg -s application/pkg/server setprop pkg/readonly=true
svccfg -s application/pkg/server setprop pkg/port=${PORT}

You'll need to edit the ${REPO}/cfg_cache file, changing the origins entry to match your system. I used a value like this:

origins = http://192.168.128.11:8181

Then you'll want to use installadm to setup an initial boot service. Here's the recipe I used:

zfs create -o compression=on ${INSTDIR}/${NAME}
installadm create-service -n ${NAME} -s /${ISODIR}/osol-dev-131-ai-x86.iso /${INSTDIR}/${NAME}

Now you need to change the default manifest file. This is the tricky part, that IMO was not terribly well explained anywhere else.

cp /${NSTDIR}/${NAME}/auto_install/default.xml /tmp

Then edit default.xml file in /tmp, changing the value of "main_url" to point to your server. I used a value like this:

<main url="http://192.168.128.11:8181" publisher="opensolaris.org"/>

Then apply this manifest to the default manifest:

installadm add -m /tmp/default.xml -n ${NAME}

Finally, I did some tweaking in my DHCP configuration. I have a macro for each service name, that provides the defaults. For example, my "os131_x86" macro looks likes this:

Include pepper
BootFile os131_x86
GrubMenu menu.lst.os131_x86

My "pepper" macro (pepper is the name of my server) sets some shared defaults, but most especially it sets BootSrvA to the IP address of the server (192.168.128.11 in my case.)

Then I just configure individual addresses for which ever version of OpenSolaris (or SXCE) I want to install using the the correct configuration macro. (For SXCE there are very different DHCP options to use. Also the SPARC version of OpenSolaris uses different options as well.)

Six Years & Counting

Its hard for me to believe that six years ago today at this hour in the morning I was getting myself ready to meet my bride. We had a wonderful wedding on the beach in front of the Del Mar powerhouse in San Diego, with our friends and family in attendance.

Looking back, its been the best six years of my life. I've truly been blessed. I'm looking forward to spending the next sixty together with my beautiful bride Deborah.

Interesting device driver work

So there are a couple of "closed" drivers that are not part of OpenSolaris, and might never be because of redistribution restrictions. However, this represents an opportunity for an enterprising software engineer to contribute. The drivers are

glm - Symbios 53x810 and similiar devices
qus - QLogic ISP 10160 and similar devices
adp - Adaptec AIC 7870P and similar devices
cadp - Adaptec AIC 7896 and similar devices

There are open source drivers for these from FreeBSD and NetBSD, which could be used as a starting point for a port. I'd probably be interested in trying one of these out myself, if time allowed -- but alas it does not, my plate is already quite full.

The best part of these drivers is that there are few, if any, "political" or "business" restrictions on integrating replacement drivers. Indeed, at one point recently each of these was considered for an EOF simply because they weren't considered strategic anymore. (The EOFs were rejected, but these will only be delivered via an extras repository or somesuch.)

So, what are you waiting for? This is a good opportunity to learn about SCSA, and provide us with superior replacement drivers. (The glm replacement looks like it could be done in as few as 2 or 3 KLOC; that is all the NetBSD version of the driver uses.)

A "modern" elxl driver

I undertook this past weekend an effort to "port" the NetBSD "ex" driver to GLDv3, as an open source replacement for "elxl".

It took a bit over 2 days.

The new sources are on line in a webrev format. I'd really appreciate feedback. I'm hoping to integrate these changes soon (and I could use help with testing!)

This new version, apart from being Open Source, also has with it:

1. Full support for VLANs (including full-MTU frames)
2. Full support for link notification on twisted pair (and hopefully also fiber)
3. Full integration with Brussels for MII and media selection.
4. Full support for Suspend/Resume (S3)
5. Full support for Quiesce (fast reboot)
6. Support for additional devices

It can also be extended fairly easily to support Cardbus and MiniPCI variants, and hardware checksum offload. (The checksum offload part is basically written and #ifdef'd out until I find a newer card to test with personally.)

What is missing is "automatic" media selection based on active probing. The old Solaris driver would "autoselect" which port (BNC, AUI, twisted pair) to use based on some active probes to look for link. These were rather complex, and not something I could take from the closed driver. These days everyone just uses twisted pair anyway, right? (The fiber and TX4 cards don't offer any choices, so there is no probing needed for them.)

If you have such a COMBO card, you can force the media using a new "driver private" property called "_media", which you can set to various values. See the driver sources in the webrev for more information.

I've done enough work on this driver that there is probably as much of my own code in it (at least) as was in the original NetBSD code. Nonetheless, I'd like to than the NetBSD Foundation for making these sources available.

I'll be posting binaries soon, stay tune. (Sun internal users can grab the binaries from /net/temecula.sfbay/data/work/gdamore/yge/yge/usr/src/uts/intel/elxl/ -- at least for now. That path is likely to work until I get the code integrated.)

Out with the Old, In with the New

Happy New Year (2010) everyone!

I thought I'd take a second to reflect on the accomplishments of the past year, and look forward to what I think is in store for my contributions to OpenSolaris this year.

It's hard to believe that I've been a member of the OpenSolaris community for over 5 years now. (I was a pilot member.)

Undoubtedly this past year my biggest contribution to OpenSolaris was the new audio framework (Boomer) and many new audio drivers.

I did a lot of other work besides, including a bunch of work on NIC drivers (including a new common MII framework and the yge driver which supports Marvell Yukon 2 parts), and various changes to the SDcard framework.

I've also developed a device driver for a very interesting (and very high performance) hybrid storage device (which won't be integrating for non-technical reasons), and a new storage framework for block oriented storage devices. (This framework, blkdev, will be integrating once we're out past the build restrictions.)

I also did a lot of cleanup of legacy and stale code, which hopefully reduces the install foot-print and shrinks compile times somewhat.

2009 was also the year I became a full voting member of PSARC, and I was privileged to serve 3 months as PSARC chair. (The chair rotates amongst all active PSARC members on a roughly quarterly schedule.)

This past year is also the year that I became the top contributor to ON in separate integrations, since the OpenSolaris project started, at least as reported by ohloh.net. (Note that the statistics only cover the open portion of the ON consolidation.)

So what's coming up in the next year? Here's what I expect to be working on:

• 10GbE ethernet for Mellanox ConnectX devices (hermon). This is probably my top priority at the moment. The work is largely being done by Mellanox, but I'm the Sun engineer ultimately responsible.
• Sun Ray audio. This is one of my biggest priorities. We want a Boomer driver for Sun Ray audio, bringing in-kernel mixing to Sun Ray appliances.
• Interrupt-less audio. This is a major rethink of the way we process audio in the kernel, and reduces a lot of complexity in device drivers and is an enabler for several other new features. The code for this is done, so expect an integration in b135 or 136.
• Better audio hotplug support. (Especially for USB audio.)
• Better audio virtualization support (especially with Trusted Extensions.)
• Additional audio device support. (Asus Xonar, via audiocmihd, for example.)
  
• Integration of blkdev, and hopefully faster, simpler, better kernel support for simple block oriented flash-storage devices. (I.e. those devices that don't natively understand the SCSI command set.)
• Fixes for a variety of network device driver bugs. I've already got a few of these changes queued up.
• Broader support for the MII framework in other NIC drivers. (I have changes queued up for rtls, already, for example.)
  
• Further cleanup of not-needed legacy code.
• Continued contributions and participation at PSARC.
• Support for SDXC card media (and possibly also development of exFAT filesystem code, dependent on licensing concerns.)
• Possibly work on various track pad bits, depending on time and resource. (Synaptics support finally?)

There are probably other things that will happen, and I'm sure that our new owners at Oracle (assuming that the deal gets finalized, which looks almost unstoppable at this point) will have some ideas about how my priorities will be spent.

One thing is that I feel very privileged to have been able to work on the OpenSolaris code base, and to continue to be able to do so. I often reflect that its amazing that I get paid to do this work -- I think I'd be far less productive if I didn't enjoy my job so much. When my management asks me to take a break and do something fun for a change, I think he has a hard time understanding that for me hacking on OpenSolaris code is fun. I genuinely hope that I continue to be so privileged for the foreseeable future.

Shortest Audio Driver?

Using my new framework changes (which have not integrated yet), along with changes in the framework designed to simplify drivers by providing certain ordering guarantees, I've been able to shrink a lot of the audio drivers (and in the process have also found a number of small bugs that I think ultimately were responsible for jitter and corruption). Ths simplest driver so far -- audiovia97, which is about as simple an AC'97 driver that can be, is only 686 lines. Several others are now less than 800 lines, and no longer need any locking primitives. As far as device drivers go, this is minuscule. In fact, I challenge anyone to identify a driver that fully services any "real" piece of hardware, fully supporting said hardware, in fewer lines of space.

(And in case you are wondering, a significant fraction of those 686 lines of audiovia97 is still comments and whitespace and even the 30+ lines of boilerplate -- the driver is written to be as readable as reasonably possible, at least to other device driver or kernel engineers.)

The forthcoming audio DDI will be about the easiest DDI for any kind of device driver in Solaris. Contrast that with the pre-Boomer SADA framework, which one could argue had the most cumbersome and painful DDI for non-nexus devices.

Hopefully the simpler framework will make it easier for others to provide new drivers, and result in far fewer "bugs" in the drivers. (And of course, the smaller drivers will result in more efficient use of kernel memory as well, although you are unlikely to notice that particular difference.)

(However, the complexity of the audiohd driver remains a serious thorn in our side -- and unfortunately the problem is made worse by a far-too-complex specification for the Intel HD audio system That said, the complexity in audiohd stems largely from offering too many codec configuration choices, and not from the part of the driver responsible for actually transporting audio data.)

Do you use IDN? (E10K inter-domain networking)

I was looking, and one of the legacy device drivers we still support is the Enterprise 10000 Inter-Domain Network. This is a memory based interconnect network that emulates an ethernet.

It occurs to me that we could update this driver to use the GLDv3, which would potentially have a significant performance boost for it. (Really, a better solution would be to create a new low level MAC type for it Nemo, but clearly that's probably too much pain for a product that is no longer shipping).

Anyway, I'd be interested to hear of any sites which use IDN on E10K class systems, and would like to see this (already high performance) network get another boost.

The other legacy SPARC drivers which are not GLDv3 compliant are:

• cassini (ce) (probably never going to be GLDv3, but that's another matter)
• gem (ge) -- this one could be easily converted *if* I had fiber cards to test it with, although there are political concerns as well. (It uses the same MAC controller as "eri", which I converted a while back.)
• scman -- This is used for the Starcat (E15K/25K) connection between the SC and domains. It is also based on the eri driver, but has some special hacks. I'm a bit paranoid about touching this one, but if someone really wants it to be GLDv3, please let me know. (This isn't supposed to be a high performance network!)

I'm not aware of any other non-GLDv3 device drivers remaining for SPARC, but if you know of any, please let me know!

New audioctl coming soon

I integrated a replacement for mixerctl (and also removed mixerctl) back in build 130. Well, build 130 is coming out really soon (I believe it is available internally in SXCE form now.)

I think therefore that it is time to provide more detail here about the new audioctl syntax:

NAME
   audioctl -    audio mixer control command line application

SYNOPSIS

   audioctl list-devices

   audioctl show-device [-v] [-d device ]

   audioctl show-control [-v] [-d device] [control ...]

   audioctl set-control [-v] [-d device] control value

   audioctl save-controls [-d device] [-f] file

   audioctl load-controls [-d device] file

DESCRIPTION
   The audioctl command is used to control various features  of
   the audio mixer and to get    information about the audio mixer
   and the audio device.

   The audioctl command operates on the following data types:

  device
      An audio device, such as "audiohd#0".   The subcommands
      that accept this do so as an argument to an option -d.
      If not supplied, then the default audio device is assumed.
      Any device node associated with an audio device will work
      as well, such as /dev/sound/0, /dev/dsp1, or /dev/audio.

  control
      A mixer control name, such as "volume".

  value
      The value of a control.  The specific format depends on
      the type of control.  Monophonic values usually use a single
      whole number between 0 and 100, inclusive.  Stereo values
      use a pair of such numbers (representing right and left
      channels.)  Boolean values indicate either "on" or "off".
      Enumerations take a single value of one or more names.

  file
      An ASCII text file of control settings.

Options:
   Each subcommand has its own set of options that it takes.
   However, some subcommands support the special flag -v, which
   indicates a request for more verbose output.

SUBCOMMANDS
   The following subcommands are supported:

   audioctl list-devices

  List all the audio devices on the system.

   audioctl show-device [-v] [-d device]

  Display general information about a device.

   audioctl show-control [-v] [-d device] [control ...]

  Display the control setting values for the device.  The named
  controls are displayed.  If no control names are provided, then
  all control values are displayed.

   audioctl set-control [-v] [-d device] control value

  Changes the value of a control to the supplied value.

   audioctl save-controls [-f] [-d device] file

  Saves the current state of all mixer control values to the named
  file.  The command will abort safely if the file already exists,
  unless -f is supplied.

   audioctl load-controls [-d device] file

  Restores previously saved state in the named file for all mixer
  controls.


ENVIRONMENT VARIABLES
   AUDIODEV     If the    -d and -a options are not specified,  the
       AUDIODEV  environment    variable is consulted. If
       set, AUDIODEV contains    the full path name of the
       user's     default  audio    device.

I expect there may be some questions or concerns about the new syntax. If they are not answered by the manual page above, please don't hesitate to ask me.

quick system identification

I am frequently needing to know "which system am I using". (I have trouble keeping track of which ones are which) -- i.e. is this the laptop, the Ultra 20, the whitebox, etc.

I had been using smbios a lot, but that's a bit awkward and non-portable to SPARC or systems that don't support smbios. Here's a quick script that can quickly tell me what model I'm working on. It works on SPARC and x86 alike. I have it stored in ~/bin/system -- it only works on Solaris of course. Feel free to borrow, reuse, whatever -- I hereby donate it to the public domain:

#!/usr/bin/ksh

sysconf=`/usr/sbin/prtdiag | head -1`
biosconf=`/usr/sbin/prtdiag | head -2 | tail -1`

sysconf=${sysconf#*:}
biosconf=${biosconf#*:}
sysconf=`echo ${sysconf} | /usr/bin/sed -e 's/^ //g'`
biosconf=`echo ${biosconf} | /usr/bin/sed -e 's/^ //g'`
if [ -z "$sysconf" ]
then
printf "BIOS: %s\n" "$biosconf"
else
printf "%s\n" "$sysconf"
fi

Sample outputs from a Toshiba laptop, a Sun V890, and a non-name Intel whitebox system:

TOSHIBA TECRA M9
Sun Microsystems sun4u Sun Fire V890
BIOS: Intel Corp. BX97520J.86A.2777.2007.0805.1747 08/05/2007

While the format isn't terribly parseable, its great for giving a quick assessment about what type of machine I'm on.

new aquarium lights

Some of you may know that I keep a salt water reef aquarium (210 gallon). Well, one of the challenges with such aquariums is lighting -- most corals need a large amount of high quality light to grow.

Most systems use metal halide lighting, which is extremely bright, and extremely hot. On my aquarium I've used three 250 watt MH lights, supplemented by a pair of 150 watt MH lights. (The 150s have been out for a while due to a bad ballast.) Yes, that's over a kilowatt in lighting. And MH lights have historically been considered some of the most efficient lighting -- i.e. most lumens or PAR per watt.

One alternative that has been up and coming has been LED based lighting. High power LED systems offer a lot of potential capabilities (although due to a patent problem that has put at least one company out of business and probably set the industry back at least five years, software based control of LEDs is only available on extremely expensive lighting systems -- systems costing thousands of dollars.

And until recently, inexpensive LED lighting simply did not exist.

I recently had cause to investigate this, as the bulbs in two of my 250s have died recently -- and I'm particularly unhappy with 250 watt MH lighting for several reasons: heat, problems with bulb fit (whoever designed the mechanical interface used for 250 and 400 watt double-ended MH lighting needs to quit mechanical engineering -- I've had numerous problems getting the bulbs to stay put in their "friction" based holding interface, and broken a couple of bulbs trying to increase tension so that they wouldn't fall out of the fixture), and power consumption (power in California is very expensive.)

Anyway, I'm happy to report that I found a good alternative. LED Wholesalers.com offers a 120 watt LED fixture intended for this application for just over $300. (They have an eBay store where the fixture can be purchased for under that, but with shipping and our ridiculous 9.25% CA sales tax, you'll still be over $300.) Compared to other options, this price is ridiculously low.

I put it on my aquarium yesterday, and comparing side by side with the remaining 250 watt MH fixture, I think the fixture "appears" to be about equivalent to a 250 watt MH. I've not measured it precisely, but the light appears about as bright as the 250 MH (the wife says brighter, but I'm not sure), although the spectrum is a bit bluer. If I had to guess I'd guess the spectrum to be somewhere around 18000K.

Anyway, I thought I'd post this here, since I think I'm one of the very early adopters of this. I'm going to monitor the health of my corals, and if it goes the way I hope it will, I'll be purchasing two more of these units to get full coverage over the entire aquarium. I might even purchase one for my 56 gallon tank. (I might supplement with another 100 - 200 watts of compact fluorescent at ~6500K though. That's still only about half as much power as my system previously consumed.)

I'll return to this post in the coming months to report my results as I get more experience with these lights.

Bad sound in Virtual Box in recent builds

Its been pointed out to me that if you use Virtual Box and Solaris guests with recent builds (say newer than 124), that you might get a bad sound in the Solaris guest.

Turns out that the problem is that the Boomer stack has increased its timing accuracy to a higher limit, beyond what Virtual Box can provide.

We're going to fix this properly soon... but in the meantime, you can just edit /kernel/drv/audio810.conf and change the play-interrupts to 30 in the guest.

This is CR 6901849 in case anyone is tracking it closely.

Jack of all trades, master of none

While I probably say this about other things (C++, perl), I think I've found something else it applies to -- the OpenSolaris Live CD.

The Live CD attempts to offer both a "Live Use" environment, and a "full" (for some value of full) installable copy of OpenSolaris.

The problem with this is that we have way, way too many things duplicated on the Live CD. For example, two copies of Mozilla, two copies of Evolution, and two copies of Thunderbird. (One copy each to run in the live environment, and one copy in packaged form for installation.)

This is crazy.

People who want to play around with an operating system don't need two different mailer packages. In fact, one could argue that if I'm going to use a Live environment, the CD is not going to be my method of choice. I'm going to want eithe a DVD, or (more likely) USB media where I can get faster performance and have more options. Trying to keep a live demonstration environment in the confines of a CD (or rather the space left on the CD after the installable portions), makes no sense at all. As a Live environment, I am going to want all the bells and whistles, and compilers too! (And -- perhaps most importantly -- office applications like Open Office, which don't fit in the CD right now.)

Conversely, there are those who just want to install the bits. The live media is just a vehicle to install from. Oh, there are potentially useful applications that can be present too, but they are always in support of installation. Having a web browser so that I can access my router's configuration, for example, or search for updated information about my hardware, is likely to be useful. But do I really need or want two different mailers? And klotski? PDA synchronization? Etc. This is crazy.

And its worse; because of the size of the Live "CD", we already have a situation where it does not fit on the CD. We have also sacrificed other things that, in my opinion, we should not have, in the name of "space". I'm a strong tcsh advocate, and its absence from the installation media is sorely felt, especially when installing to systems that don't have a Internet connectivity. I'm sure there are other similar things that could be the installation media if only there were space.

Well, there is space. Easily. We just need to be smarter about it.

Its time to separate the jobs into separate tasks.

• Live Media. (Live DVD, and probably also USB.) Give up on trying to do it in 700MB for a CD though. This should be sized for a single DVD, and have all the goodies, including OpenOffice. It should continue to offer installable bits as well.
• Install media. This should not have the Live environment, but have only those tools which are deemed supportive to installation. (Do include a web browser, but don't include games or a music player, for example.) It should fit easily within 700MB. I suspect that the installation media could even be merged into the AI media, to support a single disk that can do either network installation, or local CD-ROM based installation.
  

Eventually the 700MB limitation is going to be a problem even with a change like the above. I believe that we can go a lot longer with what we've got, if we don't try to force the jack-of-all-trades to fit into a shoebox, though.

sar graphical output - do you use sag?

In the process of trying to "clean house", one of the hiccups we've run into is "sag", which is used to generate graphical output from "sar" data. (If you don't use "sar" or "sag", you can stop reading now.)

sag generates Tek 4014 mode output, which can (through some contortions) be viewed either in xterm, or converted (using "posttek") to PostScript. The results are actually fairly unpleasant to work with, and rather ugly, as the results are generated through some rather ugly abuse of "graph" and "plot".

I'd like to eliminate sag altogether, because I believe far superior alternatives exist. One example is kSar, which is a Java application which generates a variety of graphs and has both interactive and scripted operation. (It can also process sar output from the Sysstat package used on Linux, or AIX which is a plus.) It can generate output in a variety of useful formats as well, making it useful in generation of web-based reports, etc.

If you use sag, and would be impacted by its absence, I would really like to know. I am operating on the belief that there are very few, if any, of you out there -- tell me otherwise if this case, please!

Specifically, are you using sag in programs or scripts? What does sag offer for you that kSar lacks? (Or would the transition to kSar otherwise be painful for some reason?)

Note that for the purposes of this discussion we can assume that I'd be making kSar available as an IPS package via the SFW consolidation, and that it can process sar data from any version of Solaris going all the way back to at least Solaris 2.6 (and likely all the way back to Solaris 2.0 or 2.1.)

nice ZIL device....

Working together with James McPherson, I've been able to get a driver for a really nifty device from DDRdrive -- see here -- working with OpenSolaris.



The device itself has interesting applications -- but I suspect the real killer application for it is in use as a "Logzilla" (ZIL) to accelerate synchronous write loads with ZFS. The only limitation on performance is the software and the single PCIe lane. (We're really right up against the PCIe single lane limit.)

From the driver perspective, its really interesting because it works with only modest changes to blk2scsa (which I'll be integrating into Nevada soon). The driver itself is tiny -- only about 950 lines of code. Its proven to be a great validation of the concept of blk2scsa -- while I never intended blk2scsa for use with high performance devices, I'm ecstatic that its as performant as it is. (Sorry, I can't post performance numbers here -- at least not yet.)

If you already have one of these devices, and want to test it in a non-production environment running OpenSolaris (no Solaris 10 yet!), please drop me a line. I'm willing to work with a few people to get some more testing done.

audioemu10k driver pushed

I just pushed the audioemu10k driver into Solaris -- it will be in builds 128 and higher. Here's what the flag day message said:

The push of CR 6539690 introduces support for audio for Creative Sound Blaster Live! and Audigy devices based on the EMU 10K1 and 10K2 devices. If you have such a device, you can now enjoy audio playback and record support using your audio device.

Devices supported are identifiable by the PCI ids pci1102,2, pci1102,4, and pci1102,8. This should be the full set of Sound Blaster Live! and Audigy PCI devices not already supported by the audiols or audiop16x drivers. It does not include support for any X-Fi devices.

This driver includes 5.1 and 7.1 surround sound support, and SPDIF support, for devices that are capable of it. It also includes support for the various break out boxes (Live! Drive and Platinum products), although it does not include support for MIDI or the infrared remote control found on such boxes.

I still would enjoy receiving test feedback, especially for folks who want to use it with unusual configurations or for more than just plain stereo output.

Taking myself off of laptop discuss

I'm tired of seeing bounce messages each time I reply to some thread on laptop-discuss@. The draconian list management policy is set such that when I reply to a message delivered to me, it always results in a bounce. This happens a couple of times a week when I try to reply to a message where I think I might have helpful information. (The bounces might have been due to a bogus attempt on my part to use a vanity opensolaris.org address in my subscription, I don't know. To be honest, I don't really care -- it should not matter. This has become a matter of principle to me.)

I think there is a way I can cobble my list subscription address to work, and there are solutions where I can subscribe multiple times to the list, but I refuse to go to ridiculous contortions because the list manager for this list still refuses to actually moderate the list.

(Mailman has both support for manual moderation of addresses that are not subscribed to it and the moderator can also whitelist addresses that he knows are from reasonable posters. This is the procedure I use for managing the several lists I maintain.)

Btw, this problem is probably going to impact nearly every Sun employee post Oracle acquisition, by the way, particularly if Oracle does header rewriting to rewrite outbound From: headers to be some @oracle.com address.

If you want my participation on laptop-discuss (and this will probably also happen to driver-discuss), then ask the list moderator to change his way of doing business. (Or perhaps, to actually start doing the real duty of moderating posts instead of setting a draconian policy and then ignoring it.)

SDcards and ZFS

I've gotten this question a few times. "Would SDcards make a good ZIL device?" "Would SDcards make good ZFS devices?" The answer to both is resoundingly no.

The reason for this is simple, and is the same reason you should never reformat SDcard media using your computer.

It boils down to the fact that you need to do some very "SDcard" specific optimizations when you layout your filesystems in order to get good performance. (The details for this are only available after signing an NDA with the SDcard trade association, btw.) Of course, general purpose computers use general purpose filesystem code, which is targetted for hard disks. So the optimizations, if there are any, are all wrong for SDcard media. (Note that this only matters when doing the initial filesystem layout .... i.e. when you are formatting the media.)

You can reclaim at least some of the performance you might have lost if you've made this mistake, by reformatting the media in your camera. But unless you have a much fancier camera than I do, your camera probably doesn't speak ZFS.

hacking perl to help my kids at school

So I have three kids, and they are all in elementary school. And right now all three of them are still trying to memorize their multiplication tables. Apparently the schools these days teach the kids to start by "counting up" (3, 6, 9, 12 ... etc for 3's) rather than going straight towards memorization as I recall we were taught.

So now they're still counting up, and have not memorized these basic facts.

So I had the idea that we need to give them sheets with practice problems, like the timed ones they do in school. I figure if they have to do this a lot, then eventually the facts will just "sink in".

This is when the wife pipes up: "hey can you write a program to do that?" (If only I had a nickel for every time the wife asked "can you write a program to do that?"...)

Well, this one was easy, took me about 20 minutes in perl. I wrote a script, "problems.pl" that dumps text file of some 144 problems suitable for sending to the printer. Here's sample output:

gdamore@pepper{18}> perl ~/problems.pl -r
    Practice: (# 650)
    Times tables from 1 to 12.
    All mixed up!


 4 x  7 =     5 x  7 =     1 x 10 =     2 x  9 =    10 x  9 =     6 x  7 =    

12 x 10 =     1 x  4 =     2 x  5 =     6 x  3 =     5 x 12 =    11 x  8 =    

 2 x 12 =     1 x  1 =     7 x  4 =     3 x 10 =     9 x 10 =     6 x 11 =    

Command line switches: "-r" to randomize the problems a bit (although every problem is given), "-s " to set the numbers to start from, and "-e " to set the ending problem (note that the second values are constrained from 1..12, because that's what the kids in school need), "-S " to set a seed (so can regenerate a previous sheet) and "-a" for an "answers sheet" (which is really only useful if you want the child to use it to check her own work. Because you should know all these facts so cold that you can check without an answer key, right? Right?)

Here's the code. Feel free to pass it around, give it away, modify it, whatever. It can be adapted to other kinds of problems pretty easily, I expect. Maybe you know a teacher or parent who could use this dumb little script. Maybe you just need to practice your multiplication tables on your own? I won't tell!

#!/usr/bin/perl

use Getopt::Std;

my $start = 1;
my $end = 12;

my @problems;

sub usage() {
        print STDERR basename($0), ": usage\n";
        print STDERR "problems: [ -r ] [ -a ] [ -s  ] [ -e  ]\n";
        exit(2);
}

srand();
$seed = int(rand(1000));

my $s, $e, $i, $j;
getopts('rs:e:aS:', \%opt) || usage;
$dorand = exists($opt{'r'});
$doans = exists($opt{'a'});
$start = $opt{'s'} if exists($opt{'s'});
$end = $opt{'e'} if exists($opt{'e'});
$seed = $opt{'S'} if exists($opt{'S'});

srand($seed);

$i = 0;
$s = $start;
$idx = 0;
foreach $i (1..12) {
        foreach $j (1..12) {
                $problems[$idx]->{'problem'} =
                        sprintf("%2d x %2d =    ", $s, $j);
                $problems[$idx]->{'answer'} =
                        sprintf("%2d x %2d =%3d " , $s, $j, $s * $j);
                $problems[$idx]->{'defined'} = 1;
                $idx = $idx + 1;
        }
        $s = $s + 1;
        if ($s > $end) {
                $s = $start;
        }
}

@new = ();
if ($dorand) {
        for( @problems ){
                my $r = rand @new+1;
                push(@new,$new[$r]);
                $new[$r] = $_;
        }
} else {
        @new = @problems;
}

if ($doans) {
        printf("\tAnswers: %s\n", $dorand ? "(# $seed)" : "");
} else {
        printf("\tPractice: %s\n", $dorand ? "(# $seed)" : "");
}

printf("\tTimes tables from $start to $end.\n");
if ($dorand) {
        printf("\tAll mixed up!\n");
}
printf("\n\n");
foreach $x (0 .. 1) {
        $idx = $x * (72);
        for $y ( 1 ..12) {
                if ($doans) {
                        printf("%s%s%s%s%s%s\n",
                                $new[$idx]->{'answer'},
                                $new[$idx + 12]->{'answer'},
                                $new[$idx + 24]->{'answer'},
                                $new[$idx + 36]->{'answer'},
                                $new[$idx + 48]->{'answer'},
                                $new[$idx + 60]->{'answer'},
                                $new[$idx + 72]->{'answer'});
                        printf("\n");
                } else {
                        printf("%s%s%s%s%s%s\n",
                                $new[$idx]->{'problem'},
                                $new[$idx + 12]->{'problem'},
                                $new[$idx + 24]->{'problem'},
                                $new[$idx + 36]->{'problem'},
                                $new[$idx + 48]->{'problem'},
                                $new[$idx + 60]->{'problem'},
                                $new[$idx + 72]->{'problem'});
                        printf("\n");
                }
                $idx = $idx + 1;
        }
}

Feeling Marginalized by OpenSolaris

The OpenSolaris project team has elected to leave my favorite shell (/bin/tcsh) out of the default installation media. There is "bug" on this, but I don't think it is getting any traction.

I'm feeling marginalized... it means that when folks first upgrade a system on SWAN to OpenSolaris, I won't be able to login unless they also remember to "pkg install SUNWtcsh". (This also creates a PITA for me on my network, but I've added a secondary user account ... "gdamoreb" with bash as its login shell to my NIS passwd file just so I can work around this.)

Anyway, compared to other decisions that this team has made, I definitely feel "marginalized". Look at some other stuff on the LiveCD:

• Not one, but two different e-mail packages (T-bird and Evolution)
• A number of "gnome" games that simply don't work at all.
• Two different versions of "vi" (/usr/bin/vi != /usr/xpg4/bin/vi)
• Unuseful "esd" (enlightenment sound daemon, not used by default)
• A number of not terribly useful device drivers ("pcelx", "pcram", etc.?) I'm going to work on EOF'ing some of these ancient device drivers.

I definitely feel like a 2nd class citizen, as a tcsh user in OpenSolaris. (zsh users, feel free to have the same opinion.)

If you feel the same way, please go ahead and put a comment in the bug report.

Call for testers: audioemu10K

I've just posted another update to the audioemu10k package. This update completely redesigned the underlying timing and interrupt support, and as a result it should work better with Suspend/Resume (without getting engine underruns.)

I've posted a webrev as well, and would really appreciate help with codereview.

Additionally, I've fixed the mixer a bit, to the point that I'm confident that a number of devices work perfectly. But some don't. Here's my test results so far:

• CT4670 -- (Sound Blaster Live!) verified 100% functional, including 4.0 surround.
• SB0100 -- (Sound Blaster Live! 5.1) SPDIF works, 4.0 surround works, no center/lfe output in 5.1 mode. Can't figure out why. (Would love to hear advice on this one.)
• SB0400 (Audigy 2 Value) -- works perfectly. Including full 5.1 surround, SPDIF output. I've not tested the side 7.1 channels because I don't have the necessary cable to do so.
• SB0350 (Audigy 2 ZS Platinum) -- surround sound on rear jacks works fine in 5.1 (can't test 7.1). My expansion box appears to be kaput though (doesn't work in Linux either), so I need help testing this from someone else who has a working one.

Those are the only devices I have on hand to test, but the driver can in theory support many many more. If you can expand the supported test, please let me know.

Thanks!

test audioemu10k driver posted

I've posted a test package containing the "audioemu10k" driver. (x86/amd64 platforms only).

There is also a webrev with the code.

This driver supports (in theory) a large number of cards from Creative. (Devices identified in prtconf -vp as "pci1102,2", "pci1102,4", and "pci1102,8".) I've not tested many of them though -- only the SB0100, CT4670, SB0310, and SB0360 have been tested, and I've been unable to verify SPDIF or expansion box functionality, but it should work.

To test, you'll need Boomer, which means build 115 of OpenSolaris or newer. I've only tested with build 124, and I recommend using 124 or newer if possible.

If you test it, I'd like to see the output from "cat /dev/sndstat", and as much testing of inputs and outputs (try different mixer controls, as well!) as you can. I'm hoping to integrate this code into Nevada in the next week or so.

hme shrinks by about 40%

I just removed about 40% of the code from "hme", by converting it to use the common MII layer. It also makes it use Brussels, and fixes an old bug relating to 10 Mbps functionality. Another net negative lines of code integration for me. I wonder if this confuses ohloh.net's accounting?

OpenSolaris on snv_124 Impressions

I've bitten the bullet, and finally installed "OpenSolaris" over my SXCE install, upgrading from build 105 to build 124 in the process. I thought I'd share some things I've noticed:

• Gnome's volume control Just Works, and doesn't show the legacy Sun device anymore. (Yay!)
• There is a 12 MB Core file on the CD ... er... too big... DVD ... image. (Oops!) Maybe someone was hoping the community would jointly debug the program that generated the core file. (gtk-update-icon).
• Still no /bin/tcsh installed by default. Fortunately easy to get off the network for this machine. (For other systems, not so easy. This really ought to be fixed.)
• AudioHD has a really, really annoying beep on this system. The beep volume control doesn't take effect until you lower it to 33%. (And then it gets really quiet.) This is a bug that I'll have to investigate further, there's probably some codec tweaking needed.
  
• Regular SunOS vi has been replaced with vim, which has an incredibly obnoxious bug. It doesn't scroll down properly, redrawing on the last line in the terminal window instead of the whole screen. This is IMO at least a P2 bug against the editor. The old editor in SXCE, while maybe lacking some neat colorization features and not being open source, at least worked properly.
  
• Trying to install CUPS using the package manager GUI fails with a horrible stack back trace. While the request is is made to report the problem with the stack back trace, there is no provision to save it or e-mail it. So alas, I don't have it anymore to post. It should be easily reproducible. (It looked like a problem relating to the SMF configuration.)

I have a lot more to do... so many things are at the moment still missing. But I think I'll be able to muddle on at this point. (The vim bug is going to really, really bug me though, because there isn't an easy workaround -- apart from just copying over the old vi from SXCE.)

Update: Turns out /usr/xpg4/bin/vi (which is what I had in my path anyway) doesn't have this problem. And its installed by default. Yay. But someone really needs to fix vim, because that bug is horrible.

Update 2: When I logged in using my old home dir, the Gnome panel which I had configured for auto_hide did hide itself. But unfortunately would not unhide itself with any mouse actions. The only way I could get it to unhide was to disable the auto_hide property using gconftool2. Unfortunately, it took a while to figure out how to use this.

Giving up on laptop-discuss@

Here's a message I sent to laptop-discuss-owner at opensolaris.org:

Okay, I've tried *three* different addresses to post to laptop-discuss@... each time the message bounces.

The addresses I tried:

gdamore@sun.com
garrett.damore@sun.com
garrett@damore.org

I give up. The list is *unusable* to community members, because of a draconian list policy that instead of just moderating reasonable attempts to post to the list just *bounces* them. This community (laptop-discuss@) will no longer be able to receive mail from me, despite the fact that I'm probably one of the more active contributors to the software that makes up core laptop platform support.

If you think your membership ought to be able to hear from me, then please make it easier for me (and for other people) who have a legitimate need to post to the list. Simple moderation with white and blacklisting can be used to achieve this.

- Garrett

For the record, the message I tried to post was:

I'm considering a case to remove/EOF the partial (incomplete) support we have for certain Tadpole laptops in Nevada.

I'd like feedback on this. Is anyone out there still using a Tadpole SPARCLE with OpenSolaris or Solaris Nevada?

The reason for this is the intention to remove support for graphics (its already not present in OpenSolaris). I never got enough cycles to finish the work to integrate power management or other mobility features for this platform (SPARCLE), and now it seems to be quite obsolete. We're talking about UltraSPARC-II (up to 650 MHz max cpu speed) systems here.

Would anyone here strongly object to just removing the support?

- Garrett

The bounce messages I received look like this:

You are not allowed to post to this mailing list, and your message has
been automatically rejected.  If you think that your messages are
being rejected in error, contact the mailing list owner at
laptop-discuss-owner@opensolaris.org.

The problem is that draconian list membership/posting rules make posting updates incredibly painful. If you think my contributions to the forum are or would be useful, please ask the list moderator to start actually moderating the list instead of just setting it on auto-reject. (At this point, I don't remember which vanity address I subscribed to the list using... at the moment, I'm just about to the point of not caring. I've already reached this point with driver-discuss@ - on more than one occasion I've simply decided not to keep trying to send a message to that alias because of the same stupid draconian policy.

why we might never support Dolby Digital

SPDIF (and AC3) are the (now legacy) ways that home theater systems are supposed to use to transfer high end surround sound to the receiver for amplification and distribution to individual speakers.

Many audio cards on the market can transport AC3 (aka Dolby Digital) or DTS compressed audio data over a digital cable (SPDIF). They can also transport 16-bit stereo PCM (uncompressed) to SPDIF.

Boomer can't decode or encode, or transcode, any compressed audio at the moment. While it might be nice if we could do this, there are significant and non-trivial hurdles to making that happen. Some are technical (such as computational costs and lack of hardware offload), and others are more legal (such as trademark, licensing, and patent considerations.)

So, Boomer could transport uncompressed 16-bit stereo over SPDIF. (That's all SPDIF has bandwidth for!) Which we support today in audiohd. And in the future we might extend this to other devices.

However some operating systems also support a pass-thru for compressed audio data, where data is taken from a source (such as a DVD application) and routed directly to the receiver without any changes. This works reasonably well, except that it adds a lot of complexity to make it work right, and you completely lose any control over gains or the ability to mix other audio sounds with the stream. (Want to get a notification of an incoming VoIP call while you are watching your movie? Too bad... you can't do that with AC3 passthru.) This mode is of such limited use that we're inclined not to support it at all -- its really only useful for media center PCs used in conjunction with home theater systems to watch DVDs. (And we already know about the various problems with watching commercial DVDs right? I won't talk about those legal problems here.)

The best feature for SPDIF is the ability to transfer 6 streams of data in a single cable -- using a lossy compression. The reduction in cables to tie this into your home theater system is really a good thing. But of course, we don't have an encoder, so we can't do that except in the pass-thru case we already said we weren't going to do. (So no 5.1 gaming via SPDIF for you... Dolby calls this Dolby Digital and all audio cards that support it do so in software only, which requires expensive licenses from Dolby.)

Fortunately, there are better options on the horizon, and this is where we want to spend our time.

One option is ADAT ("lightpipe") -- which can support 8 channels of uncompressed audio at 48 kHz. Its available on some motherboards even now. Supporting it would probably be fairly easy, although using ADAT might require some very high end audio equipment (ADAT is used in professional audio situations.)

A second, and far more accessible option, is the use of HDMI's audio channel. New motherboards (and some cards, such as the Asus Xonar HDAV) can support HDMI audio. HDMI supports 8 channels of uncompressed audio at up to 192 kHz. This is plenty for even the most demanding audiophile. And if you want more channels, you can always use a second HDMI cable! HDMI offers us the best audio capabilities without any compromises. (We can use it with mixing, volume controls, or use it in a bit-perfect fashion. We get full quality. And we don't have to deal with any of the licensing, technical, or legal headaches that accompany trying to get a real-time AC3 or DTS encoder working. ) So at this point, any investment spent in AC3 instead of HDMI on our part just looks kind of silly.

I expect that sometime later this year (or more likely next year), we will be looking hard at getting HDMI audio working with Boomer. (Note that I have no control over funding or project priorities -- so this is just a prediction, not a promise.)

audio driver enhancements

Okay, I've got some audio drivers that are basically ready to go, but which I'm trying to decide whether to integrate or not. Part of the problem is that integration of new drivers is a somewhat expensive process, and if there is no demand for some some of these, then I'd rather go do other things.

So, another straw poll. If you have one of these devices and you want a driver to test with, please let me know.

1. Cirrus-Logic/Crystal CS 4281 -- driver is ready to go, stereo only
2. FM-801 -- I think the driver will work, but don't have hardware at the moment to prove it.
3. C-Media 8738/8768 surround sound support (stereo already supported in OpenSolaris) -- this will probably be integrated at some point, but I can share binaries now if someone wants them.

Other drivers I can supply with just a modicum of effort, if there is demand:

1. Yamaha YMF-7 series -- driver not yet ready, but now I have hardware to work on it, if there is demand
2. C-Media 8788 - driver mostly ready, but no hardware, and its expensive hardware to purchase.

Note that I will be doing the driver for the oft-requested EMU10K (Audigy/SB Live!) devices very very soon. Don't bother asking me again about it; its pretty high on my priority list. Likewise, X-Fi is very very desired, but will probably be quite a while longer in coming due to lack of specs. Various VIA chips (Envy24HT, etc.) are in the pipe as well.

And of course, Sun Ray audio is very high priority. Stay tuned for more on that.

ohloh.net statistics

I notice that OpenSolaris has very few "users" listed on ohloh.net. Lets bump it up. If you use OpenSolaris, why not go ahead and create an account on ohloh.net, and indicate "I use this" for OpenSolaris.

While you're there, if you're grateful, you could grant kudos to your favorite developers. And if you have any contributions of your own (pushes to ON), go ahead and claim those to (search for your name or login in the "people")... it bumps your rank, and makes your "kudos" worth more when you deign to give them out.

ESS Solo-1 (audiosolo) integrated

I just integrated audiosolo -- it should be in build 125. :-) I think this driver might have set a new record for integration into ON. I started work on it Sunday afternoon, and integrated into ON on Tuesday evening. Proof that not all integrations into ON have to take forever. (Granted if this were NetBSD it probably would have integrated on Monday morning instead of tonight...)