Thursday, July 31, 2008

The TeXTing Swindle

I recently started communicating with some family via "texting", because, for reasons I don't fully understand, they will happily respond to a text message (and send them out on their own), but won't answer a regular phone.

So ignoring the annoyance of trying to draft up a message on the toy 10 key keypad on my phone -- which takes approximately 20 times as long as it would to say the same thing in a voice call, I started thinking about the economics behind the texting fad.

My provider charges me $0.20 per text message. My overlimit minutes cost about $0.25. (Picture messages cost more -- $1.95 each, I think.)

This is the biggest swindle made by phone companies in recent history.

Look at the data involved.

A phone call (voice message) takes the following bit of bandwidth:

  • Assume 8-bit ULAW or ALAW sampling (12 bits compressed to 8 bits)
  • Assume 8 kHz sampling, which is typical for "phone quality" audio
That gives up 8 * 8000 = 64,000 bits per second or 3,840,000 bits per minute

Now, consider a typical text message. Say 12 lines at 40 characters each. (That's more than my phone will display), each character at 8 bits (we'll assume ASCII for now, I can't imagine trying to conjure up CJK characters on my phone!), gives me 12 * 40 * 8 = 3,840 bits. (I didn't try to make the numbers work out as evenly as they do for this example, it just happened!) So, 3840 bits per message.

To a rough approximation a text message costs about the same to the consumer as a minute of talk time. HOWEVER, the bandwidth involved is approximately 1/1000. If everyone is texting instead of using the voice service, the service provider should have about 1,000 times more capacity!

Put another way, text bits are about 1,000 times more expensive to end user!

Now, consider pictures. $1.95 is nearly 10x the cost of a text message or a minute of talk time.

A fully decompressed 16-bit VGA (640x480) resolution picture takes about 4.9Mbits.

To a rough approximation, a picture takes about as much bandwidth as a minute of talk time. The picture is therefore 100x less expensive per bit to send than a text message, and about 10x more expensive than a voice call.

Of course the secret here is that texting is incredibly efficient use of bandwidth to send a message (compared to a voice call), especially with those wacky abbreviations (such as "c u l8r"), but the end user doesn't realize that it is costing the provider far less to send the same message via a text message. (Put another way, the end user has no idea how much network bandwidth is simply wasted with traditional voice calls.)

The mobile providers here have made a real swindle. The fact that they aren't trying to undercut each other on the text messaging does allude rather strongly to "collusion" in market pricing on them -- there should be a lot more competitive pressure to drive the cost of texting down, or even to offer it for free (especially with plans that have unlimited voice, such as mine).

In fact, the phone company should be begging me to use texting instead of voice calls, instead of charging me extra for texting.

Anyone from the mobile companies want to explain how you can get away with this swindle?

Ask your mobile provider why they aren't giving you texting for free with your unlimited voice plan!

Wednesday, July 16, 2008

New Experimental AudioHD Driver

I've just posted the latest experimental version of the OpenSolaris audiohd driver. This driver includes the latest work from the engineers in Beijing, including a widget parser that enables the driver to work on a much larger variety of audio configurations.

As most motherboards these days ship with an audiohd compliant device, this should greatly expand the support for audio on Solaris systems.

The above driver binary also includes suspend/resume support.

Note that this is not the anticipated Open Sound System driver, but rather an extension of the pre-existing Sun driver. It won't work with OSS. However, there is some chance (I've not tested it myself) that this will even work on Solaris 10.

A webrev of the changes should be posted soon.

In the meantime, if you want to give this a whirl, let me know the results. (To test it, just copy the binary objects into /kernel/drv/amd64/ or /kernel/drv and reboot. -- Yes there are ways to do this without a reboot, but the caveats are too many to list here.)

We're interested in any problem reports, as well as notification of hardware configurations where this either did or did not work. Thanks!

[ update 10/13/2008: I've changed the link to the files list so that readers can find the latest version posted up on the site ]

Tuesday, July 8, 2008

NIC driver family trees

This may be interesting to some people -- and its recorded here for posterity. A number of NIC drivers in Solaris have been developed using "copy-paste" approaches. As a result, there are a couple of family "lines" for NIC drivers.

le (ancient 10 Mbit only) driver, is the common root for one heritage. From le was developed hme. From hme were developed qfe, eri, and gem. From these was developed ce. And ultimately, from these was developed nxge (although nxge shares little with its ancestors anymore.)

Another heritage goes back to ancient dnet (tulip) drivers on x86. dnet begat dmfe, which begat bge. There are probably other drivers that bge spawned, but its hard to know for sure how much copying may have occurred.

afe and mxfe were developed pretty much independently from the others, although afe was developed first, and begat mxfe (which is for very similar hardware -- a good counter example for a case where copying the code made sense -- of course I'm also the original author of both drivers). (They started life as pure DLPI from-scratch drivers, but migrated to GLDv2, and later to GLDv3.)

Murayama's drivers (e.g. sfe) owe some ancestry to Linux, although we're told that all the code was changed so that no derivative code from Linux remains.

I believe e1000g owes its ancestry to a combination of Linux code at Intel, and unique code written just for Solaris. (There was another driver, ipge, for the same hardware, which was part of the le/hme heritage, but it died in late toddlerhood.)

The wifi drivers owe most of their existence to the work done in FreeBSD and OpenBSD.

The other older drivers (pcelx, elxl, pcn, etc.) appear to share little if any heritage with any of the other NIC drivers.

Cut & Paste Device Driver Design

A common approach to device driver design is to start with a device driver for a similar piece of hardware that someone else wrote, and copy it, then modify it. However, this approach is fraught with peril, and I thought it might be a good idea to record my thoughts on this for posterity. If this post saves just one future developer from falling into this pit, it will have been worthwhile.

The first problem is that most device drivers have a number of non-trivial "features" in them that relate to the specific issues of a particular piece of hardware. Often this means that there are internal design features of the driver that are not ideal for a new driver. (A classic example would be nxge's use of particular hacks relating to the location of it in relation to the IOMMU on certain Niagra processors. No other driver is ever going to need that.)

The second problem is that most device drivers are not 100% bug free. The copy-modify approach to coding tends to lead to bugs being copied from one driver to another.

The third, and possibly most significant problem, is that when the copy-modify approach is used, the author doing the copying rarely has a complete understanding of all of the original source driver's features, and the end result is a new driver that the original author isn't aware of (and isn't maintaining), and with code that the new author might not fully understand.

A fourth problem is that real drivers are complicated. I frequently hear advice given to new NIC driver authors to "just copy bge". That is terrible advice to give to someone who may be trying to get their head around a new DDK. As a real driver, bge carries a lot of legacy baggage with it, that new drivers definitely shouldn't repeat. All that baggage obfuscates the original code, and the would-be-copier may have little ability to contain the entire device driver in his head.

This leads to the fifth problem, which is that copying repeats all the optimization steps, but elides the measurement and experience that led to them. This violates one of my core design principles, which is KISS (keep it simple, stupid) -- design for simplicity first, and optimize only when proven necessary. (Example: does a 100MB NIC really need to have the logic associated with "buffer loan up" in it? Almost certainly not! And in fact, despite the fact that many NIC drivers have such logic in them, dating back to older architectures, the loan up actually has significant problems, and results in lower performance than the far simpler unconditional bcopy.

A sixth problem is that this approach can often lead to reproduction of design mistakes or limitations. For example, if one were to build a new driver for SPARC hardware by copying hme, the end driver would likely not be 100% portable to x86. (And even if it were made portable, the first pass would probably be quite suboptimal from a performance standpoint owing to hme's use of SPARC-only dvma interfaces.)

The upshot of all this is that I would strongly encourage prospective device driver authors not to start a new driver by copying source code from another driver.

A far better approach, IMO, is to start with a "skeleton driver" (Writing Device Drivers has some such skeleton drivers for different kinds of drivers) and flesh code as you go. Ask questions, on the mailing list as well, if you're not familiar with a particular type of driver.

For folks who might argue that starting fresh from a skeleton violates code-reuse and increases development costs, I would say that

  1. The end driver will almost certainly be more maintainable, as the author will understand every line of code.
  2. The cost of developing a new driver from scratch is probably not that high compared to the copy-modify approach (unless the source and target drivers are for devices that are very very similar. )
  3. The educational benefit of taking this approach is very great as well. I can think of no better way to truly understand the DDI/DDK than to write a device driver from scratch.
  4. For commonly reused code, a library with well defined interfaces is a far far better approach. The driver frameworks in Solaris such as SCSA and GLD are good examples of this. (Third party approaches such as Murayama's GEM also qualify in this regard.) This way there is only one real copy of the code, and clear interface boundaries make it possible for multiple parties to reuse the code safely, without depending on "implementation details".
There are some folks who "start from scratch" - I hope I'll see more of them in the future.