Earlier this month I talked about the Microchip PIC16C745 and PIC16C765, which are 8-bit microcontrollers with built-in low-speed USB. Since that time, I've got hold of one of these devices and built a simple USB peripheral with it. Click on the thumbnail for a bigger version of the image.
Summary: hardware easy, firmware tricky, software painful. Read on for more detail and some hints for anyone treading the same road in the future.
[Last updated 20050606: added firmware source code.]
As you can see from the image, the hardware is simplicitly itself. (If you're really interested, you can look at a schematic). You can see the USB B connector at the far left, carefully soldered to some wires: I did take care to continue the data connections as a twisted pair, but other than that it's really not subtle.
The PIC16C745 is in the centre; you can see that this is the ceramic /JW package from the little quartz window over the die. You uncover this when you want to reprogram the chip; it takes about ten minutes cooking in my UV eraser. Theoretically, the processor can start up badly unless you cover the window when the chip is in use, so I usually put a little orange sticky dot over it on a programmed device.
Above the processor you can see a crystal and two capacitors giving us the required 6MHz reference frequency. It turns out that low-speed USB is more flexible about frequency accuracy than the faster variants and that a ceramic resonator would have been sufficient and a little cheaper. That's probably more a concern for people interested in production quantities; you can get a bag of those things for a fiver.
Just to the right of the processor you may be able to make out a decoupling capacitor and a resistor holding the chip's reset pin high; just to the left of it you can see a resistor and capacitor that allow the device to pull up one of the USB signals at startup to indicate that it is a low-speed device. The bit of black sticky tape is there to remind me where the end of the chip goes after reprogramming it.
This is everything you need to get the device up and running; the remaining components on the breadboard are four little switches, each with an in-line resistor to help protect against static discharge, and two LEDs each with a current limiting resistor.
Things started well on the firmware front. Microchip do have standard firmware packages for these devices linked directly, for example, to the PIC16C745's index page. Even better, though, they have a series of "canned applications" that are more useful for tutorial purposes, provided as technical briefs. The only problem is that these are almost impossible to find on their web site. For reference, I found all of them by reading through the list of all application notes and picking up everything that sounded interesting.
I started by downloading Technical Brief 54 and its associated source code package. This is An Introduction to USB Descriptors - With a Gameport to USB Gamepad Translator Example and the ZIP file contains a hex format file you can burn directly to a PIC16C745 or PIC16765. Plug in the USB connector and the device should be almost instantly recognised as a gaming device: no other drivers are required, at least on Windows since the 98SE version and under Linux 2.4. You can play with your new USB "gamepad" in the Windows control panel.
This surprisingly handy effect comes about because the firmware identifies the USB device as a HID class device: HID standing for Human Interface Device, i.e., one of those terribly slow things whose data rate is limited by how fast humans can twitch. Within the HID class, there is an additional specification to allow a device to enumerate its controls, their values and the usages intended: for example, a joystick device would have a couple of linear axes and some on/off buttons.
The problem comes if you want to deviate from someone else's working setup, as you almost certainly will at some point. For example, I wanted to change the number of buttons and add a couple of LEDs. To do this, you need to build new HID report descriptors and there you run into problems with incomprehensible standards documentation and inconsistent operating systems implementations.
There are several different kinds of descriptor you need to build to make a working USB device. Some of these are described in the USB standard itself (hint: for most purposes, just read the 1.1 standard as it is substantially shorter), some in the HID adjuncts to that standard. The most difficult of all to understand are the HID report descriptors and the system of usages and usage pages: I found that apparently sensible changes to a descriptor would just silently "not work", resulting in a device that was invisible to Windows 2000. My advice is to make small changes, step by step, from a known working system to where you want to go. Back up at every stage before making a change, and revert immediately if something doesn't work. Don't be tempted to make too many descriptor changes at a time, even if it does take ten minutes to erase the device each time you want to make a change. Under no circumstances attempt to build descriptors from scratch, as most likely your device simply won't configure; you get essentially no information as to the reason for this. If you'd like to compare the TB054 source code with my modified version, you can download my firmware source code.
There is definitely an operating system consistency issue as well: for example, I have seen configurations of controls that are acceptable in Windows 2000 but cause Windows 98SE to reject the device. You need to be able to test your firmware against all the target systems in order to be sure that it will actually work there.
The USB HID page includes a tool to be used to create HID report descriptors; it can perform minimal checking on the ones it creates or be used to do the same minimal checking on ones you've constructed elsewhere. It's not infallible, though.
The other tool provided by the USB Implementers Forum that looks like it should be useful is USBCV. This only works under Windows 2000 and XP, and only works in systems with both a high speed (USB 2.0) host controller and high speed hub, even for testing low speed devices. Unfortunately, I don't have a system meeting all of those requirements at present so I can't yet say how good a diagnostic tool it is.
Other resources I found of use while debugging firmware:
lsusbcommand, which seems pickier about some things than Windows, and more verbose to boot.
One last item that fits in under "firmware" better than anywhere else is the issue of vendor IDs. Like PCI and other "plug and pray" bus systems, your device presents a vendor ID and a product ID so that the host operating system can locate appropriate drivers. Unlike those other systems, there is no "experimental" vendor ID or apparently any way to get a legitimate vendor ID other than paying the USB Implementers Forum a minimum of $1500 every couple of years. Microchip do have a vendor ID, but as far as I know they don't have a programme in place to hand out product IDs to customers as was the case for many PCI chip vendors. If you do the obvious thing and pick two random 16-bit numbers or some defunct company's vendor ID, everything will (probably) work but you can't use the USB logos on your products.
Host application software for a custom peripheral is where things get really painful. This isn't the case if you're building a joystick or a mouse or some other device for which operating systems already provide all the drivers that you need; all you need to do is fire up Flight Simulator and you're done. If you want a host application of your own talking to a collection of custom peripherals you've designed, though, things are much harder.
Ideally, I'd want to be able to write a client application in a nice safe language like Java and have some hope of it talking to my USB devices under at least some variants of Windows and Linux. The bad news for me is that there is no stable Java API for USB yet, and neither of the attempts to date appear to have been ported to Windows.
The older of the two extant APIs is Java USB, which appears moribund and is currently Linux-only. Some Windows code was submitted to the project a year or so back, but it doesn't appear to have been developed or available for use. The newer API is javax.usb, which apart from being quite active is also the subject of Java Community Process JSR-80: in other words, the aim is to make it into a semi-official standard. Unfortunately, this project is also currently Linux-only; something in the way of a Windows port is anticipated around the end of 2003 or early 2004.
One option I have considered is to make use of the fact that if I continue to make my peripheral pretend to be a gaming device of some kind (with some fake axes that never move), getting data from it can be done through Microsoft's DirectInput API. It just so happens that an enterprising German developer has something that allows just this kind of operation from Java under Windows: this is JXInput; it works by using a JNI (Java Native Interface) library to bridge across from the Java VM to the DirectInput objects. This is a promising avenue, but unfortunately DirectInput gives essentially no way to write data back to the device, apart from the "force feedback" facilities. JXInput does not even support the force feedback option: this means I could read my buttons but couldn't light my LEDs.
Microsoft's DirectInput documentation covers this eventuality neatly by suggesting that the Windows HID functionality be used directly for these purposes. Of course, this could be the route for both data directions and another option would be to write my own JNI wrapper for the appropriate Windows API calls.
Stepping back from Java, there are of course several alternative programming environments for Windows. I know both C++ and Visual Basic well, and example code of varying degrees of crudity are available for these languages in various places (start at Jan Axelson's site's USB code page). However, neither language is really as good with multi-threading or asynchrony as Java is, and I really dislike the idea of learning to program the Windows GUI in C++ at this stage, having avoided it for so many years. Microsoft's C# would be the new trendy alternative, but I'm unfamiliar with it except as being "Java like" at some level and having got the "Hello, C# world" console application running.
You can see from this that by far the most difficult part of the whole enterprise is software for the host. At present, I'm using an adapted version of some of Jan Axelson's Visual Basic code but I'm far from happy that I could scale this up to drive several similar devices in an asynchronous way, and unfortunately that's what I have in mind when I get beyond the pure experimental stage.
If I'm lucky, JSR-80 will happen before I get there.