What Rob Pike is refering to is of course Plan 9 and its derivatives (say Inferno). Whether you want to consider them Unix-likes or not is your choice (many HNers will say it's more Unix than Unix).
If you look for desktop alternatives to Unix, OSes to look at are BeOS (an open source clone is developed under the name HaikuOS) and the Windows NT kernel (don't hurt me: the Windows NT kernel is IMHO far more elegant than, say Linux or Mach. What is crappy is especially the WinAPI; please look deep below the surface: Here Windows gets nice).
The reason why you won't find many desktop/server alternatives to Unix can be read here: http://herpolhode.com/rob/utah2000.pdf
Thus lots of OS reserach is either focused on embedded stuff or virtualization instead of desktop or server.
Accepting this, especially have a look at the L4 kernel family (a family of very small and fast microkernels). If you like highly secure designs, you'll probably like seL4, the first formally verfied kernel. Also QNX is worth a look (QNX is Unix, accepted; nevertheless a very elegant kernel design).
In the context of the NT kernel, I supoose ReactOS.org might warrant a look (I don't really know how closely (or not) the kernel design matches NT beyond being compatible with drivers...).
I'm not sure what huge technical advantage the BeOS model offers over contemporary Unix-likes in this day and age, other than its file system support for extended attributes, which are a controversial topic and nonetheless still used by things like XFS.
I'd also like some clarification on what you think is good about the NT kernel. It seems far too entangled with other cruft that forms the Windows stack.
That said, I'd also like to mention the Hurd. At this point, it's really a fragile ad-hoc reimplementation of 9P file servers on top of a modded Mach, but such a model was still quite daring for general purpose computing back during the Hurd's original window of opportunity (1989-1995, or so). Probably would have advanced the state of modern OS at least a bit, if it weren't for managerial incompetence.
It's still a Unix at heart, though, despite many important extensions.
> I'm not sure what huge technical advantage the BeOS model offers over contemporary Unix-likes in this day and age, other than its file system support for extended attributes, which are a controversial topic and nonetheless still used by things like XFS.
BeOS was very optimized for multimedia, which is an interesting property I think. Also the GUI library used multithreading from beginning. This surely isn't interesting for servers, but for desktop computers.
> I'd also like some clarification on what you think is good about the NT kernel. It seems far too entangled with other cruft that forms the Windows stack.
For a lack of time for explanations I will only give one example: How to do fast asynchronous I/O (You know: Asynchronous IO: The hot thing that node.js is about ;-) ;-) ). Under FreeBSD/Linux asynchronous I/O is just synchronous non-blocking I/O (FreeBSD needed to implement kqueue to even allow this in a fast way; the Linux developers implemented epoll (which is incompatible to kqueue :-( )). Under Windows NT it is an easy problem that has been solved (from beginning?). See
> BeOS was very optimized for multimedia, which is an interesting property I think. Also the GUI library used multithreading from beginning. This surely isn't interesting for servers, but for desktop computers.
BeOS claimed to be optimized for multimedia, but that does not mean that it was. I remember, that I was able to have fluid DivX (3.11) playback on PII-300 running Windows and Linux, but not on BeOS.
What BeOS did have is DirectShow-like media architecture, using nodes and pipelines. But at the time, it was not an effective architecture.
(And yes, the BeOS engineers never managed to support VESA GTF in their display drivers, meaning that the picture on my monitor was always shifted compared to other OSes).
The original hacked codec appeared in 1998. It got boost in popularity when the movie The Matrix came out (1999).
This happened in Windows 98 timeframe. Windows 2000 was in early beta, not yet on sale and XP was unheard of yet. The current linux were Redhat 5, 5.1 and 6; BeOS 4 and 4.5.
They're of course not a desktop option at all, but there's also some very interesting work happening with unikernels, which are really only recently a viable option now that Xen can present a unified hardware layer, instead of the endless treadmill of specific drivers. I've been getting into OpenMirage [1] quite a bit recently, largely due to the energy of the team [2], but also looked a bit at HalVM [3]. I definitely recommend [2] for anyone interested in plausible ideas of what the one of the next generations of VM/apps might look like.
If you look for desktop alternatives to Unix, OSes to look at are BeOS (an open source clone is developed under the name HaikuOS) and the Windows NT kernel (don't hurt me: the Windows NT kernel is IMHO far more elegant than, say Linux or Mach. What is crappy is especially the WinAPI; please look deep below the surface: Here Windows gets nice).
The reason why you won't find many desktop/server alternatives to Unix can be read here: http://herpolhode.com/rob/utah2000.pdf Thus lots of OS reserach is either focused on embedded stuff or virtualization instead of desktop or server.
Accepting this, especially have a look at the L4 kernel family (a family of very small and fast microkernels). If you like highly secure designs, you'll probably like seL4, the first formally verfied kernel. Also QNX is worth a look (QNX is Unix, accepted; nevertheless a very elegant kernel design).