The other end of the Erlang experiment is the libraries I was working with. My application was built on the Sumo Rest stack, built on top of the Erlang Cowboy HTTP server.

I found this finicky and prone to vague runtime errors. That's not important, though. The overall style of the system was pretty reasonable, and is probably a good way to write REST services in general.

( Read more... )

The layout implications of this are useful and generic: one source file per type, where each type file defines its storage, serialization, and documentation; and one source file per route, where each route file has its own URL path, complete machine-processable documentation and metadata, and understands the standard failure cases. You just need the right high-level REST library in your language of choice to support this.

There's a new release of the KDE desktop environment out, and in particular there's a test repository with packages for Kubuntu. It's apparently not just a point-zero release but the KDE maintainers are pretty unapologetic about it being a relatively raw point-zero release; "KDE 3 is fully supported and even under reasonable development, but nobody's really using a 'beta' KDE 4 and we need field bug reports."

KDE has gone in the "shiny" direction I had been hoping some mainstream Linux desktop environment would. The Compiz/Beryl/XGL/whatever X-over-OpenGL desktop effort has been pretty but it fundamentally is a concept demo that's Different From What You're Using Now. So KDE 4 has translucent-window support, some neat abilities to dim windows that have active modal dialog boxes, and some of the Mac-ish desktop effects like a window switcher that shows the current state of every open window.

The flip side of this is, well, that a lot of "normal-user" functionality isn't entirely there. I can't find the Debian app menu (which tends to be more complete than KDE's); I can't successfully log out; I can't add keybindings for "switch to desktop #5"; I can't change any characteristic of the panel; adding applets to the panel is non-intuitive. KDE 4's version of the Konqueror Web browser won't import bookmarks from the KDE 3 Konqueror and seems to find infinite loops no other browser does reading Livejournal.

Conclusion? It's definitely shiny, and if you're a bleeding-edge person and a KDE person it's probably worth playing with. But overall KDE 3 a lot more intrinsically usable than KDE 4 is at this point.

Our new corporate masters use a well-known largely Windows-oriented mail system (thankfully not Exchange, and the client does run inder Linux). One consequence of this is that the email editor looks similar to Word or your favorite other font-aware GUI text editor. And a side consequence of this is that I'm actually happier to get HTML email...and it's hard to send good-looking text mail.

So I now understand the temptation of HTML mail: if everyone you correspond with lives in an HTML-aware mail world, then everyone's life is slightly prettier if everyone uses HTML. But I come from a world where not everyone does, so now there's the challenge (which the software doesn't help with at all) of making my GUIfied mail look good to text mail readers. I hope I'm succeeding, but this is a hard UI problem.

I think there are just too many options. Some people send their email in blue. Why blue? I'm not sure. The formatting options I want most are "monospace" and "italic", and sometimes "list", if I want more than that I'll write a document in something else and attach it. These options would be pretty easy to port over to your slightly-formatted-to-text renderer. So then you're not foisting angry fruit salad on the world, and you've made both the HTML-reader and text-reader people happy. Just as soon as I get to hacking on this closed-source heavily-legacy mail software...

I figured I'd try to write a to-do list manager as a random side project. Tasks should be hierarchical, tasks should have dependencies, data should be stored locally, should have a pretty GUI around the whole thing. This doesn't sound hard, it should be something that I can get a first pass knocked off in about a weekend. Great.

I settled on C++ as an implementation language (I want something type-safe, Debian has no Haskell GUI libraries packaged, I know C++, OO code in C sucks a lot, and I kind of like Gtk). So I sat down and wrote a pretty simple data store for it. Bottom-up design and all that. So now I have a module where I can commit a new data state to it, and undo and redo. Oh yeah, and I wound up writing some basic code to do things like reference counting and callbacks when the state changed. Not hard or long, but it needed doing.

So now I start writing the GUI (using gtkmm) and I discover that I lose: to make a tree view of my hierarchical tasks, I either need to copy all of my data (eew) or write a custom TreeStore. There is no documentation for the latter. And the only way I can pass my store into the TreeView class is via a smart pointer...in the Glib reference-counting system...which is different from mine.

My options are now to either tie my data store into Glib (yuck), figure out how to write the custom tree data store object in C and tie it in (more yuck), figure out how to make my adaptor object Glib-refcounted (possible, I suppose, but TFM is pretty unhelpful). Or learn OCaml, or figure out how to compile and install gtk2hs. The last is increasingly sounding like a better use of my brain cells...

For quite a while I've been getting mail exclusively at MIT and reading it exclusively through Gnus. Gnus has a lot of nice features, and since it's entirely in elisp it's very customizable. On the other hand, since it's entirely in elisp, it's also kinda slow, especially doing things like sorting mail and doing statistical spam detection, and reading mail stored in AFS sucks over any kind of consumer Internet connection. Is it time to move on?

Goals. I'd like the mail-sorting to happen "offline"; I don't want to have to wait for all of my mail to get sorted in between deciding to read it and being able to. I'd like to keep being able to use Gnus. I need my mail sorted by mailing list, and need some sort of spam filtering. Something IMAP-backed is probably ideal. It needs to be backed up, which I'm not set up to do at home, and have good connectivity and uptime.

Option 1: Google Mail. All of the cool kids are doing it. It might have the right features. AFAIK it only has a Web interface; you can get at a flat mail store by POP but that's not so interesting. An @gmail.com address is pretty reputable. I don't know if this would give me the level of control I want, but it's there.

Option 2: Virtual private server. There are various shops that will rent you a fraction of a machine; I've been loosely eyeing Linode's $20/month plan for a while. If I did this, I could run whatever I wanted on it, including chooblog, SMTP and IMAP servers, and some sort of mail sorting system like procmail but less sucky. This clearly meets all of the requirements, but costs money and takes some effort to set up. It also requires me to come up with a domain name, and if I screw anything up on this it's my fault and my problem.

Option 3: Suck it up. Because the current system works okay, and I can identify the particular operations that are painful (starting Gnus, exiting mail.misc.spam), and aside from the speed and the minor sketchiness of keeping an @mit.edu address as an alum it does work. Since I keep mail in AFS I get Ops' quota and backup constraints (well, really, SIPB's) and not Network's.

Does gmail DWIW? Is Linode or some other similar provider reputable, and worth the $n per month? Is maintaining a public mail/Web server that tricky?

6.170 introduced me to the Census Bureau's TIGER/Line data set, and I've been experimenting with it on and off for a couple of years now. The bike trip mapping plot has revealed several gaps in the data that it's not obvious I would have found if I wasn't trying to find routes using the data. But, for example, Farm Street in Dover is broken into two segments, with TLID 87283093 being a very short segment connecting the two labelled "Census 2000 collection block boundary not represented by existing physical feature" that happens to connect Farm Street to Farm Street. Eliot Street in Natick and Washington Street in Wellesley don't line up at the city line in spite of being the same road. That sort of thing.

In poking around at newer TIGER data, I discovered that there's a $200 million federal project to fix these sorts of inaccuracies. Which is great for people like me who use this data this way. But I'm sure the same data is available from commercial sources; it's probably not cheap, but, $200 million? Is TIGER really anything more than a data set used internally by the Census Bureau and by a small number of dedicated amateurs?

...this document discusses the scope of the project a little more. A large part of the project sounds like "redesign our internal database, it's 15 years old" more than "update the data", and also "make it possible for Census field agents to do their jobs and update the database without paper maps". And there's a requirement to support every type of address in the United States, not just the 90% or 99% case. Actually, this is a kind of interesting read if you're curious how the data got put together originally and why it has the problems it has. So the money is mostly sustaining this goofy constitutional requirement that we go around and count people every ten years; it feels a little more sensible now.

Combining the publicly available information about Google Maps, a copy of the Census Bureau's TIGER database imported into PostgreSQL, some Python hacking, and general navigational memory, I now have bike trips projected on to Google Maps.

The big problem right now is TIGER's view of the world being slightly different from Google's; if you zoom in a lot, you can see the points not lining up. Also, look at the bridge between Salem and Beverly in the Rockport trip, which follows TIGER's reality and not Google's. (I also found one bug in TIGER/Line 2003, which I should figure out how to report.) There are also limits (in some cases the Google Maps JavaScript hangs, I'm not sure when but trimming out points seems to help), and a report that this doesn't work under IE.

One of the problems with documents in XML is that getting print out of them is vaguely irritating. The "standard" way is to use XSLT to convert your input to XSL:FO, and then use a tool like FOP or PassiveTeX to convert that to PS/PDF. The obvious problems are that the stylesheets out there aren't very good (printed DocBook is "just okay" aesthetically) and that the free formatting tools suck.

But TeX is widely available, free, and has a good formatting engine. Why does nobody use XSLT to convert XML to *TeX, and then format that? The impedance mismatch between XML's character set and LaTeX's is slightly irritating, but it's not that hard to work around, even without having EXSLT available. And then if you're familiar with both XML and LaTeX, you can probably copy-and-paste your way to happiness given a stylesheet (also true of XSL:FO, but more people know LaTeX than XSL:FO).

For a proof-of-concept, I did this for my resume (not that I'm job-hunting at all, it's just a simple XML document I have kicking around). It could use some formatting tweaks, but it came out pretty well. Certainly it proves that what I'm trying to do is eminently reasonable, and so it's sane to suggest to other people that if they have data they want to query and include in a LaTeX document that they should store the data in XML and then use XSLT to create document fragments rather than trying to write TeX macros to process the data directly.