costela.net

Well, every once in a while we’re forced to do something that isn’t particularly interesting or pleasant. Last week it happened again: I had to import a few pretty big PSTs (most 2Gig, one 10Gig, with about 100.000 emails) into our dovecot IMAP.

Doing this with Outlook itself was out of the question: it took way too long, even on the local network (many hours for a 1G file) and was prone to hanging and crashes, which were obviously a pain to debug and start over.
Thunderbird was unfortunately not much better, since – at least in our tests – it didn’t import the read status of the emails (they were all marked as unread) and also wasn’t particularly good at handling folders with strange names, containing “.”, “/” or some more obscure characters. We had used it before for smaller files, where manually dealing with the problems was acceptable, but this time it required something a bit more elaborate, if we were to keep our sanity.

Enter libpst. It includes the handy readpst utility which dumps all emails in usable formats in a directory tree, one directory per folder. Unfortunately the Debian version is somewhat outdated and doesn’t support the newer Outlook formats, so I did some packaging and even a little bit of patching. It seems Thunderbird also uses this library, which would explain why it didn’t handle the Read-Status (haven’t confirmed this though; just read it somewhere).

The last step was this not-so-little script, which uses the dumped directories from readpst and imports them in IMAP. It would have probably been a bit more elegant to use libpst directly, but I unfortunately didn’t have the time to mess around with that. I did have to mess around a lot with encodings though, ergo the unholy chaos with unicode()s and str.encode()s thrown around like rice at a wedding (I could never really wrap my head around charset problems; the subject boggles my mind to this very day).

code after the jump
Read the rest of this entry »

I may not be the biggest Star Wars geek out there (by far), but I do enjoy almost all soundtracks by John Williams. Specially when it’s being conducted by none other than Darth Vader himself!
This is the Brazilian Symphonic Orchestra (Osquestra Sinfônica Brasileira – OSB) in one of the many songs in a tribute to John Williams (link in Portuguese) last year.
Also see this one for a better introduction, but worse view.

Oh, I so wish I’d been there! Huge kudos to the many who had the presence of mind to record this.

See you there!

A few weeks ago I noticed an image on my Froyo phone which appeared to be a screenshot. This came as a surprise, since I don’t have any screenshot applications installed and am generally economic when it comes to the amount of fluff on my phone. I did a few searches and there doesn’t seem to be any visible mention of this feature out there, only mentions of how to do it with the SDK or with specific apps.

Since today I finally managed to discover how to actually do it again (holding the Back button and pressing Home), I took these screenshots using different launchers to be sure it wasn’t some feature of ADW.Laucher.

I still haven’t beat my laziness to actually reset to factory settings and make sure it’s not some other random app (which would be a bit unsettling, actually) and it could also be a Samsung specific feature, implemented as one of those pesky unremovable apps.

Can anybody out there confirm this (not particularly useful) feature? Did I miss it being published somewhere?

[UPDATE]: I found a couple of “hidden” apps that probably have something to do with this (through Settings→Applications→Manage applications→All).

The first one is an obvious suspect, but since it isn’t named after Samsung (which admittedly doesn’t mean much) I figured it might be activated by the second one, which in my overly-creative and not-really-investigative mind could be a sort of hotkey-daemon. [UPDATE 2: well, that's ignoring the obvious fact that SEC stands for "Samsung Electronics Corporation". D'oh!]
I don’t think the feature is important or interesting enough for me to investigate much further, but still I thought I might leave some pointers if people want to get this on their ROMs.

So you have two geeks in a university room after a relatively late and (at least for one of them) unproductive learning session. It’s just natural that they decide to kick back and watch some mind-numbingly stupid geek series, which in this case happened to be Stargate SG-1 (so absurdly shitty its actually very entertaining).
The first lazy geek instinct is to just watch it on the laptop that has the file, which – with its 11″ display and shitty speakers – doesn’t turn out to be a great idea. The next try involves the other laptop, but a 13″ screen isn’t that big of an improvement.
Since the room our intrepid heroes are in happens to have a pretty decent built-in projector and a couple of small – but still a lot better than a laptop’s – Bose speakers, the obvious next step would be using it. The only problem is the lack of a VGA cable.

Inspired by the brief sight of MacGyver on the 11″ screen, one particularly enterprising geek comes up with the challenge of making a VGA cable out of the only material available at the time: one horribly yellow cat5 ethernet cable.
Being the helpful little extra-dimensional entity that it is, the internet happily provided all the needed information and after some slight problems trying to appropriately deprive the cat5 of its connectors (no scissors and no blades of any kind in sight) and some annoying and manual sticking-cable-to-socket action…

our reluctant hackers get it right:

The final solution looked like this:

And if you’re wondering where those white wires came from, one final touch of über-hackerdom:

This might seem like overkill, but after a nice nice 4 hour movies and series marathon, we can safely say it was totally worth it (but no, we didn’t stand 4 hours of Stargate; even geeks have their limits).

Just in case the Instructables page gets hosed at some point, here’s the invaluable connection diagram, originally scraped off of a since dead Geocities page.

Another one of these technical “reminder” posts.
Calendar Server needs extented attributes on the filesystem it uses to store calendars. This fact has been very helpfully documented by the Debian maintainer, but since there are apparently still a couple consistency checks missing in the server, I’ve spent the last couple of hours trying to find out why Sunbird (actually Lightning) would simply loop like crazy between PROPFINDs and OPTIONs, going nowhere.
It turns out the first time I started caldavd, the underlying filesystem didn’t have xattr support, which meant some files and/or directories in the spool directory tree didn’t get the needed attributes and after a restart the server would just gladly run as if nothing were amiss, driving the client slightly bonkers.

I honestly wish I had the time right now to debug it a bit better and provide some patches upstream, but for now this post will have to suffice as help for any other poor soul out there facing this issue.

Just so I don’t forget it for a second time: esmtp can be pretty obscure when it comes to error messages.
When it encounters an “.authenticate/ca.pem” file with public permissions it connects to the server and only after seeing the extensions list does it tell you “StartTLS extension not supported by MTA“. Not exactly straightforward.

Sometimes Wikipedia shows it even has a somewhat humorous side. My finding it funny may be a product of late night learning sessions and semi-random clicking-sprees, but still, worthy of a chuckle for those with the right (wrong?) inclinations.

Quantum computing could be used to effectively implement a bogosort algorithm with a time complexity of O(n). It uses true quantum randomness to randomly permute the list. By the many-worlds interpretation of quantum physics, the quantum randomization spawns 2^N (where N is the number of random bits) universes and one of these will be such that this single shuffle had produced the list in sorted order. The list is then tested for sortedness (requiring n-1 comparisons); should it be out of order, the computer destroys the universe – implementation of this step being left as an exercise for the reader. The only observers will then be in the surviving universes and will see that the randomization worked the first time and that the list is in sorted order.

Note, however, that while this algorithm is O(n) in time, permuting the list requires O(n log n) bits of quantum randomness. It also assumes that destroying the universe is O(1) in operation.

Found here.

Neil Williams recently commented on the lack of smart-bookmarks in Firefox/Iceweasel/Chromium and since the post doesn’t accept comments, counter-post FTW.
Maybe I didn’t understand exactly what was meant, but I’m personally trying to see the advantage of having a smart-bookmark sit on the toolbar as opposed to just being used via a label. Both FF/IW and Chrome can do the label thing, where you bookmark something like “http://bugs.debian.org/%s” with a shortcut like “bugs” and can then simply Ctrl-L to the address bar and type “bugs 999999″. Done.
Can the way Epiphany does this be more effective? (actually this is the way Galeon did it way back then and I used to love it before I found out I could be way quicker with the keyboard+shortcut thingy. Not to mention having less clutter in the toolbar.)

And granted: Chromium’s interface doesn’t allow the editing of this shortcut and they only work when imported from FF/IW, but I expect this to be fixed eventually. Doesn’t make much sense to have such a “hidden” feature. [UPDATE: nevermind. As handily pointed out by Chris Butler, you can edit the shortcuts under Options→Basics→Default Search→Manage. It might not be the most intuitive of places to put it, but it's there.]

As for the rest of the reasons for switching mentioned in the original post, I can certainly see where they’re coming from. No real solid counter-arguments there.

I don’t presume to know what’s going to happen – if anything – now that Apple outlawed third-party API abstractions, but it did make me think a bit about the different ways people see portability.

The whole dance seems strangely fascinating: developers have obvious reasons to want their applications to run on the most diverse number of architectures – specially in a market which still isn’t so clearly defined as the PC market – and platform vendors have an equally obvious interest in having the best applications around run on their platforms.
But it gets further muddled up when platform vendors control the gateways to the platform, something that’s – as far as I can tell – unique to the mobile market. Now the platform vendor not only wants the best applications for themselves, they can filter out competing applications and leverage the system to keep good applications out of competitors’ platforms.

But back to portability. Portability becomes progressively harder – and less efficient – the higher we go in the abstraction scale.
Hardware portability may have many pitfalls, but is homogenous enough to be achievable in a wide range of targets without sacrificing too much functionality, as is the case for the Linux kernel.
OS portability gets a bit trickier. Even though the underlying hardware may have conceptually similar interfaces, the exposed APIs may vary greatly and the problem becomes two-fold: keeping the amount of OS-specific code low and providing an integrated experience in the different OSs. This can still be somewhat solved either with additional abstraction layers, like Java’s Swing, or with using one ported toolkit, like Gtk for Windows, but both options make sacrifices.
The current mobile market takes these problems a step further, by having wildly different hardware platforms, even more incompatible APIs and to some extent even different interaction paradigms (while most other PC architectures agree on folder/file, click/drag and window/desktop paradigms).

All this culminates in the only non-dubious argument for Apple’s move: quality assurance. They’re hold on developers might turn out not to be as strong as they wanted, but at least in terms of quality, demanding applications to be written specifically for them is a certain win.

Or course, this rambling is pretty much moot when it comes to Apple, which has a force-choke grip on their specific market niche and an equally force-related hypnotic influence over masses of developers and users. They can pretty much put “receive a kick to the balls/tits” as an EULA requirement and you’d still see millions flocking to Apple Stores to buy whatever cute gadget they’ve got next.

It’s just interesting as an open-source developer, to whom portability is such an obvious advantage, to see it being used as leverage tactic.

And in the end I’m only slightly annoyed by Apple’s decision because it’s another reminder of a possible 1984-esque future where access to technology – both by users and developers – is metered out by our not-so-benevolent corporate overlords.
There are certainly other giants out there intent on keeping general-purpose computing alive, but the thought is still haunting, that a sort of K-T extinction event might creep on us without our noticing and cause all the PC dinosaurs to die of uselessness when the mobile devices become powerful enough, suddenly taking away all our hacking freedoms.
Haunting indeed.