I play Clan Lord, a multiplayer online RPG. Clan Lord provides a lot of ways for players to customize their characters' appearance, including a system for changing shirt and pants colors with an algorithmic dye and bleach system. Experimenting with dyes and bleaches in-game can be prohibitive — each application consumes in-game currency and (often rare) materials. As a result, there's a long tradition of external dye-simulation tools; the earliest was Stylus (named in the 'function-us' pattern of many Clan Lord NPCs), followed by Stylos, the OS X re-imagining of Stylus. Stylos didn't work well on Mac OS X 10.4 Tiger, so in 2005 I built the first version of Stylunk as a replacement.

Since then the number one Stylunk-related request has been for a Windows version, and the number two request is 'hey, add this new shirt color!'. Both of which were difficult enough to fill that they happened never (Windows version) and only intermittently. So I've decided to simplify cross-platform concerns and centralize updating by turning Stylunk into a web app. The beta is right here.

Palettized recoloring in JavaScript or 'the actual non-background stuff about which I plan to write'

I started playing Clan Lord in 1998 and it had already been in development for a while at that point, so it shouldn't be a surprise that the tech isn't exactly modern. Images in Clan Lord are an run-length-encoded list of indices into a per-image color map, which itself contains indices into the 'Mac standard' 256 color map. That's all pretty integral to Clan Lord, because character customization is implemented as a set of 20 color overrides for the per-image colormap: indices zero and one contain the highlight and shadow colors for the character's hair, for instance.

Images for a particular icon are arranged into a single 16x3 sprite sheet, comprising the 32 standing/walking/running sprites in eight cardinal directions as well as a 'dead' sprite and fifteen 'poses' that players can activate to emote. Here's what the 'Ghorak Zo' sprite sheet looks like:

My first stab at implementing web Stylunk was about what you'd expect: I converted each of the sprite sheets into PNG files and wrote code that draws the selected sprite into a canvas, iterates across the pixels, and replaces the 'default' colors with the mapped values calculated from the current UI selection. Then I created a smaller canvas, copied the frame I needed from the large canvas to the small canvas, and used toDataURL() to extract something that I could stick into the background-image of my buttons.

It all worked fairly well, at least at first. On my main machine (a Core i7 Mac Mini) everything was peachy. On my MacBook Air, it was sluggish. On my iPad... well, the less said the better.

My immediate response was to try some very aggressive caching, leading to the creation of a reusable JavaScript least-recently-used cache. That helped, but not as much as I'd hoped.

My next thought was: I'm doing more work than I need to: there's no point colorizing the entire sprite sheet and then only displaying a single frame of it. So I swapped things around; first extract a small canvas from the larger, then recolor only the smaller canvas. That helped too, but it was still pretty logey on the iPad.

Unfortunately there's no way to profile JavaScript in MobileSafari. But desktop Safari does have a profiler, and it showed something interesting: the slowest part of the process was reading the pixels out of the initial canvas. So I took a cue from Clan Lord and took a step back in time: instead of storing the images as PNG sprite sheets, I wrote a tool that converts each frame of the sheet into a JavaScript data structure containing a run-length encoded bunch of pixel indices. That way I can skip the 'reading from canvas' step, and jump straight to writing them. I also took the step of storing each frame as a separate JavaScript file which the main app loads on-demand via XMLHttpRequest, which simplifies the caching a lot by letting the browser do the work.

If you're curious, you can take a look at the code for a sprite here. That's my JS-encoded version of the male Ghorak Zo facing east — the top-left sprite in the image above. Yup! — despite being larger than a PNG representation or the same image, and despite needing to be parsed by the JavaScript engine, it still ends up being faster than reading data back from a canvas.

The end result is that Stylunk runs at a usable speed on my iPad (first-gen) and on my iPhone 4, which is good enough for me. There are still some minor inefficiencies as a result of the tortured path to the current architecture — if I were doing this from scratch there would also be JavaScript representations of the character color maps, which would simplify the color mapping process and also allow me to easily draw characters using the color map data from the XML data feed powering the Clan Lord Informer and iClan.

I'm actually half-tempted to do that, because it would provide a starting point for building a Socket.IO- and Node.js-powered HTML5 Clan Lord client. But that's a headache for another day.

Of course then I refactored things dramatically and it's barely used at all. But if you need a JavaScript least-recently-used cache, check it out.

class Program
{
    public static Func MakeAccumulator(T start, Func addFunction)
    {
        return inc => start= addFunction(start, inc);
    }

    static void Main(string[] args)
    {
        var intAccumulator = MakeAccumulator(0, (i, j) => i + j);

        Debug.Assert(0 == intAccumulator(0));
        Debug.Assert(1 == intAccumulator(1));
        Debug.Assert(11 == intAccumulator(10));
        Debug.Assert(55 == intAccumulator(44));

        var floatAccumulator = MakeAccumulator(0.0, (i, j) => i + j);

        Debug.Assert(0 == floatAccumulator(0.0));
        Debug.Assert(0.1 == floatAccumulator(0.1));
        Debug.Assert(1.1 == floatAccumulator(1.0));
        Debug.Assert(5.5 == floatAccumulator(4.4));

        var stringAccumulator = MakeAccumulator("", (i, j) => i + j);

        Debug.Assert("" == stringAccumulator(""));
        Debug.Assert("ZYZ" == stringAccumulator("ZYZ"));
        Debug.Assert("ZYZZY" == stringAccumulator("ZY"));
        Debug.Assert("ZYZZYVA" == stringAccumulator("VA"));

        Console.WriteLine("Success!");
        Console.ReadLine();
    }
}

So there's that. Still not terribly useful, but I do like shortening code.

Bad, old jokes aside:

I've said it before, and I'll say it again: the Bush War on Terror was never about hunting down bin Laden. To view it in that light is to characterize it as a campaign of retribution rather than one of pre-emptive self defense. It is a simplistic and reductionist misrepresentation favored primarily by those who opposed Bush in every fashion, and it's a straw man that we would all do well to stop stuffing.

Some people make the world better through their existence, some make it worse, and the vast majority of us largely break even. Osama bin Laden was clearly in the 'worse' column, and in that sense the world is a better place with him a corpse. But to claim that his death in any measurable way makes us safer or surer or more just is folly.

Including, fatefully one from the Playstation Network (link goes to a Flash-only site, because Sony's just in full-on pissing-me-off mode at the moment).

If you're reading this there's roughly a 99% probability that you already know that the PSN has been 'out' for nearly a week, and the latest news from Sony is that it ain't comin' back anytime soon, and oh, by the way, some hackers probably have your credit card numbers.

I know this the same way you do, through the magical collaborative power of the Internet. Not, say, from an email sent to me by Sony warning me that my credit card number had been stolen. Strike one, Sony. You assholes.

But as annoying — and probably actionable — as their failure to proactively warn their customers may be, that's not what really pisses me off.

No, what's really bugging me is that I can't use Netflix on my PS3.

Why? Because Netflix on the PS3 — despite being tied to my totally independent non-Sony Netflix account — requires a PSN login to function. For no reason at all except that Sony are a bunch of goddamned morons.

Thank God for the Apple TV.

As the story goes, Catherine's inspection took the form of a leisurely cruise down the Dniper River. Potemkin, being aware that the Crimea was kind of a crummy area and fearful that the Tsarina would mislike the realization that her new territory kinda sucked, ordered the hasty construction of numerous fake village façades along the river bank. The illusion held; Catherine went home thinking the Crimea was a bustling hub of non-squalid not-mud, and our modern world gained a vivid new idiom: 'Potempkin Villages'.

Of course all the actual evidence is that the story is bunk — Potemkin may have instructed the peasants to clean up a bit and maybe paint some fences, but the idea that the whole area was an elaborate set was an extreme exaggeration spread by his political rivals. But it's one of those stories that's just too good to not tell, isn't it? It's 'truthy'.

Changing direction entirely: unit testing is the new hotness in software development. The basic idea is that you decompose your software into small, manageable units and create dedicated automated tests that verify the functionality of those units. And it is, in general, a really good idea.

The problem that crops up with unit tests is that some things are just really hard to test, particularly when your code interacts with something outside your control — such as a human user, a piece of hardware, or a third-party web service. In those cases you need to get tricky; you create new layers of abstraction that separate the interaction from the main work of your module, and then you create 'mock' version of those abstraction layers and substitute in a 'simulated user' for a real user, or a 'simulated serial port' for your hardware, or a 'simulated web service' for the third-party service.

In some cases that approach may make sense. For some pieces of hardware the range of possible states is limited; a button is up or down, and it's never both. If a user is given an extremely limited set of buttons to click, you can reasonably simulate every possible interaction model. But for anything more complex, you're kind of creating a Potemkin unit test. So you've extracted out an ITwitterApiProxy that your TwitterService talks to, and you can plug in your MockTwitterApiProxy and write a whole host of tests for your TwitterService. Every failure mode you can think of can be simulated by the MockTwitterApiProxy and you've coded the TwitterService to deal with all of them reasonably — and hurray, the unit tests mean you're safe!

Now you go to production, with the RealTwitterApiProxy, and stuff goes wrong. Twitter's failing in ways you never anticipated! So you add more switches and dials to your MockTwitterApiProxy, and add more code to your TwitterService to deal with them, and write tests to exercise them... and hurray, the unit tests mean you're safe! ...at least until Twitter fails in another new way you haven't seen.

Compare that to how you would have handled things without unit tests: You code up TwitterService, handle the error cases you anticipate, go to production and discover some more error cases, update TwitterService to handle them... lather, rinse, repeat. It's essentially the same workflow.

So what does the extra effort buy you? What's the benefit of building your API abstraction and unit tests?

Well the obvious answer is that it buys you protection against regression. You have an explicit test in place for handling error type A, so if in the course of implementing support for type G errors you accidentally break type A, the unit test will fail and you'll find out. There's that, certainly. Without unit tests it's basically down to trust: I trust myself (or my fellow developers) not to break this part of the code while working on that part of the code.

But on the other hand, your test for handling error type A isn't really testing your handling of error type A, it's testing your handling of your mock simulation of error type A. How can you be sure that in the course of adding code to simulate error G to your mock you didn't accidentally break the code that simulates error type A? Isn't that the same sort of "trust yourself" approach that the non-unit-testing approach is left to? Should you add unit tests for your mocks to ensure that they're properly simulating errors?

Ultimately a lot of this is down to personal judgement, like most of the interesting questions in software development. And I'm certainly not going to argue that unit tests aren't a good idea a lot of the time. But I do believe that there are certain situations where unit tests create a lot of extra code to maintain without really adding much in the way of protection, and that committing to building such Potemkin unit tests is a foolish consistency of precisely the sort that will prove a poor substitute for Norman Osborn.

The question has a number of rather pertinent ramifications. It's good to have an answer.

If your company is a family, then when you invade my workspace for a family activity, preventing me from getting any work done and thereby causing me to miss deadlines and slip schedules, well, that's okay. We're family. We'll deal. Pass the cake.

If, on the other hand, your company is a company, then when you invade my workspace for a family activity — preventing me from getting any work done and thereby causing me to miss deadlines and slip schedules — it's on me to make up the time. Your actions have become my problems, which is ample justification for me to be more than a little peeved.

So is your company a company, or is your company a family?

If your company is like most companies, it's both — you get to do what you want, and I get screwed with no right to be angry about it.

Here's a protip about 'most companies' — people good enough to get jobs elsewhere... do.

Teenagerititude is the state of believing that everyone else in the world is phenomenally stupid — that the solution to every problem is blatantly obvious, and that everyone would be much better off if they'd just shut up and do as you say.

The interesting thing about being a teenager is that you don't actually outgrow it.

At some point a light goes on in your mind, the scales drop from your eyes, the metaphor similes upon you, whatever, and you realize you've been a teenager and you stop. You congratulate yourself on being so adult and on owning up to your past bad acts and you move on with your life.

And then a few years later you realize that you were mistaken, that you were actually still a very-slightly-more-evolved form of teenager despite that revelation, and the scales drop from your lights and you eye what you metaphor and you move on with your life.

Until it happens again.

Eventually you reach a degree of meta-awareness — you recognize that Socrates kinda had a point about the whole 'knowing you know nothing' schtick. That's when you ascend to a higher plane of existence! Then you help Teal'c and MacGuyver out a few times, and eventually you return to the show with your tail between your legs because it turns out your landlord won't accept 'art' in lieu of money. But hey, higher power. You got that going for you.

Until, damn it, it happens again.

Meta ain't enough, nor is meta-meta. Maybe there's some omega-meta state where you stop realizing that you're an idiot, and you get to draw a Batman logo on Anthro's cave wall just to reassure the idiots who didn't figure out the pattern based on Kal and Hal and whatnot. But I can't be sure of that. I think any form of personal growth boils down to suddenly recognizing what a jackass you've been and thus becoming an exciting new form of jackass.

...

So what's the moral of the story? The moral is that I'm stupid, you're stupid, he's stupid, she's stupid, and the primary differentiating point between us is our awareness of our own incompetence. So when something looks dumb to you, remember that it either is dumb, or it's smart in a way you haven't thought yet deciphered — and prudence dictates that you assume the latter until you've assembled reasonable support for the former. And even if something turns out to be legitimately dumb, don't draw conclusions from that, because oftentimes there's more to the story. It's not uncommon to find clever implementations of a piss-poor design; that may be a sign of abstractional schizophrenia, or it may be a sign of the blind leading the brilliant.

*If you currently are a teenager, shut up and get the hell off my lawn. Damned kids.

So that's the way video rentals work, and they all stock VHS cassettes, and that's that. But now a sea change occurs: somebody invents DVDs. All of the video rental shops start building their stock of DVD titles, and while doing so they take advantage of their affiliation network, so if your favored shop doesn't have Joe Dirt on DVD, they can get it from the shop across town.

But one little shop bucks the trend — even though DVD is the future, they decide to stick with VHS. And that works out okay for them, because while the other rental shops are investing heavily in building a DVD library, they're making room by selling off their VHS cassettes, and this little dark-horse shop is gobbling them up like popcorn, and their library is huge.

Now it's ten years later. Every video rental shop around the world has switched to DVD, but this little shop is still on VHS. It's impossible to buy a VHS deck in stores, so this shop rents or sells you a VHS player when you sign up for membership, but that's okay actually — some people find that kind of convenient.

But now something interesting happens — a big, new, hit movie comes out... but only for DVD. This little shop is caught-flat footed, because they simply can't compete here. So what do they do? They advertise!

This shop starts running ads, talking about how the local Blockbuster has a tiny library of titles, and it's absolutely terrible how small their library is. Now, in a sense that's true; any individual Blockbuster may have a small library available, but keep in mind the affiliate network — when you get a DVD at Blockbuster, you may be getting it from any one of dozens of DVD rental places all over the world.

---

So what does any of that have to do with anything?

Well, VHS is the creaky old CDMA digital cellular technology. DVDs are GSM, the modern cellular network architecture that almost every cellular carrier around the world uses. Our hit DVD is obviously the iPhone.

And the tiny dark-horse video shop, that's Verizon.

Now, don't read this as a defense of AT&T. Personally I haven't had all that much trouble with them, but I know plenty of people who reflexively spit upon hearing those letters, and it's pretty clear that AT&T has been using their iPhone windfall to make money hats rather than investing in their network infrastructure, and that's clearly subpar. But while I dislike incompetence and don't care for spin, I absolutely loathe deception, and that's what Verizon's 'map' ads amount to, in my opinion.

The 'Verizon map' in those ads is, by necessity, a map of Verizon-owned-and-operated coverage zones; they have to be Verizon-owned-and-operated, because nobody else is running CDMA these days, just like nobody's using VHS or Windows 95. The 'AT&T map' to which they compare themselves, meanwhile, is of AT&T-owned-and-operated coverage areas, which is not at all representative because AT&T has reciprocal carriage agreements with GSM carriers throughout the US, and roaming agreements with GSM carriers around the world.

Personally, I can't stand Verizon — they have a policy of installing Verizon-controlled firmware on every phone they support, which tends to cripple the phone by removing built-in features to encourage you to use Verizon's expensive and inferior network-based equivalents. Ever try to copy a ringtone to your Verizon phone using standard Bluetooth file transfer? If you did, I'm betting it didn't go so well. But Verizon does have a lot to offer customers, and if they want to be taken seriously they should compete by explaining the benefits of Verizon rather than deceiving people about the non-deficiencies of their competitors.

If over a year has passed and two of those three people are still on the list, your company is probably screwed. You might not actually have realized it yet, but that doesn't make it any less true. Your management or business metrics may actually be positive, but if your development team isn't improving, the technology underlying everything is stagnating and sooner or later the smell will reach your customers.

Unless you already employ the best programers in the world, of course.

And if you really and truly believe that you employ the best programmers in the world, then unless your company name rhymes with 'Foogle', 'Schnapple', or 'Sicromoft', you're probably really screwed.

Asynchronous function calls seem, at first blush, only a little more difficult — instead of calling DoSomething() and executing your follow-up code after it returns, you pass in a callback: BeginDoSomething( Action callback ).

So, problem solved, let's go home. Unless you need a return value, that is. But even then it seems simple; to turn a synchronous method like int CalculateSomething( ) into an asynchronous method, you just pass it a delegate that takes a parameter: void BeginCalculateSomething( Action<int> callback ).

So is that it? Nope. Because all of that is wrong.

Even though the original DoSomething( ) method had no return type, it still had a return path — it could throw an exception. Let's imagine that BeginDoSomething looks something like this:

public void BeginDoSomething(Action callback)
{
    PrepareForLongRunningOperation( );
    
    ThreadPool.QueueUserWorkItem(
        delegate
            {
                LongRunningOperation();

                callback();
            });
}

A handy way to think about this sort of thing is to figure out where a thrown exception would emerge.

If something goes wrong within the call to PrepareForLongRunningOperation, that happens in the same context as the calling code — any exceptions will throw up to the calling code and come out of its call to BeginDoSomething. The same applies to the call to ThreadPool.QueueUserWorkItem — no problem there.

But what if LongRunningOperation throws?

LongRunningOperation would throw up into whatever internal part of the ThreadPool implementation actually launched it. That exception can't come out of ThreadPool.QueueUserWorkItem, because by the time the asynchronous anonymous delegate is running ThreadPool.QueueUserWorkItem has already returned. And since the exception can't come out of ThreadPool.QueueUserWorkItem, it also can't come out of BeginDoSomething — which means there's no way for the calling code to get the exception.

There are two main approaches to this problem — error handlers and completion calls.

Instead of passing your begin method one callback, pass two: a callback to be invoked if everything goes to plan, and an exception-accepting callback to which errors will be passed.

public void BeginDoSomething(Action callback, Action<Exception> errorHandler)
{
    PrepareForLongRunningOperation( );
    
    ThreadPool.QueueUserWorkItem(
        delegate
            {
                try
                {
                    LongRunningOperation();
                    callback();
                }
                catch(Exception ex)
                {
                    errorHandler(ex);
                }
            });
}

// Sample usage:

BeginDoSomething(
    delegate
    {
		// Do something now that we're done
    },
    delegate(Exception ex)
    {
		// Do something with the error
    });

There are strengths and weaknesses to this approach. The biggest strength of this model is that it forces the calling code to think about error handling — the prompt for it is right there in the method signature. Error handling tends to fall through the cracks in any sort of code, but it's especially easy to overlook in an asynchronous context (It's also a lot more dangerous in an asynchronous context, because often dropping a callback invocation will cause a process to spin forever, sucking down resources and accomplishing nothing).

Separating the success case from the failure case may be either a strength or a weakness, depending on the particular task. Sometimes it makes your code much cleaner, but it often happens that your success and error handler need to share context and implementation, which can make for some very ugly code.

Instead of just invoking a parameter-less Action callback or a single-parameter Action<TReturn> callback, your code calls a single-parameter callback and passes it an Action or Func<TReturn> that the callback in turn invokes.

public void BeginDoSomething(Action<Action> callback)
{
    PrepareForLongRunningOperation( );
    
    ThreadPool.QueueUserWorkItem(
        delegate
            {
                try
                {
                    LongRunningOperation();
                    callback(delegate { });
                }
                catch(Exception ex)
                {
                    callback(delegate { throw ex; });
                }
            });
}

// Sample Usage:

BeginDoSomething(
    delegate(Action complete)
    {
        try
        {
            complete();
        }
        catch (Exception ex)
        {
            // Do something with the error
            return;
        }
        // Do something now that we're done
	})

At first blush, this seems like a much clumsier solution; you're essentially trusting the calling code to call your completion method. That's true, at least in this case.

Where completion calls really shine are for asynchronous calls returning values; instead of calling their callback and handing in an Action, you call their callback and give it a Func<TReturn>, which they then must invoke to get their result. That gives you an opportunity to throw exceptions that they can't cleverly bypass:

public void BeginCalculateSomething(Action<Func<int>> callback)
{
    PrepareForLongRunningCalculation();

    ThreadPool.QueueUserWorkItem(
        delegate
        {
            try
            {
                int result = LongRunningCalculation();
                callback(() => result);
            }
            catch (Exception ex)
            {
                callback(delegate { throw ex; });
            }
        });
}

// Sample Usage:

BeginCalculateSomething(
    delegate(Func<int> complete)
    {
        int result;
        try
        {
            result = complete();
        }
        catch (Exception ex)
        {
            // Do something with the error
            return;
        }
        // Do something with the content of result;
	})

Personally I prefer completion calls, mainly because the the pattern works so well for return values. In practice, any time you need this sort of a pattern it's because you care about return values; if you need to ensure that X happens after Y, it's generally because X depends on the result of Y. If X doesn't depend on Y, that's often a sign that you're being too linear in your analysis and that the tasks should be happening in parallel.

Microsoft seems to have collectively reached the same conclusion; IHttpAsyncHandler, the asynchronous methods off of SqlCommand, and the asynchronous forms of WebRequest all use the completion call approach.

My function takes a function and when done it calls that function, passing a function that the calling function calls for the result.

Now in fact my function calls another function taking a function accepting a function to call for its result, and to that function it passes a function which when called calls the first function passing a new function that when called calls the function that was passed to the function that my function passed to the other function, thereby returning the result of that function to the function that called my function.

And they say you can't write Lisp code in C#.

I'm trying to decide the best way to show text bubbles. Everything's pretty scrupulously iPhone-friendly this time around (that's actually what got me started on Xclan again, figuring out how to work with UDP CFSockets in a run loop), but as I understand it uploading new text textures every few frames will bog an iPhone down pretty quickly. I'm thinking of switching my NSOpenGLView to a CAOpenGLLayer and putting a bunch of CATextLayers on top for the bubbles.

There's actually an NSTextField in a layer in the game window; on Leopard it floats above the NSOpenGLView, but in Snow Leopard (where the screen shots were taken) it's invisible. I haven't pursued it because I was planning on replacing the OGL view with a layer anyway, but it's a strange difference.