Creating Frankenstein's Monster

In the familiar story of Frankenstein, Dr. F creates a monster, which later destroys him. Conventional criticism of Frankenstein refers to Prometheus, punished for stealing the gods' fire, or speaks of Dr. F's flawed relationships. But I draw a lesson about the unintended consequences of technology.

See, everyone wants to create the monster. In your mind, you see how it will be; new, and big, and so very, very cool. And you will control it. It will do your bidding. So you build the thing, all in a rush of late nights and exciting revelation. It is only when the monster rises from its slab and starts crashing around that you realize your control may be imperfect. Then the monster does something scary and altogether unexpected, and you realize that control was always an illusion. From apps with security holes to drugs with side effects to disruptive technologies that unravel social structures, unintended consequences are the dark side of innovation. When you solve a problem in a new way, you must consider whether your solution enables unintended results forbidden to previous solutions.

RFID tags are one of my favorite examples of Frankenstein technology. An RFID tag works like a paper label, only you can read it instantly, with a radio instead of your eyes. It doesn't matter if the tagged item is upside-down, on a pallet with 99 other items, or behind another object. At first, RFID looks like a very cool technology. It makes inventory or checkout a snap.

But then the monster starts to stir. RFID facilitates locating items in inventory. It also facilitates theft of valuable items without the need to hunt for them in every crate in the warehouse. RFID facilitates instant checkout, replacing human eyes, so it also enables theft by simply removing the tag. RFID provides remote reading. The walls that once kept your stuff apart from temptation suddenly might as well be glass, except that a metal box or plate is opaque, when you expected all tagged items to be visible. RFID tags on credit cards, driving licenses, and passports, and even the tags on ordinary items like subway cards and card-keys identify individuals, evaporating the anonymity of the crowd. If RFID tags cannot be turned off, they are permanent beacons of identity. If they can be turned off, that function enables a potent denial of service attack against any user dependent on the technology.

These risks emerge directly from unintended uses of the technology as designed, in a world with multiple stakeholders. These risks are quite aside from risks arising from errors in realizing the technology. Any risk might have been mitigated in the original design. Some may still be, but only if all the stakeholders voices are heeded. Dr. F may care less that the monster terrifies some peasants, and more when it kills his own wife. The central problem is that Dr. F created his monster without even considering the trouble it might get into.

The more rush in development, the fewer use cases are considered. If you're building a video game, maybe the worst that happens is the game is unplayable due to griefers. If you're embedding software in a device with a long service lifetime, the greater is the chance that someone will exploit any lack of care in a way that you (or your company) will find painful.

Let the technologist beware, lest your name go down in history as the man who created the monster.

Resumes: The 80% Rule

I've sat on both sides of the interview table. I've been a lead looking for competent hires for a complex software project. My experience hiring staff can't be that different to the mainstream.

I'm amazed at the resumes I receive. I'm amazed at the quantity; about 100 resumes back in the day when we wrote newspaper ads. Somewhat fewer for an ad on hotjobs or monster. When I'm hiring, I don't have HR filter the candidates, because if they do, I don't get any resumes. I love the folks in HR, but they just don't know enough about software. So, 100 resumes in a pile, or 100 resumes in my in-box.

About 80 per cent of the resumes I receive are unqualified on their face for the clearly described position for which I'm hiring; retreaded poets and astronomers; guys with no experience at all in the languages or technologies I specified; people with no degree and no experience and no apparent ability to write grammatical english sentences. People whose home address is on a different continent (plus they have no experience). I guess these guys send a resume to absolutely every job posting, web site, and advertisement. They might as well not bother. These resumes go in the recycling on the first pass. (HR keeps a copy for awhile, because there is a requirement that they do so).

That leaves about 20 resumes that are worth a second look, which is to say a careful, front-to-back reading. Of these resumes, about 80 per cent are unqualified in some specific way. You can understand why they applied, but they are not the droids you are looking for. Not enough experience, not the right experience, or there's just something funny about them.

That leaves five resumes that are people you want to call up and talk to and maybe have in for an in-house interview. Of these five, there are maybe one or two (20% again) that you might make an offer to.

And there is another one who is lying to you. You bring them in for an in-house interview and their experience melts away. They clearly don't know what they're talking about. They just aren't what they say they are. Posers; fakes; cheats. And not very good ones, because you bust them in a couple of hours of interviews.

In the modern age, the barriers to submitting a resume are far lower, but there are more jobs. What that means to a hiring manager is more resumes that do not make the first cut, and fewer qualified candidates from whom to choose. You might not even get one candidate you want to hire, so your job stays open longer...and longer...and longer, while your team misses milestones because they are too few.

So, if it seems like your beautiful resume disappears into a black hole every time you submit it, remember that it's a needle in a pretty big haystack. If you get called, you made the first two cuts, the 20 per cent of the 20 per cent. That's pretty good.

Coder's Block

I used to get coder's block, which is like writer's block, only for software. I'd stare at an empty screen, having no idea where to start. I'd dither between two implementation directions, unwilling to commit to either course, while the cursor blinked away the seconds in the corner of that empty screen.

I cured my coder's block, by the same technique recommended to cure writer's block; write something.

OK, initially this is hard advice to follow. By definition, if I didn't know what to write, what could I write? If I didn't know which implementation path was best, wouldn't I potentially be wasting my effort writing the wrong thing?

The solution that worked for me was to write something different. If I didn't know what code to write, I would write down a list of requirements (in prose) for the code. If I was looking at a big undefined area and I didn't know just what I'd need, I would write a structured walkthrough, again in prose. Or, I would write the function header, so that what I was doing had a name and an argument list and a return value. If I was feeling particularly virtuous, I would write a module test attempting to use the unwritten method to solve the problem.

And this is what happened. As soon as there was something on the screen, I could begin to review it. I might realize that I didn't know what the requirements were. That gave me something to think about, and pretty soon I did know. If I did a walkthrough, I would find myself writing in the passive voice, which would mean that I didn't know what object called which method. Now I had a hook on which to hang further thought. If I wrote the function name, I could begin to think about whether I really needed to do a thing with that name. Every word I wrote got thoughts out of the nebulous fog inside my skull and down in (temporarily) solid and immutable text where I could see it and analyze it and refine it.

If I couldn't write a paragraph, I'd write a bullet list. If I couldn't write a bullet list, I'd write a few conceptual phrases. What I would not do is get a cup of coffee or read my email or surf the web. That way lay madness. OK, maybe just a little surfing, but I would hold it in mind that I was procrastinating, and what I needed to do was to grind it out.

I know some people who deliberately distract themselves with some email or web surfing. Maybe that works for them, or maybe they are self-deluded. But for me doing something else is just postponing the hard stuff. I know some people who say the answer comes to them in a dream. In my dreams I just worry about being blocked, although sometimes in the morning I have amazing insights in the shower, when I'm very fresh.

When I grind it out, I almost never find I've gone down the wrong road. Perhaps that's part of being an Old Hand. Come to think of it, I conquored my coder's block about the same time I became an Old Hand.

2,000 Mondays

This post is a warning to undergrad CS majors. The stories you hear about the "real world" are not the truth.

There's this place called Xerox PARC. At the dawn of the Computer Age they invented personal computers, and Ethernet, and Smalltalk, and windows and stuff. And the conference rooms were full of beanbag chairs and the streets were paved with gold. This is what your professors think the real world looks like.

Xerox PARC, ca. 1980: A fantasy of the real world

There's this cool startup you'll hear about (a different one each year), where they give you a MacBook to develop on, and there is free pop and snacks in the fridge, and wine on Fridays, and the office is decorated in twenty-something chic, with a universal gym and foosball tables, and you can work any hours you want, and dress like a slob.

This is the "real world" you hear about in college. It's a world where nerds rule and employers are so desparate for top talent that they'll do anything to keep you happy.

It's a fantasy; a fairy tale told by people (students) who have never even seen the real world, and people (professors) who turned their backs on the real world in favor of a place where you could never ever ever be fired no matter how you behaved, once you got tenure.

Mostly, the real world looks like this.

Sorry, this is a lot more real

The real world is a very interesting place. There are lots of challenging, engaging projects in the real world. But it's not a playground. Most offices are cube farms or open bullpens. There are a whole lot of add/change/delete screens and login pages to code for each unique and challenging algorithm you get to invent. There are a thousand lines of somebody else's poorly written code to patch for every line of your own unique software stylings.

Employers really are desparate for top talent. Only the thing is, new grads aren't top talent, no matter how smart you are. You're a n00b, a novice. You won't even know how much you don't know for a couple of years. To employers, top talent means that one guy on the team who turns out 10 times as many lines of solid, functioning code as anyone else; who works 80 hour weeks for 40 hours of pay; who's been coding since he built his first computer from individual logic gates when he was six. Top talent has already worked for Microsoft and Google and Facebook. That's how your employer knows they're top talent. You haven't worked for anyone at all.

There are indeed startups with free food, and wine, and foosball, and a fancy office. They just aren't all the same startup. And most of them are in just one or two cities; cities where a decent house costs a million bucks; cities you don't currently live in. What every startup has is long hours and low wages and stock options, which 99 times out of 100 expire worthless, and 1 percent of the time make you a millionaire. Most startups only live for a year. Then they crash and take your dreams of wealth with them, leaving you unemployed, wondering why you worked so hard for so little.

In the real world, if you're lucky, you either play nice with others, or find yourself suddenly unemployed. If you aren't so lucky, your employer doesn't care if you smell like sweat socks and annoy your colleagues with conspiracy theories, because basically they don't even care if you are a carbon-based life form, as long as you crank out code.This type of employer treats you like a replaceable part, and discards you like last years' cell phone when they're done with you.

In the real world, your CEO is probably not a geek. He's probably an extraverted, glad-handing, back-slapping, ex-frat-rat, because weirdly enough that's the right skill set for a CEO. He definitely votes Republican. And he thinks you look a lot like the nerdy kids he used to stuff into lockers and trip in the cafeteria.

Working in the real world is a job. It can be a lot of fun, but it's also a lot of work, and a certain amount of tedium, and a certain amount of putting up with stuff you don't like. There are 2,000 Mondays in the average career. Anyone who tells you different doesn't know what he's talking about.

How I Became an Old Hand

One morning, about 17 years into my career, as I sat down in front of my terminal to begin a day's coding, a thought occurred to me. I knew, with the certainty of muscle memory, that any idea I could think up, I could turn into working code. What was more important, I realized that this hadn't been true a year ago. There were things I wouldn't have dared attempt because they seemed to me too hard or ill-defined.

Seventeen years is a long time to master a craft. I'm sure there are people who became master craftsmen in fewer years. I'm also sure that anyone who thinks they have mastered software development in one year, or in five, is delusional. Mastering the craft of software development is like mastering a martial art, sport, or musical instrument. There are aspects of the craft that just take time to develop. There are no shortcuts.

You are an Old Hand when you know you are one, with the certainty of completely internalized reflexes. There is no test. There is no certification. There is no course of study. You cannot buy mastery with any currency but time and practice.

That does not mean you cannot improve. Mastery is only obtained through study, but study doesn't lead to mastery. Study does lead to knowledge, and to improved practice. But mastery comes from a different well.

Mastery doesn't mean you cannot improve either. With considerably more years practice behind me, I find coding tasks come easier and easier. And I'm still learning too.

The Trouble With Lint

You all know what lint is; those little bits of white fuzz that stick to your clothes. Lint isn't exactly dirt. It mostly just makes your clothes look untidy. Mostly.

Software has lint too; those compiler warnings you see every time you do a clean build. You've seen them a thousand times before. You know they're harmless. If they're in your code, maybe you think, "I need to fix that someday. But not now."

This posting is about why it is important to remove lint from your software sooner rather than later.

You see, lint has a dark side. Enough lint in the same place is flammable; sometimes even explosive. Houses burn down each year from fires that start in the clothes dryer.

Software lint has the same nature. Too much lint; too many warning messages; and you stop paying attention. Then when an important warning appears, you don't notice. At a previous employer, a team of half a dozen engineers spent four 16-hour days (including Saturday and Sunday) hunting a bug that held up an urgently needed release, when the problem they were hunting was printing a visible compiler warning. It was one of 200 such warnings, so nobody paid any attention.

I was going through some code today, removing lint. I came across this warning.

Foo_UnitTest.h:178: warning: right-hand operand of comma has no effect

Anyone know what this means? Here's a hint; the code at line 178 read

delete [] buf1, buf2;

Still confused? Here's an explanation. C and C++ share a little-known feature called the comma operator. The expression <expr1> , <expr2> evaluates <expr1>, then throws the result away (except for side effects), then evaluates <expr2>.

I imagine the person who wrote this line of code intended to delete the storage pointed to by both buf1 and buf2, but that's not what happened. Instead, the expression delete [] buf1 was evaluated, then discarded (except for the side effect of deleting buf1), and then the expression buf2 was evaluated. So buf2 didn't get deleted.

If you say, "Well, it doesn't really matter because it's just a unit test." then you are entirely missing the point. Removing lint is a discipline that results in clean code that runs the way you expect. If buf2 has a non-trivial destructor, there's no telling how much code isn't getting run (and therefore isn't getting tested) because buf2 isn't getting deleted. You wouldn't know that without investigating what the type of buf2 was.

Even lint that doesn't affect the generated code, like unused variables, can be a problem if these unused variables confuse a future reader of the code. It's easy to comment out this code if you want it to stay in the listing. Easier yet to delete it.

C++ provides syntax for defining functions that don't use all their arguments.

void classname::func ( int parm1, int /*parm2*/ ) {...}

defines a method that takes two int arguments, but only uses the first one. Sometimes you need this for defining base-class methods meant to be overridden in derived classes. Commenting out the argument name turns off the compiler warning, and clearly expresses your intent.

Sometimes your compiler insists on issuing a warning for code that you know is ok. If you can't fix the code with an explicit cast or something, the language implementation probably provides a #pragma for temporarily turning off the warning. You can go even further, running a static checker like lint(1) or cppcheck and fixing the code to remove the warnings it produces. There's just no reason not to have a clean build, and every reason to have one.

Learning About Robustness

Something I think about from time to time is, "What do Old Hands think about code, that is different from what rookies think?" Here are some examples.

• When I was a rookie, I believed programs should be precise. If the program parsed some input, that input should conform. I believed that a program was robust if it printed a detailed error message and halted, so I could quickly fix the input.
  
  As an experienced developer, I realized that processing the input was far more important than ensuring that every comma was in the right place. I came to believe that a program was robust if it accepted a generous superset of the expected input syntax. I remember hearing this advice as a rookie and snorting with derision at the thought of anything so sloppy. This is another difference between Old Hands and rookies.
  
  When I became an Old Hand, I realized that any program you run frequently provids a service. Any time the program halts prematurely, it fails in its reason for being, which is to provide the service. I discovered that even a program that is not working correctly may be more useful than a program that won't run. I learned that a program is robust if it provides its service, all the time, under the widest range of conditions. A robust program should not fail, and if it does fail it should recover, and if it cannot recover, it should restart, and if it can't restart, another program should restart it. The code to provide all this robustness amounted to as much as 50% of the total, but I no longer viewed that as wasteful excess.
  

• When I was a student, I never checked return codes. "How could a system function possibly fail?", I thought. I was naive enough to assume that the people who write system functions never made mistakes, and hardware never failed.
  
  When I was a rookie, I discovered that even if a function failed only once in a million calls, I would see it fail. A million calls just doesn't take that long when a program is running flat out all day. Such a program would terminate unexpectedly, usually saying no more than "segmentation fault".
  
  As an experienced developer, I learned that functions fail all the time, but I hadn't known it, because I wasn't checking the return codes! Usually functions failed because the argument values were illegal. I remembered spending hours looking for why my program wasn't working, when the functions were telling me exactly what was wrong. I meticulously checked every error return, and reported failure up to higher levels of the code. I also began writing functions that did more checking and logging, because when these functions failed, they told me what I was doing wrong.
  
  Once when I was old enough to know better, I wrote a library of meticulous functions tthat checked every return code, to check out some functionality in Windows. It took days. A colleague who was an Old Hand bodged together an informal but usable tool in a couple of hours because he didn't check return codes. I learned something that day.