Creating Frankenstein's Monster

In the familiar story of Frankenstein, Dr. F creates a monster, which later destroys him. Conventional criticism of Frankenstein refers to Prometheus, punished for stealing the gods' fire, or speaks of Dr. F's flawed relationships. But I draw a lesson about the unintended consequences of technology.

See, everyone wants to create the monster. In your mind, you see how it will be; new, and big, and so very, very cool. And you will control it. It will do your bidding. So you build the thing, all in a rush of late nights and exciting revelation. It is only when the monster rises from its slab and starts crashing around that you realize your control may be imperfect. Then the monster does something scary and altogether unexpected, and you realize that control was always an illusion. From apps with security holes to drugs with side effects to disruptive technologies that unravel social structures, unintended consequences are the dark side of innovation. When you solve a problem in a new way, you must consider whether your solution enables unintended results forbidden to previous solutions.

RFID tags are one of my favorite examples of Frankenstein technology. An RFID tag works like a paper label, only you can read it instantly, with a radio instead of your eyes. It doesn't matter if the tagged item is upside-down, on a pallet with 99 other items, or behind another object. At first, RFID looks like a very cool technology. It makes inventory or checkout a snap.

But then the monster starts to stir. RFID facilitates locating items in inventory. It also facilitates theft of valuable items without the need to hunt for them in every crate in the warehouse. RFID facilitates instant checkout, replacing human eyes, so it also enables theft by simply removing the tag. RFID provides remote reading. The walls that once kept your stuff apart from temptation suddenly might as well be glass, except that a metal box or plate is opaque, when you expected all tagged items to be visible. RFID tags on credit cards, driving licenses, and passports, and even the tags on ordinary items like subway cards and card-keys identify individuals, evaporating the anonymity of the crowd. If RFID tags cannot be turned off, they are permanent beacons of identity. If they can be turned off, that function enables a potent denial of service attack against any user dependent on the technology.

These risks emerge directly from unintended uses of the technology as designed, in a world with multiple stakeholders. These risks are quite aside from risks arising from errors in realizing the technology. Any risk might have been mitigated in the original design. Some may still be, but only if all the stakeholders voices are heeded. Dr. F may care less that the monster terrifies some peasants, and more when it kills his own wife. The central problem is that Dr. F created his monster without even considering the trouble it might get into.

The more rush in development, the fewer use cases are considered. If you're building a video game, maybe the worst that happens is the game is unplayable due to griefers. If you're embedding software in a device with a long service lifetime, the greater is the chance that someone will exploit any lack of care in a way that you (or your company) will find painful.

Let the technologist beware, lest your name go down in history as the man who created the monster.