In the familiar story of Frankenstein, Dr. F creates a
monster, which later destroys him. Conventional criticism of
Frankenstein refers to Prometheus, punished for stealing the
gods' fire, or speaks of Dr. F's flawed relationships. But I draw a
lesson about the unintended consequences of technology.
See, everyone wants to create the monster. In your mind, you see
how it will be; new, and big, and so very, very cool. And you will
control it. It will do your bidding. So you build the thing, all in a
rush of late nights and exciting revelation. It is only when the
monster rises from its slab and starts crashing around that you
realize your control may be imperfect. Then the monster does
something scary and altogether unexpected, and you realize that
control was always an illusion. From apps with security holes to
drugs with side effects to disruptive technologies that unravel
social structures, unintended consequences are the dark side of
innovation. When you solve a problem in a new way, you must consider
whether your solution enables unintended results forbidden to
previous solutions.
RFID tags are one of my favorite examples of Frankenstein
technology. An RFID tag works like a paper label, only you can read
it instantly, with a radio instead of your eyes. It doesn't matter if
the tagged item is upside-down, on a pallet with 99 other items, or
behind another object. At first, RFID looks like a very cool
technology. It makes inventory or checkout a snap.
But then the monster starts to stir. RFID facilitates locating
items in inventory. It also facilitates theft of valuable items
without the need to hunt for them in every crate in the warehouse.
RFID facilitates instant checkout, replacing human eyes, so it also
enables theft by simply removing the tag. RFID provides remote
reading. The walls that once kept your stuff apart from temptation
suddenly might as well be glass, except that a metal box or plate is
opaque, when you expected all tagged items to be visible. RFID tags
on credit cards, driving licenses, and passports, and even the tags
on ordinary items like subway cards and card-keys identify
individuals, evaporating the anonymity of the crowd. If RFID tags
cannot be turned off, they are permanent beacons of identity. If they
can be turned off, that function enables a potent denial of service
attack against any user dependent on the technology.
These risks emerge directly from unintended uses of the technology
as designed, in a world with multiple stakeholders. These risks are
quite aside from risks arising from errors in realizing the
technology. Any risk might have been mitigated in the original
design. Some may still be, but only if all the stakeholders voices
are heeded. Dr. F may care less that the monster terrifies some
peasants, and more when it kills his own wife. The central problem is
that Dr. F created his monster without even considering the trouble
it might get into.
The more rush in development, the fewer use cases are considered.
If you're building a video game, maybe the worst that happens is the
game is unplayable due to griefers. If you're embedding software in a
device with a long service lifetime, the greater is the chance that
someone will exploit any lack of care in a way that you (or your
company) will find painful.
Let the technologist beware, lest your name go down in history as
the man who created the monster.