Your search for “history” did not return any results.

I often think about how to preserve data. This is mostly driven by my photography habit. My pictures are not fantastic, but they mean a lot to me, and I suspect, but am by no means certain, that they will mean something to my children and grandchildren. I certainly would love to know what the lives of my own grandparents were like, to see them in stages of life parallel to my own. But I don’t know how to make sure my kids and their kids will be able to see these photos.

A box of old pictures
A box of old pictures

This is a super difficult problem. The physical media that the images are stored on (hard drives, flash cards, etc) degrade and will fail over time, and even if they don’t, the equipment to read that media will become scarce. Furthermore, the format of the data may become undecipherable over time as well. I have high confidence that it will be possible to read jpegs in the year 2056, but when you get into some more esoteric formats, I dunno.

A commonly  proffered solution is to upload your data to a cloud service for backup. I have strong reservations about this as a method for long-term preservation. Those cloud backups are only good as long as the businesses that run them have some reason to continue to do so. Subscriptions, user accounts, and advertising driven revenue seem a poor match for permanent archival storage of anything. Who, long after I’m dead, is going to receive the email that says “your account will be closed if you do not update your credit card in 30 days”? Also, what good is a backup of data I can no longer view on my now-current quantum holographic AI companion?

All of this compares quite unfavorably with a common archival technique used for informal, family information: the shoe box. Photographs stored in a shoe box are susceptible to destruction by fire or flood, but they are fantastically resilient to general benign neglect over exceedingly long periods of time. Sure, the colors will fade if the box is left in a barn for 50 years, but, upon discovery, anyone can recognize the images using the mark-I human eyeball. (Furthermore, it’s really astounding how easy it is to use a computer to restore natural color to faded images.)

There is simply no analog to the shoe box full of negatives in today’s world. Sure, you can throw some flash memory cards into such a box, but you still have the readout problems mentioned above.

As people migrate from their first digital camera to their last digital camera to iPhoneN to iPhoneN+1, lots of images have already been lost. Because of the very short history of digital photography, you can’t even blame that loss on technological change. It’s more about plain old poor stewardship. But just to amplify my point above: the shoe box is quite tolerant of poor stewardship.

*   *   *

Okay, so, this post was not even going to be about the archival problems of families. That is, in aggregate, a large potential loss, made up of hundreds of millions of comparatively smaller losses.

The reason I decided to write today was because I saw this blog post about this article, in which it was described how the on-line archives for a major metropolitan newspaper — going back more than 200 years, are in risk of disappearing from the digital universe.

Here we have a situation in which institutions that are committed to preserving history, with (shrinking) staffs of professional librarians and archivists are failing to preserve history for future generations. In this case, the microfiche archives of the print version of the paper are safe, but the digitally accessible versions are not. The reason: you can’t just put them in a shoe box (or digital library). Someone most host them, and that someone needs to get paid. Forever.

Going forward, more and more of our history is going to happen only in the digital world. Facebook, Twitter, Hillary Clinton’s (or anyone other politician’s) email. There’s not going to be a microfilm version at the local university library. Who is going to store it? Who will access it and how?

A few years ago, it looked like companies like Google were going to — pro bono — solve this problem for us. They were ready, willing, and seemingly able to host all the data and make is available. But now things are getting in the way. Copyright is one. The demand from investors to monetize is another. It used to be thought that you could not monetize yesterday’s paper — today’s paper is tomorrow’s fish-wrap, but more wily content owners realize that if they don’t know the value of an asset, they can’t give it away for free. Even Google, which, I think, hands somewhat tied, is still committed to this sort of project, probably cannot be trusted with the permanent storage of our collective history. Will they be around in 50, 100 years? Will they migrate all their data forever? Will they get bought and sold a dozen times to owners who are not as committed to their original mission to “organize the world’s information and make it universally accessible and useful?” Will the actual owners of the information that Google is trying to index try to monetize it into perpetuity?

I think we know the answers. Right now, it all looks pretty grim to me.




Carbon Tax vs Cap-and-Trade. This is boring by now.

If you’ve spent any time in an energy economics class, you have probably seen a slide that shows the essential equivalency of a carbon tax and a cap-and-trade system, at least with respect to their ability to internalize externalities and fix a market failure. However, if you scratch the surface of the simple model used to claim this equivalency and you realize it only works if you have a good knowledge of the supply and demand curves for carbon emissions. (There are other non-equivalencies, too, like where the incidence of the costs falls.)

The equivalency idea is that for a given market clearing of carbon emissions and price, you can either set the price, and get the emissions you want, or set the emissions and you will get the price. As it turns out, nobody really has a good grip on the nature of those curves, and we live in a world of uncertainty anyway, so there actually is a rather important difference: what variable are we going to “fix” about and which one will “float,” carrying  all the uncertainty: the price of the carbon emissions quantity?

I bring this up because today I read a nice blog post by Severin Borenstein which I will reduce to its essential conclusion: A carbon tax is much better than cap-and-trade. He brings up the point above, stating that businesses just are much better able to adapt when they know what the price is going to be, but there are other advantages to a tax.

First, administratively, it is much easier to set a tax than it is to legislate an active and vibrant market into existence. If you’ve lived in the world of public policy, I hope you know that Administration Matters.

Furthermore, legislatures are not fast, closed-loop control systems. They can’t adapt their rules on the fly quickly as new information comes in, and sometimes political windows close entirely, making it impossible make corrections. As a result, the ability to adjust caps in a timely manner is, at best, difficult. This is a fundamentally harder problem then getting people to agree, a priori, on what an acceptable price — one with more than a pinch of pain, but not enough to kill the patient.

So, how did we end up with cap-and-trade rather than a carbon tax? Well, certainly a big reason is the deathly allergy legislatures have to the word “tax.” Even worse: “new tax.” Perhaps that was the show-stopper right there. But it certainly did not help that we had economists (I suspect Severin was not among them) providing the conventional wisdom that a carbon tax and a cap-and-trade system are essentially interchangeable. The latter is not true, unless a wise, active, and responsive regulator, free to pursue an agreed objective is at the controls. So pretty much never.

How to pay for the Internet, part 0xDEAF0001

Today’s Wall Street Journal had an article about Facebook, in which they promise to change the way the serve advertising in order to defeat ad blockers. This quote, from an FB spokesperson was choice:

“Facebook is ad-supported. Ads are a part of the Facebook experience; they’re not a tack on”

I’ll admit, I use an ad block a lot of the time. It’s not that I’m anti ads totally, but I am definitely utter trash, garbage, useless ads that suck of compute and network resources, cause the page to load much more slowly, and often enough, include malware and tracking. The problem is most acute on the mobile devices, where bandwidth, CPU power, and pixels are all in short supply, and yet it’s harder to block ads there. In fact, you really can’t do it without rooting your phone or doing all your browsing through a proxy.

The ad-supported Internet is just The Worst. I know, I know, I’ve had plenty of people explain to me that that ship has sailed, but I can still hate our ad-supported present and future.

  • Today’s ads suck, and they seem to be getting worse. Based on trends in the per ad revenue, it appears that most of the world agrees with this. They are less and less valuable.
  • Ads create perverse incentives for content creators. Their customer is the advertising client, and the reader is the product. In a pay for service model, you are the customer.
  • Ads are an attack vector for malware.
  • Ads use resources on your computer. Sure, the pay the content provider, but the cpu cycles on your computer are stolen.

I’m sure I could come up with 50 sucky things about Internet advertising, but I think it’s overdetermined. What is good about it is that it provides a way for content generators to make money, and so far, nothing else has worked.

The sad situation is that people do not want to pay for the Internet. We shell out $50 or more each month for access to the Internet, but nobody wants to pay for the Internet itself. Why not? The corrosive effect of an ad-driven Internet is so ubiquitous that people cannot even see it anymore. Because we don’t “pay” for anything on the Internet, everything loses its value. Journalism? Gone. Music? I have 30k songs (29.5k about which I do not care one whit) on my iThing.

Here is a prescription for a better Internet:

  1. Paywall every goddam thing
  2. Create non-profit syndicates that exist to attract member websites and collect subscription revenue on their behalf, distributing it according to clicks, or views, or whatever, at minimal cost.
  3. Kneecap all the rentier Internet businesses like Google and Facebook. They’re not very innovative and there is no justification for their outsized profits and “revenue requirements.” There is a solid case for economic regulation of Internet businesses with strong network effects. Do it.

I know this post is haphazard and touches on a bunch of unrelated ideas. If there is one idea I’d like to convey is: let’s get over our addiction to free stuff. It ain’t free.



The future of electrical engineering as a profession

The other day I was watching Dave Jones, a video blogger that I find entertaining and informative. His blog, the EEVblog, is catnip for nerds who like to solder stuff and use oscilloscopes.

Recently he did a short segment where he answered a question from a student who was upset that his teacher told him that EE was perhaps not a great field for job security, and he sort of went on a colorful rant about how wrong the professor is.

The professor is right.

Electrical engineering employment is indeed in decline, at least in the USA, and I suspect, other development countries. It’s not that EE skills are not helpful, or that understanding electronics, systems, signals, etc, are not useful. They are all useful and will continue to be. But I think more and more of the work, in particular, the high paying work, will migrate to software people who understand the hardware “well enough.” Which is fine. The fact is that EEs make good firmware engineers.

I think someone smart, with a solid EE background and a willingness to adapt throughout your entire career, should always find employment, but over time I suspect it will be less and less directly related to EE.

I mostly know Silicon Valley. Semiconductor employment is way down here. Mostly, it is through attrition, as people retire and move on, but nobody is hiring loads of young engineers to design chips anymore. It makes sense. Though chip volumes continue to grow, margins continue to shrink, and new chip design starts are way down, because “big” SOCs (systems on chip) with lots of peripherals can fill many niches that used to require custom or semi-custom parts.

I suspect that the need for EEs in circuit board design is also in decline. Not because there are fewer circuit boards, but because designing them is getting easier. One driver is the proliferation of very capable semiconductor parts with lots of cool peripherals is also obviating a lot of would-have-been design work. It’s gotten really easy to plop down a uC and hook up a few things over serial links and a few standard interfaces. In essence, a lot of board design work has been slurped into the chips, where one team designs it once rather than every board designer doing it again. There might be more boards being designed than ever, but the effort per board seems to be going down fast, and that’s actually not great for employment. Like you, I take apart a lot of stuff, and I’m blown away lately not by how complex many modern high volume boards are, but how dead simple they are.

The growth of the “maker” movement bears this out. Amateurs, many with little or no electronics knowledge, are designing circuit boards that do useful things, and they work. Are they making mistakes? Sure, they are. The boards are often not pretty, and violate rules and guidelines that any EE would know, but somehow they crank out working stuff anyway.

I do hold out some hope that as Moore’s law sunsets — and it really is sunseting this time — there will be renewed interest in creative EE design, as natural evolution in performance and capacity won’t solve problems “automatically.” That will perhaps mean more novel architectures, use of FPGAs, close HW/SW codesign, etc.

Some statistics bear all this out. The US Bureau of Labor Statistics has this to say about the 2014-2024 job outlook for EEs:

Note that over a 10 year period they are predicting essentially no growth for EE’s at all. None. Compare this to employment overall, in which they predict 7% growth.

One final note. People who love EE tend to think of EEs as the “model EE” — someone clever, curious, and energetic, and who remains so way for 40+ years. But let’s remind ourselves that 1/2 of EEs are below median.  If you know the student in question, you can make an informed assessment about that person’s prospects, but when you are answering a generic question about prospects for generic EEs, I think the right picture to have in mind is that of the middling engineer, not a particularly good one.

I’m not saying at all that EE is a bad career, and for all I know the number of people getting EE degrees is going down faster than employment, so that the prospects for an EE graduate are actually quite good, but it is important for students to know the state of affairs.

Narrow fact-checking is less than useless

Last night, Donald Trump gave a speech that included a bunch of statements about crime that the New York Times fact-checked for us. This summarizes what they found:

Many of Mr. Trump’s facts appear to be true, though the Republican presidential nominee sometimes failed to offer the entire story, or provide all of the context that might help to explain his numbers.

Putting aside the ridiculously low bar of “many facts appear to be true”, they failed to mention or explain that in every case, despite his factoids being narrowly true, the conclusions he was drawing from them, and suggesting we draw from them, were absolutely, incontrovertibly false.

This kind of reporting drives me bonkers. Crime stats, like all stats, are noisy, and from one year to another, in a specific city, you can find an increase or decrease — whatever you are looking for. But the overall trends are clear, and Trump’s assessment was utter bullshit.

Another, somewhat less savory, media outlet did a much better job, because they took the 10 minutes of Googling necessary to assemble some charts and put Trump’s facts in context.

Would it have been partisan for the NYT to put Trump’s facts into context with respect to the conclusions he was drawing from them? It just seems like journalism.


Nerding while sweating

I was slowly cranking my way of Claremont Avenue the other day on my trusty Bianchi when I started wondering why I was so slow. Well, that was easy. I’m pretty heavy and I’m somewhat out of shape. But which is more important, which would have a bigger impact if improved?

First, I used a website like this one to determine the average grade over a certain familiar portion of the route. In this case, it was 13.3%. I also have a speedometer on my bike that tells that I average about 5 mph over that stretch. Finally, I weigh about 100 kg, and my bike is another 10 kg.

So, given that the energy to raise a mass up h height is m*g*h, the power to raise a mass at r rate is m*g*r.


claremont_power: 317.67222 (watts)

That is, that’s how much power it takes to lift my mass up a hill at that rate. Note the trig to change my speed up the hill to a vertical speed. There are losses in pedaling a bike, and on the tires on the road, etc, but this is a good estimate of the overall order of how much power I can comfortably sustain. Let’s call it 300W.

Now, another thing I’ve noticed while riding is that on flat ground, I can maintain about 17 mph. In that case, I’m not adding power to climb a hill at all, all of my power is overcoming road friction and drag.

It happens that power going to aerodynamic drag goes by the cube of the velocity. (There is more going on here than wind drag, but, eh, it probably dominates at higher speeds…) So, if we assume that on level ground I’m capable of the same ~300W that I do while climbing, I can calculate the constant in:

P = c * v^3

This is a simplification of the more general equation linked above, assuming constant air density, yadda. For 17 mph and 317 W, I get about 0.72376 kg / m. kg/m is a strange dimension, but it it what it is.

So then, I wondered, how fast should I be able to go with a given power budget while climbing different grades?

I created this equation which combines the power to climb and the power to overcome drag

P = c v^3 + m g v sin(theta)

where P  is power, c  is the drag power constant calculated above, m  is mass, g  is the acceleration of gravity, and theta is the angle of the hill. (The angle is the arctangent of the grade, by the way.) Oh, and v  is my speed.

It turns out that my brain doesn’t perform the way it once did and I can’t solve that cubic equation on my own, so I resorted to a Python-based solver which is part of the sympy package.

This function gets the job done:

Note this equation has three solutions, two of which are complex. Only interested in the “real” solution.

Now, this is finally where the fun starts. Want to know how fast I can climb different grades, or how actual athletes who can summon more power than me can get up?

How fast I might get up hills if I could make more power.
How fast I might get up hills if I could make more power. (mass = 110 kg)

Like I said, I can make about 300W, but I saw a youtube video of a dude who could make about 1kW, at least for long enough to make toast.

Then I was wonder, would losing weight help much? It does. Interestingly, it helps on the middle grades. On the highest grades, I’m nearly stopped, and the numbers get small. On flat grades, drag (a function of my shape, not my size) dominates. But in the middle, yeah, there’s an effect.

Dave might go faster if we was less fat.
Dave might go faster if we were less fat.

So there you have it. If I lost 10 kg and could increase my power output by 15% I could go from about 5 mph on Claremont to about 6 mph.

Actually, that’s depressing.

What is the “Middle Class?”

On the radio today, All Things Considered did a little “Brief History of the Middle Class.” Apparently, according to Pew Research center, a family is in the middle class if its income is between about $47k/yr and $141k/yr. They also have breakdowns of wealth (if you are in net debt, you are not middle class) and consumption.


My own definition middle class has more to do with economic security than it does with wealth, income, or many types of consumption. To wit:

If you do not need to worry about food, housing, transportation, health care, education for your children, and safety, and can look forward to a retirement with those same basic needs satisfied, and you are not at risk of losing any of those things immediately on some negative shock, like an accident, negative diagnosis, or loss of job, then you have achieved middle class status. My definition doesn’t have anything to do with iDevices, high speed Internet, vacation or dining out.

It’s tricky, because I think a lot of people who do have nice things, like fancy cars, and big houses, don’t actually have solid economic security. I don’t know if they’re middle class or not.

What do you think the definition of middle class should be?

technological progress, freedom to v. freedom from

Technology progresses. Most of the time, progress is good, sometimes bad, but in all times it creates new circumstances, and those circumstances have winners and losers. Our society is not good at recognizing when circumstances have changed. We tend to take, for a long time at least, the world-as-it-is as the world-as-it-ought-to-be.

But I see no reason it must be so. I wish we were better at evaluating our reality, deciding if we like it or want something else, and then, coming to consensus on what, if anything, should be done at a policy level to control our circumstances.


For example, remote control airplanes have been around for quite some time. They were rather expensive toys, and not easy to fly. Similarly, aerial photography has existed almost since the dawn of flight. Because paying a pilot of fly over some location for you and photograph it is not cheap, it tends to be done where value of the resulting photograh is high enough to justify the expense.

For whatever reasons, we were pretty much OK with that status quo and the laws surrounding it. For example, yes, someone could photograph you through your window, and a passing plane could catch you sunning in your yard. People do not like those thigns, but it was hard enough to do and easy enough to stop, that basically, everyone but celebs and paparazzi seemed fine with the world as it was.

Enter inexpensive, simple aerial photograph with UAVs. Today, anybody with a few hundred bucks can get aerial imagery, and in a few years, that might be $10’s or even $1’s. Whole new possibilities for surveillance open up and people are suddenly uncomfortable about their vulnerability.

Does this mean we need laws to stop aerial surveillance “abuse?” Or maybe we need to adjust our expectations of privacy? I dunno. We need to evaluate the situation anew, since technology has changed circumstances. The fact that the existing laws were fine does not mean they are fine.

Totally rad UAVI can think of lots of contemporary examples of this sort of change: facial recognition along with ubiquitous video cameras make it possible to track everyone, everywhere they show their faces. License plate reader technology allows someone to track everywhere you go. You could do the same before, with detectives or private eyes, but now it can be done in bulk, cheaply. Cookies on websites allow someone to track most of what you look like on the Internet. In essence, people’s expectation of privacy was actually the complex combination of the state of technology and the law together, not either separate from the other.

None of these technologies are sinister in and of themselves, but dropped into a an environment that was in legal equilibrium without them, I think we should expect that equilibrium to shift.

Zoom!Of course, there are historical examples of such adjustments. Prior to the ubiquity of the automobile, people did not need carriage licenses, nor did they need to carry liability insurance for carriage accidents. How long after cars became popular did we realize they were dangerous enough and important enough that we should require that drivers get training? I think most (though not all) people today regard drivers’ licenses as a good idea. A few decades after that, we started requiring drivers to carry liability insurance and today most states have some requirement, though it is amazingly low in some places. (I know that agreement is hardly universal that liability requirements are a good idea, but we have them.)

One contemporary problem that is not typically considered in this light is gun violence. One might say that extremely capable weapons have been available for a long time, but that they have been expensive enough and just tricky enough to obtain, that we, as a society, were comfortable with the status quo. Collectors and sportspeople had them, and they used them safely, more or less. Enter cheap, easily available weapons, and all of a sudden the game has changed. In fact, today you can 3D print a gun at home, and maybe in a few years you’ll be able to 3D print a most of a not-too-shabby automatic weapon. The technology is not going to go away, but because of the technology change, the status quo is going to shift. Can or should we try to shift it back?


My point is that I think there are many  people who advocate for a kind of technological determinism, suggesting, “well, tech marches on.” But history tells us that we clearly do not have to accept such outcomes if we don’t want them.

Freedom-loving readers will notice a whole lot of “we’s” in this essay. I’m afraid they’re right. I’m suggesting that the group sometimes make decisions that restrict an individual’s freedoms. I know there is a cost to that. But I also see costs in letting individuals restrict the freedom (and well-being) of many other individuals.

As always, practicality and balance will be hard to achieve. We all seem fine with driver’s licensure, but pet grooming licenses seem perhaps too far. Required liability coverage for drivers is OK, but we probably would not tolerate such a requirement for many other potentially dangerous-to-others activities.

I hope we will have spirited, informed debates on issues like privacy and autonomy and that the outcome, if not new norms and laws, are new, explicit reiterations of existing norms that were previously implicit.

Gah… Apple

I use a Mac at work. It’s a fine machine and I like the screen and battery life, but I’m not a generally fan of Apple the company or its products. Sometimes I forget why, and I need to be reminded.

Like today, when I decided, even though Safari is basically a sucky product, there are probably people that use it, so I might just port my little political statement Chrome extension to Safari. I’d already done so to Firefox, so how hard could it be?

Well, it turns out, not too hard. Actually, for the minimalist version that most people are using, it required no code changes at all. It did take me awhile to figure out how everything works in the Apple extension tool, but overall, not too bad.

I knew I would have to submit to reviewers at Apple to get it published. I had to do the same at Mozilla for Firefox. But what I did not know is that in order to do that, I had to sign up to be an Apple Developer. Moreover, I could only do so under my real name (ie, not and most annoying, they wanted $99. A year. or as long as the extension is up.

I’m not going to play $99/yr to provide a free plugin for the few people who are dumb enough to use Safari on a regular basis.

In an odd way, this gets right to the heart of one of the many reasons I do not like Apple. They are constitutionally opposed to my favorite aspects of the computing and the Internet: the highly empowering ability for people to scrappily do, say, make anything they want for next to nothing, and at the level of sophistication that they want to deal with. Apple doesn’t like scrappy things in its world, and actively weeds them out.

Apple, you suck. Thanks for the reminder never to spend my own money on your polished crap.

Clever, disturbing

Apple was recently granted a new patent for technology that will disable your phone’s camera at concerts where photography is forbidden.

The patent uses an infrared signal, which could be picked up by the imaging sensor itself. This is rather ingenious and cunning, because you could not disable the shut-down sensor without disabling the camera yourself, since they are one and the same.

IPhone_5S_main_cameraDepending one how pervasive such tech became, and how closely integrated the detection, decoding, and disabling is to the actual silicon image sensor, it could become nearly impossible to defeat this tech, or to obtain a phone that doesn’t include it.

I find blocking cameras at concert venues mildly annoying, but the potential for abuse of this technology seems large. Will folks on the street use it to block being photographed? Will it be deployed in government buildinds? Outside cop-cars? Will the secret for how to disable everyone else’s phone get out?

Over the last few years we’ve seen some exciting benefits from ubiquitous deployment of cameras. People are getting caught doing things that are illegal or at least shameful. I’d be bummed to see some technology from Silicon Valley reverse this progress.