Marketing Genius

Over the past couple of months, in stolen moments and late night coding sessions, I’ve quietly been inventing a little piece of ham radio gear intended to facilitate using one’s radio remotely.

I thought it would be a good way to try my hand at designing a useful product, from start to finish, including complete documentation, packaging, etc.

I posted about it to a few ham radio forums and it turned out nobody was interested, so instead of making a product, I’m just throwing it up on github for people to ignore forever.

I’m no marketing genius.

 

 

Rigminder 2017-2017 RIP

ATIS in your kitchen

One ritual that every pilot observes before launching into the wild blue yonder (or dark gray muck) is tuning in the Automated Terminal Information Service, or ATIS. The ATIS is a recording, usually updated hourly, that contains a very terse version of the current weather and anything else new that pilots need to know.

ATIS is not the first weather information a pilot will hear before flying. In fact, it is more likely to be the last, after she has gotten a complete legal weather briefing (14 CFR 91.103), but before taking off. Pilots also listen to the ATIS at an airport at which they intend to land.

A similar system, called AWOS (Automated Weather Observation System) is like ATIS, except that it usually only carries the weather (no other info) and always sounds like a robot.

As it turns out, I have a doohickey in my home that 1) can connect to the Internet to get the weather and 2) sounds like a robot. I thought, maybe it would be fun to write an app that simulates ATIS on an Amazon Echo.

Well, here it is.

This is a rather straightforward Alexa Skill. A user specifies the airport of interest by using its four-letter ICAO identifier. Standard ICAO phonetics are supported. (alpha, bravo, charlie, …)

For example, Chicago O’Hare’s IATA code is ORD, but its complete ICAO code is KORD. You could say:

Alexa, ask airport weather to get kilo oscar romeo delta.

And it would read you the weather in Chicago. The skill also knows the names of many (but by no means all) airports, too, so you can specify an airport that way, too. And if you give only three letters (like an IATA code rather than an ICAO airport identifier), it will try to fill in that fourth letter if you. For European users, you can get the visibility and altimeter settings in metric format.

A few details of the skill:

  • written in node.js
  • Uses the Alexa Skills Kit API — Amazon handles all the voice stuff
  • Runs as a function in AWS Lambda
  • Accesses weather data from ADDS.
  • Stores user preferences in an AWS DynamoDB (a Mongo-like database thingy)
  • Caches weather info from ADDS for up to 5 minutes to reduce load on ADDS
  • Whole thing runs in the AWS “Free tier” — which is important, as I’m not going to spend money to host a free app.

One of the more fun aspects of the project was getting to maximal verisimilitude. The ADDS weather source actually provides a METAR, which has a lot of the same information as does the ATIS, but it’s not entirely the same in form or content, so I had to do some translation and adjustment. For example, wind directions in METARs are true-north references, but in ATIS, they are magnetic-north referenced. In Northern California, where I live, that’s a 16.5° difference — not trivial. The program makes the adjustment based on the location of the airport and calculations from the World Magnetic Model.

So this METAR

 becomes:

 There is even code there to try to get the pauses and pacing to be realistic.

Anyway, code is not the cleanest thing I ever did. Such is the case when things start as personal hacks and turn into “sofware.” Check it out on github.

More instructions here: http://toolsofourtools.org/alexa-metars-and-tafs

Update: Since coding this skill, I have added the Terminal Area Forecast (TAF) capability as well. Just as for the forecast.

The end of computing as a hobby?

I grew up with computers. We got our first machine, an Atari 800, when I was only 8 or 9. An 8-bitter with hardware sprites. 48 KiB of memory, and a cassette tape trive, this was only one step removed from the Atari 2600 game console. Very nearly useless, this was a machine for enthusiasts and hobbyists.

Over time, computers became less useless, as well as more “user-friendly,” but they — particularly the PC style machines — kept the doors open to hobbyists and tinkerers.

The Bad News

I think, however, that that era has come to an end, and I’m saddened. I see three basic trends that have killed it.

The first is that the network-connected world is dangerous. You can’t just fire up any old executable you find on the Internet in order to see what it does. It might do something Awful.

The second is that the closed ecosystem app stores of the world, aiming for a super smooth experience, have raised the quality bar for participation — particularly for “polish.” You simply cannot publish ugly, but highly functional software today.

The third problem is that you can’t make interesting software today without interacting with several systems in the cloud. Your app, hosted on a server, talks to a database, another app, and a half dozen other APIs: a link shortener, a video encoder, etc. And these APIs change constantly. There is no commitment to backward compatibility — something that was an iron-clad requirement of the PC era.

Trend one is a painful fact of life. Trend two could be reversed if the manufacturers had any incentive to do so. They do not. Trend three, I think is the worse, because it is wholly unnecessary. Say what you want about the “Wintel duopoly,” but they did not punish developers like modern companies do.

Together, these things pretty much lock out the casual developer. I’ve learned this the hard way as I try to push forward in my free time with a few open-source apps in a post PC world. It is one thing for a paid programmer to maintain a piece of software and deal, however grudgingly, with every email that comes from Google telling you that you need to update your code, again. But the hobbyist who wrote something cool for his friends, that worked for six months and then broke, is kind of stuck. Does he want to run a zero-revenue company that “supports” his app in perpetuity?

This makes me sad, because I wonder what we’re missing. As many of your know, I have gotten into ham radio. There’s a lot of cool ham-authored software out there. It’s ugly. It’s clunky. But some of it does amazing things, like implement modems that forward-error-correct a message and then put it into a ridiculously narrow signal that can reach around the world. Today, that software still runs on Windows, usually coded against the old Win32 or even Win16 libraries. It gets passed around in zip files and people run unsigned executables without installers. It’s the last hacky platform standing, but not for long.

The Good News

Of course, if the PC, Mac, i-device, and household gadget becomes more and more locked off, there is an exciting antidote: Arduino, Raspberry Pi, Beaglebone, and the entire maker world. People are building cool stuff. It’s cheap, it’s fun, and the barriers to entry, though intellectually a bit higher than the “PC” are pretty damn low. Furthermore, the ecosystems around these products are refreshingly chaotic and more than slightly anti-corporate.

One of the nice things about this platforms is that they are self-contained and so pose little threat to data other than what you put on them. On the other hand, they are full-fledged computer and are as exploitable as any other.

If you make something cool that runs on a Raspberry Pi, there’s still pretty little chance every kid at school will soon have it and run it, but then again, maybe there never was.

 

More election nerdism

Keeping up my streak of mildly entertaining, though basically useless Chrome Extensions, I have create a very tiny extension that keeps the Nate Silver fivethirtyeight predictions in your Chrome toolbar at all times.

You can choose which of Silver’s models is displayed, and clicking brings up more detail as well as links to a few other sites making predictions. Check it out!

For those who are interested in such things, the code is up on github. It’s actually a reasonably minimalist example of a “browser action” extension.

screen-shot-2016-10-05-at-11-09-46-am

Discouraging coding

I know I’ve written about this before, but I need to rant about tech companies pay lip service about encouraging young people to “code” but then throw up barriers to end-users (ie, regular people, not developers) writing code for their own use.

The example that’s been bugging me lately is Google Chrome, which asks you, every single time it’s started, if you want to disable “developer mode” extensions, with disable as the default, natch.

You see, you can’t run a Chrome extension unless you are in “developer mode” to start with. Then you can write some code, load it into Chrome, and you’re off to the races. This is good for, you know, developing, but also nice for people who just want to write their own extension, for their own use, and that will be the end of it.

Except they will be nagged perpetually for trying to do so. The solution is to upload your extension to the Chrome Web Store, where it can be validated by Google according to a secret formula of tests, and given a seal of approval (maybe).

But you don’t want to upload your extension to the Chrome Web Store? Well, too fscking bad, kid! Maybe you should stick to Scratch if you don’t want to run with the big boys.

It’s not just Google. If you want to run an extension on Firefox, you have to upload it to Mozilla, too — but at least if you just want to use it yourself, you can skip the human validation step. (NB: If you do want to share the extension, you will be dropped into a queue where a human being will — eventually — look at your extension. I tried upgrading Detrumpify on Firefox last week and I’m still waiting for approval.)

And don’t even get me started on Apple, where you need to shell out $99 to do any kind if development at all.

I don’t know how this works on phone apps, but I suspect it’s as complicated.

I get it: there are bad guys out there and we need to be protected from them. And these systems are maybe unavoidably complex. But, damn, I don’t hear anybody saying out loud that we really are losing something as we move to “app culture.” The home DIY hacker is being squeezed.

 

Detrumpify2 — some cleanup

Even though my short brush with Internet fame appears to be over (Detrumpify has about 920 users today, up only 30 from yesterday), pride required that I update the extension because it was a bit too quick-n-dirty for my taste. Everything in it was hard-coded and that meant that every update I made to add new sites or insults would require users to approve an update. Hard-coding stuff in your programs is a big no-no, starting from CS 101 on.

So, I have a rewritten version available, and intrepid fans can help me out by testing it. You will not find it by searching on the Chrome Web Store, instead, get it directly from here. It is substantially more complicated under the hood than before, so expect bugs. (Github here, in “v2” folder.)

An important difference between this and the classic version is that there is an options page. It looks like this:

Screen Shot 2016-06-28 at 11.33.34 AM The main thing it lets you do is specify an URL from which a configuration file will periodically be downloaded. The config file contains the actual insults as well as some other parameters. I will host and maintain several configuration files ToolsOfOurTools, but anyone who wants to make one (for example, to mock a different presidential candidate) will be able to do so and just point to it.

If you want to make changes locally, you can also load a file, click on the edit button, make changes, and then click on the lock button. From then on the extension will use your custom changes.

The format of the config file is simple. Here’s an example with most of the names removed:

Explanation:

  • actions  is a container that will hold one or more named sets of search and replace instructions. This file just has one for replacing trump variations, but one can make files that will replace many different things according to different rules
  • find_regex  inside the trump action finds a few variations of Trump, Donald Trump, Donald J. Trump.
  • monikers  section lists the alternatives.
  • randomize_mode  can be always , hourly , daily , and tells how often the insult changes. In always , it will change with each appearance in the document.
  • refresh_age  is how long to wait (in milliseconds) before hitting the server for an update.
  • run_info  tells how long to wait before running the plugin and how many times to run. This is for sites that do not elaborate their content until after some javascript runs. (ie, every site these days, apparently). Here, it runs after 1000ms, then runs four more times, each time waiting 1.8x as long as the last time.
  •   bracket  can be set to a two-element array of text to be placed before and after any trump replacement.
  • schema  is required to ID the format of this file and should look just like that.
  • whitelist  is a list of sites that are enabled to run the extension. Et voila.

Let me know if you experience issues / bugs! The code that runs this is quite a bit more complex than the version you’re running now. In particular, I’m still struggling a bit with certain websites that turn on “content security policies” that get in the way of the config fetch. Sometimes it works, sometimes it doesn’t.

 

Mental Models

I think we all make mental models constantly — simplifications of the world that help us understand it. And for services on the Internet, our mental models are probably very close — logically, if not in implementation — to the reality of what those services do. If not, how could we use them?

I also like to imagine how the service works, too. I don’t know why I do this, but it makes me feel better about the universe. For a lot of things, to a first approximation, the what and how are sufficiently close that they are essentially the same model. And sometimes a model of how it works eludes me entirely.

for example, my model of email is that an email address is the combination of a username and a system name. My mail server looks up the destination mail server, and IP routes my blob of text to the destination mail server, where that server routes it to the appropriate user’s “mailbox,” which is a file. Which is indeed how it works, more or less, with lots of elision of what I’m sure are important details.

I’ve also begun sorting my mental models of Internet companies and services into a taxonomy that have subjective meaning for me, based on how meritorious and/or interesting they are. Here’s a rough draft:

The What The How Example Dave’s judgment
obvious as in real life email Very glad these exist, but nobody deserves a special pat on the back for them. I’ll add most matchmaking services, too.
obvious non-obvious, but simple and/or elegant Google Search (PageRank) High regard. Basically, this sort of thing has been the backbone of Internet value to-date
not obvious / inscrutable nobody cares Google Buzz lack of popularity kills these. Not much to talk about
obvious obvious Facebook Society rewards these but technically, they are super-boring to me
obvious non-obvious and complex natural language, machine translation, face recognition Potentially very exciting, but not really very pervasive or economically important just yet. Potentially creepy and may represent the end of humanity’s reign on earth.

 

Google search is famously straightforward. You’re searching for some “thing,” and Google is combing a large index for that “thing.” Back in the Altavista era, that “thing” was just keywords on a page. Google’s first innovation was to use the site’s own popularity (as measured by who links to it and the rankings of those links.) to help sort the results. I wonder how many people had a some kind of mental model of how Google worked that was different than that of Altavista — aside from the simple fact that it worked much “better.” The thing about Google’s “Pagerank” was that it was quite simple, and quite brilliant, because, honestly, none of the rest of us thought of it. So kudos to them.

There have been some Internet services I’ve tried over the years that I could not quite understand. I’m not talking about how they work under the hood, but how they appear to work from my perspective. Remember Google “Buzz?” I never quite understood what that was supposed to be doing.

Facebook, in its essence is pretty simple, too, and I think we all formed something of a working mental model for what we think it does. Here’s mine, written up as SQL code. First, the system is composed of a few tables:

A table of users, a table representing friendships, and a table of posts. The tables are populated by straightforward UI actions like “add friend” or “write post.”

Generating a user’s wall when they log in is as simple as:

You could build an FB clone with that code alone. It is eye-rollingly boring and unclever.

Such an implementation would die when you got past a few thousand users or posts, but with a little work and modern databases that automatically shard and replicate, etc, you could probably handle a lot more. Helping FB is the fact they makes no promises about correctness: a post you make may or may not ever appear on your friend’s wall, etc.

I think the ridiculous simplicity of this is why I have never taken Facebook very seriously. Obviously it’s a gajillion dollar idea, but technically, there’s nothing remotely creative or interesting there. Getting it all to work for a billion users making a billion posts a day is, I’m sure, a huge technical challenge, but not requiring inspiration. (As an aside, today’s FB wall is not so simple. It uses some algorithm to rank and highlight posts. What’s the algorithm and why and when will my friends see my post? Who the hell knows?! Does this bother anybody else but me?)

The last category is things that are reasonably obviously useful to lots of people, but how they work is pretty opaque, even if you think about it for awhile. That is, things that we can form a mental model of what it is, but mere mortals do not understand how it works. Machine translation falls into that category, and maybe all the new machine learning and future AI apps do, too.

It’s perhaps “the” space to watch, but if you ask me the obvious what / simple how isn’t nearly exhausted yet — as long as you can come up with an interesting “why,” that is.

minor annoyances: debug-printing enums

This is going to be another programming post.

One thing that always annoys me when working on a project in a language like C++ is that when I’m debugging, I’d like to print messages with meaningful names for the enumerated types I’m using.

The classic way to do it is something like this:

Note that I have perhaps too-cleverly left out the break statements because each case returns.

But this has problems:

  • repetitive typing
  • maintenance. Whenever you change the enum, you have to remember to change the debug function.

It just feels super-clunky.

I made a little class in C++ that I like a bit better because you only have to write the wrapper code once even to use it on a bunch of different enums. Also you can hide the code part in another file and never see or think about it again.

C++11 lets you initialize those maps pretty nicely, and they are static const, so you don’t have to worry about clobbering them or having multiple copies. But overall, it still blows because you have to type those identifiers no fewer than three times: once in the definition and twice in the printer thing.

Unsatisfactory.

I Googled a bit and learned about how Boost provides some seriously abusive preprocessor macros, including one that can loop. I don’t know what kind of dark preprocessor magic Boost uses, but it works. Here is the template and some macros:

And here’s how you use it:

Now I only have to list out the enumerators one time! Not bad. However, it obviously only works if you control the enum. If you are importing someone else’s header with the definition, it still has the maintenance problem of the other solutions.

I understand that the C++ template language is Turing-complete, so I’m suspect this can be done entirely with templates and no macros, but I wouldn’t have the foggiest idea how to start. Perhaps one of you do?