The Touchscreen Paradigm

Programming is evolving faster than ever. In recent years, mobile platforms have broken the software market wide open, and most implications of this disruption are yet to be discovered. However, some effects are already obvious. Software has transcended the limitations of mouse/keyboard/gamepad input, since mobile devices integrate touchscreens with cameras, microphones, speakers, and wireless connections. I call this “the touchscreen paradigm” but it refers to all of those now-standard inputs and outputs.

This hardware generalizes to an unprecedented number of applications. A typing keyboard can be simulated by key images on the touchscreen, although that experience has decidedly inferior ergonomics. Mouse clicks are replaced by touchscreen taps, and while this system has no provision for “hover” interactions, other forms of mouse control are improved. Drawing is very awkward with a mouse, since the brain has to map the mousepad surface to the display in real-time. Touchscreens eliminate this problem, and in fact they are functionally similar to high-end drawing tablets with integrated screens that have been available for some time. Wacom, a manufacturer of these computer accessories, now sells a high-end stylus that integrates with mobile software.

Other applications go beyond anything that is possible with a mouse and keyboard. Multiple finger touches can be processed at once, making “Minority Report” interfaces easy to build in software. Microsoft put significant capital into a tabletop touchscreen computer called the Surface Table (re-branded as the PixelSense). However, similar interfaces can be added to mobile devices with software, such as this Photo Table application. Fortunately for independent developers, the barriers to entry in mobile software are very low because standard hardware already exists.

These examples barely begin to fill the space of possible touchscreen applications. My phone is already a feature-rich camera, an “FM” radio, a guitar tuner, an SSH client, and a flashlight. Those products have been manufactured before with dedicated hardware, but mobile software is also being used to invent completely new technology. Products which require a touchscreen, audio/video input/output, or an internet connection can be built entirely in software and sold as mobile applications.

As a software engineer, this is obviously a good thing. However, the sheer number of new applications that are possible on mobile platforms presents an intimidating problem: what to build next? Customers might be able to describe what they’d pay for, but they don’t always know what they’ll want to buy in the future. The first generation of application programmers probably experienced a similar feeling. It’s inspiring and terrifying at the same time.

THIMBL Keyboard

I recently developed a prototype music keyboard for the iPad, in order to play around with the idea. It’s called the THIMBL Keyboard and it looks like this.

Each octave-row maps the twelve semitones to six positions on each hand: Thumb, Half (between Thumb and Index), Index, Middle, Big (ring finger), and Little, thus the THIMBL acronym. This keyboard is very interesting because it has no diatonic bias like a standard piano keyboard, but it does have a bias toward certain keys, i.e. the Left Index position is always a C note. The player moves up and down octaves by moving the hands vertically, so chord inversions are very easy to find. However, this layout means that a C Major scale is not especially simple to play without knowing the right sequence of steps, or memorizing finger positions.

I’ve been practicing some basic technique with this prototype, and have discovered a few things about how it behaves. It seems unorthodox at first, but after learning the intervals between each pair of finger positions, playing music by ear becomes much easier. There are some expected problems with touchscreen controls, as the fingers can’t rest on the key surfaces, and the keys don’t overlap in tiers, but in general this prototype is more durable and easier to maintain than the last version. I’d still like to build a production-quality model but this works surprisingly well in the meantime. Check it out if you’re interested!

I’ve also put together some vertically-oriented notation paper which helps with transcribing and playing music. Time is measured in rows and finger positions correspond to columns of cells. You’ll have to find some way to indicate the octave of each note in this grid, I’d recommend color-coding notes to match the octave colors on the keyboard.

MixBall

Music is a big part of my life. I have a voracious appetite for recorded music, and I’m working on my own humble contribution to the universe of sound. Like many aspiring composers, I’ve dreamed of creating songs that touch many lives. It hasn’t been easy – profound communication through music is an especially difficult task. In today’s world where every musical idea is measured, recorded, licensed, and purchased, that task is harder than it has ever been. I attended school with several people who are now working musicians, struggling for excellence in a craft which has been commoditized to the point of disposability.

Maybe digital distribution and piracy didn’t cause this, but many of us have still forgotten to respect the artistic process. If I were to release an original music demo, it would almost certainly be lost in a sea of other free legal or illegal content, and the prospect of eventually making usable money in this way without staging live events is not great. It feels like a step backwards, if not an unexpected development.

Knowing this, I decided to make a demo which subverts the trend. My first release of original music is now available, but only in an interactive format. MixBall is a special game that mixes the music while you play. I’m not charging money yet, but you’ll have to spend time and energy “beating” each mix, so it might make you think a bit about value. If it achieves that, I will consider it a success. The catchy tunes are a bonus, hopefully you’ll enjoy them too.

You might be wondering why this countercultural experiment is hosted on Apple’s App Store and requires an iOS device. The answer has to do with hardware limitations. I was interested in building for Android as well, but low-latency sound requires considerate effort, and Apple has the whole portable music pedigree to boot. Hopefully that doesn’t offend anyone.

Get MixBall today in the App Store!

World Wide Ouija

(Skip to the demo!)

I have a new open-source software project to share today. It is an idea that has been bouncing around my head for a while: a working, real-time, multiplayer Ouija board game that runs in a web browser. I decided to go ahead and make it last week because there are some amazing new JavaScript tools maturing out of the NodeJS madness, and this kind of project has suddenly become much easier to complete than it would have been just a few years ago.

Specifically, I’m using MeteorJS, a brand-new full-stack NodeJS/MongoDB framework which abstracts away the intimidating problem of integrating a client program with its server resources. To oversimplify, a fast web app can’t wait for every bit of program logic to download from the server, and a centralized server can’t keep every bit of data organized on every client without its own reference copies, so a lot of work gets duplicated. MeteorJS attempts to minimize this issue by setting up a framework where application data can be accessed from either the client or the server at the same logical address or “collection,” using the same JavaScript semantics on either end. When it is possible to synchronize each client’s copy of its application data with the server’s version, MeteorJS automatically passes updates to the server and fetches fresh data, algorithmically eliminating conflicts that the other clients might have introduced.

It’s all wonderfully intricate stuff, but the point is that somebody else is building it and we don’t have to. So let’s cut the techno-babble and make a multiplayer Ouija game! Start by installing MeteorJS. Then, to create a new project and populate it with basic HTML, CSS, and JS files, type this at a command line:

meteor create worldwideouija

To turn off “easy mode” and better control the data that gets shared, disable autopublish:

meteor remove autopublish

Our game needs to store three kinds of “things,” so we’ll start by initializing three collections. Open the “worldwideouija.js” file in the “worldwideouija” folder and add this at the top:

Rooms = new Meteor.Collection("rooms");
Forces = new Meteor.Collection("forces");
Messages = new Meteor.Collection("messages");

The Rooms collection will store metadata about each game room, the Forces collection will store temporary impulse values that push the Ouija marker around the board, and the Messages collection will store chat messages.

We can access these collections from either client or server, but the distinction is still important for the purposes of our application, as we need our Ouija server to manage the data for each room, and update each player’s client as necessary. MeteorJS allows us to conditionally execute JavaScript depending on whether the current environment is the client or the server, which allows all our application logic to be contained in one script. Let’s declare what the client should do now:

if (Meteor.is_client) {
Meteor.startup(function() {
Meteor.subscribe("allrooms");
Meteor.subscribe("allmessages");
//...
});
//...
}

Meteor.startup() takes a callback function that should contain all the code we want to run when the application is ready to start execution. This callback will contain our game loop.

Inside, the Meteor.subscribe() methods tell the client that it should ask the server for access to “allrooms” and “allmessages” that the server hopefully has available. While the Rooms variable always refers to the collection, this “allrooms” subscription tells the client which records to actually synchronize with the server. Because autopublish is disabled, we will have to make these subscriptions available to the client. At the end of the file, add a block that tells the server to publish all Rooms to “allrooms” subscribers and all Messages to “allmessages” subscribers. Security is not a concern when consulting with spirits:

if (Meteor.is_server) {
Meteor.startup(function () {
Meteor.publish("allrooms", function() {
return Rooms.find();
});
Meteor.publish("allmessages", function() {
return Messages.find();
});
//...
});
}

I’ve neglected to publish the Forces collection, but this is on purpose. At first, I imagined that each client could just update the server’s marker position for the current room at any time and the server would sort everything out, but whenever the delay between page updates got worse, the movement became jerky and weird. The Forces collection is how I solved this. It acts as a buffer where each client can write control data. When the server is asked to update the marker position, it reads from this collection, updates the position according to all the available Forces, and then deletes them, before sending the new position back. This means that no client ever has to read directly from this collection. Each client can update its own Forces collection and send new records to the server without subscribing to any published objects. It’s probably better not to waste bandwidth synchronizing this data if the clients don’t need it.

So let’s write some code! Define a game loop that executes every 500 milliseconds by adding a Meteor.setInterval() method to the client block, inside the startup callback function:

if (Meteor.is_client) {
Meteor.startup(function() {
Meteor.subscribe("allrooms");
Meteor.subscribe("allmessages");
Meteor.setInterval(function() {
if (Session.get("room")) {
if (isNaN(Session.get("dx"))) Session.set("dx", 0);
if (isNaN(Session.get("dy"))) Session.set("dy", 0);
Session.set("dx", Session.get("dx") * 0.9);
Session.set("dy", Session.get("dy") * 0.9);
Forces.insert({
room: Session.get("room"),
x: Session.get("dx"),
y: Session.get("dy")
});
Meteor.call("updateMarker", Session.get("room"), function(e,r) {
if (r.x && r.y) {
Session.set("posX", r.x);
Session.set("posY", r.y);
}
});
}
}, 500);
});
//...
}

This loop uses Session.set() and Session.get() to manage temporary session variables, used to save the client’s current impulse value, along with the client’s copy of the marker position. Meteor.call() tells the server to execute its “updateMarker” method that we will have to define, and the callback updates the client’s position using whatever response data comes back.

The rest of the client application uses HTML templates integrated with JavaScript, like this function which returns the current Room ID:

Template.main.currentRoom = function() {
return Session.get("room") || false;
};

This return object can be accessed from the “main” template by including {{currentRoom}} anywhere in context. We won’t go into detail about the rest of the templates but they are all included with the source code and should make enough sense if you can follow this logic. Look in the .html file for all the template HTML.

Templates can also handle jQuery events, and we can use this functionality to read the mouse position and update the client’s impulse values:

Template.room.events = {
"mousemove .gameBoard": function(e) {
var theRoom = Rooms.findOne(Session.get("room"));
var trueX = e.pageX - parseInt($('.gameBoard').css('margin-left'));
Session.set("dx", (trueX - Session.get("posX"))/25);
Session.set("dy", ((e.pageY - 50) - Session.get("posY"))/25);
}
//...
};

This is the events object for the “room” template, with keys that are strings containing a jQuery event type and often a CSS selector that declares which DOM element(s) to bind. Inside, we use a little hackery to compute an impulse vector from the mouse and marker positions, and then we save the session variables that the game loop will read from.

Finally, we have to implement an “updateMarker” method on the server:

if (Meteor.is_server) {
Meteor.startup(function () {
Meteor.publish("rooms", function() {
return Rooms.find({});
});
Meteor.publish("messages", function() {
return Messages.find({});
});
Meteor.methods({
updateMarker: function(id) {
var theRoom = Rooms.findOne(id);
var position = {};
if (theRoom) {
if (isNaN(theRoom.x)) theRoom.x = 480;
if (isNaN(theRoom.y)) theRoom.y = 320;
var dx = 0;
var dy = 0;
var numForces = 0;
var theForces = Forces.find({room: id});
theForces.forEach(function(force) {
dx += parseInt(force.x);
dy += parseInt(force.y);
numForces++;
});
Forces.remove({room: id});
Rooms.update(id, {$set: {players: numForces}});
if (numForces > 0) {
var newX = theRoom.x + dx/numForces;
var newY = theRoom.y + dy/numForces;
if (newX < 100) newX = 100;
if (newX > 860) newX = 860;
if (newY < 100) newY = 100;
if (newY > 540) newY = 540;
Rooms.update(id, {$set: {x: newX}});
Rooms.update(id, {$set: {y: newY}});
position.x = newX;
position.y = newY;
}
}
return position;
}
});
});
}

And that’s (almost) all!

Visit http://github.com/guscost/worldwideouija for the complete source code, and try it out at worldwideouija.meteor.com or worldwideouija.com! I’m still tinkering around so uptime and code quality are not 100% guaranteed.

And before I forget, many thanks to AVGP for the missing chatroom example, which helped a lot.

Variables

What is the most important thing that a programmer should do? The textbook answer, “comment everything”, only ensures a minimum viable reusability, especially when nonsense appears like:

//UTF-8 Kludge
or
//DON'T REMOVE!!!

causing unsuspecting developers to detour for what could be hours (are office distractions NP-complete?) and jeopardizing the sanity of everyone involved. Don’t get me wrong, great comments are indispensable works of art, but to be able to write them, you must first write a great program.

Then what is a great program? It depends on who you ask. Rails users might argue that a great program says everything it needs to say exactly once in unreasonably concise Ruby, and includes modular unit tests for 100% of the code base. LISP enthusiasts might praise a program for its mind-boggling recursive structure, and its unreasonable efficiency. However, in both of these cases, and always for any novice programmer, the most important feature of a great program is its consistent and correct use of variables.

Put another way: a computer program’s codebase is unsustainable if its variable identifiers can’t be interpreted correctly by developers. This means that calling a variable “temp” or “theAccountNumber” is always a bad idea unless it is actually correct to think of that variable as “the temp variable”, or the only “account number” that that particular program (or section of the program) uses. We are at a point where nearly every bottleneck I encounter in everyday software development is between my mind and an unfamiliar block of code. If there is a chance of confusing anything else with whatever you’re naming, it’s the wrong name.

What is the right name? That’s another question with a multitude of answers. CamelCase with the basic, accurate name of a variable is a good place to start, meaning that if I were to create a variable to hold the text of some person’s additional info popup, it might be a good idea to start with one of the following:

InfoPopupText
PersonInfoPopupText
SarahInfoPopupText

depending on the context, i.e. what the program (or section) is supposed to do. Most developers I have met use the first letter of every variable to encode a bit of extra information, like if I decided to use lower case letters for an instance-level variable:

personInfoPopupText

or possibly prepend a special character for private variables:

_personName

As with everything, the key is to use whatever makes the most sense to the people who need to work on it. If we are to sidetrack into a bit of philosophy, accurate naming is what makes all intelligent communication possible. The only thing that allows you to understand my word (“chicane”) is the fact that it has been used in public to refer to some persistent feature of the universe, and therefore anyone else can imagine what I imagine (or check Wikipedia) when I use it. This applies to all formal and mathematical language too: the only thing that is required to understand a given theorem is an accurate understanding of the words that it contains. Be careful, though: accurately understanding some of the words that mathematicians use is a lot harder than it sounds.

Are any non-programmers still paying attention? This part applies to you too. As the old 20th-century paradigm of “files, folders and windows” starts to look more and more passé, why not ditch that menacing behemoth that you call your “backup” and start organizing files by naming them correctly? (Spelling is important!) If you do that, just Gmail them all to yourself, and then they will be archived as reliably as Google is, forever. If you picked the right name for a file, you’ll know what to search for when you need it.