Why Computer Programmers Struggle To Code For the Future

The required daily tradeoffs every business faces between efficiency and comprehensiveness. Except entire systems can crash if they get this wrong. JL

Greg Lavallee reports in Slate:

The four-digit solution to the Y2K problem only solved the next 8,000 years.There’s another giant date bug  in 2038. In Unix (and Linux) operating systems, time is  stored as the number of seconds since midnight on Jan. 1, 1970. This date was arbitrarily chosen for convenience. On Jan. 19, 2038 (2147483647) there won’t be enough space left to store the next second (2147483648) in many computer systems. The code may instead assume the current time is negative. This format for time is used in billions of lines of code, on all systems, to determine dates. It’s another example of how software engineers have to balance efficiency with quality

Y2K was the nerdy disaster that wasn’t. The fear was that the moment 1/1/00 rolled around, some computers would think it was Jan. 1, 1900. What could go wrong? Maybe highly computerized hydroelectric dams would open their floodgates! Or maybe all date math trying to subtract from 00 would end up negative, and suddenly your mortgage would have been paid off dozens of decades ago!

The world freaked out. Software engineers stayed up late. In the end, Y2K had some terrible real-life consequences, but it also didn’t turn out to be a complete catastrophe that required stockpiling ammunition and MREs. After airplanes didn’t fall out of the sky, everyone breathed a sigh of relief. The problem, as the public learned so well in the run-up to the New Year, was that for decades, software engineers had left out the century to save on space when storing dates. It was as though they had assumed their software would always run in a year that began with 19. For many who were still just getting used to dial-up internet, Y2K was their first exposure to the potential fragility of software.

Many people don’t understand how programming works—not just the literal languages, but the way programmers work. If software developers spent all their time thinking about the future, it would take such an inordinate amount of time to produce code, it would be the future by the time they were done.

A tremendous amount of coding is made up of trade-offs. Should you “hard-code” the font color of a website as dark gray in three places, meaning you’ll basically be locked into the shade, or should you make it a variable in case someone is going to ask you to change it later? How much time would you save if you made it variable now vs. doing it if/when they ask? What if it’s hard-coded in five places instead of three?

Software engineers usually work in teams in which one or two people write the code. Then, before it’s deployed to a place where it could affect other systems (including people), a few others review it in a process called, usefully, code review. In many code review systems, each line of code is up for discussion—a back-and-forth that can range from asking the developer to add a single comma to spotting a bad line of code that could crash the application or leak private data.

Pressure mounts during code review. On one side, there are product managers, users, and other stakeholders who want the new code to be deployed. On the other side, there are software engineers who are going to be held responsible and get woken up in the middle of the night if the code ships and has bugs. Sometimes one team wins, sometimes the other does. Watching code ship that you know will break someday is a curse of software engineering, but happens more often than you’d think. The trade-offs will continue as long as software is being written.

The four-digit solution to the Y2K problem only solved the next 8,000 years. When the year 10000 (Y10k) rolls around, we’ll have the Y2K problem all over again when we try to subtract 9000 from 0000.

If 8,000 years is too far in the future, don’t worry! There’s another giant date bug that’s right around the corner in 2038. In Unix (and Linux) operating systems, time is often stored as the number of seconds since midnight on Jan. 1, 1970. According to Wired’s interview with those around at the inception of Unix, this 1/1/1970 date, known as the Unix “epoch,” was arbitrarily chosen for convenience. For example, the moment you saved a document at 2:05 p.m. EDT on June 2, 1986, Unix would have given it a time of 518090700. That’s all fine and good, but on Jan. 19, 2038 (2147483647) there won’t be enough space left to store the next second (2147483648) in many computer systems. Instead, the code may instead assume that the current time is actually negative. This format for time is used in billions of lines of code, on all sorts of systems, to determine dates.

It’s another example of how software engineers have to balance efficiency with quality every day. You might know that there’s a pothole in the road up ahead, but that doesn’t mean you have time to stop and fill it in. You’re just going to have to drive over it, and hope for the best. In some jobs you make something and then move on, but software is built on top of libraries of other software that run on systems composed of even more software that run on hardware. Inefficiencies and trade-offs in every one of those systems pile up over time. Eventually, developers have to convince their superiors that they need to stop working on new features and start fixing the underlying infrastructure.

Take Twitter, for example: Every tweet that’s sent gets a unique identifier. When it was first developed, the engineers at Twitter chose a 32-bit integer to store tweets. That means the first tweet would have an ID of 1, and there would be room in the system for 4,294,967,295 tweets. When Twitter was a few folks talking about what they were eating for lunch, 4 billion probably seemed like enough for a while. But by 2009, it became clear that Twitter was going to need more space for storing IDs. Also, hundreds if not thousands of applications had been built on top of Twitter (Tweetbot, Tweetdeck, Seismic, etc.), and those would also need to get the heads-up to increase the size of their database columns to store tweet IDs. So JavaScript developers were told to stop using the “ID” field that Twitter sent them and instead use a new field called “id_str,” which wrapped the tweet IDs in quotes so that JavaScript wouldn’t try to interpret them as numbers and crash applications. Twitter crisis averted.

Software engineers are already working on the year 2038 bug. The systems will be updated, if they haven’t already, because lives and mortgages depend on them. Software that runs on updated systems will follow suit, and then the downstream libraries and various software built on those libraries will get updated as well. Some developers won’t ever even have to think about the year 2038. A minor “patch” to a library used by their database will have a version bump—and they won’t even read the release notes.

Of course, some systems just won’t be updated. As with Y2K, at the biggest risk are embedded systems that aren’t connected to a network. These would require a real live human to update them instead of a company issuing a remote update. Maybe we finally found a good reason to have refrigerators that can tweet.

In any event, spare a thought in early 2038 for the programmers who will be weighing options, fighting about priorities, and ultimately saving your mortgage.