In a few weeks it’ll be 25 years ago: June 23, 1988—the day Mathematica was launched.

Late the night before we were still duplicating floppy disks and stuffing product boxes. But at noon on June 23 there I was at a conference center in Santa Clara starting up Mathematica in public for the first time:

(Yes, that was the original startup screen, and yes, Mathematica 1.0 ran on Macs and various Unix workstation computers; PCs weren’t yet powerful enough.)

People were pretty excited to see what Mathematica could do. And there were pretty nice speeches about the promise of Mathematica from a spectrum of computer industry leaders, including Steve Jobs (then at NeXT), who was kind enough to come even though he hadn’t appeared in public for a while. And someone at the event had the foresight to get all the speakers to sign a copy of the book, which had just gone on sale that day at bookstores all over the country:

So much has happened with Mathematica in the quarter century since then. What began with Mathematica 1.0 has turned into the vast system that is Mathematica today. And as I look at the 25th Anniversary Scrapbook, it makes me proud to see how many contributions Mathematica has made to invention, discovery and education:

But to me what’s perhaps most satisfying is how the fundamental principles on which I built Mathematica have stood the test of time. And how the core ideas and language that were in Mathematica 1.0 persist today (and yes, most Mathematica 1.0 code will still run unchanged today).

But, OK, where did Mathematica come from? How did it come to be the way it is? It’s a long story, really. Deeply entwined with my own personal story. But particularly as I look to the future, I find it interesting to understand how things have evolved from all that history.

Perhaps the first faint glimmering of an orientation toward something like Mathematica came when I was about 6 years old—and realized that I could “automate” those tedious addition sums I was being given, by creating an “addition slide rule” out of two rulers. I never liked calculational math, and was never good at it. But starting around the age of 10, I became increasingly interested in physics—and doing physics required doing math.

Electronic calculators arrived on the scene when I was 12—and I immediately became an enthusiast. And around the same time, I started using my first computer—an object the size of a large desk, with 8 kilowords of 18-bit memory, programmed mostly in assembler using paper tape. I tried doing physics with it, to no great success. But by the time I was 16, I had published a few physics papers, left high school, and was working at a British government lab. “Real” theoretical physicists basically didn’t use computers in those days. But I did. Alternating between an HP desk calculator (with a plotter!) and an IBM mainframe programmed in Fortran.

I was basically just doing numerics, though. But in the physics I wanted to do, there was all sorts of algebra. And not just a little algebra. Huge amounts. Expressions from Feynman diagrams with hundreds or thousands of terms, all of which had to be precisely right if one was going to get the right answer.

I wondered what to do. I imagined spending my life chasing minus signs and factors of 2. But then I started thinking about using a computer to help. And right then someone told me that other people had had that idea too. There were three programs that I found out about—all as it turned out started some 14 years earlier from a single conversation at CERN in 1962: Reduce (written in LISP), Ashmedai (written in Fortran) and Schoonschip (written in CDC 6000 assembler).

The programs were specialized, and it wasn’t clear how many people other than their authors had ever used them seriously. They were pretty clunky to use: typically you’d submit a deck of cards, and then some time later get back a result—or more often a cryptic error message. But I managed to start doing physics with them.

Then in the summer of 1977 I discovered the ARPANET, or what’s now the internet. There were only 512 hosts on it back then. And @O 236 went to an open computer at MIT that ran a program called Macsyma—that did algebra, and could be used interactively. I was amazed so few people used it. But it wasn’t long before I was spending most of my days on it. I developed a certain way of working—going back and forth with the machine, trying things out and seeing what happened. And routinely doing weird things like enumerating different algebraic forms for an integral—then just “experimentally” seeing which differentiated correctly.

My physics papers started containing all sorts of amazing formulas. And not imagining that I could be using a computer, people started thinking that I must be some kind of great human algebraic calculator. I got more and more ambitious, trying to do more and more with Macsyma. Pretty soon I think I was its largest user. But sometime in 1979 I hit the edge; I’d outgrown it.

And then it was November 1979. I was 20 years old, and I’d just gotten my PhD in physics. I was spending a few weeks at CERN, planning my future in (as I believed) physics. And one thing I concluded was that to do physics well, I’d need something better than Macsyma. And after a little while I decided that the only way I’d really have a chance to get what I wanted was if I built it myself.

And so it was that I embarked on what would become SMP (the “Symbolic Manipulation Program”). I had a pretty broad knowledge of other computer languages of the time, both the “ordinary” ALGOL-like procedural ones, and ones like LISP and APL. At first as I sketched out SMP, my designs looked a lot like what I’d seen in those languages. But gradually, as I understood more about how different SMP had to be, I started just trying to invent everything myself.

I think I had some pretty good ideas. And actually even some of my early SMP design documents have a remarkably Mathematica-like flavor to them:

Looking back at its documentation, SMP was quite an impressive system, especially given that I was only 20 years old when I started designing it. But needless to say, not every idea in SMP was good. And as a long-time connoisseur of language design, I can’t resist at the bottom of this post mentioning a few of my “favorite” mistakes.

Even in my early designs, SMP was a big system. But for whatever reason, I didn’t find that at all daunting. I just wanted to go ahead and implement it. I wanted to make sure I did everything as well as possible. And I remember thinking: “I don’t officially know computer science; I’d better learn it”. So I went to the bookstore, and bought every book I could find on computer science—the whole half shelf of them. And proceeded to read them all.

I was working at Caltech back then. And I invited everyone I could find from around the world who’d worked on any related system to come give a talk. I put together a little “working group” at Caltech—which for a while included Richard Feynman. And I started recruiting people from around the campus to work on the “SMP Project”.

A big early decision was what language SMP should be written in. Macsyma was written in LISP, and lots of people said LISP was the only possibility. But a young physics graduate student named Rob Pike convinced me that C was the “language of the future”, and the right choice. (Rob went on to do all sorts of things, like invent the Go language.) And so it was that early in 1980, the first lines of C code for SMP were written.

The group that worked on SMP was an interesting one. My first recruit was Chris Cole, who’d worked at IBM and become an APL enthusiast, and went on to found a rather successful company called Peregrine Systems. Then there were students with a variety of different skills, and a programming-enthusiast professor who’d been a collaborator of mine on some physics papers. There was some eccentricity along the way, of course. Like the person who wrote very efficient code, all on one line, with functions colorfully named so their combinations would read as little jokes. Or the quite brilliant undergraduate who worked so hard on the project that he failed all his classes, then promised he wouldn’t touch a computer—but was soon found dictating code to someone else.

I wrote lots of code for SMP myself (about 1000 lines/day). I did the design. And I wrote most of the documentation. I’d never managed a large project before. But somehow that part never seemed very difficult. And sure enough, by June 1981, SMP Version 1 was running—and even looking a bit like Mathematica:

For its time, SMP was a very big software system (though its executable was just under a megabyte). Its original purpose was to do mathematical computation. But along the way I realized that even to do that well, I had to create a whole, rather general, symbolic language. I suppose I saw it as being a bit like physics—but instead of dealing with elementary particles, I was trying to find the elementary components of computation. I developed a kind of aesthetic: always try to pack the largest capability into the smallest number of primitives. Sometimes I would puzzle for weeks about how to do something—but in the end I’d come up with a design, then implement it.

I understood the idea that everything could be represented by symbolic expressions. Although the whole business of symbolically indexed lists prevented SMP from having the notion of “expression heads” that’s so clean in Mathematica. And there was definitely some funkiness in the internal implementation of symbolic expressions—most notably bizarre ideas about storing all numbers in floating point. (Tini Veltman, author of Schoonschip, and later winner of a physics Nobel Prize, had told me that storing numbers in floating point was one of the best decisions he ever made, because FPUs were so much faster at arithmetic than ALUs.)

Before SMP, I’d written lots of code for systems like Macsyma, and I’d realized that something I was always trying to do was to say “if I have an expression that looks like this, I want to transform it into one that looks like this”. So in designing SMP, transformation rules for families of symbolic expressions represented by patterns became one of the central ideas. It wasn’t nearly as clean as in Mathematica, and there were definitely some funky and far-out ideas. But a lot of the core elements were already there.

And in the end, the table of contents from the SMP Version 1.0 documentation from 1981 had a fair degree of modernity:

Yes, “graphical output” is relegated to a small section, alongside “memory management”. And there are the charming “programming impasses” (i.e. system hangs), as well as “statistical expression generation” (i.e. making random expressions). But “parallel processing” is already there, along with “program construction” (i.e. code generation). (SMP even had a way of creating C code, compiling it, and, very scarily, dynamically linking it into the running SMP executable.) And there were lots of mathematical functions, and mathematical operations—though vastly less powerful than in Mathematica.

But, OK. So SMP 1.0 was running. What should be done with it? It was pretty clear there were lots of people who would find it useful. It only ran on quite big computers—so-called “minicomputers”, like the VAX, that was the size of several large refrigerators, and cost a few hundred thousand dollars. But still, I knew there were plenty of research and engineering organizations that had such machines.

I really didn’t know anything about companies or business at the time. But I did understand that it cost money to pay people to work on SMP, and it seemed pretty obvious that a good way to get that money was to sell copies of SMP. My first idea was to go to what would now be called the “technology transfer office” at Caltech, and see if they could help. At the time, the office essentially consisted of one pleasant old chap. But after a few attempts, it became clear he really didn’t know what to do. I asked him how this could be, given that I assumed similar things must come up all the time at Caltech. “Well”, he said, “the thing is that faculty members mostly just go off and start companies themselves, so we never get involved”. “Oh”, I said, “can I do that?”. And he leafed through the bylaws of the university and said: “Software is copyrightable, the university doesn’t claim ownership of copyrights—so, yes, you can”.

And so off I went to start a company. But it wasn’t as simple as that. Because a little while later the university administration suddenly decided that, no, it wasn’t OK. It got very weird—and scurrilous (“give me a cut, and I’ll sign off on this”, etc.). Richard Feynman and Murray Gell-Mann interceded on my behalf. The president of the university didn’t seem to know what to do. And for a while everything was completely stuck. But eventually we agreed that the university would license whatever rights they might have—even though they were (very foolishly, as it later turned out when they tried to recruit computer science faculty) changing their bylaws about software.

As it happened there was one “last problem” though, come up with by the then-provost of the university. He claimed that having a license in place between the university and the company created a conflict of interest if I worked at the university and owned part of the company. “OK”, I said, “that’s easy to resolve: I’ll quit the university”. That seemed to come as a big surprise. But quit I did, and moved to the Institute for Advanced Study in Princeton, where, as the then-director pointed out, they’d “given away the computer” when John von Neumann died, so they couldn’t really be too worried about intellectual property.

For years, I’d wondered what had actually been going on at Caltech. And as it happens, just a couple of weeks ago, I agreed to visit Caltech again (to get a “distinguished alumnus award”), and having lunch at the faculty club there—I discovered that at the next table was none other than the former provost of Caltech, now about to turn 95. I was very impressed at his immediate and deep recall of what he called “the Wolfram Affair” (was he “warned”?), and the conversation we had finally explained things a bit better.

Frankly, it was more bizarre than I could have possibly imagined. The story in a sense began in the 1930s, when Arnold Beckman was at Caltech, invented the pH meter, and left to found Beckman Instruments. By 1981, Beckman was a major donor to Caltech, and the chairman of its board of trustees. Meanwhile, the chairman of its biology department (Lee Hood) was inventing the gene sequencer. He’s told me he tried many times to interest Beckman Instruments in it, but failed, and so started his own company (Applied Biosystems), which became very successful. At some moment, I’m told, Arnold Beckman got upset, and told the administration that they needed to “stop IP walking off campus”. Well, it turned out that the only thing of relevance happening on campus right then was none other than my SMP project. Which the then-provost said he thought he had a duty to “deal with”. (Well, he was also a chemist, who Feynman and Gell-Mann, as physicists, claimed had a “thing about physicists”, etc.)

But notwithstanding this whole adventure, the company that I named Computer Mathematics Corporation got started. At the time, I still thought of myself as a young academic, and didn’t imagine that I’d know how to run a company. So I brought in a CEO, who happened to be about twice my age. And at the behest of the CEO and some venture capitalists, the company arranged to merge with a startup that was doing what they thought was going to be really hot artificial intelligence R&D.

Meanwhile, SMP began to be sold under the banner of “mathematics by computer”:

There were horrible missteps. CEO: “Let’s build a workstation computer to run SMP”; me: “No, we’re a software company, and I’ve seen this Stanford University Network (SUN) system that’s going to be better than anything we can build”. And then there were the charmingly misguided agency-created ads:

And pretty soon I decided the whole thing was too frustrating. SMP remained something of a cash cow, and although the CEO wasn’t good at making money, he was good at raising it, going through a dizzying number of investment rounds—until there was finally an undistinguished IPO many years later.

I was meanwhile having a terrific time doing basic science, and discovering things that laid the foundations for A New Kind of Science. And in fact SMP turned out to be a crucial precursor to what I did. Because it was my success in inventing computational primitives for the language of SMP that got me thinking about inventing computational primitives for nature—and building a science from studying the consequences of those primitives.

You might ask what happened to SMP. It continued to be sold until sometime after Mathematica was released. None of its code was ever used for Mathematica. But occasionally I used to start it up, just to see how it “felt” compared to Mathematica. As time went by, it became harder to find computers that would run SMP. And perhaps 15 years ago, the last computer we had that could run SMP stopped working.

Well, I thought, I’d always been sent a personal copy of the SMP source code—though I hadn’t looked at it for ages. So now why not just recompile it on a modern system? But then I remembered: I’d had this “great” idea that we should keep the source code encrypted. But what was the key? I asked everyone I could think of. But nobody remembered.

It’s been years now, and I’d really like to see SMP run again. So here’s a challenge. This is the source for a C program encrypted like the SMP source code. Actually, it’s the source for the program that did the encryption: a version of the circa-1981 Unix crypt utility, “cleverly” modified by changing parameters etc. Can someone break the encryption? And finally free SMP from the strange digital time safe in which it’s been locked for so long. (Here’s what Wolfram|Alpha Pro has to say if one just uploads this raw file)

But back to the main story. I stopped working on SMP in 1983, and began alternating between basic science, software projects, and my (wonderfully educational) “hobby” of doing technology and strategy consulting. I used SMP a bit, but mostly I ended up writing lots and lots of C code, usually gluing together algorithms and graphics and interfaces.

The science that I’d started was going very well—and it was clear that there were lots of important things to do. But instead of trying to do it all myself, I decided I should try to get other people involved. And as part of that, I resolved to start a research institute—and got what amounted to bids from different universities for it. The University of Illinois was the winner, and so in August 1986 off I went there to start the Center for Complex Systems Research.

But by this point I was already getting concerned that my scheme of “other people doing the science” wasn’t so good. And within just a few weeks of arriving in Illinois I’d come up with plan B: build the best tools I could, and the best personal environment I could, and then try to do as much science as I could myself. And since I was pretty well plugged into the computer industry, I knew that powerful software systems would soon be able to run on the zillions of personal computers that were starting to appear. So I knew that if I could build something good, there’d be a good market for it, that would support an interesting company and environment.

And so it was that late in August 1986, I decided to try to build my ultimate computation system—that could do all the computations I wanted, or could imagine I would ever want.

And of course the result was Mathematica.

I knew a lot about what to do (and not do) from SMP and my other software experiences. But it was refreshing to be able to start from scratch, just trying to get the design right, without prior constraints. In SMP, algebraic computation had been the central goal. But in Mathematica, I wanted to cover lots of other areas too—numerics, graphics, programming, interfaces, whatever. I thought a lot about the foundations for the system, wondering for example whether things like the cellular automata I’d studied in my basic science could be relevant. But I just kept on coming back to the basic paradigm I’d already developed for SMP. Symbolic expressions and transformations for them seemed exactly right as a high-level, yet general, representation for computation.

If it hadn’t been for SMP, I would certainly have made a lot of mistakes. But SMP pretty much showed me what was important and what was not, and where the issues were. Looking through my archives today, I can see the painstaking process of puzzling through problems that I knew from SMP. And one by one coming up with solutions.

Meanwhile, just as for SMP, I’d assembled a team, and started the actual implementation of Mathematica. I’d also started a company—this time with me as CEO. Every day I’d write lots of code. (And to my chagrin, quite a bit of that code is still running in Mathematica today, especially in the pattern matcher and evaluator.) But my biggest focus was design. And following a practice I’d started with SMP, I wrote documentation as I developed the design. I figured if I couldn’t explain something clearly in documentation, nobody was ever going to understand it, and it probably wasn’t designed right. And once something was in the documentation, we knew both what to implement, and why we were doing it.

The first code for Mathematica was written in October 1986. And by the middle of 1987 Mathematica was beginning to come to life. I’d decided that the documentation should be published as a book, and hundreds of pages were already written. And I estimated that Mathematica 1.0 would be ready by April 1988.

My original plan for our company was to concentrate on R&D, and to distribute Mathematica primarily through computer manufacturers. Steve Jobs was the first to take Mathematica on, making a deal to bundle it with every one of his as-yet-unreleased NeXT computers. Deals with Sun, Silicon Graphics, IBM and a sequence of other companies followed. We started sending out a few beta copies of Mathematica. And—even though this was long before the web—word of its existence began to spread. Some media coverage started up too (I still like that kind of ice cream):

Sometime in the spring of 1988, we officially set June 23 as the release date for Mathematica (without Wolfram|Alpha, I didn’t know it was Alan Turing’s birthday, etc.). There was a lot to get ready. In those days releasing software didn’t just involve flipping a switch. Like I remember we were right down to the wire in getting The Mathematica Book printed. So I flew to Canada with a hard disk and personally babysat a phototypesetting machine for a long weekend, handing the box of film it produced to a person who met me at the airport in Boston and rushed it to the printer. But despite adventures like that, shortly before June 23 off were mailed some mysterious invitations:

And at noon on June 23 the room had filled, and we were ready to launch Mathematica into the world.

It’s been a great 25 years since then. The foundations that we laid in Mathematica 1.0—greatly informed by my earlier experiences—have proved incredibly robust, and we’ve been able to just build and build on them. My “plan B” of developing Mathematica, then using it to do science, worked out just great, and led to A New Kind of Science. And from Mathematica, we’ve been able to build a great company, as well as build things like Wolfram|Alpha. And over the course of 25 years, we’ve had the pleasure and privilege of seeing Mathematica contribute in all sorts of ways to many things in the world.

What was SMP like? Here are a few examples of SMP programs that I wrote for the SMP documentation:

In some ways these look quite similar to Mathematica programs—complete with [...] for functions, {...} for lists and -> for rules. But somehow the readability that’s a hallmark of Mathematica isn’t there, and instead the SMP programs seem quite cryptic and obscure.

One of the most obvious problems is that SMP code is littered with $ and % characters—appearing respectively as prefixes for pattern and local variables. In SMP, I hadn’t had the Mathematica idea of separating pattern constructs (such as _) from names (such as x). And I thought it was important to emphasize which variables were local—but didn’t have a subtle cue like color to do it with.

In SMP I’d already had the (good) idea of distinguishing immediate (=) and delayed (:=) assignment. But in a nod to languages like ALGOL, I indicated them by the rather obscure : and :: (For rules, -> was the immediate form, as it is Mathematica, while --> was the analog of :> and S[...] was the analog of /. )

In SMP, just like in Mathematica, I indicated built-in functions with capital letters (at the time it was a fairly new thing to distinguish upper and lowercase at all on a computer). But while Mathematica typically uses English words for function names, SMP used short—and often cryptic—abbreviations. When I was working on SMP, I was quite taken with the design of Unix, and wanted to emulate its practice of having short function names. That might have been OK if SMP had just a few functions. But with hundreds of functions with names like Ps, Mei and Uspb things began to get pretty unreadable. Of course, back then, there was another issue: lots of users couldn’t type quickly—so that provided a motivation to have short function names.

It’s interesting to look at the SMP documentation today. SMP had plenty of good ideas—most of which I used again in Mathematica. But it also had some quite bad ideas—which happily aren’t part of Mathematica. One example of a bad idea—that even sounds bad as soon as one hears it—are “chameleonic symbols”: symbols that change their name whenever they’re used. (These were an attempt at localizing things like dummy variables, a bit like an over-automated form of Module.)

There were some much more subtle mistakes too. Like here’s one that in a sense came from trying to go too far in unifying the system. Like Mathematica, SMP had a notion of lists, like {a,b,c}. It also had functions, like f[x]. And in my effort to achieve the maximum possible unification, I thought that perhaps one could combine the notion of lists and functions.

Let’s say one has a list v={a,b,c}. (In SMP assignment was done with :, so this would have been written v:{a,b,c}.) Then for example in SMP v[2] would extract the second element in the list. But this notation looks a lot like asking for the value of a function v when its argument is 2. And this gave me the idea that perhaps one could generalize lists—to have not just integer-indexed elements, but elements with arbitrary symbolic indices.

In SMP, pattern variables (x_ in Mathematica) were written $x. So when one defined a function f[$x] : $x^2 one could imagine that this was just defining f itself to have a value that was a symbolically indexed list: {[$x]: $x^2}. If you wanted to find out how a function was defined, you just typed its name—like f. And the value that came back would be the symbolically indexed list that represented the definition.

An ordinary vector-type list could be thought of as something like {[1]:a, [2]:b, [3]:c}. And one could mix in symbolic indices: {[1]: 1, [$x]:$x f[$x-1]}. There was also a certain unification with part numbering in general symbolic expressions. And at some level it all seemed rather nice. And to describe my unified concept of functions and lists, I called the f in f[x] a “projection”, and x a “filter”. (There were jokes about lists of definitions being “optical benches”.)

But gradually cracks started appearing. It got pretty weird, for example, when one started making definitions like v[2]:b, v[3]:c. According to SMP’s conventions for assignments v would then have value {[3]:c, [2]:b}. But what if one made a definition like v[1]:a? Well, then v suddenly had to reorder itself as {a, b, c}.

It got even weirder when one started dealing with multi-argument functions. It was quite nice that one could define a matrix with m:{{a,b},{c,d}}, then m[1] would be {a,b}, and either m[1,1] or m[1][1] would be a. But what if one had a function with several arguments? Would f[x, y] be the same as f[x][y]? Well, sometimes one wanted it that way, and sometimes not. So I had to come up with a property (“attribute” in Mathematica)—that I called Tier—to say for every function which way it should work. (Today more people might have heard of “currying”, but in those days this kind of distinction was really obscure.)

Symbolically indexed lists in SMP had some really powerful and elegant features. But in the end, when the whole system was built, there were just too many weirdnesses. And so when I designed Mathematica I decided not to use them. Over the years, though, I’ve kept thinking about them. And as it happens, right now, more than 30 years after SMP, I’m working on some very interesting new functionality in Mathematica that’s closely related to symbolically indexed lists.

I learned a huge amount designing SMP—and then seeing how the design played out. One particularly memorable moment for me was this. Like Mathematica, SMP had pure functions. But unlike Mathematica, it didn’t have a syntax like & to indicate them. And that meant that it needed a special object called a “mark” (written `) to indicate when a pure function was supposed to give a literal, constant, value. Well, about 5 years after SMP was released, I was looking at one of its training manuals. And out jumped at me the sentence: “Marks are the enigma of SMP”. And in that moment I realized: that’s what a language design mistake looks like.

SMP was in many ways a very radical system—a kind of extreme experiment in programming language design. It had only grudging support for most of what were then familiar programming constructs. And instead almost everything in it revolved around the idea of transformation rules for symbolic expressions. In some ways I think SMP went too far into the unfamiliar. Because in a sense what a programming language has to do is to connect the human conception of a computation to an actual computation that a computer can execute. And however powerful a language is, it doesn’t do much good if humans don’t have enough context to be able to understand it. Which is why in Mathematica, I’ve always tried to make things familiar when I can, limiting the unfamiliar to places where it’s really needed in supporting things that are fundamentally new.

One of the things about designing a system is knowing what’s going to end up being important. In SMP, we spent a lot of effort on what we called “semantic pattern matching”. Let’s say one made a definition like f[$x+$y, $x, $y] := {$x, $y}. It’s pretty clear that this would match f[a+b, a, b]. But what about f[7, 3, 4]? In SMP, that would match—even though the 7 isn’t structurally of the form $x+$y. It took lots of effort to make this work. And it was neat to see in simple examples. But in the end, it just didn’t come up very often—and when it did, it was usually something to avoid, because it typically made the operation of programs really hard to understand.

There was something similar with recursion control. I thought it was bad to have f[$x] : $x f[$x-1] (with no end condition for f[1]) go into an infinite loop trying to evaluate f[-1], f[-2], etc. Because after all, at some point there’s multiplication by 0. So why not just give 0? Well, in SMP the default was to give 0. Because instead of running all the way down the evaluation of each branch of the recursion tree, SMP would repeatedly stop and try to simplify all the unevaluated branches. It was neat and clever. But by the time one started parametrizing this behavior it was just too hard for people to understand, and nobody ended up using it.

And then there was user-defined syntax. Allowing users for example to set “U” (say, for “union”) to be an infix operator. Which worked great until one wanted to type a function with a “U” in its name. Or until one completely trapped oneself in one’s syntax, diverting the parsing of any form of escape.

SMP was a great learning experience for me. And Mathematica wouldn’t be nearly as good if I hadn’t done SMP first. And as I reflect now on “mistakes” in SMP, one thing I find quite satisfying is that I don’t think I’d make any of them today. Between SMP and 25 years of Mathematica design, most of them would now fall into the category of “easy issues” for me.

It’s funny, though, how often variations of some of the not-so-good ideas in SMP seem to come up. And actually I’m very curious with my modern design sensibilities how exactly I’d feel about them if I ran SMP today. Which is part of the reason I’m keen to release SMP from its “digital time safe”, and get it running again. Which I hope someone out there is going to help me make possible.

Today it’s exactly a quarter of a century since we launched Mathematica 1.0 on June 23, 1988. Much has come and gone in the world of computing since that time. But I’m pleased to say that through all of it Mathematica has just kept getting stronger and stronger.

A quarter century ago I worked very hard to lay the best possible foundations for Mathematica—and to define principles and structures on which I thought Mathematica could be grown far into the future. And looking back now I have to say that all that effort paid off far better than I could ever have imagined.

I have always insisted that Mathematica be a system without compromises: a system where everything is designed and built right. Often there have been immense technical challenges in doing this. And sometimes it’s taken years. But the result has been the creation of an ever more impressive and unique structure, able to grow and advance without bound.

Twenty-five years is a long time in the history of technology. And it’s extremely rare for a technology project to maintain a clear and consistent direction for that long. No doubt part of what’s made it possible for Mathematica is that I’ve personally been able to continue to provide leadership over all those years. But what’s also been critical is that we’ve been able to build Wolfram Research into a long-term company that can focus on long-term goals.

One might have thought that any technology that existed 25 years ago would by now look old and clunky. But not Mathematica. The interface to Mathematica 1.0 had to deal with dial-up connections and one-megabyte memory limitations. But from the very beginning I tried hard to make sure that the underlying language—what we now call “the Wolfram Language”—was pure and timeless. And while the Wolfram Language has grown and broadened immeasurably over the last quarter century, the core of it is still what was in Mathematica 1.0—and it looks as fresh and modern as ever.

My goal in building Mathematica was an ambitious one: to create once and for all a single unified computational system that could eventually handle all forms of algorithmic work. Mathematics was an early target area (hence the name “Mathematica”). But the real goal—and the core design of the system—was much broader. And over the past 25 years the system has grown at an accelerating rate to encompass more and more.

Throughout everything we’ve followed a principle that I defined right at the beginning: that whatever is added, everything across the whole system must always fit together in a coherent way. Often it’s taken great effort to achieve this. But the payoff has been spectacular. Not only has it kept an ever-larger system easy to learn and use; it’s also made possible a kind of combinatorial growth in the power of the whole system—with each new part routinely able to combine capabilities from every other part.

It’s been exciting to watch the growth of Mathematica over the past 25 years. To see so many new areas covered in each successive version. And to see what is now an immense algorithmic edifice emerge: a giant interconnected web of algorithms and capabilities quite unlike anything even imagined before.

Some of the algorithms in Mathematica are ones that were already known. But increasingly they’re ones we’ve invented. Sometimes by making use of sophisticated functionality from other parts of Mathematica. And sometimes by using methods like automated algorithm discovery from A New Kind of Science. But increasingly what we need are not just algorithms, but meta-algorithms—that automatically select between different algorithms based on a host of criteria from efficiency to aesthetics.

Automation has always been a guiding principle of Mathematica. Users define what they want to achieve. Then the idea is that it’s up to Mathematica to figure out—as automatically as possible—how best to achieve it. At first it wasn’t clear how far automation could go. But with every new version of Mathematica, we’ve found ways to automate more and more. In effect making Mathematica a higher and higher level system.

Back when Mathematica was young there was sometimes a tension: should one use Mathematica because it’s easy and general, or should one use some special-purpose system that’s specifically optimized for a particular kind of work? Well, over the past decade or so, even when it comes to efficiency, special-purpose systems have lost their edge. Because with all its capabilities Mathematica can just implement vastly better algorithms.

Looking at the development of Mathematica over the past 25 years, I see a mixture of inexorable progress, and surprises. Often pieces of ever more sophisticated functionality simply have to be built layer-by-layer over the course of many years. And sometimes it takes advances in hardware and interfaces—or in peoples’ familiarity with some concept or another—to make progress possible. But then there are surprises. Where given what exists, one can suddenly see some new possibility that one never imagined before.

Mathematica will never be truly finished. There will always be more to add to it, more to automate, more intellectual structures to discover. Often over the years I’ll be thinking about some area or another and wonder whether it’ll ever be possible to integrate it into Mathematica. But the remarkable experience that I’ve had over and over again is that eventually the answer will turn out to be yes. Sometimes it’ll require significant conceptual breakthroughs. But usually the result is that by building on the principles and foundations of Mathematica one can create something uniquely clear and powerful—that often for the first time “consumerizes” a particular area to the point where it can routinely be used.

As a software engineering achievement, Mathematica can undoubtedly be considered one of the great codebases of our time. But more than that, I think it stands as a major intellectual achievement: a unique representation and clarification of the scope and concept of computation.

Mathematica long outgrew its math-related name. And today in fact it stands at a turning point. Its content and capabilities make it relevant to a huge range of applications. And now the ambient technology of our times—cloud, mobile, and more—will finally allow it to be conveniently deployed in a quite ubiquitous way. Already Mathematica is at the core of CDF, Wolfram|Alpha and everything that is done with them. But there is vastly more to come.

It has been wonderful to see over the past 25 years so many ways that Mathematica has contributed to invention, discovery and education in the world. But I suspect that all that has happened so far will pale in comparison to what the future holds. We have spent more than a quarter of a century building up the unique structure that is Mathematica today. And in some ways it has taken the better part of 25 years to realize just how strong and important what we have is—and to get to the point where it can begin to realize its full potential.

For me and our team Mathematica is much more than a product. It is a mission. To create the means to bring the power of computation and computational knowledge to as much of our world as possible. And to create something that will stand as a broad and enduring contribution to civilization.

I am proud of what we have been able to achieve with Mathematica in the last 25 years. And it is a source of great encouragement to read remarks from people whose lives Mathematica has touched. But as I look to the future, I realize that in many ways we are just getting started. And humbling though it is after spending nearly half my life so far devoted to Mathematica, it is inevitable that in time the quarter century just passed will seem like just a small part of the development of Mathematica.

But today I am pleased to celebrate the first 25 years of Mathematica. And, yes, in recognition of the many seemingly impossible challenges that we have overcome in the development of Mathematica, the object at the top is what might seem like an impossible geometrical object: a 25-pointed “spikey”.

It’s been a great first 25 years for Mathematica. It’s been a pleasure and privilege to be part of it. And I look forward with great anticipation to the years to come.

Computational knowledge. Symbolic programming. Algorithm automation. Dynamic interactivity. Natural language. Computable documents. The cloud. Connected devices. Symbolic ontology. Algorithm discovery. These are all things we’ve been energetically working on—mostly for years—in the context of Wolfram|Alpha, Mathematica, CDF and so on.

But recently something amazing has happened. We’ve figured out how to take all these threads, and all the technology we’ve built, to create something at a whole different level. The power of what is emerging continues to surprise me. But already I think it’s clear that it’s going to be profoundly important in the technological world, and beyond.

At some level it’s a vast unified web of technology that builds on what we’ve created over the past quarter century. At some level it’s an intellectual structure that actualizes a new computational view of the world. And at some level it’s a practical system and framework that’s going to be a fount of incredibly useful new services and products.

I have to admit I didn’t entirely see it coming. For years I have gradually understood more and more about what the paradigms we’ve created make possible. But what snuck up on me is a breathtaking new level of unification—that lets one begin to see that all the things we’ve achieved in the past 25+ years are just steps on a path to something much bigger and more important.

I’m not going to be able to explain everything in this blog post (let’s hope it doesn’t ultimately take something as long as A New Kind of Science to do so!). But I’m excited to begin to share some of what’s been happening. And over the months to come I look forward to describing some of the spectacular things we’re creating—and making them widely available.

It’s hard to foresee the ultimate consequences of what we’re doing. But the beginning is to provide a way to inject sophisticated computation and knowledge into everything—and to make it universally accessible to humans, programs and machines, in a way that lets all of them interact at a vastly richer and higher level than ever before.

A crucial building block of all this is what we’re calling the Wolfram Language.

In a sense, the Wolfram Language has been incubating inside Mathematica for more than 25 years. It’s the language of Mathematica, and CDF—and the language used to implement Wolfram|Alpha. But now—considerably extended, and unified with the knowledgebase of Wolfram|Alpha—it’s about to emerge on its own, ready to be at the center of a remarkable constellation of new developments.

We call it the Wolfram Language because it is a language. But it’s a new and different kind of language. It’s a general-purpose knowledge-based language. That covers all forms of computing, in a new way.

There are plenty of existing general-purpose computer languages. But their vision is very different—and in a sense much more modest—than the Wolfram Language. They concentrate on managing the structure of programs, keeping the language itself small in scope, and relying on a web of external libraries for additional functionality. In the Wolfram Language my concept from the very beginning has been to create a single tightly integrated system in which as much as possible is included right in the language itself.

And so in the Wolfram Language, built right into the language, are capabilities for laying out graphs or doing image processing or creating user interfaces or whatever. Inside there’s a giant web of algorithms—by far the largest ever assembled, and many invented by us. And there are then thousands of carefully designed functions set up to use these algorithms to perform operations as automatically as possible.

Over the years, I’ve put immense effort into the design of the language. Making sure that all the different pieces fit together as smoothly as possible. So that it becomes easy to integrate data analysis here with document generation there, with mathematical optimization somewhere else. I’m very proud of the results—and I know the language has been spectacularly productive over the course of a great many years for a great many people.

But now there’s even more. Because we’re also integrating right into the language all the knowledge and data and algorithms that are built into Wolfram|Alpha. So in a sense inside the Wolfram Language we have a whole computable model of the world. And it becomes trivial to write a program that makes use of the latest stock price, computes the next high tide, generates a street map, shows an image of a type of airplane, or a zillion other things.

We’re also getting the free-form natural language of Wolfram|Alpha. So when we want to specify a date, or a place, or a song, we can do it just using natural language. And we can even start to build up programs with nothing more than natural language.

There are so many pieces. It’s quite an array of different things.

But what’s truly remarkable is how they assemble into a unified whole.

Partly that’s the result of an immense amount of work—and discipline—in the design process over the past 25+ years. But there’s something else too. There’s a fundamental idea that’s at the foundation of the Wolfram Language: the idea of symbolic programming, and the idea of representing everything as a symbolic expression. It’s been an embarrassingly gradual process over the course of decades for me to understand just how powerful this idea is. That there’s a completely general and uniform way to represent things, and that at every level that representation is immediately and fluidly accessible to computation.

It can be an array of data. Or a piece of graphics. Or an algebraic formula. Or a network. Or a time series. Or a geographic location. Or a user interface. Or a document. Or a piece of code. All of these are just symbolic expressions which can be combined or manipulated in a very uniform way.

But in the Wolfram Language, there’s not just a framework for setting up these different kinds of things. There’s immense built-in curated content and knowledge in each case, right in the language. Whether it’s different types of visualizations. Or different geometries. Or actual historical socioeconomic time series. Or different forms of user interface.

I don’t think any description like this can do the concept of symbolic programming justice. One just has to start experiencing it. Seeing how incredibly powerful it is to be able to treat code like data, interspersing little programs inside a piece of graphics, or a document, or an array of data. Or being able to put an image, or a user interface element, directly into the code of a program. Or having any fragment of any program immediately be runnable and meaningful.

In most languages there’s a sharp distinction between programs, and data, and the output of programs. Not so in the Wolfram Language. It’s all completely fluid. Data becomes algorithmic. Algorithms become data. There’s no distinction needed between code and data. And everything becomes both intrinsically scriptable, and intrinsically interactive. And there’s both a new level of interoperability, and a new level of modularity.

So what does all this mean? The idea of universal computation implies that in principle any computer language can do the same as any other. But not in practice. And indeed any serious experience of using the Wolfram Language is dramatically different than any other language. Because there’s just so much already there, and the language is immediately able to express so much about the world. Which means that it’s immeasurably easier to actually achieve some piece of functionality.

I’ve put a big emphasis over the years on automation. So that the Wolfram Language does things automatically whenever you want it to. Whether it’s selecting an optimal algorithm for something. Or picking the most aesthetic layout. Or parallelizing a computation efficiently. Or figuring out the semantic meaning of a piece of data. Or, for that matter, predicting what you might want to do next. Or understanding input you’ve given in natural language.

Fairly recently I realized there’s another whole level to this. Which has to do with the actual deployment of programs, and connectivity between programs and devices and so on. You see, like everything else, you can describe the infrastructure for deploying programs symbolically—so that, for example, the very structure and operation of the cloud becomes data that your programs can manipulate.

And this is not just a theoretical idea. Thanks to endless layers of software engineering that we’ve done over the years—and lots of automation—it’s absolutely practical, and spectacular. The Wolfram Language can immediately describe its own deployment. Whether it’s creating an instant API, or putting up an interactive web page, or creating a mobile app, or collecting data from a network of embedded programs.

And what’s more, it can do it transparently across desktop, cloud, mobile, enterprise and embedded systems.

It’s been quite an amazing thing seeing this all start to work. And being able to create tiny programs that deploy computation across different systems in ways one had never imagined before.

This is an incredibly fertile time for us. In a sense we’ve got a new paradigm for computation, and every day we’re inventing new ways to use it. It’s satisfying, but more than a little disorienting. Because there’s just so much that is possible. That’s the result of the unique convergence of the different threads of technology that we’ve been developing for so long.

Between the Wolfram Language—with all its built-in computation and knowledge, and ways to represent things—and our Universal Deployment System, we have a new kind of universal platform of incredible power. And part of the challenge now is to find the best ways to harness it.

Over the months to come, we’ll be releasing a series of products that support particular ways of using the Wolfram Engine and the Universal Platform that our language and deployment system make possible.

There’ll be the Wolfram Programming Cloud, that allows one to create Wolfram Language programs, then instantly deploy them in the cloud through an instant API, or a form-based app, or whatever. Or deploy them in a private cloud, or, for example, through a Function Call Interface, deploy them standalone in desktop programs and embedded systems. And have a way to go from an idea to a fully deployed realization in an absurdly short time.

There’ll be the Wolfram Data Science Platform, that allows one to connect to all sorts of data sources, then use the kind of automation seen in Wolfram|Alpha Pro, then pick out and modify Wolfram Language programs to do data science—and then use CDF to set up reports to generate automatically, on a schedule, through an API, or whatever.

There’ll be the Wolfram Publishing Platform that lets you create documents, then insert interactive elements using the Wolfram Language and its free-form linguistics—and then deploy the documents, on the web using technologies like CloudCDF, that instantly support interactivity in any web browser, or on mobile using the Wolfram Cloud App.

And we’ll be able to advance Mathematica a lot too. Like there’ll be Mathematica Online, in which a whole Mathematica session runs on the cloud through a web browser. And on the desktop, there’ll be seamless integration with the Wolfram Cloud, letting one have things like persistent symbolic storage, and instant large-scale parallelism.

And there’s still much more; the list is dauntingly long.

Here’s another example. Just as we curate all sorts of data and algorithms, so also we’re curating devices and device connections. So that built into the Wolfram Language, there’ll be mechanisms for communicating with a very wide range of devices. And with our Wolfram Embedded Computation Platform, we’ll have the Wolfram Language running on all sorts of embedded systems, communicating with devices, as well as with the cloud and so on.

At the center of everything is the Wolfram Language, and we intend to make this as widely accessible to everyone as possible.

The Wolfram Language is a wonderful first language to learn (and we’ve done some very successful experiments on this). And we’re planning to create a Programming Playground that lets anyone start to use the language—and through the Programming Cloud even step up to make some APIs and so on for free.

We’ve also been building the Wolfram Course Authoring Platform, that does major automation of the process of going from a script to all the elements of an online course—then lets one deploy the course in the cloud, so that students can have immediate access to a Wolfram Language sandbox, to be able to explore the material in the course, do exercises, and so on. And of course, since it’s all based on our unified system, it’s for example immediate that data from the running of the course can go into the Wolfram Data Science Platform for analysis.

I’m very excited about all the things that are becoming possible. As the Wolfram Language gets deployed in all these different places, we’re increasingly going to be able to have a uniform symbolic representation for everything. Computation. Knowledge. Content. Interfaces. Infrastructure. And every component of our systems will be able to communicate with full semantic fidelity, exchanging Wolfram Language symbolic expressions.

Just as the lines between data, content and code blur, so too will the lines between programming and mere input. Everything will become instantly programmable—by a very wide range of people, either by using the Wolfram Language directly, or by using free-form natural language.

There was a time when every computer was in a sense naked—with just its basic CPU. But then came things like operating systems. And then various built-in languages and application programs. What we have now is a dramatic additional step in this progression. Because with the Wolfram Language, we can in effect build into our computers a vast swath of existing knowledge about computation and about the world.

If we’re forming a kind of global brain with all our interconnected computers and devices, then the Wolfram Language is the natural language for it. Symbolically representing both the world and what can be created computationally. And, conveniently enough, being efficient and understandable for both computers and humans.

The foundations of all of this come from decades spent on Mathematica, and Wolfram|Alpha, and A New Kind of Science. But what’s happening now is something new and unexpected. The emergence, in effect, of a new level of computation, supported by the Wolfram Language and the things around it.

So far I can see only the early stages of what this will lead to. But already I can tell that what’s happening is our most important technology project yet. It’s a lot of hard work, but it’s incredibly exciting to see it all unfold. And I can’t wait to go from “Coming Soon” to actual systems that people everywhere can start to use…

Last week I wrote about our large-scale plan to use new technology we’re building to inject sophisticated computation and knowledge into everything. Today I’m pleased to announce a step in that direction: working with the Raspberry Pi Foundation, effective immediately there’s a pilot release of the Wolfram Language—as well as Mathematica—that will soon be bundled as part of the standard system software for every Raspberry Pi computer.

In effect, this is a technology preview: it’s an early, unfinished, glimpse of the Wolfram Language. Quite soon the Wolfram Language is going to start showing up in lots of places, notably on the web and in the cloud. But I’m excited that the timing has worked out so that we’re able to give the Raspberry Pi community—with its emphasis on education and invention—the very first chance to put the Wolfram Language into action.

I’m a great believer in the importance of programming as a central component of education. And I’m excited that with the Wolfram Language I think we finally have a powerful programming language worthy of the next generation. We’ve got a language that’s not mostly concerned with the details of computers, but is instead about being able to understand and create things on the basis of huge amounts of built-in computational ability and knowledge.

It’s tremendously satisfying—and educational. Writing a tiny program, perhaps not even a line long, and already having something really interesting happen. And then being able to scale up larger and larger. Making use of all the powerful programming paradigms that are built into the Wolfram Language.

And with Raspberry Pi there’s something else too: immediately being able to interact with the outside world. Being able to take pure code, and connect it to sensors and devices that do things.

I think it’s pretty amazing that we’re now at the point where all the knowledge and computation in the Wolfram Language can run in a $25 computer. And I think that it’s the beginning of something very important. Because it means that going forward it’s going to be technically possible to embed the Wolfram Language in pretty much any new machine or system. In effect immediately injecting high-level intelligence and capabilities.

I’ve waited a long time for this. Back in 1988 when Mathematica was first released, it could only just fit in a high-end Mac of the time, but not yet a PC. A decade later—even though it had grown a lot—it could run well on pretty much any newly sold personal computer. But embedded computers were a different story—where one expected that only specially compiled simple code could run.

But I knew that one day what would become the Wolfram Language would be able to run in its complete form on an embedded computer. And now it’s clear that finally that day has come: with the Raspberry Pi, we’ve passed the threshold for being able to run the Wolfram Language on an embedded computer anywhere.

To be clear, the Raspberry Pi is perhaps 10 to 20 times slower at running the Wolfram Language than a typical current-model laptop (and sometimes even slower when it’s lacking architecture-specific internal libraries). But for many things, the speed of the Raspberry Pi is just fine. And for example, my old test of computing 1989^1989 that used to take many seconds on the computers that existed when Mathematica was young now runs in an immeasurably short time on the Raspberry Pi.

From a software engineering point of view, what’s being bundled with the Raspberry Pi is a pilot version of our new Wolfram Engine. Then there are two applications on the Pi powered by this engine.  The first is a command-line version of the Wolfram Language. And the second is Mathematica with its notebook user interface, providing in effect a rich document-based way of interacting with the Wolfram Language.

The command-line Wolfram Language is quite zippy on the Raspberry Pi. The full notebook interface to Mathematica—requiring as it does the whole X Window stack—can be a trifle sluggish by modern standards (and we had to switch a few things off by default, like our new Predictive Interface, because they just slowed things down too much). But it’s still spectacular: the first time Mathematica has been able to run at all on anything like a $25 computer.

And it’s the whole system. Nothing is left out. All 5000+ Wolfram Language functions.  All capabilities of Mathematica and its notebook interface.

For me, one of the most striking things about having all this on the Raspberry Pi is how it encourages me to try a new style of real-world-connected computing. For a start, it’s easy to connect devices to a Pi. And a Pi is small and cheap enough that I can put it almost anywhere. And if I start a Wolfram Language program on it, it’s reliable enough that I can expect it to pretty much go on running forever—analyzing and uploading sensor data, controlling an autonomous system, analyzing and routing traffic, or whatever.

Building in as much automation as possible has been a longstanding principle of mine for the Wolfram Language. And when it comes to external devices, this means consistently curating properties of devices, and then setting up general symbolic functions for interacting with them.

Here’s how one would take this whole technology stack and use it to switch on LEDs by setting voltages on GPIO pins:

And here’s some image analysis on a selfie taken by a RaspiCam:

Something we’re releasing alongside the Raspberry Pi bundle is a Remote Development Kit, that allows one to develop code and maintain a notebook interface on a standard laptop or other computer, while seamlessly executing code on a networked remote Raspberry Pi. The current RDK connects to a copy of Mathematica (such as Mathematica Student Edition) running on any Mac, PC or Linux machine; soon there will be other options, for example on the web.

Within the Wolfram Language there’s actually a whole emerging structure for symbolically representing remote running language instances—and for collecting results, dispatching commands, doing computations in parallel, and so on. We’re also going to have the Wolfram Symbolic Transport Protocol (WSTP, derived from the MathLink protocol that we’ve used for nearly 25 years), that’ll let one exchange code, data or anything else in a very flexible way.

I’m very excited to see what kinds of things people invent with the Wolfram Language on the Raspberry Pi—and I look forward to reading about some of them in the Wolfram+Raspberry Pi section on Wolfram Community, as well as on the Raspberry Pi Foundation website.

In the next few months, it’s all going to get more and more interesting. What we’re releasing today on the Raspberry Pi is just the first pilot for the Wolfram Language. There’ll be many updates, particularly as we approach the first production release of the language.

As with Wolfram|Alpha on the web, the Wolfram Language (and Mathematica) on the Raspberry Pi are going to be free for anyone to use for personal purposes. (There’s also going to be a licensing mechanism for commercial uses, other Linux ARM systems, and so on.)

As a footnote to history, I might mention that the Raspberry Pi is only the second computer ever on which Mathematica has been bundled for free use. (Not counting, of course, all the computers at universities with site licenses, etc.) The first was Steve Jobs’s NeXT computer in 1988.

I still regularly run into people today who tell me how important Mathematica on the NeXT was for them. Not to mention the gaggle of NeXT computers that were bought by CERN for physicists to run Mathematica—but ended up being diverted to invent the web.

What will be done with the millions of instances of the Wolfram Language that are bundled on Raspberry Pi computers around the world? Maybe some amazing and incredibly important invention will be made with them. Maybe some kid somewhere will be inspired, and will go on to change the world.

But one thing is clear: with the Wolfram Language on Raspberry Pi we’ve got a new path for learning programming—and connecting it to the real world—that a great many people are going to be able to benefit from. And I am very pleased to have been able to do my part to make this happen.

I have the good fortune of knowing many people, which means I end up sending out lots of holiday cards. For many years I used to send out physical cards. But last year, convenience, timeliness and ease of reply made me finally make the switch to e-cards.

I often like to write notes on the cards I send. And when I was sending out paper cards, that was straightforward to do. But what about with e-cards?

Well, it’d be easy to type messages and have them printed on the e-cards. But that seems awfully impersonal. And anyway, I rather like having at least one time each year when I do a bunch of actual writing by hand—not least so my handwriting doesn’t atrophy completely.

So there’s an obvious solution: handwritten e-cards. Which is exactly what I did this year:

The background image was created by our (frequently award-winning!) company art department. (This year, the white “Spikey” is 25-pointed, celebrating the 25th anniversary of Mathematica.) But how did we get the handwriting onto it?

With the Wolfram Language, it was really easy.

First, I got together the list of email addresses I wanted to send e-cards to. Then we had code to print out pieces of paper like this:

Then I actually did my Christmas thing, and went through and wrote all the messages I wanted to write:

Then we took this stack of pages, and ran them through a scanner, getting a bunch of image files. And now we can go to work.

First, import the file:

Then pick out the part of the image corresponding to the handwritten message (the numbers are an approximation found using an interactive tool):

Now we image-process the message, and make it the right size:

Here are the elements of the actual card we’re trying to assemble:

Now we create a version of the card with the right amount of “internal padding” to have space to insert the particular message:

And then we’re ready to use ImageCompose to assemble the final image:

OK, so that’s the card we want to send. Now, who do we want to send it to? To get that, we just have to use TextRecognize to do OCR on the original scan:

And finally, just use SendMail to send the card to the address we’ve got.

And that’s it. Handwritten e-cards. Of course, since I have a lot of techie friends, there were quite a few responses along the lines of, “How did you do that?”

Well, now it’s not “my secret” any more. And by next holiday season, the Wolfram Cloud will let one make this a service anyone can use. And maybe I’ll have to come up with another little innovation for my own cards…

Connected devices are central to our long-term strategy of injecting sophisticated computation and knowledge into everything. With the Wolfram Language we now have a way to describe and compute about things in the world. Connected devices are what we need to measure and interface with those things.

In the end, we want every type of connected device to be seamlessly integrated with the Wolfram Language. And this will have all sorts of important consequences. But as we work toward this, there’s an obvious first step: we have to know what types of connected devices there actually are.

So to have a way to answer that question, today we’re launching the Wolfram Connected Devices Project—whose goal is to work with device manufacturers and the technical community to provide a definitive, curated, source of systematic knowledge about connected devices.

We have a couple of thousand devices (from about 300 companies) included as of today—and we expect this number to grow quite rapidly in the months ahead. For each device, there is a certain amount of structured information:

Whenever possible, this information is set up to be computable, so that it can for example be used in Wolfram|Alpha:

Soon you’ll be able to make all sorts of complex queries about devices, very much like the queries you can make now about consumer products:

We’re working hard to make the Wolfram Connected Devices Project an important and useful resource in its own right. But in the end our goal is not just to deal with information about devices, but actually be able to connect to the devices, and get data from them—and then do all sorts of things with that data.

But first—at least if we expect to do a good job—we must have a good way to represent all the kinds of data that can come out of a device. And, as it turns out, we have a great solution for this coming: WDF, the Wolfram Data Framework. In a sense, what WDF does is to take everything we’ve learned about representing data and the world from Wolfram|Alpha, and make it available to use on data from anywhere.

There’s a lot to say about WDF. But in terms of devices, it provides an immediate way to represent not just raw numbers from a device, but, say, images or geopositions—or actual measured physical quantities.

In Wolfram|Alpha we’ve, of necessity, assembled the world’s most complete system for handling physical quantities and their units. We’ve got a couple of thousand physical quantities built in (like length, or torque, or tensile strength, or clicks per impression), as well as nearly 10,000 units of measure (like inches, or meters per second or katals or micropascals per square root hertz). And in WDF we immediately get to use this whole setup.

So once we can get data out of a device, WDF provides a great way to represent it. And given the WDF form, there are lots of things we can do with the data.

For researchers, we’re building the Wolfram Data Repository, that lets people publish data—from devices or otherwise—using WDF in an immediately computable form.

We’re also building the Wolfram Data Science Platform, that lets people visualize and analyze data using all the sophistication of the Wolfram Language—and then generate complete interactive reports from the data, that can be deployed on the web, on mobile, offline, and so on.

But how can one actually interact with the device? Well, within the Wolfram Language we’ve been building a powerful framework for this. From a user’s point of view, there’s a symbolic representation of each device. Then there are a standard set of Wolfram Language functions like DeviceRead, DeviceExecute, DeviceReadBuffer and DeviceReadTimeSeries that perform operations related to the device.

Ultimately, this is implemented by having a Wolfram Language driver for each device. But the idea is that the end user never has to know about this. The appropriate driver is just automatically retrieved from the Wolfram Cloud when it’s needed. And then the general Wolfram Language framework goes from the low-level operations in the driver to all the various higher-level symbolic device functions. Like DeviceReadTimeSeries, which samples a series of data points from the device, then returns them in a symbolic TimeSeries object which can immediately be used for further visualization, analysis, etc.

There is another issue here, though: How does one actually make the connection to a particular device? It depends on the device. Some devices automatically connect to the cloud, perhaps through an intermediate mobile device. And in those cases, one typically just has to connect to an API exposed in the cloud.

But at least right now, many more devices connect in various kinds of wired or wireless ways to a specific local computer. Sometimes one may then want to interact with the data directly on that local computer.

But more often one either wants to have something autonomous happen with the data on the local computer. Or one wants to get the data into the cloud. For example so one can systematically have people or machines query it, generate reports from it, and so on.

And in both these cases, it’s often really convenient to have the basic device connect to some kind of small embeddable computer system. Like the Raspberry Pi $25 Linux computer, on which, conveniently enough, the Wolfram Language is bundled as part of its standard system software.

And if one’s running the Wolfram Language on the local machine connected to the device, there are mechanisms built into the language that allow both for immediate discovery, and for communication with the cloud. And more than that, with this setup there’s a symbolic representation of the device immediately accessible to the Wolfram Language in the cloud. Which means, for example, that parallel computation operations in the language can be used to aggregate data from networks of devices, and so on.

But, OK, so what are the kinds of devices one will be able to do all this with? Well, that’s what the Wolfram Connected Devices Project is intended to answer.

It’s certainly a very diverse list. Yes, there are lots of acceleration- and/or heart-rate-based health devices, and lots of GPS-based devices. But there are lots of other kinds of devices too, measuring scores of different physical quantities.

The devices range from tiny and cheap to huge and expensive. In the current list, about 2/3 of the devices are basically standalone, and 1/3 require continuous physical connectivity. The border of what counts as a “device”, as opposed to, for example, a component, is a bit fuzzy. Our operational definition for the Wolfram Connected Devices Project is that something can be considered a “connected device” if it measures some physical quantity, and can be connected to a general-purpose computer using some standard connector or connection technology.

For now, at least, we’ve excluded objects that in effect have complex custom electrical connectivity—for example, sensors that have the form factor of integrated circuits with lots of “legs” that have to be plugged into something. We’ve included, though, objects that have just a few wires coming out, that can for example immediately be plugged into GPIO ports, say on Raspberry Pi—or into analog ports on something like an Arduino connected to a Raspberry Pi.

The case of a device whose “interface” is just a few wires is usually one of the more straightforward. Things usually get more complicated when there are serial connections, USB, Bluetooth, and so on, involved. Sometimes devices make use of slightly higher-level protocols (like ANT+ or Bluetooth LE). But our experience so far is that ultimately there’s very little that’s truly standard. Each device requires custom work to create a driver, map properly to WDF, and so on.

The good news, of course, is that with the Wolfram Language we have an incredibly rich toolset for creating such drivers. Whether it’s by making use of the hundreds of import and export formats built into the language. Or all the mechanisms for calling external programs. Or the ways of handling time or place information. Or the algorithms for doing signal processing and time series analysis.

We’ve been interacting with many device manufacturers over the past year or so. And it’s been very encouraging. Because it seems as if the technology stack we’ve been building all these years is exactly what people need.

Countless times we’ve heard the same thing. ”We’re building this great device; now we want to do great things with the data from it—analyzing it, delivering it to customers, and so on.” Well, that’s exactly what we’re going to be set up to do. And we have both the deep technical capabilities that are needed, and the practical infrastructure.

The first step is to get a Wolfram Language driver for the device. Once that’s done, everything flows from it. Whether it’s just storing computable versions of data in the Wolfram Data Repository. Or doing analysis or reporting through the Wolfram Data Science Platform. Or creating dashboards. Or exposing the data through an API. Or an app. Or producing alerts from the data. Or aggregating lots of data. Or, for that matter, combining data from multiple devices—for example in effect to create “synthetic sensors”.

There are lots of possibilities. One can use Wolfram SystemModeler to have a model for a device, that can be used to run a simulation in real time. Or one can use the control systems functions in the Wolfram Language to create a controller with the device. Or in a quite different direction, one can use our Wolfram|Alpha-style linguistic capabilities to let end users make natural language or voice queries about data coming from a device.

There are several common end results that manufacturers of devices typically want. One is just that it should be possible to take data from the device and flow it into the Wolfram Data Science Platform, or Mathematica, or some other Wolfram Language system, for some kind of processing. Another is that the whole user infrastructure around the device is built using our technology. Say creating a portal or dashboard on the web, or on a mobile device, for every single user of a particular type of device. That can use either our cloud, or a private cloud. And instead of a dashboard, one can have a query mechanism. Say through natural language for humans—or through some structured API for machines or programs.

In some ways the situation with connected devices right now is probably something of a transient. Because we’re mostly thinking about connecting devices to computers, and having those run the Wolfram Language. But in the future, the Wolfram Language is going to be running on increasingly small and ubiquitous embedded computers. And I expect that more and more connected devices are just going to end up having the computer power to run the Wolfram Language inside—so that they can do all sorts of Wolfram Language processing completely internally.

Of course, even in this case there is still going to have to be Wolfram Language code that reads raw data from sensors and so on. So there’s no getting around building drivers, just like for the current way most connected devices are set up.

We’ve had the experience now of building quite a few drivers. For simple devices, it’s a quick process. But as devices start to have more commands, and can generate more sophisticated data, it takes longer. In many ways, it feels like a curation task. Given all the Wolfram Language tools we have, it’s rarely about the details of manipulating data. Rather it’s about knowing what the data means, and knitting it into the whole WDF and Wolfram Language framework.

We’re going to have a service for manufacturers to work with us to connect their devices to our system. We’re also planning to run a sequence of hackathon-like events where students and others can work with devices to set up connections (and often get free devices at the end!).

The goal is to get seamless integration of as many kinds of devices as possible. And the more kinds of devices we have, the more interesting things are going to get. Because it means we can connect to more and more aspects of the physical world, and be in a position to compute more and more about it.

Within the Wolfram Language we have a rich symbolic way to represent the world. And with connected devices we have a way to attach this representation to real things in the world. And to make the Wolfram Language become a complete language for the Internet of Things.

But today we’re taking a first step. Launching the Wolfram Connected Devices Project to start the process of curating just what things exist so far in the current generation of the Internet of Things.

Visit the Wolfram Connected Devices Project »

We’re getting closer to the first official release of the Wolfram Language—so I am starting to demo it more publicly.

Here’s a short video demo I just made. It’s amazing to me how much of this is based on things I hadn’t even thought of just a few months ago. Knowledge-based programming is going to be much bigger than I imagined…

Two weeks ago I spoke at SXSW Interactive in Austin, TX. Here’s a slightly edited transcript (it’s the “speaker’s cut”, including some demos I had to abandon during the talk):

Well, I’ve got a lot planned for this hour.

Basically, I want to tell you a story that’s been unfolding for me for about the last 40 years, and that’s just coming to fruition in a really exciting way. And by just coming to fruition, I mean pretty much today. Because I’m planning to show you today a whole lot of technology that’s the result of that 40-year story—that I’ve never shown before, and that I think is going to be pretty important.

I always like to do live demos. But today I’m going to be pretty extreme. Showing you a lot of stuff that’s very very fresh. And I hope at least a decent fraction of it is going to work.

OK, here’s the big theme: taking computation seriously. Really understanding the idea of computation. And then building technology that lets one inject it everywhere—and then seeing what that means.

I’ve pretty much been chasing this idea for 40 years. I’ve been kind of alternating between science and technology—and making these bigger and bigger building blocks. Kind of making this taller and taller stack. And every few years I’ve been able to see a bit farther. And I think making some interesting things. But in the last couple of years, something really exciting has happened. Some kind of grand unification—which is leading to a kind of Cambrian explosion of technology. Which is what I’m going to be showing you pieces of for the first time here today.

But just for context, let me tell you a bit of the backstory. Forty years ago, I was a 14-year-old kid who’d just started using a computer—which was then about the size of a desk. I was using it not so much for its own sake, but instead to try to figure out things about physics, which is what I was really interested in. 
And I actually figured out a few things—which even still get used today. But in retrospect, I think the most important thing I figured out was kind of a meta thing. That the better the tools one uses, the further one can get. Like I was never good at doing math by hand, which in those days was a problem if you wanted to be a physicist. But I realized one could do math by computer. And I started building tools for that. And pretty soon me with my tools were better than almost anyone at doing math for physics.

And back in 1981—somewhat shockingly in those days for a 21-year-old professor type—I turned that into my first product and my first company. And one important thing is that it made me realize that products can really drive intellectual thinking. 
I needed to figure out how to make a language for doing math by computer, and I ended up figuring out these fundamental things about computation to be able to do that. Well, after that I dived back into basic science again, using my computer tools.

And I ended up deciding that while math was fine, the whole idea of it really needed to be generalized. And I started looking at the whole universe of possible formal systems—in effect the whole computational universe of possible programs. I started doing little experiments. Kind of pointing my computational telescope into this computational universe, and seeing what was out there. And it was pretty amazing. Like here are a few simple programs.

Some of them do simple things. But some of them—well, they’re not simple at all.

This is my all-time favorite, because it’s the first one like this that I saw. It’s called rule 30, and I still have it on the back of my business cards 30 years later.

Trivial program. Trivial start. But it does something crazy. It sort of just makes complexity from nothing. Which is a pretty interesting phenomenon. That I think, by the way, captures a big secret of how things work in nature. And, yes, I’ve spent years studying this, and it’s really interesting.

But when I was first studying it, the big thing I realized was: I need better tools. And basically that’s why I built Mathematica. It’s sort of ironic that Mathematica has math in its name. Because in a sense I built it to get beyond math. In Mathematica my original big idea was to kind of drill down below all the math and so on that one wanted to do—and find the computational bedrock that it could all be built on. 
And that’s how I ended up inventing the language that’s in Mathematica. And over the years, it’s worked out really well. We’ve been able to build ever more and more on it.

And in fact Mathematica celebrated its 25th anniversary last year—and in those 25 years it’s gotten used to invent and discover and learn a zillion things—in pretty much all the universities and big companies and so on around the world. And actually I myself managed to carve out a decade to actually use Mathematica to do science myself. And I ended up discovering lots of things—scientific, technological and philosophical—and wrote this big book about them.

Well, OK, back when I was a kid something I was always interested in was systematizing information. And I had this idea that one day one should be able to automate being able to answer questions about basically anything. I figured out a lot about how to answer questions about math computations. But somehow I imagined that to do this in general, one would need some kind of general artificial intelligence—some sort of brain-like AI. And that seemed very hard to make.

And every decade or so I would revisit that. And conclude that, yes, that was still hard to make. But doing the science I did, I realized something. I realized that if one even just runs a tiny program, it can end up doing something of sort of brain-like complexity.

There really isn’t ultimately a distinction between brain-like intelligence, and this. And that’s got lots of implications for things like free will versus determinism, and the search for extraterrestrial intelligence. But for me it also made me realize that you shouldn’t need a brain-like AI to be able to answer all those questions about things. Maybe all you need is just computation. Like the kind we’d spent years building in Mathematica.

I wasn’t sure if it was the right decade, or even the right century. But I guess that’s the advantage of having a simple private company and being in charge; I just decided to do the experiment anyway.
 And, I’m happy to say, it turned out it was possible. And we built Wolfram|Alpha.

You type stuff in, in natural language. And it uses all the curated data and knowledge and methods and algorithms that we’ve put into it, to basically generate a report about what you asked. And, yes, if you’re a Wolfram|Alpha user, you might notice that Wolfram|Alpha on the web just got a new spiffier look yesterday. Wolfram|Alpha knows about all sorts of things. Thousands of domains, covering a really broad area. Trillions of pieces of data.

And indeed, every day many millions of people ask it all sorts of things—directly on the website, or through its apps or things like Siri that use it.

Well, OK, so we have Mathematica, which has this kind of bedrock language for describing computations—and for doing all sorts of technical computations. And we also have Wolfram|Alpha—which knows a lot about the world—and which people interact with in this sort of much messier way through natural language. Well, Mathematica has been growing for more than 25 years, Wolfram|Alpha for nearly 5. We’ve continually been inventing ways to take the basic ideas of these systems further and further. 
But now something really big and amazing has happened. And actually for me it was catalyzed by another piece: the cloud.

Now I didn’t think the cloud was really an intellectual thing. I thought it was just sort of a utility. But I was wrong. Because I finally understood how it’s the missing piece that lets one take kind of the two big approaches to computation in Mathematica and in Wolfram|Alpha and make something just dramatically bigger from them.

Now, I’ve got to tell you that what comes out of all of this is pretty intellectually complicated. But it’s also very very directly practical. I always like these situations. Where big ideas let one make actually really useful new products. And that’s what’s happened here. We’ve taken one big idea, and we’re making a bunch of products—that I hope will be really useful. And at some level each product is pretty easy to explain. But the most exciting thing is what they all mean together. And that’s what I’m going to try to talk about here. Though I’ll say up front that even though I think it’s a really important story, it’s not an easy story to tell.

But let’s start. At the core of pretty much everything is what we call the Wolfram Language. Which is something we’re just starting to release now.

The core of the Wolfram Language has been sort of incubating in Mathematica for more than 25 years. It’s kind of been proven there. But what just happened is that we got all these new ideas and technology from Wolfram|Alpha, and from the Cloud. And they’ve let us make something that’s really qualitatively different. And that I’m very excited about.

So what’s the idea? It’s really to make a language that’s knowledge based. A language where built right into the language is huge amounts of knowledge about computation and about the world. You see, most computer languages kind of stay close to the basic operations of the machine. They give you lots of good ways to manage code you build. And maybe they have add-on libraries to do specific things.

But our idea with the Wolfram Language is kind of the opposite. It’s to make a language that has as much built in as possible. Where the language itself does as much as possible. To make everything as automated as possible for the programmer.

OK. Well let’s give it a try.

You can use the Wolfram Language completely interactively, using the notebook interface we built for Mathematica.

OK, that’s good. Let’s do something a little harder:

Yup, that’s a big number. Kind of looks like a bunch of random digits. Might be like 60,000 data points of sensor data.

How do we analyze it? Well, the Wolfram Language has all that stuff built in.

So like here’s the mean:

And the skewness:

Or hundreds of other statistical tests. Or visualizations.

That’s kind of weird actually. But let me not get derailed trying to figure out why it looks like that.

OK. Here’s something completely different. Let’s have the Wolfram Language go to some kind volunteer’s Facebook account and pull out their friend network:

OK. So that’s a network. The Wolfram Language knows how to deal with those. Like let’s compute how that breaks into communities:

Let’s try something different. Let’s get an image from this little camera:

OK. Well now let’s do something to that. We can just take that image and feed it to a function:

So now we’ve gotten the image broken into little pieces. Let’s make that dynamic:

Let’s rotate those around:

Let’s like even sort them. We can make some funky stuff:

OK. That’s kind of cool. Why don’t we tweet it?

OK. So the whole point is that the Wolfram Language just intrinsically knows a lot of stuff. It knows how to analyze networks. It knows how to deal with images—doing all the fanciest image processing. But it also knows about the world. Like we could ask it when the sun rose this morning here:

Or the time from sunrise to sunset today:

Or we could get the current recorded air temperature here:

Or the time series for the past day:

OK. Here’s a big thing. Based on what we’ve done for Wolfram|Alpha, we can understand lots of natural language. And what’s really powerful is that we can use that to refer to things in the real world.

Let’s just type control-= nyc:

And that just gives us the entity of New York City. So now we can find the temperature difference between here and New York City:

OK.  Let’s do some more:

Let’s find the lengths of those borders:

Let’s put that in a grid:

Or maybe let’s make a word cloud out of that:

Or we could find all the former Soviet countries:

And let’s find their flags:

And let’s like find which is closest to the French flag:

Pretty neat, eh?

Or let’s take the first few former Soviet republics. And generate maps of their capital cities. With 10-mile discs marked:

I think it’s pretty amazing that you can do that kind of thing right from inside a programming language, with just a line of code.

And, you know, there’s a huge amount of knowledge built into the Wolfram Language. 
We’ve been building this for more than a quarter of a century.

There’s knowledge about algorithms. And about the world.

There are two big principles here. The first is maximum automation: automate as much as possible. You define what you want the language to do, then it’s up to it to figure out how to do it. There might be hundreds of algorithms for doing different cases of something. But what we want to do is to make a meta-algorithm that selects the best way to do it. So kind of all the human has to do is to define their goal, then it’s up to the system to do things in the way that’s fastest, most accurate, best looking.

Like here’s an example. There’s a function Classify that tries to classify things. You just type Classify. 
Like here’s a very small training set of handwritten digits:

And this makes a classifier.

Which we can then apply to something we draw:

OK, well here’s another big thing about the Wolfram Language: coherence. Unification. We want to make everything in the language fit together. Even though it’s a huge system, if you’re doing something over here with geographic data, we want to make sure it fits perfectly with what you’re doing over there with networks.

I’ve spent a decent fraction of the last 25 years of my life implementing the kind of design discipline that’s needed. It’s been fascinating, but it’s been hard work. Spending all that time to make things obvious. To make it so it’s easy for people to learn and remember and guess. But you know, having all these building blocks fit together: that’s also where the most powerful new algorithms come from. And we’ve had a great time inventing tons and tons of new algorithms that are really only possible in our language—where we have all these different areas integrated.

And there’s actually a really fundamental reason that we can do this kind of integration. It’s because the Wolfram Language has this very fundamental feature of being symbolic. If you just type x into the language, it doesn’t give some error about x being undefined. x is just a thing—symbolic x—that the language can deal with. Of course that’s very nice for math.

But as far as I am concerned, one of the big discoveries is that this idea of a symbolic language is incredibly powerful for zillions of other things too. Everything in our language is symbolic. Math expressions.

Or entities, like Austin, TX:

Or like a piece of graphics. Here’s a sphere:

Here are a bunch of cylinders:

And because everything is just a symbolic expression, we could pick this up, and, like, do image processing on it:

You know, everything is just a symbolic expression. Like another example is interfaces. Here’s a symbolic slider:

Here’s a whole array of sliders:

You know, once everything is symbolic, there’s just a whole lot you can do. Here’s nesting some purely symbolic function f:

Here’s nesting, like, a function that makes a frame:

And here’s symbolically nesting, like, an interface element:

My gosh, it’s a fractal interface!

You know, once things are symbolic, it’s really easy to hook everything up. Like here’s a plot:

And now it’s trivial to make it interactive:

You can do that with anything:

OK. Here’s another thing that can be made symbolic: documents.

The document I’m typing into here is just another symbolic expression. And you can create whatever you want in it symbolically.

Like here’s some text. We could twirl it around if we want to:

All just symbolic expressions.

OK. So here’s yet another thing that’s a symbolic expression: code. Every piece of code in the Wolfram Language is just a symbolic expression, that can be picked up and manipulated, and passed around, and run, wherever you want. That’s incredibly important for programming. Because it means you can build things in a really modular way. Every piece can stand on its own.

It’s also important for another reason: it’s a great way to deal with the cloud, sort of treating it as a giant active repository for symbolic lumps of computation. And in fact we’ve built this whole infrastructure for that, that I’m going to demo for the first time here today.

Well, let’s say we have a symbolic expression:

Now we can just deploy it to the Cloud like this:

And we’ve got a symbolic CloudObject, with a URL we can go to from anywhere. And there’s our material.

Now let’s make this not static content, but an actual program. And on the web, a good way to do that is to have an API. But with our whole notion of everything being symbolic, we can represent that as just another symbolic expression:

And now we can deploy that to the Cloud:

And we’ve got an Instant API. Now we can just fill in an API parameter ?size=150
 and we can run this from anywhere on the web:

And every time what’ll happen is that you’ll be calling that piece of Wolfram Language code in the Wolfram Cloud, and getting the result back. OK.

Here’s another thing to do: make a form. Just change the APIFunction to a FormFunction:

Now what we’ve got is a form:

Let’s add a feature:

Now let’s fill some values into the form:

And when we press Submit, here’s the result:

OK.  Let’s try a different case.  Here’s a form that takes two cities, and draws a map of the path between them:

Let’s deploy it in the Cloud:

Now let’s fill in the form:

And when we press Submit, here’s what we get:

One line of code and an actual little web app! It’s got quite a bit of technology inside it. Like you see these fields. They’re what we call smart fields. That leverage our natural language understanding stack:

If you don’t give a city, here’s what happens:

When you do give a city, the system is automatically interpreting the inputs as city entities. Let me show you what happens inside. Let’s just define a form that just returns a list of its inputs:

Now if we enter cities, we just get Wolfram Language symbolic entity objects. Which of course we can then compute with:

All right, let’s try something else.

Let’s do a sort of modern programming example. Let’s make a silly app that shows us pictures through the eyes of a cat or a dog. 
OK, let’s build the framework:

Now let’s pull in an actual algorithm for dog vision. Color channels, and acuity.

OK. Let’s deploy with that:

Now we can send that over as an app.  But first let’s build an icon for it:

And now let’s deploy it as a public app:

Now let’s go to the Wolfram Cloud app on an iPad:

And there’s the app we just published:

Now we click that icon—and there we have it: a mobile app running against the Wolfram Language in the Cloud:

And we can just use the iPad camera to input a picture, and then run the app on it:

Pretty neat, eh?

OK, but there’s more. Actually, let me tell you about the first product that’s coming out of our Wolfram Language technology stack. It should be available very soon. We call it the Wolfram Programming Cloud.

It’s all the stuff I’m showing you, but all happening in the Cloud. Including the programming. And, yes, there’s a desktop version too.

OK, so here’s the Programming Cloud:

Deploy from the Cloud. Define a function and just use CloudDeploy[]:

Or use the GUI:

Oh, another thing is to take CDF and deploy it to run in the Cloud.

Let’s take some code from the Wolfram Demonstrations Project. Actually, as it happens, this was the very first Demonstration I wrote when were originally building that site:

Now here’s the deployed Cloud CDF:

It just needs a web browser. And gives arbitrary interactivity by running against the Wolfram Engine in the Cloud.

OK, well, using this technology, another product we’re building is our Data Science Platform.

And the idea is that data comes in, from all sorts of sources. And then we have all these automatic ways to analyze it. Using sort of a giant meta-algorithm. As well as using all the knowledge of the actual world that we have.

Well, then you can program whatever you want with the Wolfram Language. And in the end you can make reports. On demand, like from an API or an app. Or just on a schedule. And we can use our whole CDF symbolic documents to set up these reports.

Like here’s a template for a report on the state of my email inbox. It’s just defined as a symbolic document. That I go ahead and edit.

And then programmatically generate reports from:

You know, there are some really spectacular things we can do with data using our whole symbolic language technology stack. And actually just recently we realized that we can use it to make a very clean unification and generalization of SQL and NoSQL databases. And we’re implementing that in sort of four transparent levels. In memory. In files. In databases. And distributed.

But OK. Another thing is that we’ve got a really good way to represent individual pieces of data.
 We call it WDF—the Wolfram Data Framework.

And basically what it is, is taking the kind of algorithmic ontology that we built for Wolfram|Alpha—and that we know works—and exposing that. And using our natural language understanding to be able to take unstructured data, and automatically convert it to something that’s structured and computable. And that for example our Data Science Platform can do really good things with.

Well, OK. Here’s another thing. A rapidly increasing source of data out there in the world are connected devices. And we’ve been pretty deeply involved with those. And actually one thing I wanted to do recently was just to find out what devices there are out there.
 So we started our Connected Devices Project, to just curate the devices out there—just like we curate all sorts of other things in Wolfram|Alpha.

We have about 2500 devices in here now, growing every day. And, yes, we’re using WDF to organize this, and, yes, all this data is available from Wolfram|Alpha.

Well, OK. So there are all these devices. And they measure things and do things. And at some point they typically make web contact. And one thing we’re doing—with our Data Science Platform and everything—is to create a really smooth infrastructure for handling things from there on. For visualizing and analyzing and computing everything that comes from that Internet of Things.

You know, even for devices that haven’t yet made web contact, it can be a bit messier, but we’ve got a framework for handling those too. Like here’s an accelerometer connected to an Arduino:

Let’s see if we can get that data into the Wolfram Language. It’s not too hard:

And now we can immediately plot this:

So that’s connecting a device to the Wolfram Language. But there’s something else coming too. And that’s actually putting the Wolfram Language onto devices. And this is where 25 years of tight software engineering pays back. Because as soon as devices run things like Linux, we can run the Wolfram Language on them. And actually there’s now a preliminary version of the Wolfram Language bundled with the standard operating system for every Raspberry Pi.

It’s pretty neat being able to have little $25 devices that persistently run the Wolfram Language. And connect to sensors and actuators and things. And every little computer out there just gets represented as yet another symbolic object in the Wolfram Language. And, like, it’s trivial to use the built-in parallel computation capabilities of the Wolfram Language to pull data from lots of such machines.

And going forward, you can expect to see the Wolfram Language running on lots of embedded processors. There’s another kind of embedding we’re interested in too. And that’s software embedding. We want to have a Universal Deployment System for the Wolfram Language.

Given a Wolfram Language program, there are lots of ways to deploy it.

Here’s one: being able to call Wolfram Language code from other languages.

And we have a really easy way to do that. There’s a GUI, but in the Wolfram Language, you can just take an API function, and say: create embed code for this for Python. Or Java. Or whatever.

And you can then just insert that code in your external program, and it’ll call the Wolfram Cloud to get a computation done. Actually, there are going to be ways to do this from inside IDEs, like Wolfram Workbench.

This is really easy to set up, and as I said, it just calls the Wolfram Cloud to run Wolfram Language code. But there’s even another concept. There’s an Embedded Wolfram Engine that you can run locally too. And essentially the same code will then work. But now you’re running on your local machine, not in the Cloud. And things get pretty interesting, being able to put Embedded Wolfram Engines inside all kinds of software, to immediately add all that knowledge-based capability, and all those algorithms, and natural language and so on. Here’s what the Embedded Wolfram Engine looks like inside the Unity Game Engine IDE:

Well, talking of embedding, let me mention yet another part of our technology stack. The Wolfram Language is supposed to describe the world. And so what about describing devices and machines and so on.

Well, conveniently enough we have a product related to our Mathematica business called SystemModeler, which does large-scale system modeling and simulation:

And now that’s all getting integrated into the Wolfram Language too.

So here’s a representation of a rectifier circuit:

And this is all it takes to simulate this device:

And to plot parameters from the simulation:

And here’s yet another thing. We’re taking the natural language understanding capabilities that we created for Wolfram|Alpha, and we’re setting them up to be customizable. Now of course that’s big when one’s querying databases, or controlling devices. It’s also really interesting when one’s interacting with simulations. Looking at some machine out in the field, and being able to figure out things about it by talking to one’s mobile device, and then getting a simulation done in the Cloud.

There are lots of possibilities. 

But OK, so how can people actually use these things? Well, in the next couple of weeks there’ll be an open sandbox on the web for people to use the Wolfram Language. We’ve got a gallery of examples that gives good places to start.

Oh, as well as 100,000 live examples in the Wolfram Language documentation.

And, OK, the Wolfram Programming Cloud is also coming very soon. And it’ll be completely free to start developing with it, and even to do small-scale deployments.

So what does this mean?

Well, I think it’s pretty exciting. Because I think we just really changed the economics of going from algorithmic ideas to deployed products. If you come by our booth at the South By trade show, we’ll be doing a bunch of live coding there. And perhaps we’ll even be able to create little products for people right there. But I think our Programming Cloud is going to open up a surge of algorithmic startups. And I’ll be really interested to see what comes out.

OK. Here’s another thing that’s going to change I think: programming education. I think the Wolfram Language is sort of uniquely good for education. Because it’s a language where you get to do real things incredibly easily. You get to see computation at work in an incredibly powerful way. And, by the way, rather effortlessly see a bunch of modern computer science ideas… and immediately connect to the real world.

And the natural language aspect makes it really easy to get started. For serious programmers, I think having snippets of natural language programming, particularly in places where one’s connecting to the real world, is very powerful. But for people getting started, it’s really nice to be able to create things just with natural language.

Like here we can just say:

And have the code generated automatically.

We’re really interested in all the educational possibilities here. Certainly there’s the raw material for a zillion great hackathon projects.

You know, every summer for the past dozen years we’ve done a very successful summer school about the new kind of science I’ve worked on:

Where we’re effectively doing real-time science. We’ve also for a few years had a summer camp for high-school students:

And we’re using our experience here to build out a bunch of ways to use the Wolfram Language for programming education. You know, we’ve been involved in education for a long time—more than 25 years. Mathematica is incredibly widely used there. Wolfram|Alpha I’m happy to say has become sort of a universal tool for students.

There’s more and more coming.

Like here’s a version of Wolfram|Alpha in Chinese that’s coming soon:

Here’s a Problem Generator created with the Wolfram Language and available through Wolfram|Alpha Pro:

And we’re going to be doing all sorts of elaborate educational analytics and things through our Cloud system. You know, there are just so many possibilities. Like we have our CDF—Computable Document Format—that people have used for quite a few years to make interactive Demonstrations.

In fact here’s our site with nearly 10,000 of them:

And now with our Cloud system we can just run all of these directly in a web browser, using Cloud CDF, so they become easy to integrate into web learning environments. Like here’s an example that just got done by Versal:

Well, OK, at kind of the other end of things from education, there’s a lot going on in the corporate area. We’ve been doing large-scale custom deployments of Wolfram|Alpha for several years. But now with our Data Science Platform coming, we’ve got a kind of infinitely customizable version of that. And of course everything is integrated between cloud and desktop. And we’re going to have private clouds too.

But all this is just the beginning. Because what we’ve got with the whole Wolfram Language stack is a kind of universal platform for creating products. And we’ve got a whole sequence of products in the pipeline. It’s an exciting feeling having all this stuff that we’ve been doing for more than a quarter of a century come together like this.

Of course, it’s big challenge dealing with all the possibilities. I mean, we’re just a little private company with about 700—admittedly very talented—people.

We’ve started spinning off companies. Like Touch Press which makes iPad ebooks.

And we’ll be doing more of that, though we need more entrepreneurs. And we might even take investors.

But, OK, what about the broader future?

I think about that a fair amount. I don’t have time to say much here. But let me say just a few things. 

In what we’ve done with computation and knowledge, we’re trying to take the knowledge of our civilization, and put it in computable form. So we can essentially inject it everywhere. In something like Wolfram|Alpha, we’re essentially doing on-demand computation. You ask for something, and Wolfram|Alpha will do it.

Increasingly, we’re going to have preemptive computation. We’re building towards that a lot with the Wolfram Language. Being able to model the world, and make predictions about what’s going to happen. Being able to tell you what you might want to do next. In fact, whenever you use the Wolfram Language interactively, you’ll see this little Suggestions Bar that’s using some fairly fancy computation to suggest what to do next.

But the real way to have that work is to use knowledge about you. I’ve been an enthusiast of personal analytics for a long time. Like here’s a 25-year history of my diurnal email rhythm:

And as we have more sensors and outsource more of our memory, our machines will be better and better at telling us what to do. And at some level the machines take over just because the humans tend to follow the auto-suggests they make.

But OK. Here’s something I realized recently. I’m interested in history, and I was visiting the archives of Gottfried Leibniz, who lived about 300 years ago, and had a lot of rather modern ideas about computing. But in his time he had only one—very primitive—proto-computer that he built:

Today we have billions of computers. So I was thinking about the extrapolation. And I realized that one day there won’t just be lots more computers—everything will actually be made of computers.

Biology has already a little bit figured out this idea. But one day it won’t be worth making anything out of dumb materials; instead everything will be made out of stuff that’s completely programmable.

So what does that mean? Well, of course it really blurs the distinction between hardware and software. And it means that these languages we create sort of become what everything is made of. You know, I’ve been interested for a long time in the fundamental theory of physics. And in fact with a bunch of science I’ve done, I think there’s a real possibility that we’ve finally got a new way to find such a theory. In effect a way to find our physical universe out in the computational universe of all possible universes.

But here’s the funny thing: once everything is made of computers, even though it’ll be really cool to find the fundamental theory of physics—and I still want to do it—it’s not going to matter so much. Because in effect that actually physics is just the machine code for the universe. But everything we deal with is on top of a layer that we can program however we want.

Well, OK, what does that mean for us humans? No doubt we’ll get to deploy in that sort of much-more-than-biology-programmable world. Where in effect you can just build any universe for yourself. I sort of imagine this moment where there’s a box of a trillion souls. Running in whatever pieces of the computational universe they want.

And what happens? Well, there’s lots of computation going on. But from the science I’ve done—and particularly the Principle of Computational Equivalence—I think it’s sort of a very Copernican situation. I don’t think there’s anything fundamentally different about that computation, from what goes on all over the universe, and even in rather simple programs.

And at some level the only thing that’s special about that particular box of a trillion souls is that it’s based on our particular history. Now, you know, I deal with all this tech stuff. But I happen to like people; I guess that’s why I’ve liked building a company, and mentoring lots of people. And in a sense seeing how much is possible, and how much can sort of be generalized and virtualized with technology, actually makes me think people are more important rather than less. Because when everything is possible, what matters is just what one wants or chooses to do.

It’s sort of a big version of what we’re doing with the Wolfram Language. Humans define the goals, then technology automatically tries to achieve them. And the more we can inject computation into everything, the more this becomes possible. And, you know, I happen to think that the injection of computation into everything will be a defining feature—perhaps the defining feature—of this time in history.

And I have to say I’m personally pleased to have lived at the right time to make some contribution to this. It’s a great privilege. And I’m very pleased to have been able to tell you a little bit about it here today.

Thank you very much.

Last weekend I gave a speech at this year’s graduation event for the Stanford Online High School (OHS) that one of my children has been attending. Here’s the transcript:

Thank you for inviting me to be part of this celebration today—and congratulations to this year’s OHS graduates.

You know, as it happens, I myself never officially graduated from high school, and this is actually the first high school graduation I’ve ever been to.

It’s been fun over the past three years—from a suitable parental distance of course—to see my daughter’s experiences at OHS. One day I’m sure everyone will know about online high schools—but you’ll be able to say, “Yes, I was there when that way of doing such-and-such a thing was first invented—at OHS.”

It’s great to see the OHS community—and to see so many long-term connections being formed independent of geography. And it’s also wonderful to see students with such a remarkable diversity of unique stories.

Of course, for the graduates here today, this is the beginning of a new chapter in their stories.

I suspect some of you already have very definite life plans. Many are still exploring. It’s worth remembering that there’s no “one right answer” to life. Different people are amazingly different in what they’ll consider an “‘A’ in life”. I think the first challenge is always to understand what you really like. Then you’ve got to know what’s out there to do in the world. And then you’ve got to solve the puzzle of fitting the two together.

Maybe you’ll discover there’s a niche that already exists; maybe you’ll have to create one.

I’ve always been interested in trajectories of peoples’ lives, and one thing I’ve noticed is that after some great direction has emerged in someone’s life, one can almost always look back and see the seeds of it very early.

Like I was recently a bit shocked actually to find some things I did when I was 12 years old—about systematizing knowledge and data—and to realize that what I was trying to do was incredibly similar to Wolfram|Alpha. And then to realize that my tendency to invent projects and organize other kids to help do them was awfully like leading an entrepreneurial company.

You know, it’s funny how things can play out. Back when I was a kid I was really interested in physics. And to do physics you have to do a lot of math calculations. Which I found really boring, and wasn’t very good at.

So what did I do? Well, I figured out that even though I might not be good at these calculations, I could make a computer be good at them. And needless to say, that’s what I did—and through a pretty straight path, that’s what brought the world Mathematica and Wolfram|Alpha.

You know, another thing was that when I was a kid I always had a hard time getting myself to do exercises from textbooks. I kept on thinking to myself, “Why am I doing this exercise when zillions of other people have already done it? Why don’t I do something different, that’s new, and mine?”

People might think: that must be really hard. But it’s not. It’s just that you have to learn not just about how to do stuff, but also about how to figure out what stuff to do. And actually one thing I’ve noticed is that in almost every area, the people who go furthest are not the ones with the best technical skills, but the ones who have the best strategy for figuring out what to do.

But I have to say that for me it’s just incredibly fun inventing new stuff—and that’s pretty much what I’ve spent my life doing.

I think most people don’t really internalize enough how stuff in our world gets made. I mean, everything we have in our civilization—our technology, our ways of doing things, whatever—had to be invented. It had to start with some person somewhere—maybe like you—having an idea. And then that idea got turned into reality.

It’s a wonderful thing going from nothing but an idea—to something real in the world. For me, that’s my favorite thing to do. And I’ve been fortunate enough to do that with a number of big projects, alternating between science, technology and business. At some level, my projects might look very different: building a new kind of science, creating a computer language, encoding the world’s knowledge in computational form.

But it turns out that at some level they’re really all the same. They’re all about taking some complicated area, drilling down to the essence of it, then doing a big project to build up to something that’s useful in the world.

And when you think about what it is you really like, and what you’re really good at, it’s important to be thematic. Maybe you like math. But why? Is it the definiteness? Problem solving? Elegance? Even at OHS you only get to learn about certain specific subjects. So to understand yourself, you have to take your reactions to them, and generalize—figure out the overall theme.

You know, something I’ve learned is that the more different areas I know about, the better. When I was a kid I learned Latin and Greek—and I was always complaining that they’d never be useful. But then I grew up—and had to make up names for products and things. And actually for years a big part of what I’ve done every day is to take ideas from very different areas that I’ve learned about—and bring them together to make new ideas.

One thing, if you want to do this, is that you really have to keep all those things you’ve learned at your fingertips. History of science can’t stay in a history of science class. It has to inform that clever social media idea you have, or that great new policy direction you come up with, or that artistic creation you’re making, or whatever. The real payoff comes not from doing well in the class, but from internalizing that way of thinking or that knowledge so it becomes part of you.

You know, as you think about what to do in the world, it’s worth remembering that some of the very best areas are ones that almost nobody’s heard about yet—and there certainly aren’t classes about. But if you get into one of those new areas, it’s great—because there’s still all this basic ground-floor stuff to do there, and as the area grows, you get propelled by that.

I’ve been pretty lucky in that regard. Because early in life I got really interested in computation, and in the computational way of thinking about things. And I think it’s becoming clear that computation is really the single most important idea that’s emerged in the past century. And that even after all the technology that’s been built with it, we’re only just beginning to see its true significance.

And today, you just have to prepend the word “computational” to almost any existing field to get something that’s an exciting growth direction: computational law, computational medicine, computational archaeology, computational philosophy, computational photography, whatever.

And yes, to be able to do all this stuff, you have to get familiar with the computational way of thinking, and with things like programming. That’s going to be an increasingly important literacy skill. And I have to say that in general, even more valuable than learning the content of specific fields is to learn general approaches and tools—and keep up to date with them.

It’s not for everybody, but I myself happen to have spent a lot of time actually building tools. And for me the most powerful thing has been being able to build a tower of tools, then to use them to figure things out, then to use those things to go on and build more tools. And I’ve been fortunate enough to be able to go on doing that for more than 30 years now.

You know, it’s always an interesting judgment call when to go on in a life direction you’re already going, and when to branch into something new, or to chase some new opportunity. For myself, I try to maintain a portfolio—continuing to build on what I’ve done, but also always making sure to add new things.

One of the consequences of that is that at any given time, there’s always an area where I’m basically a beginner—and just learning. Right now, for example, that happens to be programming education. We’ve managed to automate a lot of programming—which I think is going to be a pretty big deal in general—but for education it means there’s a much broader range of people and places where programming can be taught. But how should it be done? Math and language and areas like that have centuries of education experience to draw on. But with what’s now possible with programming education, we’ve got a completely new situation, that kind of has to be figured out from the ground up. It’s always a little scary doing something like that, and I always think, “Maybe this is finally an area I’ll never figure out.” But somehow if one has the confidence to keep going, it always seems to come together—and it’s really satisfying.

You know, when I was a kid I learned some things in school and some things on my own. I was always doing projects about this or that. And somehow I’ve just kept on doing projects and learning more and more things. You’ve been exposed to lots of interesting things at OHS. Make sure you expose yourselves to lots more things in college or wherever you’re going next. And don’t forget to do projects—to do things that are really yours, and that people can look at and really get a sense of you from.

And don’t just learn stuff. Keep thinking about strategy too. Keep trying to solve the puzzle of what your best niche is. You might find it or you might have to create it. But there will be something great out there for you. And never assume that the world won’t let you get to it. It’s all part of the puzzle to solve. And the seeds are already there in who you are; you just have to find them, nurture them, and keep pushing to let them grow as each chapter of your story unfolds…

Twenty-six years ago today we launched Mathematica 1.0. And I am excited that today we have what I think is another historic moment: the launch of Wolfram Programming Cloud—the first in a sequence of products based on the new Wolfram Language.

My goal with the Wolfram Language in general—and Wolfram Programming Cloud in particular—is to redefine the process of programming, and to automate as much as possible, so that once a human can express what they want to do with sufficient clarity, all the details of how it is done should be handled automatically.

I’ve been working toward this for nearly 30 years, gradually building up the technology stack that is needed—at first in Mathematica, later also in Wolfram|Alpha, and now in definitive form in the Wolfram Language. The Wolfram Language, as I have explained elsewhere, is a new type of programming language: a knowledge-based language, whose philosophy is to build in as much knowledge about computation and about the world as possible—so that, among other things, as much as possible can be automated.

The Wolfram Programming Cloud is an application of the Wolfram Language—specifically for programming, and for creating and deploying cloud-based programs.

How does it work? Well, you should try it out! It’s incredibly simple to get started. Just go to the Wolfram Programming Cloud in any web browser, log in, and press New. You’ll get what we call a notebook (yes, we invented those more than 25 years ago, for Mathematica). Then you just start typing code.

It’s all interactive. When you type something, you can immediately run it, and see the result in the notebook.

Like let’s say you want to build a piece of code that takes text, figures out what language it’s in, then shows an image based on the flag of the largest country where it’s spoken.

First, you might want to try out the machine-learning language classifier built into the Wolfram Language:

OK. That’s a good start. Now we have to find the largest country where it’s spoken:

Now we can get a flag:

Notebooks in the Wolfram Programming Cloud can mix text and code and anything else, so it’s easy to document what you’re doing:

We’re obviously already making pretty serious use of the knowledge-based character of the Wolfram Language. But now let’s say that we want to make a custom graphic, in which we programmatically superimpose a language code on the flag.

It took me about 3 minutes to write a little function to do this, using image processing:

And now we can test the function:

It’s interesting to see what we’ve got going on here. There’s a bit of machine learning, some data about human languages and about countries, some typesetting, and finally some image processing. What’s great about the Wolfram Language is that all this—and much much more—is built in, and the language is designed so that all these pieces fit perfectly together. (Yes, that design discipline is what I personally have spent a fair fraction of the past three decades of my life on.)

But OK, so we’ve got a function that does something. Now what can we do with it? Well, this is one of the big things about the Wolfram Programming Cloud: it lets us use the Wolfram Language to deploy the function to the cloud.

One way we can do that is to make a web API. And that’s very straightforward to do in the Wolfram Language. We just specify a symbolic API function—then deploy it to the cloud:

And now from anywhere on the web, if we call this API by going to the appropriate URL, our Wolfram Language code will run in the Wolfram Cloud—and we’ll get a result back on the web, in this case as a PNG:

There are certainly lots of bells and whistles that we can add to this. We can make a fancier image. We can make the code more efficient by precomputing things. And so on. But to me it’s quite spectacular—and extremely useful—that in a matter of seconds I’m able to deploy something to the cloud that I can use from any website, web program, etc.

Here’s another example. This time I’m setting up a URL which, every time it’s visited, gives the computed current number of minutes until the next sunset, for the inferred location of the user:

Every time you visit this URL, then, you get a number, as a piece of text. (You can also get JSON and lots of other things if you want.)

It’s easy to set it up a dashboard too. Like here’s a countdown timer for sunset, which, web willing, updates every half second:

What about forms? Those are easy too. This creates a form that generates a map of a given location, with a disk of a given radius:

Here’s the form:

And here’s the result of submitting the form:

There’s a lot of fancy technology being used here. Like even the fields in the form are “Smart Fields” (as indicated by their little icons), because they can accept not just literal input, but hundreds of types of arbitrary natural language—which gets interpreted by the same Natural Language Understanding technology that’s at the heart of Wolfram|Alpha. And, by the way, if, for example, your form needs a color, the Wolfram Programming Cloud will automatically create a field with a color picker. Or you can have radio buttons, or a slider, or whatever.

OK, but at this point, professional programmers may be saying, “This is all very nice, but how do I use this in my particular environment?” Well, we’ve gone to a lot of effort to make that easy. For example, with forms, the Wolfram Language has a very clean mechanism for letting you build them out of arbitrary XML templates, to give them whatever look and feel you want.

And when it comes to APIs, the Wolfram Programming Cloud makes it easy to create “embed code” for calling an API from any standard language:

Soon it’ll also be easy to deploy to a mobile app. And in the future there’ll be Embedded Wolfram Engines and other things too.

So what does it all mean? I think it’s pretty important, because it really changes the whole process—and economics—of programming. I’ve even seen it quite dramatically within our own company. As the Wolfram Language and the Wolfram Programming Cloud have been coming together, there’ve been more and more places where we’ve been able to use them internally. And each time, it’s been amazing to see programming tasks that used to take weeks or months suddenly get done in days or less.

But much more than that, the whole knowledge-based character of the Wolfram Language makes feasible for the first time all sorts of programming that were basically absurd to consider before. And indeed within our own organization, that’s for example how it became possible to build Wolfram|Alpha—which is now millions of lines of Wolfram Language code.

But the exciting thing today is that with the launch of the Wolfram Programming Cloud, all this technology is now available to anyone, for projects large and small.

It’s set up so that anyone can just go to a web browser and—for free—start writing Wolfram Language code, and even deploying it on a small scale to the Wolfram Cloud. There are then a whole sequence of options available for larger deployments—including having your very own Wolfram Private Cloud within your organization.

Something to mention is that you don’t have to do everything in a web browser. It’s been a huge challenge to implement the Wolfram Programming Cloud notebook interface on the web—and there are definite limitations imposed by today’s web browsers and tools. But there’s also a native desktop version of the Wolfram Programming Cloud—which benefits from the 25+ years of interface engineering that we’ve done for Mathematica and CDF.

It’s very cool—and often convenient—to be able to use the Wolfram Programming Cloud purely on the web. But at least for now you get the very best experience by combining desktop and cloud, and running the native Wolfram Desktop interface connected to the Wolfram Cloud. What’s really neat is that it all fits perfectly together, so you can seamlessly transfer notebooks between cloud and desktop.

I’ve built some pretty complex software systems in my time. But the Wolfram Programming Cloud is the most complex I’ve ever seen. Of course, it’s based on the huge technology stack of the Wolfram Language. But the collection of interactions that have to go on in the Wolfram Programming Cloud between the Wolfram Language kernel, the Wolfram Knowledgebase, the Wolfram Natural Language Understanding System, the Wolfram Cloud, and all sorts of other subsystems are amazingly complex.

There are certainly still rough edges (and please don’t be shy in telling us about them!). Many things will, for example, get faster and more efficient. But I’m very pleased with what we’re able to launch today as the Wolfram Programming Cloud.

So if you’re going to try it out, what should you actually do? First, go to the Wolfram Programming Cloud on the web:

There’s a quick Getting Started video there. Or you can check out the Examples Gallery. Or you can go to Things to Try—and just start running Wolfram Language examples in the Wolfram Programming Cloud. If you’re an experienced programmer, I’d strongly recommend going through the Fast Introduction for Programmers:

This should get you up to speed on the basic principles and concepts of the Wolfram Language, and quickly get you to the point where you can read most Wolfram Language code and just start “expanding your vocabulary” across its roughly 5000 built-in functions:

Today is an important day not only for our company and our technology, but also, I believe, for programming in general. There’s a lot that’s new in the Wolfram Programming Cloud—some in how far it’s been possible to take things, and some in basic ideas and philosophy. And in addition to dramatically simplifying and automating many kinds of existing programming, I think the Wolfram Programming Cloud is going to make possible whole new classes of software applications—and, I suspect, a wave of new algorithmically based startups.

For me, it’s been a long journey. But today I’m incredibly excited to start a new chapter—and to be able to see what people will be able to do with the Wolfram Language and the Wolfram Programming Cloud.

To comment, please visit the copy of this post at the Wolfram Blog »

We’ve got an incredible amount of new technology coming out this summer. Two weeks ago we launched Wolfram Programming Cloud. Today I’m pleased to announce the release of a major new version of Mathematica: Mathematica 10.

We released Mathematica 1 just over 26 years ago—on June 23, 1988. And ever since we’ve been systematically making Mathematica ever bigger, stronger, broader and deeper. But Mathematica 10—released today—represents the single biggest jump in new functionality in the entire history of Mathematica.

At a personal level it is very satisfying to see after all these years how successful the principles that I defined at the very beginning of the Mathematica project have proven to be. And it is also satisfying to see how far we’ve gotten with all the talent and hard work that has been poured into Mathematica over nearly three decades.

We’ll probably never know whether our commitment to R&D over all these years makes sense at a purely commercial level. But it has always made sense to me—and the success of Mathematica and our company has allowed us to take a very long-term view, continually investing in building layer upon layer of long-term technology.

One of the recent outgrowths—from combining Mathematica, Wolfram|Alpha and more—has been the creation of the Wolfram Language. And in effect Mathematica is now an application of the Wolfram Language.

But Mathematica still very much has its own identity too—as our longtime flagship product, and the system that has continually redefined technical computing for more than a quarter of a century.

And today, with Mathematica 10, more is new than in any single previous version of Mathematica. It is satisfying to see such a long curve of accelerating development—and to realize that there are more new functions being added with Mathematica 10 than there were functions altogether in Mathematica 1.

So what is the new functionality in Mathematica 10? It’s a mixture of completely new areas and directions (like geometric computation, machine learning and geographic computation)—together with extensive strengthening, polishing and expanding of existing areas. It’s also a mixture of things I’ve long planned for us to do—but which had to wait for us to develop the necessary technology—together with things I’ve only fairly recently realized we’re in a position to tackle.

When you first launch Mathematica 10 there are some things you’ll notice right away. One is that Mathematica 10 is set up to connect immediately to the Wolfram Cloud. Unlike Wolfram Programming Cloud—or the upcoming Mathematica Online—Mathematica 10 doesn’t run its interface or computations in the cloud. Instead, it maintains all the advantages of running these natively on your local computer—but connects to the Wolfram Cloud so it can have cloud-based files and other forms of cloud-mediated sharing, as well as the ability to access cloud-based parts of the Wolfram Knowledgebase.

If you’re an existing Mathematica user, you’ll notice some changes when you start using notebooks in Mathematica 10. Like there’s now autocompletion everywhere—for option values, strings, wherever. And there’s also a hovering help box that lets you immediately get function templates or documentation. And there’s also—as much requested by the user community—computation-aware multiple undo. It’s horribly difficult to know how and when you can validly undo Mathematica computations—but in Mathematica 10 we’ve finally managed to solve this to the point of having a practical multiple undo.

Another very obvious change in Mathematica 10 is that plots and graphics have a fresh new default look (you can get the old look with an option setting, of course):

And as in lots of other areas, that’s just the tip of the iceberg. Underneath, there’s actually a whole powerful new mechanism of “plot themes”—where instead of setting lots of individual options, you can for example now just specify an overall theme for a plot—like “web” or “minimal” or “scientific”.

But what about more algorithmic areas? There’s an amazing amount there that’s new in Mathematica 10. Lots of new algorithms—including many that we invented in-house. Like the algorithm that lets Mathematica 10 routinely solve systems of numerical polynomial equations that have 100,000+ solutions. Or the cluster of algorithms we invented that for the first time give exact symbolic solutions to all sorts of hybrid differential equations or differential delay equations—making such equations now as accessible as standard ordinary differential equations.

Of course, when it comes to developing algorithms, we’re in a spectacular position these days. Because our multi-decade investment in coherent system design now means that in any new algorithm we develop, it’s easy for us to bring together algorithmic capabilities from all over our system. If we’re developing a numerical algorithm, for example, it’s easy for us to do sophisticated algebraic preprocessing, or use combinatorial optimization or graph theory or whatever. And we get to make new kinds of algorithms that mix all sorts of different fields and approaches in ways that were never possible before.

From the very beginning, one of our central principles has been to automate as much as possible—and to create not just algorithms, but complete meta-algorithms that automate the whole process of going from a computational goal to a specific computation done with a specific algorithm. And it’s been this kind of automation that’s allowed us over the years to “consumerize” more and more areas of computation—and to take them from being accessible only to experts, to being usable by anyone as routine building blocks.

And in Mathematica 10 one important area where this is happening is machine learning. Inside the system there are all kinds of core algorithms familiar to experts—logistic regression, random forests, SVMs, etc. And all kinds of preprocessing and scoring schemes. But to the user there are just two highly automated functions: Classify and Predict. And with these functions, it’s now easy to call on machine learning whenever one wants.

There are huge new algorithmic capabilities in Mathematica 10 in graph theory, image processing, control theory and lots of other areas. Sometimes one’s not surprised that it’s at least possible to have such-and-such a function—even though it’s really nice to have it be as clean as it is in Mathematica 10. But in other cases it at first seems somehow impossible that the function could work.

There are all kinds of issues. Maybe the general problem is undecidable, or theoretically intractable. Or it’s ill conditioned. Or it involves too many cases. Or it needs too much data. What’s remarkable is how often—by being algorithmically sophisticated, and by leveraging what we’ve built in Mathematica and the Wolfram Language—it’s possible to work around these issues, and to build a function that covers the vast majority of important practical cases.

Another important issue is just how much we can represent and do computation on. Expanding this is a big emphasis in the Wolfram Language—and Mathematica 10 has access to everything that’s been developed there. And so, for example, in Mathematica 10 there’s an immediate symbolic representation for dates, times and time series—as well as for geolocations and geographic data.

The Wolfram Language has ways to represent a very broad range of things in the real world. But what about data on those things? Much of that resides in the Wolfram Knowledgebase in the cloud. Soon we’re going to be launching the Wolfram Discovery Platform, which is built to allow large-scale access to data from the cloud. But since that’s not the typical use of Mathematica, basic versions of Mathematica 10 are just set up for small-scale data access—and need explicit Wolfram Cloud Credits to get more.

Still, within Mathematica 10 there are plenty of spectacular new things that will be possible by using just small amounts of data from the Wolfram Knowledgebase.

A little while ago I found a to-do list for Mathematica that I wrote in 1991. Some of the entries on it were done in just a few years. But most required the development of towers of technology that took many years to build. And at least one has been a holdout all these years—until now.

On the to-do it was just “PDEs”. But behind those four letters are centuries of mathematics, and a remarkably complex tower of algorithmic requirements. Yes, Mathematica has been able to handle various kinds of PDEs (partial differential equations) for 20 years. But in Mathematica we always seek as much generality and robustness as possible, and that’s where the challenge has been. Because we’ve wanted to be able to handle PDEs in any kind of geometry. And while there are standard methods—like finite element analysis—for solving PDEs in different geometries, there’s been no good way to describe the underlying geometry in enough generality.

Over the years, we’ve put immense effort into the design of Mathematica and what’s now the Wolfram Language. And part of that design has involved developing broad computational representations for what have traditionally been thought of as mathematical concepts. It’s difficult—if fascinating—intellectual work, in effect getting “underneath” the math to create new, often more general, computational representations.

A few years ago we did it for probability theory and the whole cluster of things around it, like statistical distributions and random processes. Now in Mathematica 10 we’ve done it for another area: geometry.

What we’ve got is really a fundamental extension to the domain of what can be represented computationally, and it’s going to be an important building block for many things going forward. And in Mathematica 10 it delivers some very powerful new functionality—including PDEs and finite elements.

So, what’s hard about representing geometry computationally? The problem is not in handling specific kinds of cases—there are a variety of methods for doing that—but rather in getting something that’s truly general, and extensible, while still being easy to use in specific cases. We’ve been thinking about how to do this for well over a decade, and it’s exciting to now have a solution.

It turns out that math in a sense gets us part of the way there—because it recognizes that there are various kinds of geometrical objects, from points to lines to surfaces to volumes, that are just distinguished by their dimensions. In computer systems, though, these objects are typically represented rather differently. 3D graphics systems, for example, typically handle points, lines and surfaces, but don’t really have a notion of volumes or solids. CAD systems, on the other hand, handle volumes and solids, but typically don’t handle points, lines and surfaces. GIS systems do handle both boundaries and interiors of regions—but only in 2D.

So why can’t we just “use the math”? The problem is that specific mathematical theories—and representations—tend once again to handle, or at least be convenient in, only specific kinds of cases. So, for example, one can describe geometry in terms of equations and inequalities—in effect using real algebraic geometry—but this is only convenient for simple “math-related” shapes. One can use combinatorial topology, which is essentially based on mesh regions, and which is quite general, but difficult to use directly—and doesn’t readily cover things like non-bounded regions. Or one could try using differential geometry—which may be good for manifolds, but doesn’t readily cover geometries with mixed dimensions, and isn’t closed under Boolean operations.

What we’ve built in effect operates “underneath the math”: it’s a general symbolic representation of geometry, which makes it convenient to apply any of these different mathematical or computational approaches. And so instead of having all sorts of separate “point in polygon”, “point in mesh”, “point on line” etc. functions, everything is based on a single general RegionMember function. And similarly Area, Volume, ArcLength and all their generalizations are just based on a single RegionMeasure function.

The result is a remarkably smooth and powerful way of doing geometry, which conveniently spans from middle-school triangle math to being able to describe the most complex geometrical forms for engineering and physics. What’s also important—and typical of our approach to everything—is that all this geometry is deeply integrated with the rest of the system. So, for example, one can immediately find equation solutions within a geometric region, or compute a maximum in it, or integrate over it—or, for that matter, solve a partial differential equation in it, with all the various kinds of possible boundary conditions conveniently being described.

The geometry language we have is very clean. But underneath it is a giant tower of algorithmic functionality—that relies on a fair fraction of the areas that we’ve been developing for the past quarter century. To the typical user there are few indications of this complexity—although perhaps the multi-hundred-page treatise on the details of going beyond automatic settings for finite elements in Mathematica 10 provides a clue.

Geometry is just one new area. The drive for generality continues elsewhere too. Like in image processing, where we’re now supporting most image processing operations not only in 2D but also in 3D images. Or in graph computation, where everything works seamlessly with directed graphs, undirected graphs, mixed graphs, multigraphs and weighted graphs. As usual, it’s taken developing all sorts of new algorithms and methods to deal with cases that in a sense cross disciplines, and so haven’t been studied before, even though it’s obvious they can appear in practice.

As I’ve mentioned, there are some things in Mathematica 10 that we’ve been able to do essentially because our technology stack has now reached the point where they’re possible. There are others, though, that in effect have taken solving a problem, and often a problem that we’ve been thinking about for a decade or two. An example of this is the system for handling formal math operators in Mathematica 10.

In a sense what we’re doing is to take the idea of symbolic representation one more step. In math, we’ve always allowed a variable like x to be symbolic, so it can correspond to any possible value. And we’ve allowed functions like f to be symbolic too. But what about mathematical operators like derivative? In the past, these have always been explicit—so for example they actually take derivatives if they can. But now we have a new notion of “inactive” functions and operators, which gives us a general way to handle mathematical operators purely symbolically, so that we can transform and manipulate expressions formally, while still maintaining the meaning of these operators.

This makes possible all sorts of new things—from conveniently representing complicated vector analysis expressions, to doing symbolic transformations not only on math but also on programs, to being able to formally manipulate objects like integrals, with built-in implementations of all the various generalizations of things like Leibniz’s rule.

In building Mathematica 10, we’ve continued to push forward into uncharted computational—and mathematical—territory. But we’ve also worked to make Mathematica 10 even more convenient for areas like elementary math. Sometimes it’s a challenge to fit concepts from elementary math with the overall generality that we want to maintain. And often it requires quite a bit of sophistication to make it work. But the result is a wonderfully seamless transition from the elementary to the advanced. And in Mathematica 10, we’ve once again achieved this for things like curve computations and function domains and ranges.

The development of the Wolfram Language has had many implications for Mathematica—first visible now in Mathematica 10. In addition to all sorts of interaction with real-world data and with external systems, there are some fundamental new constructs in the system itself. An example is key-value associations, which in effect introduce “named parts” throughout the system. Another example is the general templating system, important for programmatically constructing strings, files or web pages.

With the Wolfram Language there are vast new areas of functionality—supporting new kinds of programming, new structures and new kinds of data, new forms of deployment, and new ways to integrate with other systems. And with all this development—and all the new products it’s making possible—one might worry that the traditional core directions of Mathematica would be left behind. But nothing is further from the truth. And in fact all the new Wolfram Language development has made possible even more energetic efforts in traditional Mathematica areas.

Partly that is the result of new software capabilities. Partly it is the result of new understanding that we’ve developed about how to push forward the design of a very large knowledge-based system. And partly it’s the result of continued strengthening and streamlining of our internal R&D processes.

We’re still a fairly small company (about 700 people), but we’ve been steadily ramping up our R&D output. And it’s amazing to see what we’ve been able to build for Mathematica 10. In the 19 months (588 days) since Mathematica 9 was released, we’ve finished more than 700 new functions that are now in Mathematica 10—and we’ve added countless enhancements to existing functions.

I think the fact that this is possible is a great tribute to the tools, team and organization we’ve built—and the strength of the principles under which we’ve been operating all these years.

To most people in the software business, if they knew the amount of R&D that’s gone into Mathematica 10, it would seem crazy. Most would assume that a 26-year-old product would be in a maintenance mode, with tiny updates being made every few years. But that’s not the story with Mathematica at all. Instead, 26 years after its initial release, its rate of growth is still accelerating. There’s going to be even more to come.

But today I’m pleased to announce that the fruits of a “crazy” amount of R&D are here: Mathematica 10.

To comment, please visit the copy of this post at the Wolfram Blog »

For as long as I can remember, my all-time favorite activity has been creating ideas and turning them into reality—a kind of “entrepreneurism of ideas”. And over the years—in science, technology and business—I think I’ve developed some pretty good tools and strategies for doing this, that I’ve increasingly realized would be good for a lot of other people (and organizations) too.

So how does one spread idea entrepreneurism—entrepreneurism centered on ideas rather than commercial enterprises? Somewhat unwittingly I think we’ve developed a rather good vehicle—that’s both a very successful educational program, and a fascinating annual adventure for me.

Twelve years ago my book A New Kind of Science had just come out, and we were inundated with people wanting to learn more, and get involved in research around it. We considered various alternatives, but eventually we decided to organize a summer school where we would systematically teach about our methodology, while mentoring each student to do a unique original project.

From the very beginning, the summer school was a big success. And over the years we’ve gradually improved and expanded it. It’s still the Wolfram Science Summer School—and its intellectual core is still A New Kind of Science. But today it has become a broader vehicle for passing on our tools and strategies for idea entrepreneurism.

This year’s summer school just ended last week. We had 63 students from 21 countries—with a fascinating array of backgrounds and interests. Most were in college or graduate school; a few were younger or older. And over the course of the three weeks of the summer school—with great energy and intellectual entrepreneurism—each student worked towards their own unique project.

The summer school is part idea incubator, part course, part hackathon and part mentoring event. And it’s become a tradition for me to open it with a concentrated burst of idea entrepreneurism: a live experiment in which over the course of an hour or so I try—live and in public—to discover or invent something new.

Realistically, what makes this—and indeed much of what’s done at the summer school—possible is what’s now the Wolfram Language, with all its built-in knowledge and automation, as well as immediate presentation capabilities.

My rule for live experiments is that apart from spending a few minutes beforehand coming up with a topic (sometimes just by opening A New Kind of Science at random), I don’t think at all about what I’m going to do. The experiment is always fresh and spontaneous—and quite an adventure for all concerned. It’s a strange kind of intellectual performance, and it takes quite a bit of concentration. But I think it’s pretty educational to watch—not least because most people have never seen something done from scratch in real time like this.

There are always ups and downs in the course of a live experiment—and sometimes it seems that all is lost. But so far, in dozens of live experiments I’ve done, I’ve always found a way to navigate them to some kind of success. And seeing this always seems quite empowering to people; and makes this kind of idea entrepreneurism feel like something close at hand, that they can do too.

This year I actually did two live experiments. My first one was a piece of pure science that involved numbers. The idea is pretty elementary: just take a number, write it in base 2, manipulate its digits, then add it to the original number. Then iterate this many times. Here’s the little piece of Wolfram Language code I wrote during the live experiment to do this:

In A New Kind of Science, I did a version of this where the digit manipulation consisted of reversing the whole sequence of digits. But now I wanted to try something simpler: just rotating the digits by some number of positions. I wasn’t sure this would do anything interesting. But in the spirit of the live experiment, I wrote a little piece of code to find out:

What happened was quintessential NKS (New Kind of Science). Even though the underlying rule was incredibly simple, the behavior was far from simple—and in many ways looked quite random.

Here’s the whole notebook from the hour or so of live experiment:

It’s got quite a few interesting results. And indeed—like many previous summer school live experiments—it’s got the core of what would make a nice research paper.

In addition to this pure science experiment I decided this year to do a second—more practical—live experiment. In a sense it was a meta experiment. Because it consisted of analyzing code of pretty much the type used to do live experiments. Specifically, I read in lots of code from the Wolfram Demonstrations Project, then started doing statistics on it.

At first I looked at the general distribution of functions used, and started analyzing correlations and so on. But then, following a suggestion from the audience, I decided to focus on one simple example, and just started looking specifically at the use of named colors. The result was this bar chart, showing that (for whatever reason) red and black are the most common named colors in this corpus of code:

It’s always important to get visualizations at every step along the way. And in this particular live experiment, we quickly decided to visualize correlations between colors, generating bar charts showing what the distribution of colors is if one already knows that a certain color (shown as the background) appears:

My original idea for the live experiment was to look for repeated patterns of code that might suggest functions we’d want to name and implement. But we never quite got there—and when we ran out of time we were instead looking at how one could use “code corpus analysis” to develop good color palettes: a quite different, but interesting, direction that emerged from the experiment.

Doing live experiments in a sense provides a way to illustrate the spirit of idea entrepreneurism—as well as letting one introduce some specific methodologies and tools. But another important element of successful idea entrepreneurism is choosing a direction to pursue. And it’s become a tradition at our summer school that after I do my live experiment, I talk about the directions we’re currently pursuing—and about what science and technology I’m currently most excited about.

After that we launch into the most important business of the summer school: defining a project for each student. Over the years we’ve developed and steadily refined a whole process for doing this. I’m always accumulating lists of interesting problems and projects. And students often come in with definite ideas for projects. But I’ve found the best results come from pure real-time creativity: from learning about each student and then creatively coming up with a project that matches their skills and interests.

And this year, over the course of three fairly long days, we did that 63 times, defining a unique original project for each of our students.

There’s quite a bit of structure to the summer school. For example, there’s always “homework” done during the first week. Usually we pick some previously unexplored area of the computational universe, and ask students to find something interesting in it. This year lots of students found lots of interesting things—that are actually now being assembled into a paper.

Every day there are a few hours of lectures. About how to do a good computational experiment. About the types of systems studied in A New Kind of Science. About computational techniques. About implications for philosophy, music, engineering and more. About perception. About natural language. About what’s worked in previous student projects. About principles that emerge from A New Kind of Science. About all sorts of other things.

Being an instructor at the Wolfram Science Summer School has become a favorite activity for some of our top R&D employees. And as it happens, this year all the instructors were also summer school alumni from previous years (yes, we’ve done a lot of recruiting from the summer school). For three weeks they worked with students, bringing to bear on each project the kind of idea entrepreneurism that we’ve taught at the summer school—and practice at our company.

This year—not least because we’d just finished launching Wolfram Programming Cloud days before—I had the pleasure of spending plenty of time at the summer school, getting to know all the students (by the end, I knew everyone by name!), and watching lots of projects take shape. I’ve been involved with my share of hackathons and incubators. But the summer school is something different. It’s really about entrepreneurism of ideas: about the process of creating ideas and turning them into reality.

And at the end of three weeks, there were 63 projects to present—and lots of interesting things discovered and invented:

Over the years at the Wolfram Science Summer School, there have been many hundreds of great projects done, as well as many careers launched.

There’s a lot to say about education. But I think for many people, doing a unique original project is the single most powerful and useful form of education there is. Doing this successfully is quintessential idea entrepreneurism. And I think that in the long run the most important achievement of the Wolfram Science Summer School may just be the framework it’s developed for spreading entrepreneurism of ideas.

The Wolfram Science Summer School is just one of a growing constellation of education initiatives that we’re involved in. (Another that runs alongside the summer school is our two-week Mathematica Summer Camp for high-school students—that directly uses ideas from the summer school.) And particularly with all the new technologies that we’ve been developing, there are vast new opportunities for education. For me the Wolfram Science Summer School is an important and fascinating success story about innovation in education—and an encouragement for us to do more.

Every four years for more than a century there’s been an International Congress of Mathematicians (ICM) held somewhere in the world. In 1900 it was where David Hilbert announced his famous collection of math problems—and it’s remained the top single periodic gathering for the world’s research mathematicians.

This year the ICM is in Seoul, and I’m going to it today. I went to the ICM once before—in Kyoto in 1990. Mathematica was only two years old then, and mathematicians were just getting used to it. Plenty already used it extensively—but at the ICM there were also quite a few who said, “I do pure mathematics. How can Mathematica possibly help me?”

Twenty-four years later, the vast majority of the world’s pure mathematicians do in fact use Mathematica in one way or another. But there’s nevertheless a substantial core of pure mathematics that still gets done pretty much the same way it’s been done for centuries—by hand, on paper.

Ever since the 1990 ICM I’ve been wondering how one could successfully inject technology into this. And I’m excited to say that I think I’ve recently begun to figure it out. There are plenty of details that I don’t yet know. And to make what I’m imagining real will require the support and involvement of a substantial set of the world’s pure mathematicians. But if it’s done, I think the results will be spectacular—and will surely change the face of pure mathematics at least as much as Mathematica (and for a younger generation, Wolfram|Alpha) have changed the face of calculational mathematics, and potentially usher in a new golden age for pure mathematics.

The whole story is quite complicated. But for me one important starting point is the difference in the typical workflows for calculational mathematics and pure mathematics. Calculational mathematics tends to involve setting up calculational questions, and then working through them to get results—just like in typical interactive Mathematica sessions. But pure mathematics tends to involve taking mathematical objects, results or structures, coming up with statements about them, and then giving proofs to show why those statements are true.

How can we usefully insert technology into this workflow? Here’s one simple way. Think about Wolfram|Alpha. If you enter 2+2, Wolfram|Alpha—like Mathematica—will compute 4. But if you enter new york—or, for that matter, 2.3363636 or cos(x) log(x)—there’s no single “answer” for it to compute. And instead what it does is to generate a report that gives you a whole sequence of “interesting facts” about what you entered.

And this kind of thing fits right into the workflow for pure mathematics. You enter some mathematical object, result or structure, and then the system tries to tell you interesting things about it—just like some extremely wise mathematical colleague might. You can guide the system if you want to, by telling it what kinds of things you want to know about, or even by giving it a candidate statement that might be true. But the workflow is always the Wolfram|Alpha-like “what can you tell me about that?” rather than the Mathematica-like “what’s the answer to that?”

Wolfram|Alpha already does quite a lot of this kind of thing with mathematical objects. Enter a number, or a mathematical expression, or a graph, or a probability distribution, or whatever, and Wolfram|Alpha will use often-quite-sophisticated methods to try to tell you a collection of interesting things about it.

But to really be useful in pure mathematics, there’s something else that’s needed. In addition to being able to deal with concrete mathematical objects, one also has to be able to deal with abstract mathematical structures.

Countless pure mathematical papers start with things like, “Let F be a field with such-and-such properties.” We need to be able to enter something like this—then have our system automatically give us interesting facts and theorems about F, in effect creating a whole automatically generated paper that tells the story of F.

So what would be involved in creating a system to do this? Is it even possible? There are several different components, all quite difficult and time consuming to build. But based on my experiences with Mathematica, Wolfram|Alpha, and A New Kind of Science, I am quite certain that with the right leadership and enough effort, all of them can in fact be built.

A key part is to have a precise symbolic description of mathematical concepts and constructs. Lots of this now already exists—after more than a quarter century of work—in Mathematica. Because built right into the Wolfram Language are very general ways to represent geometries, or equations, or stochastic processes or quantifiers. But what’s not built in are representations of pure mathematical concepts like bijections or abstract semigroups or pullbacks.

Over the years, plenty of mathematicians have implemented specific cases. But could we systematically extend the Wolfram Language to cover the whole range of pure mathematics—and make a kind of “Mathematica Pura”? The answer is unquestionably yes. It’ll be fascinating to do, but it’ll take lots of difficult language design.

I’ve been doing language design now for 35 years—and it’s the hardest intellectual activity I know. It requires a curious mixture of clear thinking, aesthetics and pragmatic judgement. And it involves always seeking the deepest possible understanding, and trying to do the broadest unification—to come up in the end with the cleanest and “most obvious” primitives to represent things.

Today the main way pure mathematics is described—say in papers—is through a mixture of mathematical notation and natural language, together with a few diagrams. And in designing a precise symbolic language for pure mathematics, this has to be the starting point.

One might think that somehow mathematical notation would already have solved the whole problem. But there’s actually only a quite small set of constructs and concepts that can be represented with any degree of standardization in mathematical notation—and indeed many of these are already in the Wolfram Language.

So how should one go further? The first step is to understand what the appropriate primitives are. The whole Wolfram Language today has about 5000 built-in functions—together with many millions of built-in standardized entities. My guess is that to broadly support pure mathematics there would need to be something less than a thousand other well-designed functions that in effect define frameworks—together with maybe a few tens of thousands of new entities or their analogs.

Take something like function spaces. Maybe there’ll be a FunctionSpace function to represent a function space. Then there’ll be various operations on function spaces, like PushForward or MetrizableQ. Then there’ll be lots of named function spaces, like “CInfinity”, with various kinds of parameterizations.

Underneath, everything’s just a symbolic expression. But in the Wolfram Language there end up being three immediate ways to input things, all of which are critical to having a convenient and readable language. The first is to use short notations—like + or —as in standard mathematical notation. The second is to use carefully chosen function names—like MatrixRank or Simplex. And the third is to use free-form natural language—like trefoil knot or aleph0.

One wants to have short notations for some of the most common structural or connective elements. But one needs the right number: not too few, like in LISP, nor too many, like in APL. Then one wants to have function names made of ordinary words, arranged so that if one’s given something written in the language one can effectively just “read the words” to know at least roughly what’s going on in it.

But in the modern Wolfram Language world there’s also free-form natural language. And the crucial point is that by using this, one can leverage all the various convenient—but sloppy—notations that actual mathematicians use and find familiar. In the right context, one can enter “L2” for Lebesgue Square Integrable—and the natural language system will take care of disambiguating it and inserting the canonical symbolic underlying form.

Ultimately every named construct or concept in pure mathematics needs to have a place in our symbolic language. Most of the 13,000+ entries in MathWorld. Material from the 5600 or so entries in the MSC2010 classification scheme. All the many things that mathematicians in any given field would readily recognize when told their names.

But, OK, so let’s say we manage to create a precise symbolic language that captures the concepts and constructs of pure mathematics. What can we do with it?

One thing is to use it “Wolfram|Alpha style”: you give free-form input, which is then interpreted into the language, and then computations are done, and a report is generated.

But there’s something else too. If we have a sufficiently well-designed symbolic language, it’ll be useful not only to computers but also to humans. In fact, if it’s good enough, people should prefer to write out their math in this language than in their current standard mixture of natural language and mathematical notation.

When I write programs in the Wolfram Language, I pretty much think directly in the language. I’m not coming up with a description in English of what I’m trying to do and then translating it into the Wolfram Language. I’m forming my thoughts from the beginning in the Wolfram Language—and making use of its structure to help me define those thoughts.

If we can develop a sufficiently good symbolic language for pure mathematics, then it’ll provide something for pure mathematicians to think in too. And the great thing is that if you can describe what you’re thinking in a precise symbolic language, there’s never any ambiguity about what anything means: there’s a precise definition that you can just go to the documentation for the language to find.

And once pure math is represented in a precise symbolic language, it becomes in effect something on which computation can be done. Proofs can be generated or checked. Searches for theorems can be done. Connections can automatically be made. Chains of prerequisites can automatically be found.

But, OK, so let’s say we have the raw computational substrate we need for pure mathematics. How can we use this to actually implement a Wolfram|Alpha-like workflow where we enter descriptions of things, and then in effect automatically get mathematical wisdom about them?

There are two seemingly different directions one can go. The first is to imagine abstractly enumerating possible theorems about what has been entered, and then using heuristics to decide which of them are interesting. The second is to start from computable versions of the millions of theorems that have actually been published in the literature of mathematics, and then figure out how to connect these to whatever has been entered.

Each of these directions in effect reflects a slightly different view of what doing mathematics is about. And there’s quite a bit to say about each direction.

Let’s start with theorem enumeration. In the simplest case, one can imagine starting from an axiom system and then just enumerating true theorems based on that system. There are two basic ways to do this. The first is to enumerate possible statements, and then to use (implicit or explicit) theorem-proving technology to try to determine which of them are true. And the second is to enumerate possible proofs, in effect treeing out possible ways the axioms can be applied to get theorems.

It’s easy to do either of these things for something like Boolean algebra. And the result is that one gets a sequence of true theorems. But if a human looks at them, many of them seem trivial or uninteresting. So then the question is how to know which of the possible theorems should actually be considered “interesting enough” to be included in a report that’s generated.

My first assumption was that there would be no automatic approach to this—and that “interestingness” would inevitably depend on the historical development of the relevant area of mathematics. But when I was working on A New Kind of Science, I did a simple experiment for the case of Boolean algebra.

There are 14 theorems of Boolean algebra that are usually considered “interesting enough” to be given names in textbooks. I took all possible theorems and listed them in order of complexity (number of variables, number of operators, etc). And the surprising thing I found is that the set of named theorems corresponds almost exactly to the set of theorems that can’t be proved just from ones that precede them in the list. In other words, the theorems which have been given names are in a sense exactly the minimal statements of new information about Boolean algebra.

Boolean algebra is of course a very simple case. And in the kind of enumeration I just described, once one’s got the theorems corresponding to all the axioms, one would conclude that there aren’t any more “interesting theorems” to find—which for many mathematical theories would be quite silly. But I think this example is a good indication of how one can start to use automated heuristics to figure out which theorems are “worth reporting on”, and which are, for example, just “uninteresting embellishments”.

Of course, the general problem of ranking “what’s interesting” comes up all over Wolfram|Alpha. In mathematical examples, one’s asking what region is interesting to plot?, “what alternate forms are interesting?” and so on. When one enters a single number, one’s also asking “what closed forms are interesting enough to show?”—and to know this, one for example has to invent rankings for all sorts of mathematical objects (how complicated should one consider relative to log(343) relative to Khinchin’s Constant, and so on?).

So in principle one can imagine having a system that takes input and generates “interesting” theorems about it. Notice that while in a standard Mathematica-like calculational workflow, one would be taking input and “computing an answer” from it, here one’s just “finding interesting things to say about it”.

The character of the input is different too. In the calculational case, one’s typically dealing with an operation to be performed. In the Wolfram|Alpha-like pure mathematical case, one’s typically just giving a description of something. In some cases that description will be explicit. A specific number. A particular equation. A specific graph. But more often it will be implicit. It will be a set of constraints. One will say (to use the example from above), “Let F be a field,” and then one will give constraints that the field must satisfy.

In a sense an axiom system is a way of giving constraints too: it doesn’t say that such-and-such an operator “is Nand”; it just says that the operator must satisfy certain constraints. And even for something like standard Peano arithmetic, we know from Gödel’s Theorem that we can never ultimately resolve the constraints–we can never nail down that the thing we denote by “+” in the axioms is the particular operation of ordinary integer addition. Of course, we can still prove plenty of theorems about “+”, and those are what we choose from for our report.

So given a particular input, we can imagine representing it as a set of constraints in our precise symbolic language. Then we would generate theorems based on these constraints, and heuristically pick the “most interesting” of them.

One day I’m sure doing this will be an important part of pure mathematical work. But as of now it will seem quite alien to most pure mathematicians—because they are not used to “disembodied theorems”; they are used to theorems that occur in papers, written by actual mathematicians.

And this brings us to the second approach to the automatic generation of “mathematical wisdom”: start from the historical corpus of actual mathematical papers, and then make connections to whatever specific input is given. So one is able to say for example, “The following theorem from paper X applies in such-and-such a way to the input you have given”, and so on.

So how big is the historical corpus of mathematics? There’ve probably been about 3 million mathematical papers published altogether—or about 100 million pages, growing at a rate of about 2 million pages per year. And in all of these papers, perhaps 5 million distinct theorems have been formally stated.

So what can be done with these? First, of course, there’s simple search and retrieval. Often the words in the papers will make for better search targets than the more notational material in the actual theorems. But with the kind of linguistic-understanding technology for math that we have in Wolfram|Alpha, it should not be too difficult to build what’s needed to do good statistical retrieval on the corpus of mathematical papers.

But can one go further? One might think about tagging the source documents to improve retrieval. But my guess is that most kinds of static tagging won’t be worth the trouble; just as one’s seen for the web in general, it’ll be much easier and better to make the search system more sophisticated and content-aware than to add tags document by document.

What would unquestionably be worthwhile, however, is to put the theorems into a genuine computable form: to actually take theorems from papers and rewrite them in a precise symbolic language.

Will it be possible to do this automatically? Eventually I suspect large parts of it will. Today we can take small fragments of theorems from papers and use the linguistic understanding system built for Wolfram|Alpha to turn them into pieces of Wolfram Language code. But it should gradually be possible to extend this to larger fragments—and eventually get to the point where it takes, at most, modest human effort to convert a typical theorem to precise symbolic form.

So let’s imagine we curate all the theorems from the literature of mathematics, and get them in computable form. What would we do then? We could certainly build a Wolfram|Alpha-like system that would be quite spectacular—and very useful in practice for doing lots of pure mathematics.

But there will inevitably be some limitations—resulting in fact from features of mathematics itself. For example, it won’t necessarily be easy to tell what theorem might apply to what, or even what theorems might be equivalent. Ultimately these are classic theoretically undecidable problems—and I suspect that they will often actually be difficult in practical cases too. And at the very least, all of them involve the same kind of basic process as automated theorem proving.

And what this suggests is a kind of combination of the two basic approaches we’ve discussed—where in effect one takes the complete corpus of published mathematics, and views it as defining a giant 5-million-axiom formal system, and then follows the kind of automated theorem-enumeration procedure we discussed to find “interesting things to say”.

So, OK, let’s say we build a wonderful system along these lines. Is it actually solving a core problem in doing pure mathematics, or is it missing the point?

I think it depends on what one sees the nature of the pure mathematical enterprise as being. Is it science, or is it art? If it’s science, then being able to make more theorems faster is surely good. But if it’s art, that’s really not the point. If doing pure mathematics is like creating a painting, automation is going to be largely counterproductive—because the core of the activity is in a sense a form of human expression.

This is not unrelated to the role of proof. To some mathematicians, what matters is just the theorem: knowing what’s true. The proof is essentially backup to ensure one isn’t making a mistake. But to other mathematicians, proof is a core part of the content of the mathematics. For them, it’s the story that brings mathematical concepts to light, and communicates them.

So what happens when we generate a proof automatically? I had an interesting example about 15 years ago, when I was working on A New Kind of Science, and ended up finding the simplest axiom system for Boolean algebra (just the single axiom ((pq)r)(p((pr)p))==r, as it turned out). I used equational-logic automated theorem-proving (now built into FullSimplify) to prove the correctness of the axiom system. And I printed the proof that I generated in the book:

It has 343 steps, and in ordinary-size type would be perhaps 40 pages long. And to me as a human, it’s completely incomprehensible. One might have thought it would help that the theorem prover broke the proof into 81 lemmas. But try as I might, I couldn’t really find a way to turn this automated proof into something I or other people could understand. It’s nice that the proof exists, but the actual proof itself doesn’t tell me anything.

And the problem, I think, is that there’s no “conceptual story” around the elements of the proof. Even if the lemmas are chosen “structurally” as good “waypoints” in the proof, there are no cognitive connections—and no history—around these lemmas. They’re just disembodied, and apparently disconnected, facts.

So how can we do better? If we generate lots of similar proofs, then maybe we’ll start seeing similar lemmas a lot, and through being familiar they will seem more meaningful and comprehensible. And there are probably some visualizations that could help us quickly get a good picture of the overall structure of the proof. And of course, if we manage to curate all known theorems in the mathematics literature, then we can potentially connect automatically generated lemmas to those theorems.

It’s not immediately clear how often that will possible—and indeed in existing examples of computer-assisted proofs, like for the Four Color Theorem, the Kepler Conjecture, or the simplest universal Turing machine, my impression is that the often-computer-generated lemmas that appear rarely correspond to known theorems from the literature.

But despite all this, I know at least one example showing that with enough effort, one can generate proofs that tell stories that people can understand: the step-by-step solutions system in Wolfram|Alpha Pro. Millions of times a day students and others compute things like integrals with Wolfram|Alpha—then ask to see the steps.

It’s notable that actually computing the integral is much easier than figuring out good steps to show; in fact, it takes some fairly elaborate algorithms and heuristics to generate steps that successfully communicate to a human how the integral can be done. But the example of step-by-step in Wolfram|Alpha suggests that it’s at least conceivable that with enough effort, it would be possible to generate proofs that are readable as “stories”—perhaps even selected to be as short and simple as possible (“proofs from The Book”, as Erdős would say).

Of course, while these kinds of automated methods may eventually be good at communicating the details of something like a proof, they won’t realistically be able to communicate—or even identify—overarching ideas and motivations. Needless to say, present-day pure mathematics papers are often quite deficient in communicating these too. Because in an effort to ensure rigor and precision, many papers tend to be written in a very formal way that cannot successfully represent the underlying ideas and motivations in the mind of the author—with the result that some of the most important ideas in mathematics are transmitted through an essentially oral tradition.

It would certainly help the progress of pure mathematics if there were better ways to communicate its content. And perhaps having a precise symbolic language for pure mathematics would make it easier to express concretely some of those important points that are currently left unwritten. But one thing is for sure: having such a language would make it possible to take a theorem from anywhere, and—like with a typical Wolfram Language code fragment—immediately be able to plug it in anywhere else, and use it.

But back to the question of whether automation in pure mathematics can ultimately make sense. I consider it fairly clear that a Wolfram|Alpha-like “pure math assistant” would be useful to human mathematicians. I also consider it fairly clear that having a good, precise, symbolic language—a kind of Mathematica Pura that’s a well-designed follow-on to standard mathematical notation—would be immensely helpful in formulating, checking and communicating math.

But what about a computer just “going off and doing math by itself”? Obviously the computer can enumerate theorems, and even use heuristics to select ones that might be considered interesting to human mathematicians. And if we curate the literature of mathematics, we can do extensive “empirical metamathematics” and start trying to recognize theorems with particular characteristics, perhaps by applying graph-theoretic criteria on the network of theorems to see what counts as “surprising” or a “powerful” theorem. There’s also nothing particularly difficult—like in WolframTones—about having the computer apply aesthetic criteria deduced from studying human choices.

But I think the real question is whether the computer can build up new conceptual frameworks and structures—in effect new mathematical theories. Certainly some theorems found by enumeration will be surprising and indicative of something fundamentally new. And it will surely be impressive when a computer can take a large collection of theorems—whether generated or from the literature—and discover correlations among them that indicate some new unifying principle. But I would expect that in time the computer will be able not only to identify new structures, but also name them, and start building stories about them. Of course, it is for humans to decide whether they care about where the computer is going, but the basic character of what it does will, I suspect, be largely indistinguishable from many forms of human pure mathematics.

All of this is still fairly far in the future, but there’s already a great way to discover math-like things today—that’s not practiced nearly as much as it should be: experimental mathematics. The term has slightly different meanings to different people. For me it’s about going out and studying what mathematical systems do by running experiments on them. And so, for example, if we want to find out about some class of cellular automata, or nonlinear PDEs, or number sequences, or whatever, we just enumerate possible cases and then run them and see what they do.

There’s a lot to discover like this. And certainly it’s a rich way to generate observations and hypotheses that can be explored using the traditional methodologies of pure mathematics. But the real thrust of what can be done does not fit into what pure mathematicians typically think of as math. It’s about exploring the “flora and fauna”—and principles—of the universe of possible systems, not about building up math-like structures that can be studied and explained using theorems and proofs. Which is why—to quote the title of my book—I think one should best consider this a new kind of science, rather than something connected to existing mathematics.

In discussing experimental mathematics and A New Kind of Science, it’s worth mentioning that in some sense it’s surprising that pure mathematics is doable at all—because if one just starts asking absolutely arbitrary questions about mathematical systems, many of them will end up being undecidable.

This is particularly obvious when one’s out in the computational universe of possible programs, but it’s also true for programs that represent typical mathematical systems. So why isn’t undecidability more of a problem for typical pure mathematics? The answer is that pure mathematics implicitly tends to select what it studies so as to avoid undecidability. In a sense this seems to be a reflection of history: pure mathematics follows what it has historically been successful in doing, and in that way ends up navigating around undecidability—and producing the millions of theorems that make up the corpus of existing pure mathematics.

OK, so those are some issues and directions. But where are we at in practice in bringing computational knowledge to pure mathematics?

There’s certainly a long history of related efforts. The works of Peano and Whitehead and Russell from a century ago. Hilbert’s program. The development of set theory and category theory. And by the 1960s, the first computer systems—such as Automath—for representing proof structures. Then from the 1970s, systems like Mizar that attempted to provide practical computer frameworks for presenting proofs. And in recent times, increasingly popular “proof assistants” based on systems like Coq and HOL.

One feature of essentially all these efforts is that they were conceived as defining a kind of “low-level language” for mathematics. Like most of today’s computer languages, they include a modest number of primitives, then imagine that essentially any actual content must be built externally, by individual users or in libraries.

But the new idea in the Wolfram Language is to have a knowledge-based language, in which as much actual knowledge as possible is carefully designed into the language itself. And I think that just like in general computing, the idea of a knowledge-based language is going to be crucial for injecting computation into pure mathematics in the most effective and broadly useful way.

So what’s involved in creating our Mathematica Pura—an extension to the Wolfram Language that builds in the actual structure and content of pure math? At the lowest level, the Wolfram Language deals with arbitrary symbolic expressions, which can represent absolutely anything. But then the language uses these expressions for many specific purposes. For example, it can use a symbol x to represent an algebraic variable. And given this, it has many functions for handling symbolic expressions—interpreted as mathematical or algebraic expressions—and doing various forms of math with them.

The emphasis of the math in Mathematica and the Wolfram Language today is on practical, calculational, math. And by now it certainly covers essentially all the math that has survived from the 19th century and before. But what about more recent math? Historically, math itself went through a transition about a century ago. Just around the time modernism swept through areas like the arts, math had its own version: it started to consider systems that emerged purely from its own formalism, without regard for obvious connection to the outside world.

And this is the kind of math—through developments like Bourbaki and beyond—that came to dominate pure mathematics in the 20th century. And inevitably, a lot of this math is about defining abstract structures to study. In simple cases, it seems like one might represent these structures using some hierarchy of types. But the types need to be parametrized, and quite quickly one ends up with a whole algebra or calculus of types—and it’s just as well that in the Wolfram Language one can use general symbolic expressions, with arbitrary heads, rather than just simple type descriptions.

As I mentioned early in this blog post, it’s going to take all sorts of new built-in functions to capture the frameworks needed to represent modern pure mathematics—together with lots of entity-like objects. And it’ll certainly take years of careful design to make a broad system for pure mathematics that’s really clean and usable. But there’s nothing fundamentally difficult about having symbolic constructs that represent differentiability or moduli spaces or whatever. It’s just language design, like designing ways to represent 3D images or remote computation processes or unique external entity references.

So what about curating theorems from the literature? Through Wolfram|Alpha and the Wolfram Language, not to mention for example the Wolfram Functions Site and the Wolfram Connected Devices Project, we’ve now had plenty of experience at the process of curation, and in making potentially complex things computable.

But to get a concrete sense of what’s involved in curating mathematical theorems, we did a pilot project over the last couple of years through the Wolfram Foundation, supported by the Sloan Foundation. For this project we picked a very specific and well-defined area of mathematics: research on continued fractions. Continued fractions have been studied continually since antiquity, but were at their most popular between about 1780 and 1910. In all there are around 7000 books and papers about them, running to about 150,000 pages.

We chose about 2000 documents, then set about extracting theorems and other mathematical information from them. The result was about 600 theorems, 1500 basic formulas, and about 10,000 derived formulas. The formulas were directly in computable form—and were in effect immediately able to join the 300,000+ on the Wolfram Functions Site, that are all now included in Wolfram|Alpha. But with the theorems, our first step was just to treat them as entities themselves, with properties such as where they were first published, who discovered them, etc. And even at this level, we were able to insert some nice functionality into Wolfram|Alpha.

But we also started trying to actually encode the content of the theorems in computable form. It took introducing some new constructs like LebesgueMeasure, ConvergenceSet and LyapunovExponent. But there was no fundamental problem in creating precise symbolic representations of the theorems. And just from these representations, it became possible to do computations like this in Wolfram|Alpha:

An interesting feature of the continued fraction project (dubbed “eCF”) was how the process of curation actually led to the discovery of some new mathematics. For having done curation on 50+ papers about the Rogers–Ramanujan continued fraction, it became clear that there were missing cases that could now be computed. And the result was the filling of a gap left by Ramanujan for 100 years.

There’s always a tradeoff between curating knowledge and creating it afresh. And so, for example, in the Wolfram Functions Site, there was a core of relations between functions that came from reference books and the literature. But it was vastly more efficient to generate other relations than to scour the literature to find them.

But if the goal is curation, then what would it take to curate the complete literature of mathematics? In the eCF project, it took about 3 hours of mathematician time to encode each theorem in computable form. But all this work was done by hand, and in a larger-scale project, I am certain that an increasing fraction of it could be done automatically, not least using extensions of our Wolfram|Alpha natural language understanding system.

Of course, there are all sorts of practical issues. Newer papers are predominantly in TeX, so it’s not too difficult to pull out theorems with all their mathematical notation. But older papers need to be scanned, which requires math OCR, which has yet to be properly developed.

Then there are issues like whether theorems stated in papers are actually valid. And even whether theorems that were considered valid, say, 100 years ago are still considered valid today. For example, for continued fractions, there are lots of pre-1950 theorems that were successfully proved in their time, but which ignore branch cuts, and so wouldn’t be considered correct today.

And in the end of course it requires lots of actual, skilled mathematicians to guide the curation process, and to encode theorems. But in a sense this kind of mobilization of mathematicians is not completely unfamiliar; it’s something like what was needed when Zentralblatt was started in 1931, or Mathematical Reviews in 1941. (As a curious footnote, the founding editor of both these publications was Otto Neugebauer, who worked just down the hall from me at the Institute for Advanced Study in the early 1980s, but who I had no idea was involved in anything other than decoding Babylonian mathematics until I was doing research for this blog post.)

When it comes to actually constructing a system for encoding pure mathematics, there’s an interesting example: Theorema, started by Bruno Buchberger in 1995, and recently updated to version 2. Theorema is written in the Wolfram Language, and provides both a document-based environment for representing mathematical statements and proofs, and actual computation capabilities for automated theorem proving and so on.

No doubt it’ll be an element of what’s ultimately built. But the whole project is necessarily quite large—perhaps the world’s first example of “big math”. So can the project get done in the world today? A crucial part is that we now have the technical capability to design the language and build the infrastructure that’s needed. But beyond that, the project also needs a strong commitment from the world’s mathematics community—as well as lots of contributions from individual mathematicians from every possible field. And realistically it’s not a project that can be justified on commercial grounds—so the likely $100+ million that it will need will have to come from non-commercial sources.

But it’s a great and important project—that promises to be pivotal for pure mathematics. In almost every field there are golden ages when dramatic progress is made. And more often than not, such golden ages are initiated by new methodology and the arrival of new technology. And this is exactly what I think will happen in pure mathematics. If we can mobilize the effort to curate known mathematics and build the system to use and generate computational knowledge around it, then we will not only succeed in preserving and spreading the great heritage of pure mathematics, but we will also thrust pure mathematics into a period of dramatic growth.

Large projects like this rely on strong leadership. And I stand ready to do my part, and to contribute the core technology that is needed. Now to move this forward, what it takes is commitment from the worldwide mathematics community. We have the opportunity to make the second decade of the 21st century really count in the multi-millennium history of pure mathematics. Let’s actually make it happen!

It’s been many years in the making, and today I’m excited to announce the launch of Mathematica Online: a version of Mathematica that operates completely in the cloud—and is accessible just through any modern web browser.

In the past, using Mathematica has always involved first installing software on your computer. But as of today that’s no longer true. Instead, all you have to do is point a web browser at Mathematica Online, then log in, and immediately you can start to use Mathematica—with zero configuration.

Here’s what it looks like:

It’s a notebook interface, just like on the desktop. You interactively build up a computable document, mixing text, code, graphics, and so on—with inputs you can immediately run, hierarchies of cells, and even things like Manipulate. It’s taken a lot of effort, but we’ve been able to implement almost all the major features of the standard Mathematica notebook interface purely in a web browser—extending CDF (Computable Document Format) to the cloud.

There are some tradeoffs of course. For example, Manipulate can’t be as zippy in the cloud as it is on the desktop, because it has to run across the network.  But because its Cloud CDF interface is running directly in the web browser, it can immediately be embedded in any web page, without any plugin, like right here:

Another huge feature of Mathematica Online is that because your files are stored in the cloud, you can immediately access them from anywhere. You can also easily collaborate: all you have to do is set permissions on the files so your collaborators can access them.  Or, for example, in a class, a professor can create notebooks in the cloud that are set so each student gets their own active copy to work with—that they can then email or share back to the professor.

And since Mathematica Online runs purely through a web browser, it immediately works on mobile devices too. Even better, there’s soon going to be a Wolfram Cloud app that provides a native interface to Mathematica Online, both on tablets like the iPad, and on phones:

There are lots of great things about Mathematica Online. There are also lots of great things about traditional desktop Mathematica. And I, for one, expect routinely to use both of them.

They fit together really well.  Because from Mathematica Online there’s a single button that “peels off” a notebook to run on the desktop. And within desktop Mathematica, you can seamlessly access notebooks and other files that are stored in the cloud.

If you have desktop Mathematica installed on your machine, by all means use it.  But get Mathematica Online too (which is easy to do—through Premier Service Plus for individuals, or a site license add-on).  And then use the Wolfram Cloud to store your files, so you can access and compute with them from anywhere with Mathematica Online. And so you can also immediately share them with anyone you want.

By the way, when you run notebooks in the cloud, there are some extra web-related features you get—like being able to embed inside a notebook other web pages, or videos, or actually absolutely any HTML code.

Mathematica Online is initially set up to run—and store content—in our main Wolfram Cloud. But it’ll soon also be possible to get a Wolfram Private Cloud—so you operate entirely in your own infrastructure, and for example let people in your organization access Mathematica Online without ever using the public web.

A few weeks ago we launched the Wolfram Programming Cloud—our very first full product based on the Wolfram Language, and Wolfram Cloud technology. Mathematica Online is our second product based on this technology stack.

The Wolfram Programming Cloud is focused on creating deployable cloud software. Mathematica Online is instead focused on providing a lightweight web-based version of the traditional Mathematica experience. Over the next few months, we’re going to be releasing a sequence of other products based on the same technology stack, including the Wolfram Discovery Platform (providing unlimited access to the Wolfram Knowledgebase for R&D) and the Wolfram Data Science Platform (providing a complete data-source-to-reports data science workflow).

One of my goals since the beginning of Mathematica more than a quarter century ago has been to make the system as widely accessible as possible. And it’s exciting today to be able to take another major new step in that direction—making Mathematica immediately accessible to anyone with a web browser.

There’ll be many applications. From allowing remote access for existing Mathematica users. To supporting mobile workers. To making it easy to administer Mathematica for project-based users, or on public-access computers. As well as providing a smooth new workflow for group collaboration and for digital classrooms.

But for me right now it’s just so neat to be able to see all the power of Mathematica immediately accessible through a plain old web browser—on a computer or even a phone.

And all you need do is go to the Mathematica Online website…

To comment, please visit the copy of this post at the Wolfram Blog »

In the Wolfram Language a little code can go a long way. And to use that fact to let everyone have some fun, today we’re introducing Tweet-a-Program.

Compose a tweet-length Wolfram Language program, and tweet it to @WolframTaP. Our Twitter bot will run your program in the Wolfram Cloud and tweet back the result.

One can do a lot with Wolfram Language programs that fit in a tweet. Like here’s a 78-character program that generates a color cube made of spheres:

It’s easy to make interesting patterns:

Here’s a 44-character program that seems to express itself like an executable poem:

Going even shorter, here’s a little “fractal hack”, in just 36 characters:

Putting in some math makes it easy to get all sorts of elaborate structures and patterns:

You don’t have to make pictures. Here, for instance, are the first 1000 digits of π, sized according to their magnitudes (notice that run of 9s!):

The Wolfram Language not only knows how to compute π, as well as a zillion other algorithms; it also has a huge amount of built-in knowledge about the real world. So right in the language, you can talk about movies or countries or chemicals or whatever. And here’s a 78-character program that makes a collage of the flags of Europe, sized according to country population:

We can make this even shorter if we use some free-form natural language in the program. In a typical Wolfram notebook interface, you do this using , but in Tweet-a-Program, you can do it just using =[...]:

The Wolfram Language knows a lot about geography. Here’s a program that makes a “powers of 10” sequence of disks, centered on the Eiffel Tower:

There are many, many kinds of real-world knowledge built into the Wolfram Language, including some pretty obscure ones. Here’s a map of all the shipwrecks it knows in the Atlantic:

The Wolfram Language deals with images too. Here’s a program that gets images of the planets, then randomly scrambles their colors to give them a more exotic look:

Here’s an image of me, repeatedly edge-detected:

Or, for something more “pop culture” (and ready for image analysis etc.), here’s an array of random movie posters:

The Wolfram Language does really well with words and text too. Like here’s a program that generates an “infographic” showing the relative frequencies of first letters for words in English and in Spanish:

And here—just fitting in a tweet—is a program that computes a smoothed estimate of the frequencies of “Alice” and “Queen” going through the text of Alice in Wonderland:

Networks are good fodder for Tweet-a-Program too. Like here’s a program that generates a sequence of networks:

And here—just below the tweet length limit—is a program that generates a random cloud of polyhedra:

What’s the shortest “interesting program” in the Wolfram Language?

In some languages, it might be a “quine”—a program that outputs its own code. But in the Wolfram Language, quines are completely trivial. Since everything is symbolic, all it takes to make a quine is a single character:

Using the built-in knowledge in the Wolfram Language, you can make some very short programs with interesting output. Like here’s a 15-character program that generates an image from built-in data about knots:

Some short programs are very easy to understand:

It’s fun to make short “mystery” programs. What’s this one doing?

Or this one?

Or, much more challengingly, this one:

I’ve actually spent many years of my life studying short programs and what they do—and building up a whole science of the computational universe, described in my big book A New Kind of Science. It all started more than three decades ago—with a computer experiment that I can now do with just a single tweet:

My all-time favorite discovery is tweetable too:

If you go out searching in the computational universe, it’s easy to find all sorts of amazing things:

An ultimate question is whether somewhere out there in the computational universe there is a program that represents our whole physical universe. And is that program short enough to be tweetable in the Wolfram Language?

But regardless of this, we already know that the Wolfram Language lets us write amazing tweetable programs about an incredible diversity of things. It’s taken more than a quarter of a century to build the huge tower of knowledge and automation that’s now in the Wolfram Language. But this richness is what makes it possible to express so much in the space of a tweet.

In the past, only ordinary human languages were rich enough to be meaningfully used for tweeting. But what’s exciting now is that it seems like the Wolfram Language has passed a kind of threshold of general expressiveness that lets it, too, be meaningfully tweetable. For like ordinary human languages, it can talk about all sorts of things, and represent all sorts of ideas. But there’s also something else about it: unlike ordinary human languages, everything in it always has a precisely defined meaning—and what you write is not just readable, but also runnable.

Tweets in an ordinary human language are (presumably) intended to have some effect on the mind of whoever reads them. But the effect may be different on different minds, and it’s usually hard to know exactly what it is. But tweets in the Wolfram Language have a well-defined effect—which you see when they’re run.

It’s interesting to compare the Wolfram Language to ordinary human languages. An ordinary language, like English, has a few tens of thousands of reasonably common “built-in” words, excluding proper names etc. The Wolfram Language has about 5000 built-in named objects, excluding constructs like entities specified by proper names.

And one thing that’s important about the Wolfram Language—that it shares with ordinary human languages—is that it’s not only writable by humans, but also readable by them. There’s vocabulary to acquire, and there are a few principles to learn—but it doesn’t take long before, as a human, one can start to understand typical Wolfram Language programs.

Sometimes it’s fairly easy to give at least a rough translation (or “explanation”) of a Wolfram Language program in ordinary human language. But it’s very common for a Wolfram Language program to express something that’s quite difficult to communicate—at least at all succinctly—in ordinary human language. And inevitably this means that there are things that are easy to think about in the Wolfram Language, but difficult to think about in ordinary human language.

Just like with an ordinary language, there are language arts for the Wolfram Language. There’s reading and comprehension. And there’s writing and composition. Always with lots of ways to express something, but now with a precise notion of correctness, as well as all sorts of measures like speed of execution.

And like with ordinary human language, there’s also the matter of elegance. One can look at both meaning and presentation. And one can think of distilling the essence of things to create a kind of “code poetry”.

When I first came up with Tweet-a-Program it seemed mostly like a neat hack. But what I’ve realized is that it’s actually a window into a new kind of expression—and a form of communication that humans and computers can share.

Of course, it’s also intended to be fun. And certainly for me there’s great satisfaction in creating a tiny, elegant gem of a program that produces something amazing.

And now I’m excited to see what everyone will do with it. What kinds of things will be created? What popular “code postcards” will there be? Who will be inspired to code? What puzzles will be posed and solved? What competitions will be defined and won? And what great code artists and code poets will emerge?

Now that we have tweetable programs, let’s go find what’s possible…

To develop and test programs for Tweet-a-Program, you can log in free to the Wolfram Programming Cloud, or use any other Wolfram Language system, on the desktop or in the cloud. Check out some details here.

To comment, please visit the copy of this post at the Wolfram Blog »

Where should data from the Internet of Things go? We’ve got great technology in the Wolfram Language for interpreting, visualizing, analyzing, querying and otherwise doing interesting things with it. But the question is, how should the data from all those connected devices and everything else actually get to where good things can be done with it? Today we’re launching what I think is a great solution: the Wolfram Data Drop.

When I first started thinking about the Data Drop, I viewed it mainly as a convenience—a means to get data from here to there. But now that we’ve built the Data Drop, I’ve realized it’s much more than that. And in fact, it’s a major step in our continuing efforts to integrate computation and the real world.

So what is the Wolfram Data Drop? At a functional level, it’s a universal accumulator of data, set up to get—and organize—data coming from sensors, devices, programs, or for that matter, humans or anything else. And to store this data in the cloud in a way that makes it completely seamless to compute with.

Our goal is to make it incredibly straightforward to get data into the Wolfram Data Drop from anywhere. You can use things like a web API, email, Twitter, web form, Arduino, Raspberry Pi, etc. And we’re going to be progressively adding more and more ways to connect to other hardware and software data collection systems. But wherever the data comes from, the idea is that the Wolfram Data Drop stores it in a standardized way, in a “databin”, with a definite ID.

Here’s an example of how this works. On my desk right now I have this little device:

Every 30 seconds it gets data from the tiny sensors on the far right, and sends the data via wifi and a web API to a Wolfram Data Drop databin, whose unique ID happens to be “3pw3N73Q”. Like all databins, this databin has a homepage on the web: wolfr.am/3pw3N73Q.

The homepage is an administrative point of presence that lets you do things like download raw data. But what’s much more interesting is that the databin is fundamentally integrated right into the Wolfram Language. A core concept of the Wolfram Language is that it’s knowledge based—and has lots of knowledge about computation and about the world built in.

For example, the Wolfram Language knows in real time about stock prices and earthquakes and lots more. But now it can also know about things like environmental conditions on my desk—courtesy of the Wolfram Data Drop, and in this case, of the little device shown above.

Here’s how this works. There’s a symbolic object in the Wolfram Language that represents the databin:

And one can do operations on it. For instance, here are plots of the time series of data in the databin:

And here are histograms of the values:

And here’s the raw data presented as a dataset:

What’s really nice is that the databin—which could contain data from anywhere—is just part of the language. And we can compute with it just like we would compute with anything else.

So here for example are the minimum and maximum temperatures recorded at my desk:
(for aficionados: MinMax is a new Wolfram Language function)

We can convert those to other units (% stands for the previous result):

Let’s pull out the pressure as a function of time. Here it is:

Of course, the Wolfram Knowledgebase has historical weather data. So in the Wolfram Language we can just ask it the pressure at my current location for the time period covered by the databin—and the result is encouragingly similar:

If we wanted, we could do all sorts of fancy time series analysis, machine learning, modeling, or whatever, with the data. Or we could do elaborate visualizations of it. Or we could set up structured or natural language queries on it.

Here’s an important thing: notice that when we got data from the databin, it came with units attached. That’s an example of a crucial feature of the Wolfram Data Drop: it doesn’t just store raw data, it stores data that has real meaning attached to it, so it can be unambiguously understood wherever it’s going to be used.

We’re using a big piece of technology to do this: our Wolfram Data Framework (WDF). Developed originally in connection with Wolfram|Alpha, it’s our standardized symbolic representation of real-world data. And every databin in the Wolfram Data Drop can use WDF to define a “data semantics signature” that specifies how its data should be interpreted—and also how our automatic importing and natural language understanding system should process new raw data that comes in.

The beauty of all this is that once data is in the Wolfram Data Drop, it becomes both universally interpretable and universally accessible, to the Wolfram Language and to any system that uses the language. So, for example, any public databin in the Wolfram Data Drop can immediately be accessed by Wolfram|Alpha, as well as by the various intelligent assistants that use Wolfram|Alpha. Tell Wolfram|Alpha the name of a databin, and it’ll automatically generate an analysis and a report about the data that’s in it:

Through WDF, the Wolfram Data Drop immediately handles more than 10,000 kinds of units and physical quantities. But the Data Drop isn’t limited to numbers or numerical quantities. You can put anything you want in it. And because the Wolfram Language is symbolic, it can handle it all in a unified way.

The Wolfram Data Drop automatically includes timestamps, and, when it can, geolocations. Both of these have precise canonical representations in WDF. As do chemicals, cities, species, networks, or thousands of other kinds of things. But you can also drop things like images into the Wolfram Data Drop.

Somewhere in our Quality Assurance department there’s a camera on a Raspberry Pi watching two recently acquired corporate fish—and dumping an image every 10 minutes into a databin in the Wolfram Data Drop:

In the Wolfram Language, it’s easy to stack all the images up in a manipulable 3D “fish cube” image:

Or to process the images to get a heat map of where the fish spend time:

We can do all kinds of analysis in the Wolfram Language. But to me the most exciting thing here is how easy it is to get new real-world data into the language, through the Wolfram Data Drop.

Around our company, databins are rapidly proliferating. It’s so easy to create them, and to hook up existing monitoring systems to them. We’ve got databins now for server room HVAC, for weather sensors on the roof of our headquarters building, for breakroom refrigerators, for network ping data, and for the performance of the Data Drop itself. And there are new ones every day.

Lots of personal databins are being created, too. I myself have long been a personal data enthusiast. And in fact, I’ve been collecting personal analytics on myself for more than a quarter of a century. But I can already tell that March 2015 is going to show a historic shift. Because with the Data Drop, it’s become vastly easier to collect data, with the result that the number of streams I’m collecting is jumping up. I’ll be at least a 25-databin human soon… with more to come.

A really important thing is that because everything in the Wolfram Data Drop is stored in WDF, it’s all semantic and canonicalized, with the result that it’s immediately possible to compare or combine data from completely different databins—and do meaningful computations with it.

So long as you’re dealing with fairly modest amounts of data, the basic Wolfram Data Drop is set up to be completely free and open, so that anyone or any device can immediately drop data into it. Official users can enter much larger amounts of data—at a rate that we expect to be able to progressively increase.

Wolfram Data Drop databins can be either public or private. And they can either be open to add to, or require authentication. Anyone can get access to the Wolfram Data Drop in our main Wolfram Cloud. But organizations that get their own Wolfram Private Clouds will also soon be able to have their own private Data Drops, running inside their own infrastructure.

So what’s a typical workflow for using the Wolfram Data Drop? It depends on what you’re doing. And even with a single databin, it’s common in my experience to want more than one workflow.

It’s very convenient to be able to take any databin and immediately compute with it interactively in a Wolfram Language session, exploring the data in it, and building up a notebook about it

But in many cases one also wants something to be done automatically with a databin. For example, one can set up a scheduled task to create a report from the databin, say to email out. One can also have the report live on the web, hosted in the Wolfram Cloud, perhaps using CloudCDF to let anyone interactively explore the data. One can make it so that a new report is automatically generated whenever someone visits a page, or one can create a dashboard where the report is continuously regenerated.

It’s not limited to the web. Once a report is in the Wolfram Cloud, it immediately becomes accessible on standard mobile or wearable devices. And it’s also accessible on desktop systems.

You don’t have to make a report. Instead, you can just have a Wolfram Language program that watches a databin, then for example sends out alerts—or takes some other action—if whatever combination of conditions you specify occur.

You can make a databin public, so you’re effectively publishing data through it. Or you can make it private, and available only to the originator of the data—or to some third party that you designate. You can make an API that accesses data from a databin in raw or processed form, and you can call it not only from the web, but also from any programming language or system.

A single databin can have data coming only from one source—or one device—or it can have data from many sources, and act as an aggregation point. There’s always detailed metadata included with each piece of data, so one can tell where it comes from.

For several years, we’ve been quite involved with companies who make connected devices, particularly through our Connected Devices Project. And many times I’ve had a similar conversation: The company will tell me about some wonderful new device they’re making, that measures something very interesting. Then I’ll ask them what’s going to happen with data from the device. And more often than not, they’ll say they’re quite concerned about this, and that they don’t really want to have to hire a team to build out cloud infrastructure and dashboards and apps and so on for them.

Well, part of the reason we created the Wolfram Data Drop is to give such companies a better solution. They deal with getting the data—then they just drop it into the Data Drop, and it goes into our cloud (or their own private version of it), where it’s easy to analyze, visualize, query, and distribute through web pages, apps, APIs, or whatever.

It looks as if a lot of device companies are going to make use of the Wolfram Data Drop. They’ll get their data to it in different ways. Sometimes through web APIs. Sometimes by direct connection to a Wolfram Language system, say on a Raspberry Pi. Sometimes through Arduino or Electric Imp or other hardware platforms compatible with the Data Drop. Sometimes gatewayed through phones or other mobile devices. And sometimes from other clouds where they’re already aggregating data.

We’re not at this point working specifically on the “first yard” problem of getting data out of the device through wires or wifi or Bluetooth or whatever. But we’re setting things up so that with any reasonable solution to that, it’s easy to get the data into the Wolfram Data Drop.

There are different models for people to access data from connected devices. Developers or researchers can come directly to the Wolfram Cloud, through either cloud or desktop versions of the Wolfram Language. Consumer-oriented device companies can choose to set up their own private portals, powered by the Wolfram Cloud, or perhaps by their own Wolfram Private Cloud. Or they can access the Data Drop from a Wolfram mobile app, or their own mobile app. Or from a wearable app.

Sometimes a company may want to aggregate data from many devices—say for a monitoring net, or for a research study. And again their users may want to work directly with the Wolfram Language, or through a portal or app.

When I first thought about the Wolfram Data Drop, I assumed that most of the data dropped into it would come from automated devices. But now that we have the Data Drop, I’ve realized that it’s very useful for dealing with data of human origin too. It’s a great way to aggregate answers—say in a class or a crowdsourcing project—collect feedback, keep diary-type information, do lifelogging, and so on. Once one’s defined a data semantics signature for a databin, the Wolfram Data Drop can automatically generate a form to supply data, which can be deployed on the web or on mobile.

The form can ask for text, or for images, or whatever. And when it’s text, our natural language understanding system can take the input and automatically interpret it as WDF, so it’s immediately standardized.

Now that we’ve got the Wolfram Data Drop, I keep on finding more uses for it—and I can’t believe I lived so long without it. As throughout the Wolfram Language, it’s really a story of automation: the Wolfram Data Drop automates away lots of messiness that’s been associated with collecting and processing actual data from real-world sources.

And the result for me is that it’s suddenly realistic for anyone to collect and analyze all sorts of data themselves, without getting any special systems built. For example, last weekend, I ended up using the Wolfram Data Drop to aggregate performance data on our cloud. Normally this would be a complex and messy task that I wouldn’t even consider doing myself. But with the Data Drop, it took me only minutes to set up—and, as it happens, gave me some really interesting results.

I’m excited about all the things I’m going to be able to do with the Wolfram Data Drop, and I’m looking forward to seeing what other people do with it. Do try out the beta that we launched today, and give us feedback (going into a Data Drop databin of course). I’m hoping it won’t be long before lots of databins are woven into the infrastructure of the world: another step forward in our long-term mission of making the world computable…

Pictures from Pi Day now added »

This coming Saturday is “Pi Day of the Century”. The date 3/14/15 in month/day/year format is like the first digits of π=3.1415… And at 9:26:53.589… it’s a “super pi moment”.

Between Mathematica and Wolfram|Alpha, I’m pretty sure our company has delivered more π to the world than any other organization in history. So of course we have to do something special for Pi Day of the Century.

A Corporate Confusion

One of my main roles as CEO is to come up with ideas—and I’ve spent decades building an organization that’s good at turning those ideas into reality. Well, a number of weeks ago I was in a meeting about upcoming corporate events, and someone noted that Pi Day (3/14) would happen during the big annual SXSW (South by Southwest) event in Austin, Texas. So I said (or at least I thought I said), “We should have a big pi to celebrate Pi Day.”

I didn’t give it another thought, but a couple of weeks later we had another meeting about upcoming events. One agenda item was Pi Day. And the person who runs our Events group started talking about the difficulty of finding a bakery in Austin to make something suitably big. “What are you talking about?” I asked. And then I realized: “You’ve got the wrong kind of pi!”

I guess in our world pi confusions are strangely common. Siri’s voice-to-text system sends Wolfram|Alpha lots of “pie” every day that we have to specially interpret as “pi”. And then there’s the Raspberry Pi, that has the Wolfram Language included. And for me there’s the additional confusion that my personal fileserver happens to have been named “pi” for many years.

After the pi(e) mistake in our meeting we came up with all kinds of wild ideas to celebrate Pi Day. We’d already rented a small park in the area of SXSW, and we wanted to make the most interesting “pi countdown” we could. We resolved to get a large number of edible pie “pixels”, and use them to create a π shape inside a pie shape. Of course, there’ll be the obligatory pi selfie station, with a “Stonehenge” pi. And a pi(e)-decorated Wolfie mascot for additional selfies. And of course we’ll be doing things with Raspberry Pis too.

A Piece of Pi for Everyone

I’m sure we’ll have plenty of good “pi fun” at SXSW. But we also want to provide pi fun for other people around the world. We were wondering, “What can one do with pi?” Well, in some sense, you can do anything with pi. Because, apart from being the digits of pi, its infinite digit sequence is—so far as we can tell—completely random. So for example any run of digits will eventually appear in it.

How about giving people a personal connection to that piece of math? Pi Day is about a date that appears as the first digits of pi. But any date appears somewhere in pi. So, we thought: Why not give people a way to find out where their birthday (or other significant date) appears in pi, and use that to make personalized pi T-shirts and posters?

In the Wolfram Language, it’s easy to find out where your birthday appears in π. It’s pretty certain that any mm/dd/yy will appear somewhere in the first 10 million digits. On my desktop computer (a Mac Pro), it takes 6.28 seconds (2π?!) to compute that many digits of π.

Here’s the Wolfram Language code to get the result and turn it into a string (dropping the decimal point at position 2):

Now it’s easy to find any “birthday string”:

So, for example, my birthday string first appears in π starting at digit position 151,653.

What’s a good way to display this? It depends how “pi lucky” you are. For those born on 4/15/92, their birthdate already appears at position 3. (Only about a certain fraction of positions correspond to a possible date string.) People born on November 23, 1960 have the birthday string that’s farthest out, appearing only at position 9,982,546. And in fact most people have birthdays that are pretty “far out” in π (the average is 306,150 positions).

Our long-time art director had the idea of using a spiral that goes in and out to display the beginnings and ends of such long digit sequences. And almost immediately, he’d written the code to do this (one of the great things about the Wolfram Language is that non-engineers can write their own code…).

Next came deploying that code to a website. And thanks to the Wolfram Programming Cloud, this was basically just one line of code! So now you can go to MyPiDay.com…

…and get your own piece of π!

And then you can share the image, or get a poster or T-shirt of it:

The Science of Pi

With all this discussion about pi, I can’t resist saying just a little about the science of pi. But first, just why is pi so famous? Yes, it’s the ratio of circumference to diameter of a circle. And that means that π appears in zillions of scientific formulas. But it’s not the whole story. (And for example most people have never even heard of the analog of π for an ellipse—a so-called complete elliptic integral of the second kind.)

The bigger story is that π appears in a remarkable range of mathematical settings—including many that don’t seem to have anything to do with circles. Like sums of negative powers, or limits of iterations, or the probability that a randomly chosen fraction will not be in lowest terms.

If one’s just looking at digit sequences, pi’s 3.1415926… doesn’t seem like anything special. But let’s say one just starts constructing formulas at random and then doing traditional mathematical operations on them, like summing series, doing integrals, finding limits, and so on. One will get lots of answers that are 0, or 1/2, or . And there’ll be plenty of cases where there’s no closed form one can find at all. But when one can get a definite result, my experience is that it’s remarkably common to find π in it.

A few other constants show up too, like e (2.1718…), or Euler gamma (0.5772…), or Catalan’s constant (0.9159…). But π is distinctly more common.

Perhaps math could have been set up differently. But at least with math as we humans have constructed it, the number that is π is a widespread building block, and it’s natural that we gave it a name, and that it’s famous—now even to the point of having a day to celebrate it.

What about other constants? “Birthday strings” will certainly appear at different places in different constants. And just like when Wolfram|Alpha tries to find closed forms for numbers, there’s typically a tradeoff between digit position and obscurity of the constants used. So, for example, my birthday string appears at position 151,653 in π, 241,683 in e, 45,515 in , 40,979 in ζ(3) … and 196 in the 1601th Fibonacci number.

Randomness in π

Let’s say you make a plot that goes up whenever a digit of π is 5 or above, and down otherwise:

It looks just like a random walk. And in fact, all statistical and cryptographic tests of randomness that have been tried on the digits (except tests that effectively just ask “are these the digits of pi?”) say that they look random too.

Why does that happen? There are fairly simple procedures that generate digits of pi. But the remarkable thing is that even though these procedures are simple, the output they produce is complicated enough to seem completely random. In the past, there wasn’t really a context for thinking about this kind of behavior. But it’s exactly what I’ve spent many years studying in all kinds of systems—and wrote about in A New Kind of Science. And in a sense the fact that one can “find any birthday in pi” is directly connected to concepts like my general Principle of Computational Equivalence.

SETI among the Digits

Of course, just because we’ve never seen any regularity in the digits of pi, it doesn’t mean that no such regularity exists. And in fact it could still be that if we did a big search, we might find somewhere far out in the digits of pi some strange regularity lurking.

What would it mean? There’s a science fiction answer at the end of Carl Sagan’s book version of Contact. In the book, the search for extraterrestrial intelligence succeeds in making contact with an interstellar civilization that has created some amazing artifacts—and that then explains that what they in turn find remarkable is that encoded in the distant digits of pi, they’ve found intelligent messages, like an encoded picture of a circle.

At first one might think that finding “intelligence” in the digits of pi is absurd. After all, there’s just a definite simple algorithm that generates these digits. But at least if my suspicions are correct, exactly the same is actually true of our whole universe, so that every detail of its history is in principle computable much like the digits of pi.

Now we know that within our universe we have ourselves as an example of intelligence. SETI is about trying to find other examples. The goal is fairly well defined when the search is for “human-like intelligence”. But—as my Principle of Computational Equivalence suggests—I think that beyond that it’s essentially impossible to make a sharp distinction between what should be considered “intelligent” and what is “merely computational”.

If the century-old mathematical suspicion is correct that the digits of pi are “normal”, it means that every possible sequence eventually occurs among the digits, including all the works of Shakespeare, or any other artifact of any possible civilization. But could there be some other structure—perhaps even superimposed on normality—that for example shows evidence of the generation of intelligence-like complexity?

While it may be conceptually simple, it’s certainly more bizarre to contemplate the possibility of a human-like intelligent civilization lurking in the digits of pi, than in the physical universe as explored by SETI. But if one generalizes what one counts as intelligence, the situation is a lot less clear.

Of course, if we see a complex signal from a pulsar magnetosphere we say it’s “just physics”, not the result of the evolution of a “magnetohydrodynamic civilization”. And similarly if we see some complex structure in the digits of pi, we’re likely to say it’s “just mathematics”, not the result of some “number theoretic civilization”.

One can generalize from the digit sequence of pi to representations of any mathematical constant that is easy to specify with traditional mathematical operations. Sometimes there are simple regularities in those representations. But often there is apparent randomness. And the project of searching for structure is quite analogous to SETI in the physical universe. (One difference, however, is that π as a number to study is selected as a result of the structure of our physical universe, our brains, and our mathematical development. The universe presumably has no such selection, save implicitly from the fact the we exist in it.)

I’ve done a certain amount of searching for regularities in representations of numbers like π. I’ve never found anything significant. But there’s nothing to say that any regularities have to be at all easy to find. And there’s certainly a possibility that it could take a SETI-like effort to reveal them.

But for now, let’s celebrate the Pi Day of our century, and have fun doing things like finding birthday strings in the digits of pi. Of course, someone like me can’t help but wonder what success there will have been by the next Pi Day of the Century, in 2115, in either SETI or “SETI among the digits”…

This Just In…

Pictures from the Pi Day event:

Last week I spoke at SXSW Interactive 2015 in Austin, Texas. Here’s a slightly edited transcript:

A Most Productive Year

Well, hello again. I’ve actually talked about computation three times before at SXSW. And I have to say when I first agreed to give this talk, I was worried that I would not have anything at all new to say. But actually, there’s a huge amount that’s new. In fact, this has probably been the single most productive year of my life. And I’m excited to be able to talk to you here today about some of the things that I’ve figured out recently.

It’s going to be a fairly wild ride, sort of bouncing between very conceptual and very practical—from thousand-year-old philosophy issues, to cloud technology to use here and now.

Basically, for the last 40 years I’ve been building a big tower of ideas and technology, working more or less alternately on basic science and on technology. And using the basic science to figure out more technology, and technology to figure out more science.

I’m happy to say lots of people have used both the science and the technology that I’ve built. But I think what we’ve now got is much bigger than before. Actually, talking to people the last couple of days at SXSW I’m really excited, because probably about 3/4 of the people that I’ve talked to can seriously transform—or at least significantly upgrade—what they’re doing by using new things that we’ve built.

The Wolfram Language

OK. So now I’ve got to tell you how. It all starts with the Wolfram Language. Which actually, as it happens, I first talked about by that name two years ago right here at SXSW.

The Wolfram Language is a big and ambitious thing which is actually both a central piece of technology, and a repository and realization of a bunch of fundamental ideas. It’s also something that you can start to use right now, free on the web. Actually, it runs pretty much everywhere—on the cloud, desktops, servers, supercomputers, embedded processors, private clouds, whatever.

From an intellectual point of view, the goal of the Wolfram Language is basically to express as much as possible computationally—to provide a very broad way to encapsulate computation and knowledge, and to automate, as much as possible, what can be done with them.

I’ve been working on building what’s now the Wolfram Language for about three decades. And in Mathematica and Wolfram|Alpha, many many people have used precursors of what we have now.

But today’s Wolfram Language is something different. It’s something much broader, that I think can be the way that lots of computation gets done sort of everywhere, in all sorts of systems and devices and whatever.

So let’s see it in action. Let’s start off by just having a little conversation with the language, in a thing we invented 26 years ago that we call a notebook. Let’s do something trivial.

Good. Let’s try something different. You may know that it was Pi Day on Saturday: 3/14/15. And since we are the company that I think has served more mathematical pi than any other in history, we had a little celebration about Pi Day. So let’s have the Wolfram Language compute pi for us; let’s say to a thousand places:

There. Or let’s be more ambitious; let’s calculate it to a million places. It’ll take a little bit longer…

But not much. And there’s the result. It goes on and on. Look how small the scroll thumb is.

For something different we could pick up the Wikipedia article about pi:

And make a word cloud from it:

Needless to say, in the article about pi, pi itself features prominently.

Or let’s get an image. Here’s me:

So let’s go ahead and do something with that image—for example, let’s edge-detect it. % always means the most recent thing we got, so…

…there’s the edge detection of that image. Or let’s say we make a morphological graph from that image, so now we’ll make some kind of network:

Oh, that’s quite fetching; OK. Or let’s automatically make a little user interface here that controls the degree of edginess that we have here—so there I am:

Or let’s get a table of different levels of edginess in that picture:

And now for example we can take all of those images and stack them up and make a 3D image:

A Language for the Real World

The Wolfram Language has zillions of different kinds of algorithms built in. It’s also got real-world knowledge and data. So, for example, I could just say something like “planets”:

So it understood from natural language what we were talking about. Let’s get a list of planets:

And there’s a list of planets. Let’s get pictures of them:

Let’s find their masses:

Now let’s make an infographic of planets sized according to mass:

I think it’s pretty amazing that it’s just one line of code to make something like this.

Let’s go on a bit. This is where the internet thinks my computer is right now:

We could say, “When’s sunset going to be, at this position on this day?”

How long from now?

OK, let’s get a map of, say, 10 miles around the center of Austin:

Or, let’s say, a powers of 10 sequence:

Or let’s go off planet and do the same kind of thing. We ask for the Apollo 11 landing site, and let’s show a thousand miles around that on the Moon:

We can do all kinds of things. Let’s try something in a different kind of domain. Let’s get a list of van Gogh’s artworks:

And let’s take, say, the first 20 of those, and let’s get images of those:

And now, for example, we can take those and say, “What were the dominant colors that were used in those images?”

And let’s plot those colors in a chromaticity diagram, in 3D:

Philosophy of the Wolfram Language

I think it’s fairly amazing what can be done with just tiny amounts of Wolfram Language code.

It’s really a whole new situation for programming. I mean, it’s a dramatic change. The traditional idea has been to start from a fairly small programming language, and then write fairly big programs to do what you want. The idea in the Wolfram Language is to make the language itself in a sense as big as possible—to build in as much as we can—and in effect to automate as much as possible of the process of programming.

These are the types of things that the Wolfram Language deals with:

And by now we’ve got thousands of built-in functions, tens of thousands of models and methods and algorithms and so on, and carefully curated data on thousands of different domains.

And I’ve basically spent nearly 30 years of my life keeping the design of all of this clean and consistent.

It’s been really interesting, and the result is really satisfying, because now we have something that’s incredibly powerful—that we’re also able to use to develop the language itself at an accelerating rate.

Tweetable Programs

Here’s something we did recently to have some fun with all of this. It’s called Tweet-a-Program.

The idea here is you send a whole program as a tweet, and get back the result of running it. If you stop by our booth at the tradeshow here, you can pick up one of these little “Galleries of Tweetable Programs”. And here’s an online collection of some tweetable programs—and remember, every one of these programs is less than 140 characters long, and does all kinds of different types of things.

So to celebrate tweetable programs, we also have a deck of “code cards“, each with a tweetable program:

Computational Thinking for Kids

You know, if you look even at these tweetable programs, they’re surprisingly easy to understand. You can kind of just read the words to get a good idea how they work.

And you might think, OK, it’s like kids could do this. Well, actually, that’s true. And in fact I think this is an important moment for programming, where the same thing has happened as has happened in the past for things like video editing and so on: We’ve automated enough that the fancy professionals don’t really have any advantage over kids—now in learning programming.

So one thing I’m very keen on right now is to use our language as a way to teach computational thinking to a very broad range of people.

Soon we’ll have something we call Wolfram Programming Lab—which you can use free on the web. It’s kind of an immersion language learning for the Wolfram Language, where you see lots of small working examples of Wolfram Language programs that you get to modify and run.

I think it’s pretty powerful for education. Because it’s not just teaching programming: It’s immediately bringing in lots of real-world stuff, integrating with other things kids are learning, and really teaching a computational-thinking approach to everything.

So let’s take a look at a couple of examples. It’s been Pi Day; let’s look at Pi Necklaces:

The basic idea is that here’s a little piece of code—you can run it, see what it does, modify it. You can say to show the details, and it’ll tell you what’s going on. And so on.

And maybe we can try another example. Let’s say we do something a little bit more real-world… Where can you see from a particular skyscraper?

This will show us the visible region from the Empire State Building. And we could go ahead and change lots of parameters of this and see what happens, or you can go down and look at challenges, where it’s asking you to try and do other kinds of related computations.

I hope lots of people—kids and otherwise—will have fun with the explorations that we’ve been making. I think it’s great for education: a kind of mixture of sort of the precise thinking of math, and the creativity of something like writing. And, by the way, in the Programming Lab, we can watch programs people are trying to write, and do all kinds of education analytics inside.

I might mention that for people who don’t know English, we’ll soon be able to annotate any Wolfram Language programs in lots of other languages.

I think some amazing things are going to happen when lots more people learn to think computationally using the Wolfram Language.

Natural Language as Input

Of course, many millions of people already use our technology every day without any of that. They’re just typing pure natural language into Wolfram|Alpha, or saying things to Siri that get sent to Wolfram|Alpha.

I guess one big breakthrough has been being able to use our very precise natural language understanding, using both new kinds of algorithms and our huge knowledgebase.

And using all our knowledge and computation capabilities to generate automated reports for things that people ask about. Whether it’s questions about demographics:

Or about airplanes—this shows airplanes currently overhead where the internet thinks my computer is:

Or for example about genome sequences. It will go look up whether that particular random base-pair sequence appears somewhere on the human genome:

So those are a few sort of things that we can do in Wolfram|Alpha. And we’ve been covering thousands of different domains of knowledge, adding new things all the time.

By the way, there are now quite a lot of large organizations that have internal versions of Wolfram|Alpha that include their own data as well as our public data. And it’s really nice, because all types of people can kind of make “drive-by” queries in natural language without ever having to go to their IT departments.

You know, being able to use natural language is central to the actual Wolfram Language, too. Because when you want to refer to something in the real world—like a city, for example—you can’t be going to documentation to find out its name. You just want to type in natural language, and then get that interpreted as something precise.

Which is exactly what you can now do. So, for example, we would type something like:

and get:

And that would be understood as the entity “New York City”. And we could go and ask things like what’s the population of that, and it will tell us the results:

Big Idea: Symbolic Programming

There’s an awful lot that goes into making the Wolfram Language work—not only tens of millions of lines of algorithmic code, and terabytes of curated data, but also some big ideas.

Probably the biggest idea is the idea of symbolic programming, which has been the core of what’s become the Wolfram Language right from the very beginning.

Here’s the basic point: In the Wolfram Language, everything is symbolic. It doesn’t have to have any particular value; it can just be a thing.

If I just typed “x” in most computer languages, they’d say, “Help, I don’t know what x is”. But the Wolfram Language just says, “OK, x is x; it’s symbolic”.

And the point is that basically anything can be represented like this. If I type in “Jupiter”, it’s just a symbolic thing:

Or, for example, if I were to put in an image here, it’s just a symbolic thing:

And I could have something like a slider, a user-interface element—again, it’s just a symbolic thing:

And now when you compute, you can do anything with anything. Like you could do math with x:

Or with an image of Jupiter:

Or with sliders:

Or whatever.

It’s taken me a really long time, actually, to understand just how powerful this idea of symbolic programming really is. Every few years I understand it a little bit more.

Language for Deployment

Long ago we understood how to represent programs symbolically, and documents, and interfaces, so that they all instantly become things you can compute with. Recently one of the big breakthroughs has been understanding how to represent not only operations and content symbolically, but also their deployments.

OK, there’s one thing I need to explain first. What I’ve been showing you here today has mostly been using a desktop version of the Wolfram Language, though it’s going to the cloud to get things from our knowledgebase and so on. Well, with great effort we’ve also built a full version of the whole language in the cloud.

So let me use that interface there, just through a web browser. I’m going to have the exact same experience, so to speak. And we can do all these same kinds of things just purely in the cloud through a web browser.

You know, in my 40 years of writing software, I don’t believe there’s ever been a development environment as crazy as the web and the cloud. It’s taken us a huge amount of effort to kind of hack through the jungle to get the functionality that we want. We’re pretty much there now. And of course the great news for people who just use what we’ve built is that they don’t have to hack through the jungle themselves, because we’ve already done that.

But OK, so you can use the Wolfram Language directly in the cloud. And that’s really useful. But you can also deploy other things in the language through the cloud.

Like, cat pictures are popular on the internet, so let’s deploy a cat app. Let’s define a form that has a field that asks for a breed of cat, then shows a picture of that. Then let’s deploy that to the cloud.

Now we get a cloud object with a URL. We just go there, and we get a form. The form has a “smart field”, that understands natural language—in this particular case, the language for describing cat breeds. So now we can type in, let’s say, “siamese”. And it will go back and run that code… OK. There’s a picture of a cat.

We can make our web app a little more complicated. Let’s add in another field here.

Again, we deploy to the cloud, and now have a cat at an angle there:

OK. So that’s how we can make a web app, which we can also deploy on mobile and so on. We can also make an API if we want to. Let’s use the same piece of code. Actually, the easiest thing to do would be just to edit that piece of code there, and change this from being a form to being an API:

And now the thing that we have will be an API that we can go fill in parameters to; we can say “cat=manx”, “angle=300”, and now we can run that, and there’s another cat at an angle.

So that was an API that we just created, that could be used by anybody in the cloud. And we can call the API from anywhere—a website, a program, whatever. And actually we can automatically generate the code to call this from all sorts of other languages—let’s say inside Java.

So in effect you can knit Wolfram Language functionality right into any project you’re doing in any language.

In this particular case, you’re calling code in our cloud. I should mention that there are other ways you can set this up, too. You can have a private cloud. You can have a version of the Wolfram Engine that’s on your computer. You can even have the Wolfram Engine in a library that can be explicitly linked into a program you’ve written.

And all this stuff works on mobile too. You can deploy an app that works on mobile; even a complete APK file for Android if you want.

There’s lots of depth to all this software engineering stuff. And it’s rather wonderful how the Wolfram Language manages to simplify and automate so much of it.

The Automation of Programming

You know, I get to see this story of automation up close at our company every day. We have all these projects—all these things we’re building, a huge amount of stuff—that you might think we’d need thousands of people to do. But you see, we’ve been automating things, and then automating our automation and so on, for a quarter of a century now. And so we still only have a little private company with about 700 people—and lots of automation.

It’s fairly spectacular to see: When we automate something—like, say, a type of web development—projects that used to be really painful, and take a couple of months, suddenly become really easy, and take like a day. And from a management point of view, it’s great how that changes the level of innovation that you attempt.

Let me give you a little example from a couple of weeks ago. We were talking about what to do for Pi Day. And we thought it’d be fun to put up a website where people could type in their birthdays, and find out where in the digits of pi those dates show up, and then make a cool T-shirt based on that.

Well, OK, clearly that’s not a corporately critical activity. But if it’s easy, why not do it? Well, with all our automation, it is easy. Here’s the code that got written to create that website:

It’s not particularly long. Somewhere here it’ll deploy to the cloud, and there it’s calling the Zazzle API, and so on. Let me show you the actual website that got made there:

And you can type in your birthday in some format, and then it’ll go off and try and find that birthday in the digits of pi. There we go; it found my birthday, at that digit position, and there’s a custom-created image showing me in pi and letting me go off and get a T-shirt with that on it.

And actually, zero programmers were involved in building this. As it happens, it was just done by our art director, and it went live a couple of days ago, before Pi Day, and has been merrily serving hundreds of thousands of custom T-shirt designs to pi enthusiasts around the world.

Large-Scale Programs

It’s interesting to see how large-scale code development happens in the Wolfram Language. There’s an Eclipse-based IDE, and we’re soon going to release a bunch of integration with Git that we use internally. But one thing that’s very different from other languages is that people tend to write their code in notebooks.

They can put the whole story of their code right there, with text and graphics and whatever right there with the code. They can use notebooks to make structured tests if they want to; there’s a testing notebook with various tests in it we could run and so on:

And they can also use notebooks to make templates for computable documents, where you can directly embed symbolic Wolfram Language code that’ll get executed to make static or interactive documents that you can deliver as reports and so on.

By the way, one of the really nice things about this whole ecosystem is that if you see a finished result—say an infographic—there’s a standard way to include a kind of “compute-back link” that goes right back to the notebook where that graphic was made. So you see everything that’s behind it, and, say, start being able to use the data yourself. Which is useful for things like data publishing for research, or data journalism.

Internet of Things

OK, so, talking of data, a couple of weeks ago we launched what we call our Data Drop.

The idea is to let anything—particularly connected devices—easily drop data into our cloud, and then immediately make it meaningful, and accessible, to the Wolfram Language everywhere.

Like here’s a little device I have that measures a few things… actually, I think this particular one only measures light levels; kind of boring.

But in any case, it’s connected via wifi into our cloud. And everything it measures goes into our Data Drop, in this databin corresponding to that device.

We’re using what we call WDF—the Wolfram Data Framework—to say what the raw numbers from the device mean. And now we can do all kinds of computations.

OK, it hasn’t collected very much data yet, but we could go ahead and make a plot of the data that it’s collected:

That was the light level as seen by that device, and I think it just sat there, and the lights got turned on and then it’s been a fixed light level—sorry, not very exciting. We can just make a histogram of that data, and again it’s going to be really boring in this particular case.

You know, we have all this data about the world from our knowledgebase integrated right into our language. And now with the Data Drop, you can integrate data from any device that you want. We’ve got a whole inventory of different kinds of devices, which we’ve been making for the last couple of years.

Once you get data into this Data Drop, you can use it wherever the Wolfram Language is used. Like in Wolfram|Alpha. Or Siri. Or whatever.

It’s really critical that the Wolfram Language can represent different types of data in a standard way, because that means you can immediately do computations, combine databins, whatever. And I have to say that just being able to sort of “throw data” into the Wolfram Data Drop is really convenient.

Like of course we’re throwing data from the My Pi Day website into a databin. And that means, for example, it’s just one line of code to see where in the world people have been interested in pi and generating pi T-shirts from, and so on.

Some of you might know that I’ve long been an enthusiast of personal analytics. In fact, somewhat to my surprise, I think I’m the human who has collected more data on themselves than anyone else. Like here’s a dot for every piece of outgoing email that I’ve sent for the past quarter century.

But now, with our Data Drop, I’m starting to accumulate even more data. I think I’m already in the double digits in terms of number of databins. Like here’s my heart rate on Pi Day, from a databin. I think there’s a peak there right at the pi moment of the century.

Machine Learning

So, with all this data coming in, what are we supposed to do with it? Well, within the Wolfram Language we’ve got all this visualization and analysis capability. One of our goals is to be able to do the best data science automatically—without needing to take data scientists’ time to do it. And one area where we’ve been working on that a lot is in machine learning.

Let’s say you want to classify pictures into day or night. OK, so here I’ve got a little tiny training set of pictures corresponding to scenes that are day or night, and I just have one little function in the Wolfram Language, Classify, which is going to build a classifier to determine whether a picture is a day or a night one:

So there I’ve got the classifier. Now I can just apply that classifier to a collection of pictures, and now it will tell me, according to that classifier, are those pictures day or night.

And we’re automatically figuring out what type of machine learning to use, and setting up so that you have a classifier that you can use, or can put in an app, or call in an API, or whatever, so that it is just one function to do this.

We’ve got lots of built-in classifiers as well; all sorts of different kinds of things. Let me show you a new thing that’s just coming together now, which is image identification. And I’m going to live dangerously and try and do a live demo on some very new technology.

I asked somebody to go to Walmart and buy a random pile of stuff to try for image identification. So this is probably going be really horrifying. Let’s see what happens here. First of all, let’s set it up so that I can actually capture some images. OK. I’m going to give it a little bit of a better chance by not having it have too funky of a background.

OK. Let us try one banana. Let’s try capturing a banana, and let us see what happens if I say ImageIdentify in our language…

OK! That’s good!

All right. Let’s tempt fate, and try a couple of other things. What’s this? It appears to be a toy plastic triceratops. Let’s see what the system thinks it is. This could get really bad here.

Oops. It says it’s a goat! Well, from that weird angle I guess I can see how it would think that.

OK, let’s try one more thing.

Oh, wow! OK! And the tag in the flower pot says the exact same thing! Which I certainly didn’t know. That’s pretty cool.

This does amazingly well most of the time. And what to me is the most interesting is that when it makes mistakes, like with the triceratops, the mistakes are very human-like. I mean, they’re mistakes that a person could reasonably make.

And actually, I think what’s going on here is pretty exciting. You know, 35 years ago I wanted to figure out brain-like things and I was studying neural nets and so on, and I did all kinds of computer experiments. And I ended up simplifying the underlying rules I looked at—and wound up studying not neural nets, but things called cellular automata, which are kind of like the simplest possible programs.

Mining the Computational Universe

And what I discovered is that if you look at that in the computational universe of all those programs, there’s a whole zoo of possible behaviors that you see. Here’s an example of a whole bunch of cellular automata. Each one is a different program showing different kinds of behavior.

Even when the programs are incredibly simple, there can be incredibly complex behavior. Like, there’s an example; we can go and see what it does:

Well, that discovery led me on the path to developing a whole new kind of science that I wrote a big book about a number of years ago.

That’s ended up having applications all over the place. And for example, over the last decade it’s been pretty neat to see that the idea of modeling things using programs has been winning out over the idea that’s dominated exact science for about 300 years, of modeling things using mathematical equations.

And what’s also been really neat to see is the extent to which we can discover new technology just by kind of “mining” this computational universe of simple programs. Knowing some goal we have, we might sample a trillion programs to find one that’s good for our particular purposes.

That purpose could be making art, or it could be making some new image processing or some new natural-language-understanding algorithm, or whatever.

Finally, Brain-Like Computing

Well, OK, so there’s a lot that we can model and build with simple programs. But people have often said somehow the brain must be special; it must be doing more than that.

Back 35 years ago I could get neural networks to make little attractors or classifiers, but I couldn’t really get them to do anything terribly interesting. And over the years I actually wasn’t terribly convinced by most of the applications of things like neural nets that I saw.

But just recently, some threshold has been passed. And, like, the image identifier I was showing is using pretty much the same ideas as 35 years ago—with a bunch of good engineering tweaks, and perhaps a nod to cellular automata too. But the amazing thing is that just doing pretty much the obvious stuff, with today’s technology, just works.

I couldn’t have predicted when this would happen. But looking at it now, it’s sort of shocking. We’re now able to use millions of neurons, tens of millions of training images and thousands of trillions of the equivalent of neuron firings. And although the engineering details are almost as different as birds versus airplanes, the orders of magnitude are pretty much just the same as for us humans when we learn to identify images.

For me, it’s sort of the missing link for AI. There are so many things now that we can do vastly better than humans, using computers. I mean, if you put a Wolfram|Alpha inside a Turing test bot, you’ll be able to tell instantly that it’s not a human, because it knows too much and can compute too much.

But there’ve been these tasks like image identification that we’ve never been able to do with computers. But now we can. And, by the way, the way people have thought this would work for 60 years is pretty much the way it works; we just didn’t have the technology to see it until now.

So, does this mean we should use neural nets for everything now? Well, no. Here’s the thing: There are some tasks, like image identification, that each human effectively learns to do for themselves, based on what they see in the world around them.

Language as Symbolic Representation

But that’s not everything humans do. There’s another very important thing, pretty much unique to our species. We have language. We have a way of communicating symbolically that lets us take knowledge acquired by one person, and broadcast it to other people. And that’s kind of how we’ve built our civilization.

Well, how do we make computers use that idea too? Well, they have to have a language that represents the world, and that they can compute with. And conveniently enough, that’s exactly what the Wolfram Language is trying to be, and that’s what I’ve been working on for the last 30 years or so.

You know, there’s all this abstract computation out there that can be done. Just go sample cellular automata out in the computational universe. But the question is, how does it relate to our human world, to what we as humans know about or care about?

Well, what’s happened is that humans have tried to boil things down: to describe the world symbolically, using language and linguistic constructs. We’ve seen what’s out there in the world, and we’ve come up with words to describe things. We have a word like “bird”, which refers abstractly to a large collection of things that are birds. And by now in English we’ve got maybe 30,000 words that we commonly use, that are the raw material for our description of the world.

Well, it’s interesting to compare that with the Wolfram Language. In English, there’s been a whole evolution over thousands of years to settle on the perhaps convenient, but often incoherent, language structure that we have. In the Wolfram Language, we—and particularly I—have been working hard for many many years keeping everything as consistent and coherent as possible. And now we’ve got 5,000 or so “core words” or functions, together with lots of other words that describe specific entities.

And in the process of developing the language, what I’ve been doing explicitly is a little like what’s implicitly happened in English. I’ve been looking at all those computational things and processes out there, and trying to understand which of them are common enough that it’s worth giving names to them.

You know, this idea of symbolic representation seems to be pretty critical to human rational thinking. And it’s really interesting to see how the structure of a language can affect how people think about things. We see a little bit of that in human natural languages, but the effect seems to be much larger in computer languages. And for me as a language designer, it’s fascinating to see the patterns of thinking that open up when people start really understanding the Wolfram Language.

Some people might say, “Why are we using computer languages at all? Why not just use human natural language?” Well, for a start, computers need something to talk to each other in. But one of the things I’ve worked hard on in the Wolfram Language is making sure that it’s easy not only for computers, but also for humans, to understand—kind of a bridge between computers and humans.

And what’s more, it turns out there are things that human natural language, as it’s evolved, just isn’t very good at expressing. Just think about programs. There are some programs that, yes, can easily be represented by a little piece of English, but a lot of programs are really awkward to state in English. But they’re very clean in the Wolfram Language.

So I think we need both. Some things it’s easier to say in English, some in the Wolfram Language.

Post-Linguistic Concepts

But back to things like image identification. That’s a task that’s really about going from all the stuff out there, in this case in the visual world, and finding how to make it symbolic—how to describe things abstractly with words.

Now, here’s the thing: Inside the neural net, one thing that’s happening is that it’s implicitly making distinctions, in effect putting things in categories. In the early layers of the net those categories look remarkably like the categories we know are used in the early stages of human visual processing, and we actually have pretty decent words for them: “round”, “pointy”, and so on.

But pretty soon there are categories implicitly being used that we don’t have words for. It’s interesting that in the course of history, our civilization gradually does develop new words for things. Like in the last few decades, we’ve started talking about “fractal patterns”. But before then, those kind of tree-like structures didn’t tend to get identified as being anything in particular, because we didn’t have words for them.

So our machines are going to discover a lot of categories that our civilization has not come up with. I’ve started calling these things a rather pretentious name: “post-linguistic emergent concepts”, or PLECs for short. I think we can make a metaframework for things like this within the Wolfram Language. But I think PLECs are part of the way our computers can start to really extend the traditional human worldview.

By the way, before we even get to PLECs, there are issues with concepts that humans already understand perfectly well. You see, in the Wolfram Language we’ve got representations of lots of things in the world, and we can turn the vast majority of things that people ask Wolfram|Alpha into precise symbolic forms. But we still can’t turn an arbitrary human conversation into something symbolic.

So how would we do that? Well, I think we have to break it down into some kind of “semantic primitives”: basic structures. Some, like fact statements, we already have in the Wolfram Language. And some, like mental statements, like “I think” or “I want”, we don’t.

The Ancient History

It’s a funny thing. I’ve been working recently on designing this kind of symbolic language. And of course people have tried to do things like this before. But the state of the art is actually mostly from a shockingly long time ago. I mean, like the 1200s, there was a chap called Ramon Llull who started working on this; in the 1600s, people like Gottfried Leibniz and John Wilkins.

It’s quite interesting to look at what those guys figured out with their “philosophical languages” or whatever. Of course, they never had an implementation substrate like we do today. But they understood quite a lot about ontological categories and so on. And looking at what they wrote really highlights, actually, what’s the same and what changes in the course of history. I mean, all their technology stuff is of course horribly out of date. But most of their stuff about the human condition is still just as valid as then, although they certainly had a lot more focus on mortality than we do today.

And today one interesting change is that we really need to attribute almost person-like internal state to machines. Not least because, as it happens, the early applications of all this everyday discourse stuff will be to things we’re building for people talking to consumer devices and cars and so on.

I could talk some about very practical here-and-now technology that’s actually going to be available starting next week, for making what we call PLIs, or Programmable Linguistic Interfaces. But instead let’s talk more about the big picture, and about the future.

The way I see things, throughout history there’s been a thread of using technology to automate more and more of what we do. Humans define goals, and then it’s the job of technology to automatically achieve those goals as well as possible.

A lot of what we’re trying to with the Wolfram Language is in effect to give people a good way to describe goals. Then our job is to do the computations—or make the external API requests or whatever—to have those goals be achieved.

What Will the AIs Do?

So, in any possible computational definition of the objectives of AI, we’re getting awfully close to achieving them—and actually, in many areas we’ve gone far beyond anything like human intelligence.

But here’s the point: Imagine we have this box that sits on our desk, and it’s able to do all those intelligent things humans can do. The problem is, what does the box choose to do? Somehow it has to be given goals, or purposes. And the point is that there are no absolute goals and purposes. Any given human might say, “The purpose of life is to do X”. But we know that there’s nothing absolute about that.

Purpose ends up getting defined by society and history and civilization. There are plenty of things people do or want to do today that would have seemed absolutely inconceivable 300 years ago. It’s interesting to see the complicated interplay between the progress of technology, the progress of our descriptions of the world—through memes and words and so on—and the evolution of human purposes.

To me, the path of technology seems fairly clear. The evolution of human purposes is a lot less clear.

I mean, on the technology side, more and more of what we do ourselves we’ll be able to outsource to machines. We’ve already outsourced lots of mechanical thinking, like say for doing math. We’re well on the way to outsourcing lots of things about memory, and soon also lots of things about judgment.

People might say, “Well, we’ll never outsource creativity.” Actually, some aspects of that are among the easier things to outsource: We can get lots of inspiration for music or art or whatever just by looking out into the computational universe, and it’s only a matter of time before we can automatically combine those things with knowledge and judgment about the human world.

You know, a lot of our use of technology in the past has been “on demand”. But we’re going to see more and more preemptive use—where our technology predicts what we will want to do, then suggests it.

It’s sort of amusing to me when people talk about the machines taking over; here’s the scenario that I think will actually happen. It’s like with GPSs in cars: Most people—like me—just follow what the GPS tells them to do. Similarly, when there’s something that’s saying, you know, “Pick out that food on the menu”, or “Talk to that person in the crowd”, much of the time we’ll just do what the machine tells us to—partly because the machine is basically able to figure out a lot more than we can.

It’s going to get complicated when machines are acting collectively across a whole society, effectively implementing in software all those things that political philosophers have talked about theoretically. But even at an individual level, it’s very complicated to understand the goal structure.

Yes, the machines can help us “be ourselves, but better”, amplifying and streamlining things we want to do and directions we want to go.

Immortality & Beyond

You know, in the world today, we’ve got a lot of scarce resources. In many parts of the world much less scarce than in the past, but some resources are still scarce. The most notable is probably time. We have finite lives, and that’s a key part of lots of aspects of human motivation and purpose.

It’s surely going to be the biggest discontinuity ever in human history when we achieve effective human immortality. And, by the way, I have absolutely no doubt that we will achieve it. I just wish more progress would get made on things like cryonics to give my generation a better probability of making it to that.

But it’s not completely clear how effective immortality will happen. I think there are several paths; that will probably in practice be combined. The first is that we manage to reverse-engineer enough of biology to put in patches that keep us running biologically indefinitely. That might turn out to be easy, but I’m concerned that it’ll be like trying to keep a server that’s running complex software up forever—which is something for which we have absolutely no theoretical framework, and lots of potential to run into undecidable halting problems and things like that.

The second immortality path is in effect uploading to some kind of engineered digital system. And probably this is something that would happen gradually. First we’d have digital systems that are directly connected to our brains, then these would use technology—perhaps not that different from the ImageIdentify function I was showing you—to start learning from our brains and the experiences we have, and taking more and more of the “cognitive load”… until eventually we’ve got something that responds exactly the same as our brain does.

Once we’ve got that, we’re dealing with something that can evolve quite quickly, independent of the immediate constraints of physics and chemistry, and that can for example explore different parts of the computational universe—inevitably sampling parts that are far from what we as humans currently understand.

Box of a Trillion Souls

So, OK, what is the end state? I often imagine some kind of “box of a trillion souls” that’s sort of the ultimate repository for our civilization. Now at first we might think, “Wow, that’s going to be such an impressive thing, with all that intelligence and consciousness and knowledge and so on inside.” But I’m afraid I don’t think so.

You see, one of the things that emerged from my basic science is what I call the Principle of Computational Equivalence, which says in effect that beyond some low threshold, all systems are equivalent in the sophistication of the computations they do. So that means that there’s not going to be anything abstractly spectacular about the box of a trillion souls. It’ll just be doing a computation at the same level as lots of systems in the universe.

Maybe that’s why we don’t see extraterrestrial intelligence: because there’s nothing abstractly different about something that came from a whole elaborate civilization as compared to things that are just happening in the physical world.

Now, of course, we can be proud that our box of a trillion souls is special, because it came from us, with our detailed history. But will interesting things be happening in it? Well, to define “interesting” we then need a sense of purpose—so things become pretty circular, and it’s a complicated philosophy discussion.

Back to 2015

I’ve come pretty far from talking about practical things for 2015. The way I like to work, understanding these fundamental issues is pretty important in not making mistakes in building technology here and now, because that’s how I’ve figured out one can build the best technology. And right now I’m really excited with the point we’ve reached with the Wolfram Language.

I think we’ve defined a new level of technology to support computational thinking, that I think is going to let people rather quickly do some very interesting things—going from algorithmic ideas to finished apps or new companies or whatever. The Wolfram Cloud and things around it are still in beta right now, but you can certainly try them out—and I hope you will. It’s really easy to get started—though, not surprisingly, because there are actually new ideas, there are things to learn if you really want to take the best advantage of this technology.

Well, that’s probably all I have to say right now. I hope I’ve been able to communicate a few of the exciting things that we’ve got going on right now, and a few of the things that I think are new and emerging in computational thinking and the technology around it. So, thanks very much.

The Wolfram Cloud Needs to Be Perfect

The Wolfram Cloud is coming out of beta soon (yay!), and right now I’m spending much of my time working to make it as good as possible (and, by the way, it’s getting to be really great!). Mostly I concentrate on defining high-level function and strategy. But I like to understand things at every level, and as a CEO, one’s ultimately responsible for everything. And at the beginning of March I found myself diving deep into something I never expected…

Here’s the story. As a serious production system that lots of people will use to do things like run businesses, the Wolfram Cloud should be as fast as possible. Our metrics were saying that typical speeds were good, but subjectively when I used it something felt wrong. Sometimes it was plenty fast, but sometimes it seemed way too slow.

We’ve got excellent software engineers, but months were going by, and things didn’t seem to be changing. Meanwhile, we’d just released the Wolfram Data Drop. So I thought, why don’t I just run some tests myself, maybe collecting data in our nice new Wolfram Data Drop?

A great thing about the Wolfram Language is how friendly it is for busy people: even if you only have time to dash off a few lines of code, you can get real things done. And in this case, I only had to run three lines of code to find a problem.

First, I deployed a web API for a trivial Wolfram Language program to the Wolfram Cloud:

Then I called the API 50 times, measuring how long each call took (% here stands for the previous result):

Then I plotted the sequence of times for the calls:

And immediately there seemed to be something crazy going on. Sometimes the time for each call was 220 ms or so, but often it was 900 ms, or even twice that long. And the craziest thing was that the times seemed to be quantized!

I made a histogram:

And sure enough, there were a few fast calls on the left, then a second peak of slow calls, and a third “outcropping” of very slow calls. It was weird!

I wondered whether the times were always like this. So I set up a periodic scheduled task to do a burst of API calls every few minutes, and put their times in the Wolfram Data Drop. I left this running overnight… and when I came back the next morning, this is what I saw:

Even weirder! Why the large-scale structure? I could imagine that, for example, a particular node in the cluster might gradually slow down (not that it should), but why would it then slowly recover?

My first thought was that perhaps I was seeing network issues, given that I was calling the API on a test cloud server more than 1000 miles away. So I looked at ping times. But apart from a couple of weird spikes (hey, it’s the internet!), the times were very stable.

Something’s Wrong inside the Servers

OK, so it must be something on the servers themselves. There’s a lot of new technology in the Wolfram Cloud, but most of it is pure Wolfram Language code, which is easy to test. But there’s also generic modern server infrastructure below the Wolfram Language layer. Much of this is fundamentally the same as what Wolfram|Alpha has successfully used for half a dozen years to serve billions of results, and what webMathematica started using even nearly a decade earlier. But being a more demanding computational system, the Wolfram Cloud is set up slightly differently.

And my first suspicion was that this different setup might be causing something to go wrong inside the webserver layer. Eventually I hope we’ll have pure Wolfram Language infrastructure all the way down, but for now we’re using a webserver system called Tomcat that’s based on Java. And at first I thought that perhaps the slowdowns might be Java garbage collection. Profiling showed that there were indeed some “stop the world” garbage-collection events triggered by Tomcat, but they were rare, and were taking only milliseconds, not hundreds of milliseconds. So they weren’t the explanation.

By now, though, I was hooked on finding out what the problem was. I hadn’t been this deep in the trenches of system debugging for a very long time. It felt a lot like doing experimental science. And as in experimental science, it’s always important to simplify what one’s studying. So I cut out most of the network by operating “cloud to cloud”: calling the API from within the same cluster. Then I cut out the load balancer, that dispatches requests to particular nodes in a cluster, by locking my requests to a single node (which, by the way, external users can’t do unless they have a Private Cloud). But the slowdowns stayed.

So then I started collecting more-detailed data. My first step was to make the API return the absolute times when it started and finished executing Wolfram Language code, and compare those to absolute times in the wrapper code that called the API. Here’s what I saw:

The blue line shows times before the Wolfram Language code is run; the gold line after. I collected this data in a period when the system as a whole was behaving pretty badly. And what I saw was lots of dramatic slowdowns in the “before” times—and just a few quantized slowdowns in the “after” times.

Once again, this was pretty weird. It didn’t seem like the slowdowns were specifically associated with either “before” or “after”. Instead, it looked more as if something was randomly hitting the system from the outside.

One confusing feature was that each node of the cluster contained (in this case) 8 cores, with each core running a different instance of the Wolfram Engine. The Wolfram Engine is nice and stable, so each of these instances was running for hours to days between restarts. But I wondered if perhaps some instances might be developing problems along the way. So I instrumented the API to look at process IDs and process times, and then for example plotted total process time against components of the API call time:

And indeed there seemed to be some tendency for “younger” processes to run API calls faster, but (particularly noting the suppressed zero on the x axis) the effect wasn’t dramatic.

What’s Eating the CPU?

I started to wonder about other Wolfram Cloud services running on the same machine. It didn’t seem to make sense that these would lead to the kind of quantized slowdowns we were seeing, but in the interest of simplifying the system I wanted to get rid of them. At first we isolated a node on the production cluster. And then I got my very own Wolfram Private Cloud set up. Still the slowdowns were there. Though, confusingly, at different times and on different machines, their characteristics seemed to be somewhat different.

On the Private Cloud I could just log in to the raw Linux system and start looking around. The first thing I did was to read the results from the “top” and “ps axl” Unix utilities into the Wolfram Language so I could analyze them. And one thing that was immediately obvious was that lots of “system” time was being used: the Linux kernel was keeping very busy with something. And in fact, it seemed like the slowdowns might not be coming from user code at all; they might be coming from something happening in the kernel of the operating system.

So that made me want to trace system calls. I hadn’t done anything like this for nearly 25 years, and my experience in the past had been that one could get lots of data, but it was hard to interpret. Now, though, I had the Wolfram Language.

Running the Linux “strace” utility while doing a few seconds of API calls gave 28,221,878 lines of output. But it took just a couple of lines of Wolfram Language code to knit together start and end times of particular system calls, and to start generating histograms of system-call durations. Doing this for just a few system calls gave me this:

Interestingly, this showed evidence of discrete peaks. And when I looked at the system calls in these peaks they all seemed to be “futex” calls—part of the Linux thread synchronization system. So then I picked out only futex calls, and, sure enough, saw sharp timing peaks—at 250 ms, 500 ms and 1s:

But were these really a problem? Futex calls are essentially just “sleeps”; they don’t burn processor time. And actually it’s pretty normal to see calls like this that are waiting for I/O to complete and so on. So to me the most interesting observation was actually that there weren’t other system calls that were taking hundreds of milliseconds.

The OS Is Freezing!

So… what was going on? I started looking at what was happening on different cores of each node. Now, Tomcat and other parts of our infrastructure stack are all nicely multithreaded. Yet it seemed that whatever was causing the slowdown was freezing all the cores, even though they were running different threads. And the only thing that could do that is the operating system kernel.

But what would make a Linux kernel freeze like that? I wondered about the scheduler. I couldn’t really see why our situation would lead to craziness in a scheduler. But we looked at the scheduler anyway, and tried changing a bunch of settings. No effect.

Then I had a more bizarre thought. The instances of the Wolfram Cloud I was using were running in virtual machines. What if the slowdown came from “outside The Matrix”? I asked for a version of the Wolfram Cloud running on bare metal, with no VM. But before that was configured, I found a utility to measure the “steal time” taken by the VM itself—and it was negligible.

By this point, I’d been spending an hour or two each day for several days on all of this. And it was time for me to leave for an intense trip to SXSW. Still, people in our cloud-software engineering team were revved up, and I left the problem in their capable hands.

By the time my flight arrived there was already another interesting piece of data. We’d divided each API call into 15 substeps. Then one of our physics-PhD engineers had compared the probability for a slowdown in a particular substep (on the left) to the median time spent in that substep (on the right):

With one exception (which had a known cause), there was a good correlation. It really looked as if the Linux kernel (and everything running under it) was being hit by something at completely random times, causing a “slowdown event” if it happened to coincide with the running of some part of an API call.

So then the hunt was on for what could be doing this. The next suspicious thing noticed was a large amount of I/O activity. In the configuration we were testing, the Wolfram Cloud was using the NFS network file system to access files. We tried tuning NFS, changing parameters,  going to asynchronous mode, using UDP instead of TCP, changing the NFS server I/O scheduler, etc. Nothing made a difference. We tried using a completely different distributed file system called Ceph. Same problem. Then we tried using local disk storage. Finally this seemed to have an effect—removing most, but not all, of the slowdown.

We took this as a clue, and started investigating more about I/O. One experiment involved editing a huge notebook on a node, while running lots of API calls to the same node:

The result was interesting. During the period when the notebook was being edited (and continually saved), the API times suddenly jumped from around 100 ms to 500 ms. But why would simple file operations have such an effect on all 8 cores of the node?

The Culprit Is Found

We started investigating more, and soon discovered that what seemed like “simple file operations” weren’t—and we quickly figured out why. You see, perhaps five years before, early in the development of the Wolfram Cloud, we wanted to experiment with file versioning. And as a proof of concept, someone had inserted a simple versioning system named RCS.

Plenty of software systems out there in the world still use RCS, even though it hasn’t been substantially updated in nearly 30 years and by now there are much better approaches (like the ones we use for infinite undo in notebooks). But somehow the RCS “proof of concept” had never been replaced in our Wolfram Cloud codebase—and it was still running on every file!

One feature of RCS is that when a file is modified even a tiny bit, lots of data (even several times the size of the file itself) ends up getting written to disk. We hadn’t been sure how much I/O activity to expect in general. But it was clear that RCS was making it needlessly more intense.

Could I/O activity really hang up the whole Linux kernel? Maybe there’s some mysterious global lock. Maybe the disk subsystem freezes because it doesn’t flush filled buffers quickly enough. Maybe the kernel is busy remapping pages to try to make bigger chunks of memory available.  But whatever might be going on, the obvious thing was just to try taking out RCS, and seeing what happened.

And so we did that, and lo and behold, the horrible slowdowns immediately went away!

So, after a week of intense debugging, we had a solution to our problem. And repeating my original experiment, everything now ran cleanly, with API times completely dominated by network transmission to the test cluster:

The Wolfram Language and the Cloud

What did I learn from all this? First, it reinforced my impression that the cloud is the most difficult—even hostile—development and debugging environment that I’ve seen in all my years in software. But second, it made me realize how valuable the Wolfram Language is as a kind of metasystem, for analyzing, visualizing and organizing what’s going on inside complex infrastructure like the cloud.

When it comes to debugging, I myself have been rather spoiled for years—because I do essentially all my programming in the Wolfram Language, where debugging is particularly easy, and it’s rare for a bug to take me more than a few minutes to find. Why is debugging so easy in the Wolfram Language? I think, first and foremost, it’s because the code tends to be short and readable. One also typically writes it in notebooks, where one can test out, and document, each piece of a program as one builds it up. Also critical is that the Wolfram Language is symbolic, so one can always pull out any piece of a program, and it will run on its own.

Debugging at lower levels of the software stack is a very different experience. It’s much more like medical diagnosis, where one’s also dealing with a complex multicomponent system, and trying to figure out what’s going on from a few measurements or experiments. (I guess our versioning problem might be the analog of some horrible defect in DNA replication.)

My whole adventure in the cloud also very much emphasizes the value we’re adding with the Wolfram Cloud. Because part of what the Wolfram Cloud is all about is insulating people from the messy issues of cloud infrastructure, and letting them instead implement and deploy whatever they want directly in the Wolfram Language.

Of course, to make that possible, we ourselves have needed to build all the automated infrastructure. And now, thanks to this little adventure in “scientific debugging”, we’re one step closer to finishing that. And indeed, as of today, the Wolfram Cloud has its APIs consistently running without any mysterious quantized slowdowns—and is rapidly approaching the point when it can move out of beta and into full production.

My goal with the Wolfram Language is to take programming to a new level. And over the past year we’ve been rolling out ways to use and deploy the language in many places—desktop, cloud, mobile, embedded, etc. So what about wearables? And in particular, what about the Apple Watch? A few days ago I decided to explore what could be done. So I cleared my schedule for the day, and started writing code.

My idea was to write code with our standard Wolfram Programming Cloud, but instead of producing a web app or web API, to produce an app for the Apple Watch. And conveniently enough, a preliminary version of our Wolfram Cloud app just became available in the App Store—letting me deploy from the Wolfram Cloud to both mobile devices and the watch.

To some extent it was adventure programming. The Apple Watch was just coming out, and the Wolfram Cloud app was still just preliminary. But of course I was building on nearly 30 years of progressive development of the Wolfram Language. And I’m happy to say that it didn’t take long for me to start getting interesting Wolfram Language apps running on the watch. And after less than a day of work—with help from a handful of other people—I had 25 watch-ready apps:

All of these I built by writing code in the Wolfram Programming Cloud (either on the web or the desktop), then deploying to the Wolfram Cloud, and connecting to the Apple Watch via the Wolfram Cloud app. And although the apps were designed for the Apple Watch, you can actually also use them on the web, or on a phone. There are links to the web versions scattered through this post. To get the apps onto your phone and watch, just go to this page and follow the instructions. That page also has all the Wolfram Language source code for the apps, and you can use any Wolfram Language system—Wolfram Programming Cloud (including the free version), Mathematica etc.—to experiment with the code for yourself, and perhaps deploy your own version of any of the apps.

My First Watch App

So how does it all work? For my first watch-app-writing session, I decided to start by making a tiny app that just generates a single random number. The core Wolfram Language code to do that is simply:

For the watch we want the number to look nice and bold and big, and it might as well be a random color:

We can immediately deploy this publicly to the cloud by saying:

And if you go to that URL in any web browser, you’ll get to a minimal web app which immediately gives a web page with a random number. (The Delayed in the code says to delay the computation until the moment the page is accessed or refreshed, so you get a fresh random number each time.)

So what about getting this to the Apple Watch? First, it has to get onto an iPhone. And that’s easy. Because anything that you’ve deployed to the Wolfram Cloud is automatically accessible on an iPhone through the Wolfram Cloud app. To make it easy to find, it’s good to add a recognizable name and icon. And if it’s ultimately headed for the watch, it’s good to put it on a black background:

And now if you go to this URL in a web browser, you’ll find a public version of the app there. Inside the Wolfram Cloud app on an iPhone, the app appears inside the WatchApps folder:

And now, if you touch the app icon, you’ll run the Wolfram Language code in the Wolfram Cloud, and back will come a random number, displayed on the phone:

If you want to run the app again, and get a fresh random number, just pull down from the top of the phone.

To get the app onto the watch, go back to the listing of apps, then touch the watch icon at the top and select the app. This will get the app listed on the watch that’s paired with your phone:

Now just touch the entry for the RandomNumber app and it’ll go to the Wolfram Cloud, run the Wolfram Language code, and display a random number on the watch:

Randomness Apps

It’s simple to make all sorts of “randomness apps” with the Wolfram Language. Here’s the core of a Coin Flip app:

And this is all it takes to deploy the app, to the web, mobile and watch:

One might argue that it’s overkill to use our sophisticated technology stack to do this. After all, it’s easy enough to flip a physical coin. But that assumes you have one of those around (which I, for one, don’t any more). Plus, the Coin Flip app will make better randomness.

What about playing Rock, Paper, Scissors with your watch? The core code for that is again trivial:

There’s a huge amount of knowledge built in to the Wolfram Language—including, in one tiny corner, the knowledge to trivially create a Random Pokemon app:

Here it is running on the watch:

Let’s try some slightly more complex Wolfram Language code. Here’s a Word Inventor that makes a “word” by alternating random vowels and consonants (and often the result sounds a lot like a Pokemon, or a tech startup):

Watches Tell Time

If nothing else, one thing people presumably want to use a watch for is to tell time. And since we’re in the modern internet world, it has to be more fun if there’s a cat or two involved. So here’s the Wolfram Language code for a Kitty Clock:

Which on the watch becomes:

One can get pretty geeky with clocks. Remembering our recent very popular My Pi Day website, here’s some slightly more complicated code to make a Pi Clock where the digits of the current time are displayed in the context where they first occur in pi:

Or adding a little more:

Where Are You?

So long as you enable it, the Apple Watch uses GPS, etc. on its paired phone to know where you are. That makes it extremely easy to have a Lat-Long app that shows your current latitude and longitude on the watch (this one is for our company HQ):

I’m not quite sure why it’s useful (prove location over Skype?), but here’s a Here & Now QR app that shows your current location and time in a QR code:

Among the many things the Wolfram Language knows a lot about is geography. So here’s the code to find the ten volcanoes closest to you:

A little more code shows them on a map, and constructs a Nearest Volcanoes app:

Here’s the code for a 3D Topography app, that shows the (scaled) 3D topography for 10 miles around your location:

Data Flows In

Since the watch communicates with the Wolfram Cloud, it can make use of all the real-time data that’s flowing into the Wolfram Knowledgebase. That data includes things like the current (x,y,z,t) position of the International Space Station:

Given the position, a little bit of Wolfram Language graphics programming gives us an ISS Locator app:

As another example of real-time data, here’s the code for an Apple Quanting app that does some quant-oriented computations on Apple stock:

And here’s the code for a Market Word Cloud app that shows a stock-symbols word cloud weighted by fractional price changes in the past day (Apple up, Google down today):

Here’s the complete code for a geo-detecting Currency Converter app:

It’s easy to make so many apps with the Wolfram Language. Here’s the core code for a Sunrise/Sunset app:

Setting up a convenient display for the watch takes a little more code:

The Wolfram Language includes real-time weather feeds:

Which we can also display iconically:

Here’s the data for the last week of air temperatures:

And with a little code, we can format this to make a Temperature History app:

Sometimes the easiest way to get a result in the Wolfram Language is just to call Wolfram|Alpha. Here’s what Wolfram|Alpha shows on the web if you ask about the time to sunburn (it detects your current location):

Now here’s a real-time Sunburn Time app created by calling Wolfram|Alpha through the Wolfram Language (the different rows are for different skin tones):

Reports & Data Drops

The Wolfram Language has access not only to all its own curated data feeds, but also to private data feeds, especially ones in the Wolfram Data Drop.

As a personal analytics enthusiast, I maintain a databin in the Wolfram Data Drop that tells me my current backlog of unprocessed and unread email messages. I have a scheduled task that runs in the cloud and generates a report of my backlog history. And given this, it’s easy to have an SW Email Backlog app that imports this report on demand, and displays it on a watch:

And, yes, the recent increase in unprocessed and unread email messages is at least in part a consequence of work on this blog.

There are now lots of Wolfram Data Drop databins around, and of course you can make your own. And from any databin you can immediately make a watch app that shows a dashboard for it. Like here’s a Company Fridge app based on a little temperature sensor sitting in a break-room refrigerator at our company HQ (the cycling is from the compressor; the spike is from someone opening the fridge):

Databins often get data from just a single source or single device. But one can also have a databin that gets data from an app running on lots of different devices.

As a simple example, let’s make an app that just shows where in the world that app is being accessed from. Here’s the complete code to the deploy such a “Data Droplets” app:

The app does two things. First, whenever it’s run, it adds the geo location of the device that’s running it to a central databin in the Wolfram Data Drop. And second, it displays a world map that marks the last 20 places in the world where the app has been used:

Making Things Happen

A typical reason to run an app on the watch is to be able to see results right on your wrist. But another reason is to use the app to make things happen externally, say through APIs.

As one very simple example, here’s the complete code to deploy an app that mails the app’s owner a map of a 1-mile region around wherever they are when they access the app:

Apps to Generate Apps

So far, all the apps we’ve talked about are built from fixed pieces of Wolfram Language code that get deployed once to the Apple Watch. But the Wolfram Language is symbolic, so it’s easy for it to manipulate the code of an app, just like it manipulates any other data. And that means that it’s straightforward to use the Wolfram Language to build and deploy custom apps on the fly.

Here’s a simple example. Say we want to have an app on the watch that gives a countdown of days to one’s next birthday. It’d be very inconvenient to have to enter the date of one’s birthday directly on the watch. But instead we can have an app on the phone where one enters one’s birthday, and then this app can in real time build a custom watch app that gives the countdown for that specific birthday.

Here we enter a birthday in a standard Wolfram Language “smart field” that accepts any date format:

And as soon as we touch Submit, this app runs Wolfram Language code in the Wolfram Cloud that generates a new custom app for whatever birthday we entered, then deploys that generated app so it shows up on our watch:

Here’s the complete code that’s needed to make the Birthday Countdown app-generating app.

And here is the result from the generated countdown app for my birthday:

We can make all sorts of apps like this. Here’s a World Clocks example where you fill out a list of any number of places, and create an app that displays an array of clocks for all those places:

You can also use app generation to put you into an app. Here’s the code to deploy a “You Clock” app-generating app that lets you take a picture of yourself with your phone, then creates an app that uses that picture as the hands of a clock:

And actually, you can easily go even more meta, and have apps that generate apps that generate apps: apps all the way down!

More Than I Expected

When I set out to use the Wolfram Language to make apps for the Apple Watch I wasn’t sure how it would go. Would the deployment pipeline to the watch work smoothly enough? Would there be compelling watch apps that are easy to build in the Wolfram Language?

I’m happy to say that everything has gone much better than I expected. The watch is very new, so there were a few initial deployment issues, which are rapidly getting worked out. But it became clear that there are lots and lots of good watch apps that can be made even with tiny amounts of Wolfram Language code (tweet-a-watch-app?). And to me it’s very impressive than in less than one full day’s work I was able to develop and deploy 25 complete apps.

Of course, what ultimately made this possible is the whole Wolfram Language technology stack that I’ve been building for nearly 30 years. But it’s very satisfying to see all the automation we’ve built work so nicely, and make it so easy to turn ideas into yet another new kind of thing: watch apps.

It’s always fun to program in the Wolfram Language, and it’s neat to see one’s code deployed on something like a watch. But what’s ultimately more important is that it’s going to be very useful to lots of people for lots of purposes. The code here is a good way to get started learning what to do. But there are many directions to go, and many important—or simply fun—apps to create. And the remarkable thing is that the Wolfram Language makes it so easy to create watch apps that they can become a routine part of everyday workflow: just another place where functionality can be deployed.

To comment, please visit the copy of this post at the Wolfram Blog »