View transcript
- Susan Schwartzenberg, who's here, invited my friend Scott Snibbe and I to do a project in 2005 called Cabspotting, which was about the... part of an invisible dynamics program that she was starting that was all about ways of representing data that were invisible, and basically making things that were ineffable and difficult to understand accessible to the public. So, in that capacity, I'm here again, and I just want to thank you, Susan, for believing in me all that time ago. Really meant a lot and means a lot, and I'm just grateful to be here. So I wanna talk about objects to think with, and talk about some of the projects that we've been putting together and some thoughts about the future of data visualization. I also wanna welcome you to San Francisco. This is a picture of me on Friday night here at the Exploratorium. We do things a little differently here. So I'm just glad to be in this room with you all. So, just jumping straight into Cabspotting, and I'll go quickly. And I apologize, I have some things I want to tell you. Cabspotting is, as I said, a project that Susan Schwartzenberg invited Scott Snibbe and I to put together. It's a map of taxicabs in San Francisco. It was made in 2005 when the idea of a live, real-time map of California and San Francisco was a kind of radical idea. You could see all the cabs. You could see where they were. You could see who was empty, who was full. The project had a good, long run. This is part of the project that we put together for inclusion in the Museum of Modern Art in New York, where it's part of the permanent collection. So we've got a nice San Francisco-New York connection going on here. It's been shown in Shanghai and Chicago and other places. It also started for me a kind of engagement with this idea that data could be used for multiple- The same data could be used in multiple ways for multiple reasons. This is a project that, part of the project that Eddie Elliott put together, who's known to some of you here at the Exploratorium. It's 31 days of taxi data in San Francisco. And it's sparked some interesting conversation. So, this idea that there would be the official version, and then that there would be the museum version, and then there would be the version that the artists could make. This idea that there would be APIs to data was a new idea. So, it introduced some kind of fun things. This is, again, 31 day of the data. He just plotted it, really simple, but you can see the freeways are really clearly outlined. You can see that there's that big blob on the righthand side on the bottom. That's the Yellow Cab parking lot where the taxis go to hang out. The freeways are clearly defined because there's not a lot of interference, and everybody's driving the same way. You can see, though, if you start to look at the downtown area that the lines get kind of fuzzy and broken apart, and this doesn't mean that the taxi drivers are driving onto the roofs of all these giant apartment buildings. It means that GPS, as it bounces back and forth between large buildings gets all fuzzy. I just love this idea, and I've latched onto it, and I haven't let it go, this idea that you can have this object to think with. That's it not just about a clear and accurate representation of the data, but that the messiness and the fuzziness can start to become interesting, and that you can use your eyes to explore new kinds of ideas, and that really the fuzzy parts are just as important as the crisp parts. The errors are just as important as the clean parts, and maybe they're more interesting. Maybe the errors, maybe the gaps are the places where we can find interesting work to do. Another thing that I'm proud about with the project is that it was used as part of Uber's pitch deck. This is slide number 16 of the initial finance round that Uber used to raise all its money. So, Susan, maybe we should go talk to those guys at some point. Yeah. Moving forward 15 years to a recent project that we put together for the Victoria and Albert Museum. This is the the Big Glass Microphone. It's another project about data and infrastructure, and it's less about, kind of, things that are moving around sensors, but more sensors that are embedded in the ground. So this is a five-kilometer long fiber optic cable that's buried underneath Stanford University. And what the researchers there have discovered is that if you attach a computer to one end of a fiber optic cable and you measure the vibrations in the cable from the signal that goes through it, that you can make really, really hyper-accurate gigantic maps of very, very long distances. So this is being used for mostly seismic analysis. But what they've found is that if you measure those parts of the cable that are visible to the auditory spectrum, you can start to find things, and if you can also find things that are less visible to the auditory spectrum. So this, for example, is a visualization of an electric car driving by. It's not audible in any way, but it sends out an electromagnetic impulse that the cable can pick up. And so what we've got here now is this possibility of not just sensors kind of driving around, but also these kind of massively scaled sensors that are embedded in the basic fabric of our communications infrastructure. There are fiber optic cables underneath every major road in the United States, and what we're hearing from the researchers is that, based on the signals passing through a fiber optic cable, they can tell the difference between a Ford and a Volvo. They can tell the difference between a Volvo with two people in it and a Volvo with one person in it at scale, in real time, always throughout the universe. So, we're in an interesting time. We're in a time when the basic fabric of our infrastructure is not just about sending signals from one place to the next. It's also about a sensing device. And so kind of catapulting another year forward, I wanna mention a project that came out of MIT late last year. It's about using smartphones and cameras to see around corners and to detect things in the world, and it's kind of freaking me out. So what they're basically doing is pointing cameras at shadows that are around corners, and using those to find out what's around the corner. So it started with this scene in a hotel room where they basically- You can see it on slides two and three and four, basically, they're using the windows in the room as a giant pinhole camera. And by emphasizing the contrast and using machine learning, they're basically able to construct a photograph of what's outside the window from the reflection of the inside of the building. They take this further. This is about being able to detect the movement of people and other things around the corner from a space. So there's a camera on the upper left-hand side there, and then there's, they basically are taking pictures of the shadows that are reflecting around and being able to build visualizations and maps of what they can't see. This gets worse. This is a- The same researchers took cameras and pointed them at potato chips that had speakers playing on them, and based on the vibrations of the potato chip bags and the napkins were able to construct the sound that was being played on the speaker from the vibrations on the potato chip bag. There's a whole taxonomy of objects that can be used to detect things that you're seeing. When we first started talking about Cabspotting, people were really nervous that, you know, the location of the taxi was gonna be used to tell something secret and private about your life, and I'm here to show you some slides that say that the potato chip bags are microphones. I think we're hurtling, like, really really quickly into this kind of crazy future where, you know, essentially all bets are off. Honda is giving cars the ability to see around corners to avoid accidents, so we're kind of coming full circle from Cabspotting, where there's this kind of network of cabs that's being visualized into a world where the cabs are all sensors themselves, and they're seeing around corners, and basically the entire world is being wired up into this gigantic... It's not just about sort of finding... Basically the whole entire world is a gigantic sensing mechanism, and the whole world is a big data visualization collection machine. And so, at this point the world is the network, and the network is the world. The whole thing kind of collapses into one frame. And I think this is something that we need to consider as visualization practitioners, that the amount of data is kind of off the chain. This is a photograph of a bird flying in front of a nest-cam, and in some ways the bird- You see the- It's only taking photographs when the bird's wings are down. This is a visualization of the frame-rate of the camera. This stuff is all over the place. It's gonna get worse and better, and it's just gonna fly off the handle. So, what do we do now? I think this is important for us to sort of deal with the kind of vertigious nature of what's happening. I mean, it's not just about a cab here and a cab there. It's about the whole world, everything in it, being used as a sensing device, and nothing ever going away, ever. So, I started digging around and into kind of where the future is going and where the kind of optimistic dreams that I certainly had around getting involved and engaged in the internet, and I started looking into papers, and this one- I've got two things to reference here. One is a paper put together by the National Mining Association and the American Coalition for Clean Coal Electricity, and it's called "The Cloud Begins with Coal." So all the stuff that we're putting onto the internet, all the stuff that's going into the cloud is powered by these gigantic infrastructures. And I'll just start with this one sentence. So, "although charging up a single tablet or smartphone requires a negligible amount of electricity, using either to watch an hour of video weekly consumes annually more electricity in the remote networks than two new refrigerators use in a year." So those cat videos are expensive. I watched "Chinatown" on Netflix last night, and I basically bought myself a new refrigerator for a year. It's a lot. And it's, I think it's important that we deal with this in an intense way. This is a really intense situation. Look at the amount of electricity that is being consumed by the cloud is greater than the total electrical output of the country of India, and it's not slowing down. We're kind of in the middle between those two tics, so we're between 2010 and 2035, and all the electricity that we're using even to have this conversation is happening in the cloud, and this is all about coal. And it's not gonna slow down any time soon. When you upload your keynote to come to the Exploratorium so that it can be downloaded, you're using coal. I did mine onto Google, which is about midway, or towards the bottom there, which is good, but the next time you download something from Apple, think about the fact that 60% of the coal- 60% of the energy that's used to run iTunes is based on coal. A lot of coal. Here's another little nugget from that paper. "Hourly internet traffic will soon exceed the annual traffic of the year 2000." And "historically, demand for bits has grown faster than the energy efficiency of using them." So if history is any guide, we're not gonna have some gigantic breakthrough in efficiency. This is gonna just continue. So, my question to us is does anybody think this is gonna slow down? You know, this whole sort of great, fabulous journey that we're on with the internet and visualization. Does anybody think that this is being planned in any kind of central way? It doesn't seem like it is to me. It's just kind of happening. And so what are we supposed to do, right? So this is sort of a dark vision, and I'm sorry, but I'm among friends, and so I have to tell the truth. I started reading this great book that came out earlier this year called "The New Dark Age" by James Bridle, who's a really terrific artist. I encourage you to look him up, encourage you to buy this book. I have a copy in my bag if you wanna read it, or borrow it I guess, I'd like to keep it. There's a bunch here, so I hope you'll bear with me, and again, I'm bringing out some new things to talk to friends and people that I care about. The basic idea is that we've kind of had, for a long time, this idea that the more data that we had and the better models we had, the more better things would get. But his premise is that "as the world is increasing in technological complexity, our understanding of it diminishes," right? This idea that "our existence is understandable through computation" is an ideology, and one that I'm firmly in the grip of, but I think what we're starting to see is that "we're lost in a sea of information," and that we're increasingly divided by these things like fundamentalism and conspiracy theories, and you know, all this stuff. So, again, this idea that, sort of, the more... My faith has been, and the reason that I grabbed onto visualization with my whole everything 20 years ago, was because I had this faith that if we could kind of get more data and get better models, we could make the world better, and that belief has been tested in the last little while. So he's got these great nuggets in the book. I really encourage you read it. It's accessible. He's got things like, "Thinking about climate change is degraded by climate change itself." There's a whole chapter in the book about how, you know, we think that our infrastructure is kind of solid and unmoving, but it's actually not. The project of using data, and so this is my assertion, is that the project of using data and data-vis to clearly and objectively inform the world about climate change has failed. We are not in a position where we can speak clearly about these things. I'm being asked to wrap up, so I wanna just get straightly to this idea that we're not thinking the network. We're not thinking the world. This is, I'll just jump straight into this. "When one has a hammer, goes the saying, everything looks like a nail." That's the saying, right? You've got a hammer, everything looks like a nail, and I wanna problematize that a little bit, so just give you this last little metaphor. "This is not to think the hammer. The hammer, properly conceived, has many uses. It can pull nails, drive them, forge iron, shape wood, reveal fossils, fix anchors." That idea that "prehistoric hammers and axes, turned up by the plows of later generations were called thunderstones. Basically, they're original purposes passed away. They were capable of taking on new symbolic meaning. I think this is something that we need to think about and have a conversation about. That the ways we've been thinking about data visualization is the way- is thinking about is as a hammer with a nail. And I think that there's this idea that we can re-enchant our hammers, that we can re-enchant our tools so that they're less like carpenter's hammers and more like Thor's hammers, because I think the project of using the hammer to beat the nail of climate change in the face of this kind of radical new situation where the whole world is being electrified and everything is a network is doomed to fail. And so my great gratitude of having been involved with the Exploratorium all this is time is to have been introduced to this idea that the fuzzy bits are the places, maybe, where we can do this most good. That it's not just about accurate representations, but it's about thunderstones. It's about magic. And it's about re-enchanting our tools. So I would encourage us all to try to re-enchant our tools. Thank you again for having me.