View transcript
- [Woman] Here, place your card.

- Yeah.

- Thank you for joining us tonight for After Dark Online, Representation Through Visualization. My name is Kathleen Maguire and I'm part of the team that puts on these weekly After Dark programs. While tonight's program has been recorded remotely I would like to acknowledge that the home of After Dark, the Exploratorium at Pier 15 in San Francisco, is located on unseated territory, traditionally belonging to the Ramaytush Ohlone people. We pay our respects to elders both past and present for their care-taking of this land. Throughout our programs in February we'll be honoring Black History Month by sharing histories from Black communities and cutting edge work from Black scientists, historians, artists and community leaders. Tonight's program considers representation in visual systems through two distinct lenses, graphic design and technologies driven by machine learning. Later on, we'll be hearing from Deb Raji who researches bias in machine learning driven systems with a particular focus on facial recognition technologies. In her talk she'll be sharing some of this groundbreaking research as well as advocating for the need for algorithmic justice and to adopt a guiding principle of "If it doesn't work for everyone, it doesn't work" as these systems are deployed. Up first, a conversation with graphic designer and educator, Silas Munro about his recently developed class Black Design in America and his thoughts on the power of graphic design as well as the many pieces that are missing from dominant design history. Silas Munro engages in practices that that inspire people to elevate themselves and improve society. His design studio, poly-mode has designed identities and publications for exhibitions of Jacob Lawrence at MoMA and Mark Bradford at the Venice Biennale. His writing has appeared in "Slanted," "the Walker Reader," and the book "W. E. B. Du Bois's Data Portraits: "Visualizing Black America," which is a subject that comes up a little bit in this talk. He is particularly interested in the often unaddressed post-colonial relationship between design and marginalized communities. He is Assistant Professor at Otis College of Art and Design as well as Advisor and Chair Emeritus at Vermont College of Fine Arts. I'm excited that Silas is joined in this conversation by my Exploratorium colleague, Nina Fujikawa. Nina is Lead Graphic Designer at the Exploratorium and she currently focuses on marketing, branding, and exhibition design history. I'll pass this over to Nina and Silas.

- Hey, Silas, thanks so much for joining us. I wanted to start with a quick question about your history. And I was wondering if you could tell us a bit more about yourself and your journey to your career as a graphic designer and educator.

- Sure, it's really nice to meet you Nina and really excited to be here. For me, I was always drawn to libraries growing up as a kid. So there was this little community library the Woodrow Wilson Public Library that was part of my neighborhood in suburban Virginia. And I just was always drawn to how quiet it was and the books like the smell of the books, the physicality of them always just really resonated for me. And as I grew up and was interested in so many subjects there's something about design and art that just kind of made everything make sense. And so now I'm a book designer, a graphic designer and I write about design and I just there's something about that particular object, that's this combination of text and image and even the physicality of it that I think still is very soothing for me to turn the pages of a book. So that's kind of where I got connected to design.

- Yeah, I totally feel you. I feel like I have the very similar just longing and nostalgia for libraries and bookstores and the smell of the glue and the ink and the paper. It definitely was like a big turning point when I realized that, that was something that I could make too. So pretty cool. What was your initial experience of design history and when did you begin to see the missing pieces and missing parts of the history?

- My first experience with design history was, well, there's art history first, right? I think I had an amazing high school art teacher name Mrs Monroe, who was always bringing in examples of amazing artists. And she also had like really amazing calligraphy. So she would put quotes of artists like that she was, you know, drawing basically and like pin them up. And one of her parting gifts for me when I went to RISD was a book by Edward Tufte called "Visual Explanations." She could kind of tell... Yeah, exactly. Tome of information, design history. And, but kind of that, once I declared graphic design as a major and I've always been someone who's been kind of like wasn't sure like am I painter, am I poet? And Doug Scott has a class at RISD that he still teaches, which is kind of amazing because he's actually one of my students-

- Oh, wow.

- Design history class. He comes to every class. But Doug Scott teaches this Intro to Graphic Design History and he is very captivating, is an amazing lecturer. He's got this encyclopedic knowledge and he assigns every student a designer to research when you're in this class. And Herbert Bayer of the Bauhaus was the designer that I was given. And I think, I don't know as an undergrad I kind of even was like, "Well like, where are the designers that look like me?" Where are designers of color, where are queer designers? I also had the good fortune of taking a design history class at Keller, it's when I was in grad school. That was a year long that Lorraine Wild did. And it just like added more richness and dimension but it's still like, didn't see myself. And I think as I began particularly to be a teacher myself and had students who also were not reflected in Philip Meggs or in, kind of, the discourse of design I started like asking these questions and trying to do my own research into that.

- I was actually in preparing for this conversation of ours. I was flipping through my copy of Meggs and goodness this is not a good example of decolonized history. There is like, no, I was looking for Dubois and Douglas and just no mention of... anyways, it was pretty stunning for me personally to revisit a part of my education and a tool that I really depended on and to realize that a significant piece of the narrative was totally missing. It really kind of sent me on a crazy spiral. Which I think is what you are addressing with your BIPOC Design History courses. But before we get there, I just wanted to ask you if you could set some context for non-designers who are watching this, and if you could tell us what is the power of graphic design and visual communication.

- That's a really great question. I think what is sometimes hard to do about setting the context of graphic design is because it's literally in everything in the built environments all over our world particularly now in COVID where we spend a lot of time looking at a screen. Everything that we encounter has been touched or influenced in some way by a graphic designer. And I think we're now a lot more savvy. Like the general public is more savvy about design and knows about design, but for many years it was kind of this invisible art. And I guess if you're doing your job correctly as a graphic designer, like no one notices they just get the feeling like, "Oh, this letter has this kind of form" or this like, or like this experience of, I don't know, a website is usable or like a book makes sense or feels right. Or that app like, "Oh, I wanna buy that thing." Like it moves me to take an action. So I think design has a lot of power and it's very much tied to economics, right? It's tied to capitalism and this idea of desire, consumer desire, emotion, sensory experience, I think maybe a little bit more than like some of the other design disciplines like, architecture does too, in product design too, but like, there's something about graphic designing thing, particularly in terms of commerce, that's very much tied to that. And so that's why I think sometimes it's hard to sort of like notice it if you're not tuned into it. 'Cause it's got that power to be sort of transparent or to be like ipso facto or like feel like it's like meant to be the way that it's meant to be. And that's totally designed on purpose.

- What a great answer. Thanks so much. So now we get into the beef of the conversation.

- Yes.

- How do we decolonize graphic design? Knowing all that what you just explained about graphic design. How do we go about doing that?

- I love that you brought up Meggs and asked me about that because I was thinking about one of the co-authors for this course on BIPOC Design History, Pierre Bowins, in his research and his lectures. He shows a series of diagrams that were made by Brandon Waybright, who's also a contemporary design historian, where he counts the number of people of color, women designers in Meggs. And it's like notrocious. It's like three black designers for you. So I think part of doing the decolonizing is doing what you were saying Nina, and like looking at what's there now or what has historically been there in terms of design history and then radically expanding that, and I feel really lucky to do the research I did about Dubois and his diagrams that were recently digitized I think in 2016, 2017, and was able to write a book about these amazing graphics made in 1900 by bunch of people of color, display it at this world's fair. Millions of people saw these graphics and then it kind of basically disappeared and sort of was erased both by white supremacy, but also reach Black audiences in a way. And, but just didn't, wasn't part of this sort of like significant quote unquote canon of design history or art and design history. And I think part of what the class we're working on the kind of work that I've been interested in is just starting to show designers, show people there's so much more that I think hasn't been as widely recognized or talked about. It's been there and it's been in conversations maybe with people of color or maybe there's a long history of decolonization in the global South, like Central, South America of scholars in other fields, literature, art that have been doing that. I think it's just design we're a little bit slow on the uptake because of that. And probably because of the commercial factor too. Like it's not something that we've been digging into as deeply as the, I think the way we are even though there have been many design historians, people like Martha Scotford who have been talking about the lack of women representation, lack of other kinds of representations, but we're just, we're at this tipping point where everyone's ready to do the work now.

- I recently watched a presentation you gave online and it was about Dubois's... Jeez, gosh, I'm totally blanking on it. The plates that was presented at the World's Fair and at the start of the presentation you talked about Meggs and other designers and critics of the day. And it just got me thinking about what it is just to have that title of designer. And is it that Dubois didn't have that? And that's part of the reason why his body of design work has just been kind of left out of graphic design history? And I was thinking about that because my personal background, I went to art school to study design but I'm very fortunate to work with colleagues who came to graphic design from all sorts of different career paths, and so there's this great diversity of experiences. And some are very formal like mine and some are very much self-taught and they just needed to be a graphic designer. So they became a graphic designer. I just, I think that there's sort of like this club of the cool kids and then there's this group of people like Dubois who maybe don't get included in that club because they don't have the pedigree or whatever even though he was a very educated man. So I don't know. That's something that I've been that I've been chewing on lately. Thanks to a lot of the content that you have online right now.

- It's such a great question. I really love the way that you're framing that, about like who gets to have that title. And I feel like what you were saying about the club is spot-on too. I feel like I feel pretty privileged to have the schooling that I did like to be kind of considered part of the club. And I think you're right. And I've also had a good fortune to teach students who come from other disciplines and I find them amazing designers, like the kind of...right?

- That's the thing, it's like, "Oh man you didn't get a formal design education "but you're such a good designer."

- Yes, and I think that that's, you're right. There's been both in terms of who's given that title and who's allowed to practice. And then like who are referenced in the history, like those are related sets of gatekeeping and exclusion that have happened. And it unfortunately lines up with a lot of the ways that America has been had its levels of exclusion, from white supremacy to misogyny, to all the bell hooks things: patriarchy, capitalism white supremacy, misogyny, homophobia. I think maybe a little bit of a kind of insularness that graphic design not sort of being willing to sort of like, "Oh yeah here's a bunch of sociologists." They do field research. They deal with like numbers and texts and form and it's communicated in graphic print and in exhibition design. So I was like, "How is that not graphic design?" And made 19 years before the coining of the term graphic design happened.

- And like when you look at those plates, he's establishing design techniques I don't know, it's just, it's very inspiring to look at. And I think hopefully also enraging as well. Just understanding the context of it all. I guess that sort of, kind of brings me into the next question that I have for you, which is, if we sorta just talked about it but if we could elaborate a little bit more about how graphic design is used as a means of oppression both historically and still currently. And I would just love to get your take on that and kind of thinking about all the protest posters and protest art and protest vandalism that we've seen lately too.

- I think that's an amazing question. And it is enraging to kind of feel like that design is still an oppressive force and to kind of bridge your question, I would say, if you're not mad, you're not paying attention. And if you're not upset, then you're not decolonizing your curriculum. Or even kind of like, for practitioners to I think this is really important as well. And I think what happened last summer with George Floyd and this resurgence, reinforcement, re-echoing of Black Lives Matter and how because of that happening the same time as COVID-19. I have a colleague at the National African American Museum of History and Culture, Raya Coombs, who has been chillingly calling it COVID 1619. So you had this kind of like combination of social justice and this pandemic which I think because of that and the economic conditions of that we've really got a big wake up call where we realized because we're having this global reset and shut down of things as we expected that we really can reconsider like everything that we're doing. And I think part of how that kind of breakdown of life, as we know it made us aware of how much design like design of social networks, design of logos all of it reinforces notions of white supremacy, reinforces notions of class problems and distinctions. We saw also in the infection rates of people with COVID-19 and how predominantly that affected people of color, communities of color because of historic redlining, gerrymandering, other kinds of physical, spatial designed oppression and Amari Susa who does talk about design thinking and streamlining, and like connects it to eugenics, which is, it's super chilling. It's really hard to look at but you then see this origin and design thinking comes out of that study of human factors. And a lot of the people like Norman Bel Geddes, he talks a lot about him who are part of this like a utopia of the automobile industry and highways and how that has historically designed, literally, barriers. Literally put oppression into the design physical and graphic and urban fabric of our country. And so-

- Atlanta is an example of that, right?

- Okay, I have not visited in months.

- No, I have been through Atlanta a little bit but like I lived in Miami, New York. So like Miami 95 and the way that it was went through Overtown, it destroyed this mecca of jazz and Black entertainers, the BQE with Robert Moses in New York. And you can just see all these different...Los Angeles, Chicago, where, so I feel like the class has made me even more aware. Like all these amazing scholars have like added to kinda my own research. And so I think that by seeing the clear picture of it now we can start to sort of say, what are alternatives? Like, what are different that we can teach design and talk about it. I think because of COVID and these current conditions, like I don't think we would have been able to put this class together the way, 'cause there's people from all over the country and then the participants too are like all over the world. So it makes this very cool critical mass of people who wanna make changes. Both students, faculty members, professional designers all in this kind-of amazing container where we're really unpacking and questioning design as we've known it, or at least graphic design and how we might go about teaching that and how might go about practicing that.

- How did you... how is that going, putting these classes together? I know you're at the tail end of your curriculum right now but I know you are a professor at Otis so I'm assuming you had that background to fall back on how to bring these courses online but what was the process like and how did how were you able to get all these people from around the world together? Is it your global network of design friends?

- I think it's a combination of that, of like I think that Dubois thing give me a lot of visibility and access and to speak about 'cause it felt very revolutionary and the timing it is to rediscover him and position into the canon. And so through that process, I was able to meet a lot of other amazing scholars who are doing this work. One other space that I think is important is this... couple of other spaces, Vermont College of Fine Arts, where I also advise students and help build a program. There were three black faculty that happened to be teaching together, for a few years which is pretty unusual in a design program. So Ziddi Msangi and Tasheka Arceneaux-Sutton. And Tasheka and I and then one of our students, Pierre, who also happens to be a Black design educator who started doing research and work with the three of us, Ziddi, Tasheka, myself and then has contributed to this class. And now it's gonna be a book. So this kinda critical mass of like connectedness helped. And then I think people just really hungry for this kind of information and just the timing of it. And the fact that it was something you could do either rapidly, accelerated over the month or also asynchronously. So people are still coming back and watching and participating. And you can actually, there's a Discord channel space where you can, people can continue to dialogue after the fact. And so I think it was just all these kinds of forces that came together. And part of what I think also makes it exciting is the framework of trying to have a kind of more anticapitalist way of setting up the course. And like we're sharing the proceeds with the lecturers in a different way than you would if you had a contributors. And we're also going to be donating to Black educational initiatives in design and otherwise. So I think this idea of new models are, I think that's part of why it was so successful and why there was so much excitement. And I had been on social media kinda like teasing the idea like asking people, like, "Do you would you want a BIPOC Centered design history?" And there was a people were like, "Yes, yes." And like, "This is what we want." And so we felt like this has been like one pilot in that but there's so many other stories. This has been focusing on Black design and African diaspora, but there's like Indigenous histories of design and Latinx experiences. I think there's the whole, we've had like one talk focus on queer Black experiences but I feel like that could be a whole other sequence of courses. And so I think the process has also the classes themselves, have the students in there in the chat like there's really a range of learners. So the chat and the resources like it's not just the lecture and like us who are organized in a class, I heard there's so much learning coming from people participating in the class. And I feel like that is the most interesting part of it. And we're sharing some of the resources for that for people who have like signed up for classes. And we're also trying to share them freely on like platforms like Arena and like other social media is as a kind of way of like we want this knowledge to get out in the world. So like how do we make it available and make it sustainable for us, but not gatekeep that information the way that it has been before?

- Well, I suppose my last question then would be for folks who are interested about this class where should they go?

- Awesome question, bipocdesignhistory.com is where the class is hosted. And people can still sign up. You can sign up for individual al a cart lectures. Like if there's just one class, say about Dubois and Black data that you want to take or emancipation and its connection to design history. I had mentioned that like Black, queer stories in print so you can kind of pick and choose like a few of them. And then there's also like a class pass where you kind of get access to all the classes. And there's also a sliding scale for students, educators and also for BIPOC professional designers, BIPOC students and educators. And there's also scholarships too. So if you're in need or you want to take the course and you can't you can receive a scholarship, which is pretty amazing. So that's kind of the best place. And then I know a few schools and design students are also now doing like group licenses or institutional licenses. So a number of people are encountering it that way, like in their class.

- That's so amazing what an incredible resource to have in a classroom.

- I think it's really, and the discussions are a part of it too. So you get to see the talk, but then you can see the real-time Q&A, and then, like I had mentioned on Discord, you can join and be part of the conversation. One of the things, one of our talks is totally free. So you can watch, there's a talk that Colette Gaiter gave around civil rights graphics called "Strike Through." And it's amazing. And that's, anyone can download that. You don't need to buy a pass or anything.

- I actually did download that one, but not because it was free, but because I happened to be interested in the content and I think, besides it being it a great lecture and just a really informative hour and a half I think it's worth visiting bipocdesignhistory.com just to see a really beautifully designed website and congrats on that part. It just seems from top to bottom of really thoughtful design process that you and your team have rolled out here.

- I think that's one of the advantages of being someone who's interested in history and scholarship and having the formal skills of being a graphic designer is you can also make it a good experience to view. And just like, 'cause it's kinda nerdy and not about the type, the typeface is a modern-

- Is it inspired by Dubois?

- Yeah, inspired by Dubois. So Tre Seals of Vocal Type who is an amazing young type designer who has been doing revivals of typography tied to movements of protest or liberation, women's suffrage, civil rights, queer liberation. But this typeface is based off Dubois lettering and it's called William. So it's very much it feels really cool to sort of have a graphic container and graphic language that does justice to sort of like the origin point. At least for me, the team around doing this kind of history.

- Well, thanks so much, Silas. I really enjoyed our conversation.

- Me too. Thank you, Nina. That was really fun.

- Hope we get to do it again sooner rather than later.

- Totally I would love that.

- Thank you to Nina and Silas for such a beautiful conversation. And a reminder, if you'd like to learn more about Silas's research you can head to bipocdesignhistory.com. Up next we'll hear from Deb Raji. Deb is currently a fellow with Mozilla where she aims to re-imagine algorithmic auditing and evaluation practice in order to hold those that deploy artificial intelligence systems accountable. As the first intern of the Algorithmic Justice League, a nonprofit that raises public awareness around the social implications of AI, she emerged as one of its star researchers. She was previously a research fellow at the Partnership on AI and a tech fellow at the AI Now Institute at New York University. Deb was recently named an innovator under 35 by the MIT Review. In her talk she'll touch upon one of her major research projects with the Algorithmic Justice League where she worked on an audit on Amazon recognitions deployed facial recognition product, discovering it was significantly less accurate for darker skinned women than for white men. In addition to sharing that research, she'll be sharing her work to advocate, to improve and de-bias these technologies as well as trace some of the real stories of people who experienced misidentification through deployed facial recognition technologies. Here's Deb.

- Thanks so much for that introduction. Hi, I'm Deb Raji, otherwise known as Inioluwa Deborah Raji. And I'm here to talk about the struggle for algorithmic injustice. And this is work I'm doing as a Mozilla Fellow and also a Fellow at the Algorithmic Justice League. So first I'd like you to meet Robert Williams. Robert Williams is a regular man with a loving wife and a beautiful daughter. And unfortunately he was falsely arrested for a crime he did not commit due to a false facial recognition match. This means that police officers were able to target Mr. Williams and escalate the biometric evidence to the point of arresting him for a crime he did not commit. When you think about the situation and what happened to him, it's incredibly unjust. But what I want to do in this presentation is really trace the story of Robert Williams and understand the different lessons we can learn from his story and others as well as how challenging it is to actually accumulate evidence to demonstrate it and discuss this injustice. First, I'm gonna start by just explaining what facial recognition is and how widespread it is. I think a lot of people don't realize how prevailing and technology it is. If you're going to the airport, you probably encountered a facial recognition system to match your face to the passport. If you are an American citizen, just walking around you're probably surveilled through police use of facial recognition. If you go to a school in New York there's facial recognition in those schools, and even going into a Rite Aid where it was discovered in about 200 stores across the US they deployed facial recognition systems. And we know that there's a challenge or there's other potential justice issues with facial recognition because people are suing. So there's been a set of class action lawsuits in Illinois and other states around biometric access and trying to understand how to protect our personal information, our face data from different firms, trying to access it for the use of the development or use of facial recognition. We have some tenants suing their landlord, Uber who makes use of facial recognition or to confirm the identities of their driver for their internal app phased two lawsuits went from a Black driver and a Trans driver about how this technology did not work for them. And then a student suing Apple due to a false face recognition arrest. And this really just highlights all of these issues, highlights one of the big misconceptions around the way that machine learning models work in the real world versus how it works and sort of sci-fi narrative. To the left, you have Sophia the Robot which is this very famous robot very humanoid, looks very complex, and there's only maybe one at most three Sophia's across the entire world not a lot of robots that are that convincing and that aesthetically pleasing. And you know, on the other side you have the Roomba and the Roomba looks super simple. It is very simple, not very complicated but this is deployed in millions of households across the US so the simplicity of the Roomba doesn't actually compromise the ability to impact many, many people, millions of Americans. So the most impactful technology here is not necessarily a Sophia but the Roomba and the real world is full of Roomba models, models that are very simple or seem very straightforward in terms of their technical functionality and how they're constructed but incredibly prevalent and incredibly widespread. So algorithmic auditing is this idea of looking at these systems that are deployed and trying to hold them accountable. Unfortunately, there's things that get in the way of us being able to understand how these systems permeate our lives and actually hold them accountable when they fail us in ways that cause injustice. The first thing we learned doing this work is that if it doesn't work for everyone, it doesn't actually work at all. Gender Shades was a study that we did where it was an external, multi target, black box audit of commercial models, facial recognition models in this particular case trying to identify whether an image was male presenting or female presenting. And what I mean by what I mean by multi target and blackbox and external just means that as auditors we had no insider information to how the model worked at all. We were completely looking at these models from the outside. And we understood that there was potentially something to learn about these models because every other dataset used in the field of facial recognition we understood were skewed towards males. So usually you would have more male representation in a dataset and also skew towards lighter subjects. So more lighter-skinned individuals were in that dataset. So the data set that we had to create which is the PPB benchmark, the Pilot Parliaments Benchmark, that's the highlighted row was much more balanced with respect to male, female representation and darker and lighter skin representation compared to all the other dominant benchmarks at the time. And when you look at PPB you can see that it's quite balanced with respect to male, female representation and darker and lighter skin representation. And this was done intentionally in order to represent different communities that we were worried about. We were worried these systems did not function for. And that we wanted to test and evaluate for. What we did was, an intersectional, gender and skin type evaluation for gender classification. So we looked at the performance of these, of different models, on these different subgroups of darker male, darker female, lighter male, and lighter female. What we found, we did two audits, one in 2017 which we published and then another published audit in 2018. And what we found in the 2017 results there was a huge discrepancy between the performance of these models on the darker female subgroup and the lighter male subgroup. So here you can see that the performance on the darker females subgroup is around 80% and the performance on the lighter male subgroup is around a hundred percent. So that's about a 20% performance difference. Compare that to the next year after they've already been publicly called out in an audit. The company redeployed the model and updated the version of their application program interface. And were able to minimize the disparities between the lowest performing subgroup which was still the darker females, and the lighter males group. And you can see that the gap between the performance of the two is around is less than 2% versus 20% a year ago. We saw this again, happened with Face++ which was a Chinese facial recognition company where the performance on the darker female subgroup goes from about 65% to 95% to over 95% while the performance on the lighter male subgroup only improves by about 0.3%. We see a similar thing with IBM. Finally we audited in 2018 companies that we had never audited before, companies that we knew were probably aware of the initial audits. But we were curious if as an industry the field had actually evolved and moved forward. And what we found was that other actors, other companies in the industry outside of the targeted companies, did not actually make much of an improvement in the gap between the performance and the lighter male subgroup and the darker female subgroup was still quite substantial. What does this say about this about a year or so? I would say a year after we did our initial our second or follow up audit the National Institute of Standards Technology, which is really the group that does most of the auditing the formal official auditing work on behalf of the government for facial recognition tools, commercial facial recognition tools. They were able to sort of validate our results. So they audited for different tasks of face verification and face identification. And they found that these products were, that Asian and African-American people were up to a hundred times more likely to be misidentified than white men using these tools. So it indicated to us that there's a huge performance disparity that still persists between darker skin individuals specifically darker skinned female individuals and lighter skin, male individuals. And another interesting sort of development that came up after audit was this idea of a pilot audit which kind of gained popularity where people would take their facial recognition system and they would ask the question of, well, you know how does this actually work on the ground if we were to deploy it or pseudo deploy the system? How many instances would we have sort of achieved our objective? And what was discovered was a pilot that happened in the UK had an 81% error rate meaning it was incorrect 81% of the time. And another pilot in New York on the MTA system actually failed a hundred percent of the time and was not able to get a single face correct. So it's clear that once you actually deploy these systems under real world conditions the technology is lacking, is not functional for everyone and is not functional at all. So the second lesson we learnt just thinking about algorithmic auditing and trying to hold the systems accountable is that ignorance is bliss but you're responsible. Before our audit work and particularly before our follow-up work in 2018 and 2019, we would always, constantly get responses like this. And here's a direct quote from Thorsten Thies who's the director of algorithmic development at Cognitec, Cognitec and NEC are both incredibly influential players in the facial recognition industry. And he said, "It is harder to take a good picture "of a person with dark skin than it is for a white person." And here he's implying that darker skin individuals are just a fundamentally harder technological problem. And as a result of that, this is why the technology does not work for them as well as it does for lighter skin individuals. However, we can see from our audit results that after a year a year after we audited the initial company, IBM, Face++ and Microsoft, those companies were all able to minimize the performance disparities between the darker female subgroup and the lighter male subgroup. In the case of Microsoft, they went from about a 20% disparity to less than 2% and Face++ went from over 35% disparity to less than a 5% disparity. So we can see that the companies are capable of making better decisions. And they just did not. Also all the companies that we audited in 2017 were able to redeploy their system within seven months in time for the audit in 2018. So this has clearly been debunked as a very kind of ignorant response. And that's something that we learned through the process that there is accountability to be had when your systems perform differently on different groups. The other sort of project I was involved in at the time was really trying to reflect on how do we remind engineers building these systems about their responsibility as they develop these tools, as they develop these models. So how do we remind them of their responsibility to build things that work for everyone? So here is one idea, which is this idea of a model card and a model card is really a way of reinforcing to those constructing and building these systems. So engineers, or even product managers or other internal stakeholders within a company and point them to different questions and prompts of what they need to answer about the model before they can reasonably deploy it. So here's an example of a model card for Google's perspective API product which supports the filtering of comments from public forums. And you'll see here that in the initial sort of version of a prospective API product it would disproportionately flag specific words such as black gay or black homosexual as a toxic and filter those words out of public forums. And in a later version, you can see that they address a lot of the issues and the disparity between the performance of these systems on identity terms like Black or gay are not necessarily flagged as toxic in the same way that any other identity term would be flagged. So even to demonstrate progress or to evaluate and assess the situation for particular product, we found that the model card set up was a really good way of reminding engineers about their responsibility to make these systems work for everyone. The model cards effort exploded into a more robust framework that we call the SMACTR Framework which was pretty much an attempt at an end to end audit framework throughout the entire product development cycle for an AI or machine learning product. That means from the moment of conceding the product and the idea all the way until reflecting on a post-deployment plan or design mitigation we would have all these different document and documentation templates borrowed from adjacent disciplines where internal audits are very common. And we try to sort of see if we could articulate different ideas of how to capture some of the decisions that these engineers were making and get them to be a little bit more reflective about their responsibility to uphold just systems. Sort of final note on this idea of documentation and engineering responsibility. I also worked with a group called Partnership on AI where Google is a member of the Partnership on AI. And this is a amalgamation of different companies. So Google, Amazon, Apple, and other companies where all of these companies actually sat down and recognize the importance of documentation and engineering responsibility. And they collaborated on this idea called the About ML project where these different companies would sort of put forward their different proposals for documentation and discuss the pros and cons of these different plans and things that could happen together and strategies on how to move forward together. Unfortunately though not every corporate response was perfect. There was a lot of challenges the corporate responses around diversifying data. So as we pointed out that these models, these facial recognition models did not work as well for darker skin individuals. We had a situation where Chinese companies were then trying, attempting to collect or purchase data of faces from different African countries. In this case Zimbambwe in an attempt to diversify their dataset. There was also the situation with IBM where IBM in attempt to build a more diverse dataset just grabbed images from Flicker without really properly informing the individuals whose images they grabbed or getting their consent in any way. So there's a lot to look out for in terms of how the companies actually respond to the realization that their algorithms are discriminatory. The other thing as well is how they guide or restrict the use of this technology. So at the time that we had audited Amazon the ACLU is actually investigating Amazon for its sale of facial recognition to ICE which is an immigration enforcement agency and different law departments, different law enforcement departments. So there was a lot of public pressure. There's a huge sort of amount of public scrutiny on the fact that Amazon's product was not very functional for darker skin individuals and particularly darker skinned females. However, Amazon's excuse at the time, and here's a direct quote from Matt Woods, he says, "We clearly recommend "that facial recognition results should only be used "when the results are at least 99%. "And even then only as one artifact and of many "in a human driven decision." But we can see through some secret work by Georgetown Law called Garbage In, Garbage Out where they demonstrate that police departments almost never used these technologies appropriately. They often put in sketches or images of celebrities to get leads using this facial recognition technology. But also more importantly, there was an investigation by Gizmodo where they actually discussed and went directly to one of Amazon's known police clients. And that client asked the question of "what's a threshold?" They had no clue what Matt Woods was referring to. And thus we're likely just using the default threshold of 80%. The lack of restrictions or guidance for those making use of that technology can be just as dangerous as flaws the technology itself. And then finally, those making use of the technology can definitely misuse, abuse and weaponize that tool. Like I mentioned, Amazon in particular when we audited them was under investigation for how they were making, how they were pitching that technology for use, to oppress minorities under different circumstances such as law enforcement and ICE. And as we can see with the case of Robert Williams, when you are in a minority group and you're misidentified, the prejudice of that situation actually makes it more likely for that false match to escalate to arrest. So it makes that vulnerable group even more at risk. And like I mentioned, the situation with the tenants, with a landlord trying to install facial recognition to monitor the tenants, there's a lot of scary situations where people can actually manipulate or weaponize the tool in order to harass a minority group. Lesson three, we can resist technology that doesn't work for us. And this is one of the most important lessons we learned from this entire saga. A lot of these bills citing our work, which was very exciting, but these there's a bunch of bills throughout the entire country calling for the ban or the pause on the use of facial recognition due to its lack of functionality, due to the privacy risk and the surveillance risk. And we can see that a lot of these bills, some of them are local, from the Bay Area to Berkeley, San Francisco and enough local participation actually prompts a statewide discussion. So that's happening in California and Massachusetts and Portland, Oregon. And after that level, hopefully there's some level of national discussion. And we do see that with facial recognition, there's a privacy act bill that's in place and some discussion around an audit bill to come. But most importantly, the public pressure around the discussion of this topic actually cause public resistance to the use of the sale of facial recognition by these companies. So after we had initially audited these companies the company's sort of PR response was, "Oh of course we support legislation. "We support regulation in theory "but we're gonna continue selling these products." And this was really a lot of the public pressure really dialed up following the George Floyd protest where people began to, again sort of question the best interest of the police use of facial recognition for surveillance purposes. And in that moment, it became clear that these companies had to take an action to step away from the tool because it was that dangerous and put that many lives at risk. So you can see here that like Amazon, Microsoft and IBM all backed away from the use, of the sale and the dissemination of facial recognition tools. But I'll mention sort of this downside of that, which was it was a two year fight. It was a very long journey to get Amazon to go from this place of refusing to quit selling their technology, to finally pausing the use, pausing the police use of its software. So all of this, to say that the pressure to actually execute that change took an incredibly long time but we feel it was definitely worth it. For every Robert Williams there's so many other individuals. The widespread use of this technology really necessitates very careful attention to bias and abuse. We have AI driven dermatology tools that don't work on darker skin patients. We have all kinds of algorithms in use during the criminal justice system making all kinds of mistakes actually sending people to jail or extending their sentence. We have algorithms that systematically discriminated against women, voice assistants that don't work on accents, an algorithm during the COVID era that had misassigned grades to lower income students. So there's a clear need for us to pay attention to this topic, to care about this. Not just for Robert William as an individual, but everyone that he represents. Thank you very much.