Who’s allowed to write about technology?

I recently published an article about explainability in machine learning systems for the Harvard Business Review. The article argues that many businesses get stuck applying machine learning because they worry about black boxes; that they should think about what matters for a given use case, as sometimes other governance and assessment metrics are more relevant than an explanation (e.g., precision and recall for information retrieval); and that a close reading of recital 71 in the EU GDPR suggests that an individual’s right to an explanation applies to the procedures used to build and govern the entire system, not to which input features, with which weights, lead to which outputs.[1]

The article’s goal is to help businesses innovate. It seeks to empower people by helping them ask the right questions. The battle cry is: There’s no silver bullet. You have to think critically. Compliance and business teams should align with data scientists early in the machine learning system-development process to align on constraints required for a given use case. Businesses should be as clear as possible on what algorithms actually optimize for, as ethical pitfalls arise between what we can and can’t measure, what our data do and do not index about the world.[3]

The day it was published, I received two comically opposite responses from data scientists working in executive positions in technology companies. The first complimented me, mentioning that they were pleasantly surprised to see someone with my educational background writing so cogently about machine learning. The second condemned me, mentioning that someone with my educational background had no right to write about machine learning and that I was peddling dangerous hype.

I didn’t learn much from the compliment. My Mr. Peanut Butter labrador ego enjoyed being stroked.

I learned a few things from the critique. It helped clarify some of my own tacit assumptions, my ideology, my ethics, the grey matter between the words, the stuff that makes it hard to write because it feels vulnerable and exposed, the implicit stuff that signals community acceptance and alignment and that we rarely sit back, unpack, analyze, and articulate.

Here are some of the lessons.

1. Precision always matters

As with (close to) all tricky situations, I wonder if I did the right thing at the time. Upon being attacked, I chose to diffuse rather than ignite. I thanked the person for voicing their critique and disengaged. The only thing I mentioned was that it appeared that they drew their conclusion from the title of my article rather than its content. They stated that scientists have long had ways to interpret the output of neural networks and that I was peddling hype to write an article entitled “When is it important for an algorithm to explain itself?” I was surprised to see the narrow focus on algorithmic interpretability, as I felt the work my article did was to expand the analytical framework of explainability to systems and procedures, not just algorithm. So, underneath the amygdala’s attack response, my mind said “Did they even read it?”

It took me a few minutes to put (what I assume are) the pieces together. I don’t write the titles for my HBR articles and hadn’t taken the time to internalize how the title could be interpreted. When my editor suggested it, I quickly approved. Had I felt it was important that the title precisely reflect the content, I would have recommended that we say “an algorithmic system,” not an “algorithm” and say “when is it important for businesses to consider explainability in machine learning systems” versus implying that algorithms have agency. (Although it is a thought-provoking and crucial task to think about how we can and should design system front-ends to translate math-speak into people-speak, be that to communicate and quantify uncertainty or to indicate other performance metrics in a way that is meaningful and useful to developers and users.)

Let me be clear on the lesson here. I try to take as much responsibility as possible for outcomes, especially negative ones. I approved this title very quickly because I was excited to see the piece go live. Next time, I’ll think more deeply. I’m not blaming the HBR editorial team. They thought about the title and considered a few different options. I love working with my editor. He’s a wonderful partner. We bounce topics by one another. He’ll push back on stuff he’s not excited about; I’ll do the same for him. What I enjoy most is giving him feedback on questions unrelated to my writing. I like giving back, as he has done much to help me build my reputation as a writer.

The reason I wonder if I did the right thing is that I wonder if it is my duty to other writers, to other professionals, to have stood up for myself as opposed to stepping back and disengaging. But attacks beget strong emotions. For all of us. I needed time to think and let the lessons sink in. This is, obviously, my response.

2. Read things before sharing them and commenting on them

I’m guilty of having shared things without reading them, or having only skimmed an abstract. Mostly because I’m busy and can be impulsive. This is a good lesson about why that’s always a bad idea. I’d feel ashamed if people suspected I hadn’t read an article before critiquing it. There’s a lot at stake here, like democracy. It’s meaningful to engage deeply and charitably with another person’s ideas. To take the time to understand what they are trying to communicate, to find an opportunity to refine an idea, challenge an idea, improve the structure or flow. To teach one another.

3. Don’t judge someone based on their resume

The person who critiqued me seemed to draw conclusions about what I could and could not know based on my LinkedIn profile. That doesn’t reveal that much. You don’t see that much of my college education was funded by Siemens because I was one of 2 female students in New England awarded for having the highest scores on our math and science AP tests. You don’t see that I was a math major at U Chicago who always got straight As in math and struggled much more in humanities, but was ultimately more interested in literature so decided to pursue that path. You don’t see how much analysis, complex analysis, number theory, linear algebra, and group theory I studied. You don’t see how I came to understand how important that training would be once I ended up working in machine learning. You don’t see that I focused on history of philosophy and math in the 17th and 18th century in graduate school, and, while the specific math of that period is no outdated, have thought deeply about the philosophical questions associated with statistics, empirical science, and the diffusion of knowledge in the wake of new scientific discovery. You don’t see how my fellow literature graduate students told me reading my writing was like being in a prison because I was always trying to prove things, as in a math proof.

You might see that I’m not interested in competing with machine learning researchers in their own field. I want to drink in as much of their thinking as possible, want to learn everything they can teach me, want to understand why and what it means for the field, want to experience the immense joy of recognizing structural similarities between two disciplines or applications that can be the seat of innovation, that place where you realize that a mathematical technique originally explored for problem A comes into its own in the world in problem B.

You might see that the role I’ve come to accept is that of the translator, the generalist curious enough to dive deeply into whatever subject matter I’m working on, but will never be the disciplined expert. There will always be questions and gaps. Always more to learn and explore. Always people who can go deeper and narrower. Like Sheldon Levy, I viciously and vibrantly admire those whose creative minds will discover things, will reframe problems to uncover solutions stuck for centuries. They are the heroes. All we do is to sing their song, and help others hear its beauty.

4. Allow people to learn

This is the most important lesson. The one I care about. The one that puts fire into my heart and makes my fingers type quickly.

We must allow people to learn from experiences after school. We must not accept a world where the priests alone are allowed to understand, where the experts alone have the authority to write about, talk about, and share ideas about a subject. Technologies like artificial intelligence are already impacting us all. Work will change. Jobs will change. New jobs and new opportunities will arise. If people are not given the space to change, to learn, if the only people we deem qualified to do this work, to write about this work, are those who come with a certain PhD, a certain educational certificate, a certain type of social rubric of authority, we are fucked. We must trust that people can learn new things and find ways to give them opportunities. We must engage with one another so as to promote openness, to probe and push without the searing pain of judgment, to provide people with the confidence required to ask the simple questions needed to get to the heart of the matter, to give people the breathing room to embrace the initial anxiety of change so they can come to do something new.

No, I did not do my PhD in machine learning. It’s not impossible that I won’t go back and do a second PhD in reinforcement learning, as I find the epistemological questions associated with that subfield incredibly rich, incredibly akin to the perennial questions I have loved in the Greeks, in the early moderns, and again today. Time will tell. I have, however, worked in the field over the past few years. I was fortunate enough to have been granted the opportunity to learn a lot at Fast Forward Labs. I will forever be grateful to Hilary Mason for giving me a chance to help her build her business, and for believing that I could learn. I learned. I am still learning. I write about what I’ve come to learn, and accept criticism, feedback, refinements, all the stuff other people can share with me to expand my understanding and help us all grow. I’m decent at recognizing what I know with precision and where my knowledge starts to falter into fuzziness, and I tell people that. I have made mistakes, thought about them deeply, and try my best not to make them a second time (which I don’t always succeed it). I love enabling others to build sound intuitions about mathematical concepts and technology. To feel empowered, feel like they get it in ways they hadn’t before. And not because it’s dumbed down. Not because we resort to the Platonic blindfolds for the masses. Because we can all do it. It’s just that we have to break down the power walls, break down the barriers, break down our egos, and do our best to make something meaningful.

I will fight for it. There are too many people who hold themselves back because they are excluded from circles protecting themselves within elitism. Everyone deserves a voice. Everyone deserves a chance to understand.

5. Everyone should be allowed to write about technology

It’s not all going to be good. There is a lot of hype. I’m not sure hype is all bad, as it has the power to mobilize large groups of people who wouldn’t otherwise be interested. Nothing like the fine print of precise qualifications to dampen the mood of disruptive innovation. There is damage when the hype breeds unnecessary fear, rather than unbounded excitement. And there’s certainly lots of work to be done to help businesses bring expectations down to earth to capitalize on what’s possible. But there’s wonderful satisfaction that emerges when businesses start to get traction with a narrowly-focused, real-world application. And it takes different people with different viewpoints from different teams to make that happen, in particularly in established enterprises with their processes and people and quirks and habits and culture.

We may not all want to write, but we all have a part to play. And I’ll always subscribe to the Ratatouille philosophy: it’s not that everyone can cook, but that the greatest chef in the world may not necessarily have a resume our priors deem likely to succeed.

---

[1] Peter Sweeney has written many great articles about epistemology and AI, and argued that we should conceptualize the outputs of machine learning algorithms as observations, not explanations. He was responding to David Weinberger, who has argued that we should focus governance efforts on optimization, not explanation. I’m partial to that, but again think it depends on the use case. Nick Frosst, who wrote the capsule network paper with Sara Sabour[2] and Geoff Hinton, thinks that interpretability (I must admit that I use the words interpretability and explainability interchangeably, and should take the time to parse the two, both philosophically and technically) is important because those creating systems and those impacted by systems should have the right to intervene to change their behaviour or change the system to change outcomes. So, for example, if a system denies an individual a mortgage because they missed their last 3 credit card payments, then that gives the individual meaningful recourse to act differently to meet the requisite rules in the future. It does indeed get dicey if there are so many dimensions that end up correlated to some output that impacts big deal opportunities for real people living real lives. I analyzed a couple of examples in this podcast.

[2] I’m 99% confident that Sara wasn’t able to attend NIPS in Los Angeles last year because she is Iranian. Knowledge, and credit for new knowledge, is cosmopolitan.

[3] My company recently published a framework to help consumer enterprises develop responsible machine learning systems. It’s practical and breaks down the different privacy, security, governance, and ethics questions cross-functional teams should ask and address at different points in the machine learning system-development process. We worked hard on it. I’m proud of it.

The featured image is of Remy the rat chef. He has a heightened sense of taste and smell but is naturally overlooked as an awesome chef because he’s a rat. He ends up making a ratatouille that softens the curmudgeonly critique because it brings him back to his childhood like Proust’s madeleine. So worth watching over and over again.