Skip to content

Rana el Kaliouby

Abstract

Rana el Kaliouby (born 1978) built her Cambridge doctorate around a problem that computers had been allowed to ignore: a face carries most of what a person means, and a machine at the other end of a screen sees none of it. Her 2005 thesis read six mental states from video in real time. At the MIT Media Lab she turned it into a wearable “emotional hearing aid” for people on the autism spectrum, and in 2009 she and Rosalind Picard spun the work out as Affectiva, which ended up measuring how strangers’ faces moved while they watched television commercials. Two months from missing payroll she turned down $40 million from an investor who wanted the technology aimed at surveillance. Affectiva was sold to the Swedish eye-tracking firm Smart Eye in 2021 for $73.5 million, against $53 million of venture capital raised.

Rana el Kaliouby
Rana el Kaliouby. Image: Web Summit, CC BY 2.0, via Wikimedia Commons.

Cairo, Kuwait, Cambridge

El Kaliouby was born in Cairo in 1978 and grew up mostly in Kuwait, where her father worked. Her mother was one of the first women to work as a computer programmer in the Middle East, which put a machine in the house and removed the argument about whether computing was women’s work. Her father was traditional about how daughters should live and demanding about how well they should do. The family left Kuwait in a hurry when Iraq invaded in August 1990 and resettled in Egypt.

She took a bachelor’s and a master’s in computer science at the American University in Cairo, then went to Newnham College, Cambridge for a doctorate, leaving a husband and an extended family behind in the Gulf for most of each year. Her supervisor was Peter Robinson, and the group next door belonged to Simon Baron-Cohen at the Autism Research Centre, whose work on mind-reading supplied both the theory and, in the form of his Mind Reading DVD corpus, the labelled video she needed.

Mind-Reading Machines

The thesis, Mind-reading machines: automated inference of complex mental states, was accepted in 2005 and published as a Cambridge computer laboratory technical report that July. It deliberately avoided the six basic emotions that the field had been classifying since the 1970s and went after states that matter in a conversation: agreeing, concentrating, disagreeing, interested, thinking and unsure. The system tracked facial feature points and head motion in video, fed them into dynamic Bayesian networks that modelled how expressions unfold over time, and ran in real time with no manual preprocessing. On the test sets its accuracy was comparable to human judges asked to do the same task.

Robinson and el Kaliouby showed the work at the Royal Society’s Summer Science Exhibition in 2006 under the title “Mind reading machines”. The press took it as a lie detector story. The intended application was the opposite: a system that could tell someone what everyone else in the room could already see.

The Emotional Hearing Aid

El Kaliouby moved to the MIT Media Lab as a postdoctoral researcher in Picard’s affective computing group and helped start the Autism and Communication Technology Initiative. The project was a wearable camera and a small display that would tell a wearer on the autism spectrum whether the person opposite looked confused, bored or interested, the signal that neurotypical people read without noticing they are reading it.

Field work with the Groden Center in Providence produced Self-Cam, a chest-mounted camera pointed back at the wearer’s own face, and the first corpus of naturally occurring expressions recorded from people with and without autism (see The Accessibility Revolution in Computing). The assistive framing is what made the research fundable. It is also the part of it the market never bought. Media Lab sponsors kept asking the same question, which was whether the software could score how an audience reacted to an advertisement.

Affectiva

Picard and el Kaliouby incorporated Affectiva in 2009, with offices in Boston and, later, Cairo. The product that paid the bills was Affdex: consumers watched a commercial through their own webcam at home, the software reported second by second what their faces did, and the agency learned where the joke landed. By the company’s own count it analysed more than 19 million face videos from over 90 countries and tested more than 100,000 advertisements, which it described as the largest emotion dataset in existence. In 2018 it added automotive sensing, watching drivers for drowsiness and distraction.

Picard left the chairman and chief-scientist roles in 2013. El Kaliouby became chief executive on 25 May 2016, running a company of 100 to 150 people on $53 million of venture money. In May 2021 Smart Eye, a Gothenburg firm that had been selling eye tracking to carmakers, bought Affectiva for $73.5 million, mostly in stock.

The Offer She Turned Down

The decision el Kaliouby is known for inside the industry is one she declined to take. Affectiva was about two months from missing payroll when a venture arm connected to an intelligence agency offered $40 million on the condition that the company pursue lie detection, security and surveillance work. She refused, and her public account of it has not moved in a decade: “We’re not interested in applications where you’re spying on people.” A client that accepted the company’s privacy terms signed a $7 million deal days before the money ran out.

The refusal cost Affectiva the market. Emotion recognition went into hiring and policing anyway, sold by other firms; Emotient, a San Diego competitor built on the same anatomical coding scheme, was bought by Apple in January 2016 (see Affective Computing and The Privacy War).

What the Face Does Not Say

In 2019 a review led by the psychologist Lisa Feldman Barrett in Psychological Science in the Public Interest went through the evidence for reading emotion from facial movements and concluded that the mapping is far weaker and more context-dependent than commercial systems assume: people scowl when they are concentrating, smile when they are uncomfortable, and do different things in different cultures and situations. The finding cut at the premise of the entire industry, el Kaliouby’s included.

Her response has been to argue for narrower claims rather than to dispute the science: that the systems read facial expressions and vocal signals, not inner states; that they need multiple channels and context; and that the applications worth building are the ones where being wrong is cheap. She has argued consistently for consent as a design requirement, which is easy to say and was expensive for her company to mean.

After Affectiva

El Kaliouby stayed on as deputy chief executive of Smart Eye through the integration and left in 2024 to launch Blue Tulip Ventures with Gabi Zijderveld, Affectiva’s former chief marketing officer, investing in what she calls human-centric AI. Her memoir, Girl Decoded, written with Carol Colman, appeared from Currency in April 2020 and tells the Kuwait-to-Boston story alongside the technical one; by then she was divorced and raising two children in the United States. Her 2015 TEDWomen talk, “This app knows how you feel, from the look on your face”, remains the short public version of the argument.

📚 Sources