Stream Loses Face Detection Feature

No More People Timelines in Stream

On May 1, Microsoft announced in message center notification MC211744 that Stream would no longer generate a people timeline for videos after June 1, 2020 and that they would retire the feature and remove all people timeline data permanently by July 15, 2020.

The reason Microsoft cites for the decision is “low feature usage.” Microsoft also points out that this is during a time of “unprecedented growth” for Stream, largely driven by the growth of Teams to 75 million users and the consequential demand for processing of Teams meetings stored in Stream, including the recent addition of recordings for 1:1 Teams meetings.

Generation of People Timeline

The people timeline is one of the AI-powered features available in Stream to all Office 365 plans. When Stream processes a video, it applies a face recognition process to create a people timeline. The people timeline is accessible in the People link under the video (occasionally Stream can’t process a video to find faces and no People link is available).

The people timeline shows when the individuals (or rather, their faces) appear in the video as detected by Stream when it processed the video.  Figure 1 shows Stream playing a video recorded at AvePoint’s ShiftHappens event in 2019 when I was interviewed by Paul Thurrott. Stream detected two faces. You can see how the slider is used in the timeline to go to different places in the video where a selected individual appears.

Although people timeline generation worked acceptably well for videos taped in studio or other controlled settings, it had its problems. Sometimes Stream failed to detect faces or missed some of the speakers in a video. Sometimes it found multiple examples of the same person, and sometimes it just failed to work and didn’t generate a people timeline. There’s no way to request Stream to reprocess a video if it fails to create a people timeline on the first pass.

Many Videos Don’t Include Faces to Process

I’m sure Microsoft’s famous telemetry guided this decision and it’s probably true that people timelines are not a heavily used feature, especially when so many videos processed by Stream are recordings of Teams meetings when not everyone turns their video on. Figure 2 shows what a recording of a Teams meeting between 3 people often looks like. One person has a background effect enabled (but the camera is blocked) while the other two don’t have video turned on. Face detection doesn’t perform well in these circumstances.

Blurring and Backgrounds Don’t Help Either

It’s also possible that using custom background effects in Teams meetings creates processing challenges for Stream. Due to the way background blur and effects work, peoples’ faces can “float” in and out of the video feed as they move backwards and forwards (the same thing happens with other platforms like Zoom). Having a face disappear and reappear probably drives the face timeline algorithm crazy.

It’s also likely that face detection absorbs too many valuable resources at a time when every CPU cycle is needed to meet growing customer demand. As part of its initial response to throttle consumption of resources within Office 365, Microsoft reduced the standard definition for Teams recordings to 720p; cutting the people timeline seems like it might be another casualty of the boom in Teams usage.

---

Putting all the pieces together to make Office 365 work takes a lot of effort and knowledge. Learn from the best by subscribing to the Office 365 for IT Pros eBook. We do the heavy lifting so you don’t need to.