Paddy Galloway dives deep with us on Video A/B, Ask Studio, and evolving over time


Listen to article

Paddy Galloway lives and breathes YouTube analytics, so you know when he sat down with us and YouTube VP of Creator Product, Amjad Hanif, it was going to be an insightful conversation. We had the opportunity to go deep on Ask Studio, Dynamic Thumbnails, Video A/B testing, and more.

“It's always a good sign when you're adding things that people like me have been trying to manufacture and do behind the scenes for years.”

- Paddy Galloway

Ask Studio and Dynamic Thumbnails

Paddy points out that one of the most useful parts of Ask Studio is the storytelling assistant, which lets you workshop ideas in a draft state before uploading a completed video. Ask Studio will give you suggested edits based on everything it knows about your channel and audience.

Once your video is ready to go, Dynamic Thumbnails come in. This goes beyond regular "A/B" testing: you provide three thumbnails and they will be served to different audience segments automatically based on which one performs best for that group. We've seen how different audiences — regular viewers, new viewers, or some other pocket audience — respond to different types of thumbnails.

"That's what I'm really excited about," Paddy says, because "different audiences sometimes do require a different thumbnail." He continued, "It's always a good sign when you're adding things that people like me have been trying to manufacture and do behind the scenes for years."

Another feature creators may not know about is back catalog optimization, where Ask Studio will identify older videos that could benefit from a new modernized thumbnail to help them perform their best.

“When you have a sea of metrics, it's easy to get distracted by something that may not actually be in your interest as a creator. That's why we really focused on the retention curves, because that is the true insight from the A/B testing.”

- Amjad Hanif, YouTube VP of Creator Product

Video A/B testing, intros, and retention curves

Paddy notes that he's put so much emphasis on thumbnail testing and optimization because it's one of the few things you can actually change to impact performance after uploading. Now with the prospect of video A/B testing, it expands that opportunity further into the video itself.

"We all know how much of a difference an intro can make," Paddy says. "There are some times when I'll try an intro and realize immediately afterwards that it didn't resonate. The idea that we're moving towards video A/B testing is very exciting for me."

Video A/B testing will run a test, provide results, and give the creator seven days to choose which edit to go with. Amjad notes that his expectation based on discussions with creators is that video A/B testing will be applied most to intros, understanding that there are many ways to craft the start of a storyline.

Those test results will include retention curves for both edits, which is particularly valuable when testing different introductions and tells you far more than just testing thumbnails and following clickthrough rate. Amjad explains the thought process, "When you have a sea of metrics, it's easy to get distracted by something that may not actually be in your interest as a creator. That's why we really focused on the retention curves, because that is the true insight from the A/B testing."

“The spirit behind the work on the new insights is to bring out what's important for you and give you a chance to ask Studio to explain these things and interpret them.”

- Amjad Hanif, YouTube VP of Creator Product

Actionable insights, learning, and evolving

We all recognize that YouTube Studio provides a wealth of data, but until now it hasn't been as easy to parse or action on as it should be. Actionable insights is meant to change that and start the analysis automatically and highlight what's important for the creator.

Amjad explains the thought process behind bringing insights to Studio: "What we heard time and again from creators was it's a lot to interpret, it's great if you have a team of analysts, so the spirit behind the work on the new insights is to bring out what's important for you and give you a chance to ask Studio to explain these things and interpret them."

Paddy says his analysis always starts with view counts, then moves into retention curves as "one of the easiest ways to speed-run learning what your audience likes or doesn't like." He notes that the retention curves have helped him understand that audiences tend not to like scripted or manufactured moments and prefer authentic ideas and personal stories.

Doing this ongoing analysis, staying curious, and being willing to adjust and evolve is important to the longevity of your channel. Probing questions with Ask Studio and exploring in the research tab can broaden your horizons and bring you new ideas.

More like this

More like this

Subscribe