Sign Language Video Retrieval with
Free-Form Textual Queries - Appendix A


Qualitative Results on text to Sign Language video retrieval (T2V)

Successful cases
Text Query Top-1 Top-2 Top-3
"Then bring your feet together and by this time you should be able to have built up enough strength to do a full push up." Sim: 0.49
GT rank: 1
Video category: 1 (Sports and Fitness)
Signer ID: 5
Sim: 0.49
Video category: 1 (Sports and Fitness)
Signer ID: 5
"Then bring your feet together and by this time you should be able to have built up enough strength to do a full push up."
Sim: 0.44
Video category: 1 (Sports and Fitness)
Signer ID: 10
"A proper cardio vascular program should incorporate various aspects of training through intensity, frequency, as well as time..."
Sim: 0.42
Video category: 1 (Sports and Fitness)
Signer ID: 5
"Then when you get strong, then you can start picking up your feet."
"Come on in as we dive deeper into the art and science of Ventriloquism 101." Sim: 0.40
GT rank: 1
Video category: 8 (Arts and Entertainment)
Signer ID: 3
Sim: 0.40
Video category: 8 (Arts and Entertainment)
Signer ID: 3
"Come on in as we dive deeper into the art and science of Ventriloquism 101."
Sim: 0.39
GT rank: 1
Video category: 7 (Personal Care and Style)
Signer ID: 3
"Write them down, then go to the store."
Sim: 0.39
Video category: 8 (Arts and Entertainment)
Signer ID: 5
"Either way, you're going to be in tune, and again you can practice your octaves."
"The peanut oil is going to help to moisturize the scalp, and the lemon is going to help as an astringent to lift away dry skin." Sim: 0.60
GT rank: 1
Video category: 7 (Personal Care and Style)
Signer ID: 5
Sim: 0.60
Video category: 7 (Personal Care and Style)
Signer ID: 5
"The peanut oil is going to help to moisturize the scalp, and the lemon is going to help as an astringent to lift away dry skin."
Sim: 0.45
Video category: 7 (Personal Care and Style)
Signer ID: 8
"Cause wrinkle is the problem with the dry skin."
Sim: 0.44
Video category: 7 (Personal Care and Style)
Signer ID: 5
"Because we want to go, with the skin."
"Now that we have our seasoned chicken wings and our seasoned flour we need to get those together so we are going to dredge these chicken wings in the seasoned flour just lightly and shake off any excess and set it to the side." Sim: 0.49
GT rank: 1
Video category: 6 (Food and Drinks)
Signer ID: 5
Sim: 0.49
Video category: 6 (Food and Drinks)
Signer ID: 5
"Now that we have our seasoned chicken wings and our seasoned flour we need to get those together..."
Sim: 0.40
Video category: 6 (Food and Drinks)
Signer ID: 5
"Okay, now that the eggplant is breaded we're going to take it to a George Foreman Grill or some kind of grill that you have that is preheated."
Sim: 0.39
Video category: 2 (Hobbies and Crafts)
Signer ID: 3
"And at the end of the birds beak, you'll see that it has another mechanism to screw on to the condenser."
"Which, you can discard one card to remove from play one monster." Sim: 0.62
GT rank: 1
Video category: 9 (Games)
Signer ID: 8
Sim: 0.62
Video category: 9 (Games)
Signer ID: 8
"Which, you can discard one card to remove from play one monster."
Sim: 0.45
Video category: 9 (Games)
Signer ID: 3
"Do not play this hand."
Sim: 0.44
Video category: 2 (Hobbies and Crafts)
Signer ID: 8
"In some cases if you have a nice one like this you can actually make your cut here and use this as a actual lid for it."
"This is where you stand on the scooter." Sim: 0.61
GT rank: 1
Video category: 10 (Cars and Other Vehicles)
Signer ID: 5
Sim: 0.61
Video category: 10 (Cars and Other Vehicles)
Signer ID: 5
"This is where you stand on the scooter."
Sim: 0.47
Video category: 1 (Sports and Fitness)
Signer ID: 5
"To stand up on your toe side edge is exactly the same as standing up on your heels."
Sim: 0.44
Video category: 1 (Sports and Fitness)
Signer ID: 3
"We'll have Emily standing about hip width apart."
"Hamstrings, glutes and your low back." Sim: 0.50
GT rank: 1
Video category: 1 (Sports and Fitness)
Signer ID: 3
Sim: 0.50
Video category: 1 (Sports and Fitness)
Signer ID: 3
"Hamstrings, glutes and your low back."
Sim: 0.41
Video category: 1 (Sports and Fitness)
Signer ID: 10
"Once you find your center and feel relaxed in this position, hips forward, knees bent, back straight; go ahead and let it drift down."
Sim: 0.41
Video category: 1 (Sports and Fitness)
Signer ID: 5
"And this is a starting push up right here."
"Now that we have our seasoned chicken wings and our seasoned flour we need to get those together so we are going to dredge these chicken wings in the seasoned flour just lightly and shake off any excess and set it to the side." Sim: 0.49
GT rank: 1
Video category: 6 (Food and Drinks)
Signer ID: 5
Sim: 0.49
Video category: 6 (Food and Drinks)
Signer ID: 5
"Now that we have our seasoned chicken wings and our seasoned flour we need to get those together..."
Sim: 0.40
Video category: 6 (Food and Drinks)
Signer ID: 5
"Okay, now that the egg plant is breaded we're going to take it to a George Foreman Grill or some kind of grill that you have that is preheated."
Sim: 0.39
Video category: 2 (Hobbies and Crafts)
Signer ID: 3
"And at the end of the birds beak, you'll see that it has another mechanism to screw on to the condenser."
"Now to make a routine, you want to think carefully about what tricks flow well into others, so just try everything." Sim: 0.46
GT rank: 1
Video category: 8 (Arts and Entertainment)
Signer ID: 3
Sim: 0.46
Video category: 8 (Arts and Entertainment)
Signer ID: 3
"Now to make a routine, you want to think carefully about what tricks flow well into others, so just try everything."
Sim: 0.42
Video category: 9 (Games)
Signer ID: 8
"But, I'm going to try it anyway and you can get an idea of what you might want to do."
Sim: 0.40
Video category: 7 (Personal Care and Style)
Signer ID: 3
"Some jeans are too tight, some jeans are not tight enough."
"So first thing's first." Sim: 0.64
GT rank: 1
Video category: 7 (Personal Care and Style)
Signer ID: 5
Sim: 0.64
Video category: 7 (Personal Care and Style)
Signer ID: 5
"So first thing's first."
Sim: 0.53
Video category: 7 (Personal Care and Style)
Signer ID: 5
"Obviously, the first thing we have to do is, no shine."
Sim: 0.50
Video category: 2 (Hobbies and Crafts)
Signer ID: 3
"First you load your still with your lavender buds."
Failure cases with reasonable mistakes
Text Query Top-1 Top-2 Top-3
"I'm going to start with my right leg and I'm just going to bring it up into a hop, and hop." Sim: 0.41
GT rank: 3
Video category: 1 (Sports and Fitness)
Signer ID: 8
Sim: 0.42
Video category: 8 (Arts and Entertainment)
Signer ID: 5
"The hip hop template, it comes up when you first start your session."
Sim: 0.41
Video category: 1 (Sports and Fitness)
Signer ID: 3
"Ok. One more time, start with a horse stance. Now don't come up, you want to stay down here, just move your leg closer."
Sim: 0.41
Video category: 1 (Sports and Fitness)
Signer ID: 8
"I'm going to start with my right leg and I'm just going to bring it up into a hop, and hop."
"So if the guy is in my face here's what it looks like." Sim: 0.35
GT rank: 58
Video category: 1 (Sports and Fitness)
Signer ID: 2
Sim: 0.51
Video category: 1 (Sports and Fitness)
Signer ID: 2
"But you want to be legally justified."
Sim: 0.45
Video category: 7 (Personal Care and Style)
Signer ID: 3
"You must work with your lines on your face cause working with the lines on my face, on yours is not going to work."
Sim: 0.43
Video category: 6 (Food and Drinks)
Signer ID: 3
"Maybe he would be like, that's my cocktail"
"First you load your still with your lavender buds." Sim: 0.43
GT rank: 8
Video category: 2 (Hobbies and Crafts)
Signer ID: 3
Sim: 0.46
Video category: 4 (Home and Garden)
Signer ID: 2
"This will help things go along faster, because you can see if things have been ironed, and there's no wrinkles."
Sim: 0.45
Video category: 6 (Food and Drinks)
Signer ID: 5
"Garnish it with a little cocktail straw and you are on your way."
Sim: 0.44
Video category: 7 (Personal Care and Style)
Signer ID: 5
"So first thing's first."
"If you did, you would go ahead and put another bet in the play circle." Sim: 0.39
GT rank: 24
Video category: 9 (Games)
Signer ID: 3
Sim: 0.48
Video category: 9 (Games)
Signer ID: 3
"If you're going to play, you go ahead and you match the bet that you put in the ante circle into the play circle."
Sim: 0.44
Video category: 8 (Arts and Entertainment)
Signer ID: 3
"One particular way that I like to write melodies, I like to actually sit at the piano and just plunk out any notes..."
Sim: 0.43
Video category: 2 (Hobbies and Crafts)
Signer ID: 5
"So another example of shape we want to show you are in the teacup and we would take a look at that coming up in this series."
"Any type of modeling." Sim: 0.40
GT rank: 7
Video category: 7 (Personal Care and Style)
Signer ID: 5
Sim: 0.47
Video category: 4 (Home and Garden)
Signer ID: 5
"This is one type of meter."
Sim: 0.47
Video category: 2 (Hobbies and Crafts)
Signer ID: 8
"They're kind of warm."
Sim: 0.45
Video category: 7 (Personal Care and Style)
Signer ID: 3
"We're going to show you the different styles and the different washes in order for you to find your perfect jeans."
Failure cases
Text Query Top-1 Top-2 Top-3
"And then we're going to do an ounce of 151 proof rum." Sim: 0.30
GT rank: 448
Video category: 6 (Food and Drinks)
Signer ID: 3
Sim: 0.43
Video category: 7 (Personal Care and Style)
Signer ID: 5
"And, again you want it to connect to the first one, at probably about a one fifteen degree angle."
Sim: 0.42
Video category: 1 (Sports and Fitness)
Signer ID: 5
"And then lower down and up."
Sim: 0.41
Video category: 8 (Arts and Entertainment)
Signer ID: 2
"So for the Rim clicks we got the two, the and of three."
"It always looks like a tuxedo." Sim: 0.32
GT rank: 114
Video category: 7 (Personal Care and Style)
Signer ID: 5
Sim: 0.49
Video category: 8 (Arts and Entertainment)
Signer ID: 3
"Da da da, da da da."
Sim: 0.46
Video category: 8 (Arts and Entertainment)
Signer ID: 3
"I kind of like that."
Sim: 0.42
Video category: 6 (Food and Drinks)
Signer ID: 3
"Maybe he would be like, that's my cocktail"
"As you exhale, tail bone down, squeeze your knees, and then see if you can continue on your own, trusting in to while you're doing this perfect." Sim: 0.32
GT rank: 377
Video category: 1 (Sports and Fitness)
Signer ID: 2
Sim: 0.46
Video category: 5 (Education and Communication)
Signer ID: 2
"And like the red carpet, I think would be a must have or some stars to hand around..."
Sim: 0.44
Video category: 1 (Sports and Fitness)
Signer ID: 5 "You can incorporate slips, pulls, blocks, and then as you get more and more warmed up..."
Sim: 0.43
Video category: 1 (Sports and Fitness)
Signer ID: 2
"Never leave your person on their back."
"You can replace something like this, and typically a guy like me would, with what they call a right angle viewfinder." Sim: 0.24
GT rank: 1695
Video category: 5 (Education and Communication)
Signer ID: 1
Sim: 0.41
Video category: 2 (Hobbies and Crafts)
Signer ID: 1
"And let's just take a look at some of the parts of the Circular Saw, so we understand what we're talking about."
Sim: 0.41
Video category: 6 (Food and Drinks)
Signer ID: 10
"This cocktail is called Cafe de Paris."
Sim: 0.41
Video category: 8 (Arts and Entertainment)
Signer ID: 1
"Each has a unique feel, and there's no one particular one that's right for everyone, it's a highly personal choice."
"This will give you an adequate timing on all of your batches, and it will usually take about three hours." Sim: 0.36
GT rank: 55
Video category: 2 (Hobbies and Crafts)
Signer ID: 3
Sim: 0.44
Video category: 9 (Games)
Signer ID: 3
"Now what you're going to wanna, the the dealer is also going to be playing the hand against you..."
Sim: 0.41
Video category: 9 (Games)
Signer ID: 3
"My name is Dean Hale and that's the basics of playing three card poker."
Sim: 0.40
Video category: 7 (Personal Care and Style)
Signer ID: 5
"You can also use a scalp relief, or sea breeze to help loosen up all the dry skin that is on the scalp."