core

Core functionality of YoutubeTimestamper

The program flow for generating timestamps is relatively simple:

Get the id from the youtube video url passed
Fetch the transcripts from the id and concatenate them
Use deepmultilingualpunctuation to restore punctuations
Use spacy to split into senteces
Get sentences that have ? and match them up with the timestamps

YoutubeTimestamper

 YoutubeTimestamper (video_url:str)

A class that extracts the transcript from a youtube video, identifies the questions in it and attaches timestamps to them.

	Type	Details
video_url	str	A url of a YouTube Video

sample_url = "https://www.youtube.com/watch?v=QGCvycOXs2M"
yt_timestamper = YoutubeTimestamper(sample_url)
yt_timestamper

__main__.YoutubeTimestamper(video_url='https://www.youtube.com/watch?v=QGCvycOXs2M')

yt_timestamper.video_url

'https://www.youtube.com/watch?v=QGCvycOXs2M'

test_eq(yt_timestamper._get_yt_id(), "QGCvycOXs2M")

Get transcripts from the id

yt_timestamper._get_transcript()

yt_timestamper.transcript[:3]

[{'text': '[Music]', 'start': 1.28, 'duration': 12.54},
 {'text': "hey this is sayam bhutani and you're",
  'start': 13.92,
  'duration': 2.56},
 {'text': 'listening to', 'start': 15.679, 'duration': 3.68}]

This is the raw unpunctuated transcript

yt_timestamper._transcript_str[:500]

"[Music] hey this is sayam bhutani and you're listening to chai time data science a podcast for data science enthusiasts where i interview practitioners researchers and calculus about their journey experience and talk all things about data science [Music] hello and welcome to quarantine chai with kaggle heroes with this new look i am interviewing the new 4x kaggle grandmaster chris dude chris has an amazing very diverse and very rich background and we connect the dots of his journey his professio"

assert len(yt_timestamper.transcript) > 0
assert "," not in yt_timestamper._transcript_str[:500]
assert "." not in yt_timestamper._transcript_str[:500]

Restore Punctuations

yt_timestamper._restore_punctuations()

/home/ilangurudev/anaconda3/envs/ds/lib/python3.9/site-packages/transformers/pipelines/base.py:1036: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset
  warnings.warn(

The punctuations have now been added.

yt_timestamper._transcript_punct[:500]

"[Music]. hey, this is sayam bhutani and you're listening to chai time- data science, a podcast for data science enthusiasts where i interview practitioners, researchers and calculus about their journey experience and talk all things about data science. [Music]. hello and welcome to quarantine chai with kaggle heroes. with this new look, i am interviewing the new 4x kaggle grandmaster, chris dude. chris has an amazing, very diverse and very rich background and we connect the dots of his journey, "

assert "," in yt_timestamper._transcript_punct[:500]
assert "." in yt_timestamper._transcript_punct[:500]

Getting questions

yt_timestamper._get_sentences()
yt_timestamper.transcript_sents[5:10]

(#5) [chris has an amazing, very diverse and very rich background and we connect the dots of his journey, his professional journey, with data science.,we talk about his previous life.,how did he transition into data science and cargill and his journey on kaggle?,chris at the time of recording has just become a forex kaggle ground master.,he's ranked 32 on the competition steer, two on the data sets tier and is currently number one in kernels and discussions.]

assert len(yt_timestamper.transcript_sents) > 0

yt_timestamper._get_questions()
yt_timestamper.questions[:3]

(#3) ['how did he transition into data science and cargill and his journey on kaggle?','can you tell us a bit more about that, chris, now that you remember of it the secret?','did you get your invite to the fight clubs yet, or can you share a bit about those?']

assert len(yt_timestamper.questions) > 0

yt_timestamper._get_timestamps_for_questions()
assert len(yt_timestamper.timestamps) > 0

yt_timestamper.timestamps

(#36) [(0, 'Introduction'),(68.64, 'how did he transition into data science and cargill and his journey on kaggle?'),(143.52, 'can you tell us a bit more about that, chris, now that you remember of it the secret?'),(168.56, 'did you get your invite to the fight clubs yet, or can you share a bit about those?'),(284.479, "then i, um i graduated with a bachelor's degree in mathematics and then, um, immediately afterwards, i?"),(439.599, 'when did kaggle come into the picture? when did you find your addiction for kagan?'),(508.879, "you know you build a model and is your model more accurate than the other guy's model?"),(615.12, 'how did you go from just starting your journey to today being the forex grandmaster?'),(709.36, 'was it part of the enjoyment process, just getting involved, or were you making a conscious effort to? you know, maybe list down points where you need to improve?'),(760.079, 'and even if a competition or something looks similar to a previous one, i say to myself: you know, what new angle can i do here? you know what new thing can i learn?')...]

Outputs

yt_timestamper._render_timestamps(limit=10)

00:00:00 Introduction
00:01:08 how did he transition into data science and cargill and his journey on kaggle?
00:02:23 can you tell us a bit more about that, chris, now that you remember of it the secret?
00:02:48 did you get your invite to the fight clubs yet, or can you share a bit about those?
00:04:44 then i, um i graduated with a bachelor's degree in mathematics and then, um, immediately afterwards, i?
00:07:19 when did kaggle come into the picture? when did you find your addiction for kagan?
00:08:28 you know you build a model and is your model more accurate than the other guy's model?
00:10:15 how did you go from just starting your journey to today being the forex grandmaster?
00:11:49 was it part of the enjoyment process, just getting involved, or were you making a conscious effort to? you know, maybe list down points where you need to improve?
00:12:40 and even if a competition or something looks similar to a previous one, i say to myself: you know, what new angle can i do here? you know what new thing can i learn?

Created using youtube-timestamper - https://ilangurudev.github.io/youtube-timestamper/

source

YoutubeTimestamper.suggest_question_timestamps

 YoutubeTimestamper.suggest_question_timestamps (next_q_thresh:int=15)

Suggest timestamps based on questions found in the transcripts.

	Type	Default	Details
next_q_thresh	int	15	The number of tokens within a question which if the next question is present, it’ll be considered part of the same question
Returns	None

yt_timestamper.suggest_question_timestamps()

00:00:00 Introduction
00:01:08 how did he transition into data science and cargill and his journey on kaggle?
00:02:23 can you tell us a bit more about that, chris, now that you remember of it the secret?
00:02:48 did you get your invite to the fight clubs yet, or can you share a bit about those?
00:04:44 then i, um i graduated with a bachelor's degree in mathematics and then, um, immediately afterwards, i?
00:07:19 when did kaggle come into the picture? when did you find your addiction for kagan?
00:08:28 you know you build a model and is your model more accurate than the other guy's model?
00:10:15 how did you go from just starting your journey to today being the forex grandmaster?
00:11:49 was it part of the enjoyment process, just getting involved, or were you making a conscious effort to? you know, maybe list down points where you need to improve?
00:12:40 and even if a competition or something looks similar to a previous one, i say to myself: you know, what new angle can i do here? you know what new thing can i learn?
00:13:49 any favorite battle stories you that you might remember?
00:14:33 so there's many competitions and you see the leaderboard and there's a bunch of scores and there's, you know, the top 50, they have a little bit of a jump, and there's and people in the forum say: you know magic, what's the magic?
00:16:03 um, do you, do you have any uh, rigid approach to competitions now, or did you compete in any in parallel, or do you just stick to one? and what's your approach in general to competitions now?
00:22:05 so, and that's something that i also enjoy very much- on the flip side, do you feel you're also losing a competitive edge?
00:25:45 thank you, um, what advice would you have for uh, newbies who are looking to write kernels?
00:30:58 what, what advice would you have for them? they're just chasing uh, maybe the title, since it's gamified, does it annoy you? do you have any uh advice for such people?
00:32:51 uh, any favorite memories from discussions that you might have?
00:34:37 and then someone says, wow, that's awesome, hey, have you thought about doing this?
00:35:35 um, what advice would you have for people who people, many people, seek teaming up because they may be asking to just compete by themselves? but what teaming advice would you have for people in general? or what's your strategy?
00:38:47 what's your reason for investing so much time into kagan?
00:38:58 you know people might see it and say, why is he doing so much for the community?
00:39:05 you know, how have i, how have i, you know, learned these ideas so well?
00:43:49 pandas build psychic learn, okay, uh, but what, what the wrap is team doing is?
00:45:38 is the desk data plotted on top of the train data or is the test data over here? like: is this group of images the same as this group of images?
00:47:11 how do you, chris, how do you manage your time?
00:47:18 what's, what's your secret to time management?
00:49:44 but, as i said, you know, if someone's like, hey, can you? you know, can you follow the?
00:50:30 uh, is it okay if we do a quick rapid fire questions? uh, a set of questions? so what would you like? so, would you like to me give short answers?
00:51:12 okay, a favorite course that you've taught? uh, i'm not sure if you've taught a few or was it just a single one? oh, okay, yeah, so, uh, favorite course?
00:52:11 do you know that? the derivative is exactly what your brain uses for you to learn something?
00:52:38 okay, um, favorite algorithm?
00:52:54 it's just how do you classify this guy? well, how do you class with this data point?
00:53:55 okay, um, favorite competition that you've taken part in?
00:55:20 you're refreshing your computer, refreshing your computer, how did i do? how did i do? it's unbelievable, right?
01:00:28 um, what would be your favorite spot to surf? wait, favorite what sport? to windsurf?
01:01:24 uh, your favorite way to unwind, uh, from data science or kaggle?

Created using youtube-timestamper - https://ilangurudev.github.io/youtube-timestamper/