Within the final submit we took a take a look at downloading songs from Soundcloud. On this submit we are going to check out Fb and the way we will create a downloader for Fb movies. It began with me desirous to obtain a video from Fb which I had the copyrights to. I needed to automate the method in order that I may obtain a number of movies with only one command.
Now there are instruments like
youtube-dl which might do that job for you however I needed to discover Fb’s API myself. With none additional ado let me present you step-by-step how I approached this venture. On this submit we are going to cowl downloading public movies. Within the subsequent submit I’ll check out downloading personal movies.
Step 1: Discovering a Video
Discover a video which you personal and have copyrights to. Now there are two sorts of movies on Fb. The primary sort is the general public movies which may be accessed by anybody after which there are personal movies that are accessible solely by a sure subset of individuals on Fb. Simply to maintain issues straightforward, I initially determined to make use of a public video with plans on increasing the system for personal movies afterwards.
Step 2: Recon
On this step we are going to open up the video in a brand new tab the place we aren’t logged in simply to see whether or not we will entry these public movies with out being logged in or not. I attempted doing it for the video in query and that is what I bought:
Apparently we will’t entry the globally shared video as effectively with out logging in. Nevertheless, I remembered that I lately noticed a video with out being logged in and that piqued my curiosity. I made a decision to discover the unique video a bit extra.
I right-clicked on the unique video simply to examine it’s supply and to determine whether or not the video url was reconstructable utilizing the unique web page url. As a substitute of discovering the video supply, I discovered a unique url which can be utilized to share this video. Check out these photos to get a greater understanding of what I’m speaking about:
I attempted opening this url in a brand new window with out being logged in and increase! The video opened! Now I’m not certain whether or not it labored simply by sheer luck or it truly is a legitimate option to view a video with out being logged in. I attempted this on a number of movies and it labored each single time. Both Manner, we have now bought a option to entry the video with out logging in and now it’s time to intercept the requests which Fb makes once we attempt to play the video.
Open up Chrome developer instruments and click on on the XHR button identical to this:
XHR stands for XMLHttpRequest and is utilized by the web sites to request further information utilizing Javascript as soon as the webpage has been loaded. Mozilla docs has an excellent clarification of it:
Use
XMLHttpRequest(XHR) objects to work together with servers. You possibly can retrieve information from a URL with out having to do a full web page refresh. This allows a Net web page to replace simply a part of a web page with out disrupting what the consumer is doing.XMLHttpRequestis used closely in Ajax programming.
Filtering requests utilizing XHR permits us to chop down the variety of requests we must look by means of. It may not work at all times so in the event you don’t see something attention-grabbing after filtering out requests utilizing XHR, check out the “all” tab.
The XHR tab was attention-grabbing, it didn’t include any API request. As a substitute the very first requested hyperlink was the mp4 video itself.
This was surprizing as a result of normally corporations like Fb prefer to have an intermediate server in order that they don’t need to hardcore the mp4 hyperlinks within the webpage. Nevertheless, whether it is straightforward for me this fashion then who am I to complain?
My very subsequent step was to seek for this url within the unique supply of the web page and fortuitously I discovered it:
This confirmed my suspicions. Fb hardcores the video url within the unique web page in the event you view the web page with out signing in. We are going to late see how that is completely different when you find yourself signed in. The url in present case is present in a <script> tag.
Step 3: Automating it
Now let’s write a Python script to obtain public movies. The script is fairly easy. Right here is the code:
import requests as r
import re
import sys
url = sys.argv[-1]
html = r.get(url)
video_url = re.search('hd_src:"(.+?)"', html.textual content).group(1)
print(video_url)
Save the above code in a _videoobtain.py file and use it like this:
$ python video_download.py video_url
Don’t neglect to interchange video_url with precise video url of this type:
https://www.fb.com/username/movies/10213942282701232/
The script will get the video url from the command line. It then opens up the video web page utilizing requests after which makes use of common expressions to parse the video url from the web page. This may not work if the video isn’t out there in HD. I depart that as much as you to determine learn how to deal with that case.
That’s all for in the present day. I’ll cowl the downloading of your personal movies within the subsequent submit. That is a little more concerned and requires you logging into Fb. Observe the weblog and keep tuned! If in case you have any questions/feedback/ideas please use the remark type or e mail me.
Have a terrific day!
