It is well known fact that nowadays more and more ISPs in the world use Ubuntu driven servers as proxies, domain name servers, routers etc. Squid is one the major open source web caching proxy software for Ubuntu and Linux as a whole. Term ‘caching‘ means a way to store Internet objects on locally deployed proxy server for sake of reducing bandwidth consumption and access time to popular web content. You may read more about squid caching software at wikipedia or squid-cache.org.
Youtube videos can also be cached and squid web caching proxy under Ubuntu is reasonable choice for this purpose. It’s common practice when one user downloads some very popular youtube video and then shares its URL to other users withing the same organization. Caching of such youtube video using squid will definitely save Internet connection bandwidth and traffic as every time users download that video it will be fetched from local web cache rather than from youtube servers.
Use Synaptic package manager to install squid package, here is the line to get it installed using terminal:
sudo aptitude install squid
(please notice that youtube caching requires squid of at least 2.7STABLE6 version).
Once installed you should edit /etc/squid/squid.conf (main configuration of squid proxy) to apply configuration necessary to cache video from youtube. Below is an example of such configuration file for squid-2.7STABLE9 so you are welcome to open squid.conf with your favorite text file editor and copy/paste below config into it.
1. Open terminal and type the following commands to get started:
sudo echo -n > /etc/squid/squid.conf
(this cleans default squid.conf configuration file)
sudo gedit /etc/squid/squid.conf
(paste below configuration example and save changes)
http_port 3128 access_log none coredump_dir none cache_dir ufs /var/spool/squid/ 10000 16 256 acl all src all acl manager proto cache_object acl localhost src 127.0.0.1/32 acl to_localhost dst 127.0.0.0/8 acl ssl_ports port 443 acl safe_ports port 80 21 443 1025-65535 acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access deny !safe_ports http_access deny CONNECT !ssl_ports http_access allow all icp_access deny all acl QUERY2 urlpath_regex get_video\? videoplayback\? videodownload\? cache allow QUERY2 acl QUERY urlpath_regex cgi-bin \? cache deny QUERY acl youtube dstdomain .youtube.com cache allow youtube acl shoutcast rep_header X-HTTP09-First-Line ^ICY.[0-9] upgrade_http0.9 deny shoutcast acl apache rep_header Server ^Apache broken_vary_encoding allow apache refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern -i \.flv$ 10080 90% 999999 ignore-no-cache override-expire ignore-private refresh_pattern (get_video\?|videoplayback\?|videodownload\?) 5259487 99999999% 5259487 override-expire ignore-reload negative-ttl=0 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern . 0 0% 4320 quick_abort_min -1 KB maximum_object_size 4 GB minimum_object_size 512 bytes acl store_rewrite_list urlpath_regex \/(get_video\?|videodownload\?|videoplayback.*id) storeurl_access allow store_rewrite_list storeurl_access deny all storeurl_rewrite_program /etc/squid/storeurl.pl storeurl_rewrite_children 1 storeurl_rewrite_concurrency 10
2. Create /etc/squid/storeurl.pl file:
sudo gedit /etc/squid/storeurl.pl
(paste below configuration example and then save changes)
#!/usr/bin/perl $|=1; while (<>) { @X = split; $x = $X[0]; $_ = $X[1]; if (m/^http:\/\/([0-9.]{4}|.*\.youtube\.com|.*\.googlevideo\.com|.*\.video\.google\.com).*?\&(itag=[0-9]*).*?\&(id=[a-zA-Z0-9]*)/) { print $x . "http://video-srv.youtube.com.SQUIDINTERNAL/" . $2 . "&" . $3 . "\n"; } else { print $x . $_ . "\n"; } }
3. Apply execution bit to /etc/squid/storeurl.pl, here is terminal command for this:
sudo chmod +x /etc/squid/storeurl.pl
4. Now it’s time to start squid daemon:
sudo service squid startIn order to check it squid was started successfully type the following command:
sudo netstat -lnp | grep 3128
It should show one line in case of success. You can also point Ubuntu browser to 127.0.0.1:3128 as a proxy and try loading some web page like www.ubuntuka.com which could also set as a start page there 
From this point squid running in your Ubuntu can cache youtube videos. You can check this by visiting the same youtube via different browsers or point several other hosts in the same LAN to squid proxy and try visiting some youtube videos. Each time user downloads some youtube video it becomes cached by squid proxy so next time the same user or other LAN subscriber tries to visit the same video it will be downloaded from squid cache rather than from youtube servers — you can see the difference in access time to cached video.
P.S. According to squid.conf mentioned in this the maximum file size of cached video is 4GB and web cache size is 10GB (make sure you have enough free space on the disk for this). You can tune these parameters by editing maximum_object_size and cache_dir lines in /etc/squid/squid.conf respectively.
Slither
Posted on July 19, 2010 at 10:45 amThis config won’t work properly cause it’s outdated, the storeurl script doesn’t rewrite the videoplayback link (that is now equivalent of the old videodownload link).
iMadalin
Posted on July 29, 2010 at 8:40 pmi haven't heard and used squid for about 4-5 years. with todays broadband speed (20-100mbit) i don't see the point, and ISPs from my country like others from other countries colaborate with google for Youtube hosting (a kind of special CDN, most of content that you watch on youtube is hosted by your ISP, if it's a serious major isp)
For example in Romania, Romtelecom plays a big role with it's 2 huge datacenters.
For many years bandwith has gone so low that cacheing content became more expensive than direct connection of client to the internet (i've worked on a small local isp as a sysadmin and been through this steps.
Guillermo Garron
Posted on August 30, 2010 at 10:57 amI live in Bolivia (South America) and here bandwidth is too expensive, we are just too far away from the big Datacenters, so this config make sense if it works.
Actually I use squid since some 7 years ago in my office, and it helps specially with the people reading the same newspaper online every morning, while some others are making Skype calls.
B James
Posted on August 13, 2010 at 7:02 pmYou would be better off looking into nginx's reverse proxy caching, though in most cases caching is generally done in reverse now. Products like nginx or varnish are used to cache high performance versions of content on busy web sites leaving the application servers to handle less direct traffic. With that said, setting up caching systems when you have say a business with 5,000 employees and 100mbps to the net, it can help greatly. All a CDN is is a cache, but your ISPs cache is not going to help you serve the video internally to 50 people concurrently, its there to save Google and your ISP money on data transfer. The only bandwidth in the world that is cheap is "consumer" connections which are not meant to actually provide that data transfer rate 24/7/365, data center bandwidth while cheaper than it use to be is truly not "cheap" when you consider the networking equipment and redundancies involved.
Leslie Viljoen
Posted on December 13, 2010 at 6:20 pmTHANKS, this works great! The only things I had to change were:
1. I did the whole thing as root, so "sudo su" at the start. This avoids the error with "sudo echo -n > /etc/squid/squid.conf"
2. The perl script has been messed with by the web filter. Replace each & with an ampersand and the while at the top needs to be: while(<>)
How to Cache Youtube videos using Squid in Ubuntu Linux | Ubuntu - infomediaku.com
Posted on February 11, 2011 at 7:47 am[...] : http://www.ubuntuka.com/squid-cache-youtube-ubuntu-linux/ [...]
ubuntu
Posted on February 21, 2011 at 8:05 pmi'm replace that, and the squid didn't work fine, and can't caching the youtube. i'm use ubuntu 8.04
when see the log don't see TCP_HIT only TCP_MISS.
any help please ???
Setting up a Squid Proxy Caching co-resident with bandwidth controller. « NetEqualizer News Blog
Posted on May 1, 2011 at 10:28 pm[...] More on caching Youtube with Squid [...]
Nawshad Ahmed
Posted on May 13, 2011 at 8:03 amwill you please post the correct perl script? it will be very helpful. I tried but all the time it said squid fail
Squid Command Does Not Work, Cannot Change Permission
Posted on September 17, 2011 at 12:23 pm[...] permissions of `/etc/squid/squid.conf': Operation not permitted What doesnt gedit work? http://www.ubuntuka.com/squid-cache-…-ubuntu-linux/ [...]
El baby
Posted on October 17, 2011 at 1:14 pmThe 'sudo echo -n' problem can resolved like this:
sudo sh -c "echo -n > /etc/squid/squid.conf"
this is easier
Posted on December 13, 2011 at 2:07 pmthis is easier:
sudo > /etc/squid/squid.conf
joe lawandos
Posted on January 9, 2012 at 5:36 pmhi guys is there a way to cache http://testmy.net/ like speedtest.net its cachable but not testmy.net any way of helping thanks
the reason is i serve example 256k up to 2meg from my cache hit and i want to show the clients the highest bandwith all wat they see is 256k so most of thee speed test ar cachable some ar not i need help to setup the code in refresh pattern or the storeurl if its posible
joe lawandos
Posted on January 9, 2012 at 5:36 pmhi guys is there a way to cache http://testmy.net/ like speedtest.net its cachable but not testmy.net any way of helping thanks
the reason is i serve example 256k up to 2meg from my cache hit and i want to show the clients the highest bandwith all wat they see is 256k so most of thee speed test ar cachable some ar not i need help to setup the code in refresh pattern or the storeurl if its posible
joe lawandos
Posted on January 9, 2012 at 5:36 pmhi guys is there a way to cache http://testmy.net/ like speedtest.net its cachable but not testmy.net any way of helping thanks
the reason is i serve example 256k up to 2meg from my cache hit and i want to show the clients the highest bandwith all wat they see is 256k so most of thee speed test ar cachable some ar not i need help to setup the code in refresh pattern or the storeurl if its posible
joe lawandos
Posted on January 9, 2012 at 5:36 pmhi guys is there a way to cache http://testmy.net/ like speedtest.net its cachable but not testmy.net any way of helping thanks
the reason is i serve example 256k up to 2meg from my cache hit and i want to show the clients the highest bandwith all wat they see is 256k so most of thee speed test ar cachable some ar not i need help to setup the code in refresh pattern or the storeurl if its posible
joe lawandos
Posted on January 9, 2012 at 5:36 pmhi guys is there a way to cache http://testmy.net/ like speedtest.net its cachable but not testmy.net any way of helping thanks
the reason is i serve example 256k up to 2meg from my cache hit and i want to show the clients the highest bandwith all wat they see is 256k so most of thee speed test ar cachable some ar not i need help to setup the code in refresh pattern or the storeurl if its posible
joe lawandos
Posted on January 9, 2012 at 5:36 pmhi guys is there a way to cache http://testmy.net/ like speedtest.net its cachable but not testmy.net any way of helping thanks
the reason is i serve example 256k up to 2meg from my cache hit and i want to show the clients the highest bandwith all wat they see is 256k so most of thee speed test ar cachable some ar not i need help to setup the code in refresh pattern or the storeurl if its posible
Bill Riemers
Posted on January 10, 2012 at 3:09 pmEpic failure. The videos themselves do not proxy. Someone else commented the the storeurl.pl script is obsolete. Can someone provide the correct script?
Kumpulan PORT « FUCKUP
Posted on January 16, 2012 at 2:10 pm[...] used by web caches and the default port for the Squid Cache proxy [...]
BASIT KHAN
Posted on January 18, 2012 at 8:47 pmhi guys… I found LUSCA CACHE on internet search, it work perfactly fine on all video website… I think you should also try it.
Its fantastics