Tech Site & Blog Blog about technology & programming

ffmpeg and x264

How to install ffmpeg with x264 codec support on Debian Lenny (5.0)?

Why is this useful? By using x264 codec rather than a FLV video format you will gain on video and audio quality.

1. We need to remove any previous ffmpeg installations.

2. Add additional repository to /etc/apt/sources.list

Open file with nano editor:

nano /etc/apt/sources.list

and add following line on the end of the file:

deb http://www.debian-multimedia.org stable main

3. Install needed libraries:

apt-get install build-essential subversion git-core checkinstall yasm texi2html libfaac-dev libfaad-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libsdl1.2-dev libx11-dev libxfixes-dev libxvidcore4-dev zlib1g-dev libogg-dev

4. Install x264

git clone git://git.videolan.org/x264.git
cd x264
./configure
make
sudo checkinstall --pkgname=x264 --pkgversion "1:0.svn`date +%Y%m%d`" --backup=no --default

5. Install libtheora

wget http://downloads.xiph.org/releases/theora/libtheora-1.1.1.tar.gz
tar xzvf libtheora-1.1.1.tar.gz
cd libtheora-1.1.1
./configure
make
sudo checkinstall --pkgname=libtheora --pkgversion "1.1.1" --backup=no --default

6. Install ffmpeg

svn checkout svn://svn.ffmpeg.org/ffmpeg/trunk ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-version3 --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-x11grab
make
sudo checkinstall --pkgname=ffmpeg --pkgversion "4:0.5+svn`date +%Y%m%d`" --backup=no --default

7. Install qt-faststart

Normal mp4 file needs to be fully downloaded before it can be played by Flash Player. qt-faststart fixes this.

Type following command from ffmpeg source directory:

make tools/qt-faststart

Example:

ffmpeg -i input.mov -acodec libfaac -ab 128k -ac 2 -vcodec libx264 -vpre hq -crf 22 -threads 0 output.mp4
./tools/qt-faststart output.mp4 output2.mp4

Now you can play output2.mp4 by any Flash player. LongTail's freeware JW Player is quite good, has many options and can play both mp4 and flv.

Now you can use ffmpeg to compress videos with x264 codec. Read more about converting video with PHP

Warning: Latest ffmpeg version has some problems with FLV format and it's impossible to convert video to .flv with it! If you need flv support try installing older version of ffmpeg (if you install it by apt-get install ffmpeg you will be able to convert to FLV but there is no x264 support).

Source: http://ubuntuforums.org/showthread.php?t=786095

Use MySQL database in your C++ projects

How to use MySQL DB in your C++ projects. This tutorial is written for Debian Lenny/Ubuntu 9.10. With a few minor changes, it should work on other distributions also.

What will you need?

- gcc <-- c++ compiler

- MySQL server 5.0 installed

- MySQL C API

To install above programs run this command as root:

apt-get install build-essential mysql-server mysql-client libsoci-mysql-gcc

When you have everything installed you need to create database and add user (MySQL Add User).

Now you will need to add a new table. The easiest approach is to use phpMyAdmin.

Simple program to test MySQL C API (copy it and save as filename.cpp):

#include <iostream>
#include <mysql.h>

using namespace std;

//MySQL variables
MYSQL *connection, mysql;
MYSQL_RES *result;
MYSQL_ROW row;
int query_state;

int main() {
//Connects to MySQL DB
mysql_init(&mysql);

connection = mysql_real_connect(&mysql, "localhost", "username", "password", "database", 0, 0, 0);

//Checks if there was a problem with connection
if(connection == NULL) {
cout <<mysql_error(&mysql);
return 1;
}

//Execute query
query_state = mysql_query(connection, "SELECT * FROM table");

if(query_state != 0) {
cout << mysql_error(connection);
}

//Loads query data
result = mysql_use_result(connection);

//Writes data to stdio from first column on all rows in table
while((row = mysql_fetch_row(result)) != NULL) {
cout << row[0] << endl;
}

mysql_free_result(result);
mysql_close(connection);
}

You can compile it with following command:

g++ filename.cpp -L/usr/include/mysql -lmysqlclient -I/usr/include/mysql

If there was no error displayed then your program compiled successfully. You can run by typing:

./a.out

And now you made your first C++ program that gets data from MySQL DB. You can do a lot more than just that.

You will probably need to read MySQL C API Manual to learn others commands.

Next time I will write about preparing Apple Xcode and Microsoft Visual Studio for using MySQL C API.

How to remove Linux from a Mac?

I don't really see why would you do that, but here is a how to about removing Linux from a Mac.

Linux creates few partitions (usually / and swap, it's also a common practice to have /home partition). Everything would be easy if only Mac OS's built-in Disk Utility would support deleting Linux partitions. But sadly that's not the case (even on Snow Leopard).

Here you have two options how to delete partition. First option is to use Windows Vista/7 Install DVD.

1. Put DVD in Super Drive and boot from a DVD (hold alt key on bootup).

2. Select regional settings and fresh install.

3. At the partition editor select every Linux partition and delete it. Windows installation actually deletes partitions immediately as opposite to Linux's practice where this is done on the end of the installation wizard.

4. Turn off Mac and remove DVD.

Second, more known approach is to use GParted Live CD which can be downloaded at http://gparted.sourceforge.net/livecd.php. You can also "burn" it on USB key.

1. Put DVD in Super Drive and boot from a DVD (hold alt key on boot up).

2. Go trough the text-based wizard (probably you won't need to do anything than just pressing Enter).

2. Select partitions you want to remove (be careful not to delete OS X's partitions) and click on Remove (it may take a few seconds).

4. Turn off Mac and remove DVD.

Now that you deleted partitions you can resize your OS X's partition with Disk Utility (/Applications/Utilities/Disk Utility.app).

Click on your HD and select Partition tab.

Resize the partition by dragging its left corner to the end of volume. Click Apply.

In the pop-up click Partition to complete the task (I'm not responsible for any problems that may occur during this step).

And after successful partitioning you will end up with only one OS X's partition (actually there is another one, but it's hidden from Disk Utility).

mod_python and MySQLdb

How to install mod_python and MySQLdb in Debian Lenny?

MySQL is one of greatest databases for web sites, and for applications and it's probably most used with PHP. Python is a good language for a lot of things. In my case, I needed some daemons, to do certain tasks. Firstly, I tried with PHP, but it is language for web pages, and it don't perform well as a daemon. So then I have to choose which language to use. C++ is powerful but very complex and there are other languages, that could fit me better for this project. I don't have a lot of idea about Perl and so I tried programming Python.

I faced two problems in last two days:

1. I had problems with installing MySQLdb

2. How to make and publish a simple website in Python?

MySQLdb

I got a few errors while trying to install MySQLdb by using python file.py install. As far as I searched the Internet, there are some workarounds but neither of them worked for me. And then it hit me. Doesn't Debian comes with apt, a great package tool?

Installing this plug-in is very easy with apt. You just need to run following command:

apt-get install python-mysqldb

And now you can use MySQL from Python.

Example:

#! /usr/bin/python

import MySQLdb
import sys #for exit

try:
conn = MySQLdb.connect(host="localhost", user="root", passwd="", db = "database", charset = "utf8", use_unicode = True)
except MySQLdb.Error, e:
print "Error %d: %s" % (e.args[0], e.args[1])
sys.exit(1)

cursor = conn.cursor()
cursor.execute("SET NAMES utf8;SET CHARACTER SET utf8;SET character_set_connection=utf8;")
cursor.close()

cursor = conn.cursor()

while(1):
cursor.execute("SELECT * FROM table")
if cursor.rowcount > 0:
rows = cursor.fetchall()
for row in rows:
if row == None:
break
print "Data: %s %s" % (row[0], row[1])

This is an example for UTF-8 charset encoding. I still don't know why I needed to close cursor after executing SET NAMES utf8, but this code should work without a problem.
After my daemon feetched all needed data, I need to display it through web browser. PHP came to my mind, but if one thing is in Python, why not make other thing to? So for that I needed mod_python.

How to install mod_python and use it?

It can be installed by typing following command:

apt-get install libapache2-mod-python

You need to put following code in .htaccess or your Apache's configuration file to use Python:

AddHandler mod_python .py
PythonHandler mod_python.publisher

Now just write Python code, save it as something.py and put it in your website's root.

How to speed up loading of your website

Is your website slow? Do you even realize that it could load faster?

Every webmaster tries to make web sites faster, so people will get better user experience. But there are few tricks that not everyone is aware of.

How to improve loading speed?
- cache static content (pictures, javascripts, css files)
- remove ETag from pictures
- compress content

First thing is to cache static content so that it can be loaded from visitor's computer and not from server on every visit. Caching will also decrease bandwidth. It's good to set expire tag for at least a year.

Every picture sent by a server include ETag by which browser can check if a picture from a server match picture from browser's cache. This is not needed on static content, which will never change (for example, uploaded pictures).

And don't forget about compressing content. Loading will be faster because a browser needs to download a compressed file which is smaller and it will save a little bit of bandwidth. Compression is good for static and also dynamic content (.htm/.html/.css/.js/.php).

Example .htaccess file:

Header unset ETag
FileETag None

<FilesMatch "^.*\\.(ico|flv|jpg|jpeg|png|gif|js|css)$">
Header unset Last-Modified
Header set Expires "Fri, 21 Dec 2012 00:00:00 GMT"
Header set Cache-Control "public, no-transform"
</FilesMatch>

<FilesMatch "\\.(js|css)$">
SetOutputFilter DEFLATE
</FilesMatch>

You need to put this file in a root directory of your website. Be careful, file doesn't have any name, only extension "htaccess". If you're using nice/SEO URLs then there is a good chance of you having this file already in a root directory. Only thing you need to do is to open existing file and copy this example code on the bottom of the file. It will only work with Apache HTTPD Server (most popular http server).

Compressing dynamic PHP file

To compress PHP file you need to add following code on the top of the file:
<?php
ob_start("ob_gzhandler");
?>

Or a bit safer method, that will turn off GZIP compression if visitor's web browser doesn't support it:
<?php
if(!ob_start("ob_gzhandler")) ob_start();
?>

You can check site's performance with YSlow add-on for Firefox's developer plugin named Firebug.
Download: http://developer.yahoo.com/yslow/

Convert video to FLV with PHP

How to convert video to .flv with php so that it can be played by Flash Player?

This is a simple approach to converting videos to .flv with PHP. For example, users are allowed to upload videos, and then you can play it by Flash Player.

Installing FFmpeg

Probably you already have PHP installed, so let's continue with FFmpeg. Following commands works only in Debian/Ubuntu distribution.

To install FFmpeg you simply need to type:

apt-get install ffmpeg

FFmpeg is a library for converting video and capturing screenshots from video.

Now you need some sort of a plug in to allow PHP to access ffmpeg. So we need ffmpeg-php, which can be installed by following command:

apt-get install ffmpeg-php

Now you should be able to use it.

Example PHP script:

<?php
if($_FILES['file']) {
if (!$_FILES["file"]["error"] > 0) {
move_uploaded_file($_FILES["file"]["tmp_name"], $_FILES["file"]["name"]);

$movie = new ffmpeg_movie($_FILES["file"]["name"]);

$srcWidth = $movie->getFrameWidth(); //Movie width
$srcHeight = $movie->getFrameHeight(); //Movie height
$srcFPS = $movie->getFrameRate(); //Movie frame rate
$srcAB = intval($movie->getAudioBitRate()/1000); //Audio bit rate
$srcAR = $movie->getAudioSampleRate(); //Audio sample rate

exec("/usr/bin/ffmpeg -i ".$_FILES["file"]["name"])." -ar 22050 -ab 32 -f flv -s ".$srcWidth."*".$srcHeight." new_movie.flv");
//Convert a movie from original type to FLV

exec("/usr/bin/ffmpeg -i new_movie.flv -an -ss 00:00:05 -an -r 1 -vframes 1 -y new_movie%d.jpg ");
//Make screenshot of a movie at 5th second
}
}
?>
<form action="upload.php" method="post" enctype="multipart/form-data">
File: <input type="file" name="file" /> <input type="submit" value="Upload" />
</form>

This script saves an uploaded file, converts it to FLV and makes a screenshot of the movie at 5th second.

VPN or how to access US sites abroad

If you live outside US you cannot watch Hulu/CBS/etc. videos on demand because you need US IP. The other reason to use VPN is a secure connection to VPN provider. So your ISP doesn't know what are you browsing until you are connected to VPN.

There is some free and some payable services. Payable usually have no problems with speed and blocked IPs.

1. AnchorFree Hotspot Shield

It is a great service and it's free too. Downsides are that their IPs are blocked from VoD (video on demand) web sites and connection is not the fastest. Another downside is that you need to install their program in order to get service to work.

2. Payable service HappyVPN

I decided to try a payable service and I heard a lot good about HappyVPN, so I decided to give it a try.

It's 14.99$ per month and you will get unlimited access. Nice offer. So I payed for the first month and watched a few of episodes of various TV shows. Sometimes I got disconnected but I weren't worry about it too much. And after a week or a liitle bit more I am not able to even connect to this service.

So I decided to cancel my subcription to them.

You can try if you want. They offer payment by credit card or PayPal.

3. The best option is it's own server or VPS (Virtual Private Server)

They are from 20$ per month and you will really get unlimited access. There is only one downside, they have limited bandwidth.

So I bought one node from VPS.Net and try it out. You can pay with PayPal (I prefer this method, however they do not offer automatic payment through PayPal, like GoDaddy does) or credit card.

How to buy a node and create VPS at VPS.net

1. Go to http://vps.net/, select one node (or more if you need more bandwidth or multiple VPS for some reason) and click Buy Now!

2. Now you need to type your information and select your payment type. Then pay for it.

3. Go to your account and select Create new VPS.

4. Type VPS label and host name. For host name you can use your domain or sub domain. You don't actually need a domain.

5. Then select your cloud. UK to get access to UK sites (like BBC iPlayer) or US cloud to get access to US web sites (like CBS or FOX).

6. Select Debian 5.0 (Lenny) x64 and in the bottom box select VPN image. Then click Create.

7. You will be redirected to a page with your VPS's information. Wait a minute or two and then hit refresh.

8. Your VPS should be ready now. Use SSH program from Terminal (Mac/Linux) or Putty for Windows.

9. Connect to IP of your VPS and login as root with default password written on VPS's information page.

10. Type following command to edit username/password to access VPN server:

nano /etc/ppp/chap-secrets

11. Go to the end of file and type something like:

username pptpd password *

(change username and password with your desired username/password)

12. Save file and exit (Ctrl + C and then type "Y" (without quotes))

13. Type following command to reboot pptpd service in order to make VPN work:

/etc/init.d/pptpd restart

That's it for server side.

How to connect to the VPN network with Mac

Now that you bought VPN access or created VPS with VPN service you need to prepare your Mac to send all your Internet connections trought your VPN.

1. Go to System Preferences.

2. Go to Network and press on "+" button in the bottom left corner. Then change interface to VPN, select PPTP as VPN Type and enter your desired VPN Service Name.

3. Now type your IP address and username:

4. Click on Advanced and set up like in following picture:

5. Click Ok and then Apply. Now you need to type your password and hope everything is allright.

Everything should work now. You can check by going to http://www.ip-adress.com/ and if IP on site match your VPS's IP then you're good.

I'll write more about connecting to VPN on Windows in next blog post.

Essential security tips for servers

In this tutorial you will find out some of the necessary steps to secure you VPS or dedicated server.

1. First thing you need to do is to change your root password. Connect to server with SSH and type:

passwd

Then type in your new password, press enter, and retype it again.

2. It's not safe to allow root user to login throught SSH. So you need to create a new user:

useradd user_name

Then set the password for this user with:

passwd user_name

3. Now let's go to SSH settings.

Open file by typing:

nano /etc/ssh/sshd_config

And find/change following:

PermitRootLogin no
X11Forwarding no
AllowUsers user_name
Port 10000

It's important to change your SSH port to higher number (for example 10000).

4. Configure iptables:

iptables-save > /etc/iptables.rules
nano /etc/iptables.rules

Example rules:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 -i ! lo -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 10000 -j ACCEPT #Change this port to SSH server's port
-A INPUT -p icmp -m icmp --icmp-type 8 -j DROP
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
-A INPUT -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j ACCEPT
COMMIT

Be careful to change your SSH port to the one you set in sshd_config, otherwise you wouldn't be able to log in to SSH server.

Import rules to iptables:

iptables-restore < /etc/iptables.rules

5. Set new iptables rules to reset during reboots:

sudo nano /etc/network/interfaces

...
auto lo
iface lo inet loopback
pre-up iptables-restore < /etc/iptables.rules
...

6. And now reload SSH server:

sudo /etc/init.d/ssh reload

These are only essential steps to secure you server. Your server should now be a little bit more secure, but there is no such thing as 100% security.

Install Sun Java on Debian Lenny

Installing Java on Windows is easy. Simply download .exe file and run it. But on Debian Linux it's a little bit harder because it's non-free software.

It's free as in beer. So you can actually "taste" it, but cannot get the source code.

How to install Sun Java on Debian Lenny?

1. Open Terminal and login as root (type "su" and then type a password)

2. Type:

nano /etc/apt/sources.list

3. In file change:

deb http://ftp.uk.debian.org/debian/ lenny main
deb-src http://ftp.uk.debian.org/debian/ lenny main

to:

deb http://ftp.uk.debian.org/debian/ lenny main non-free
deb-src http://ftp.uk.debian.org/debian/ lenny main non-free

Save it.

4. Refresh aptitude packages by typing:

apt-get update

5. Type:

apt-get install sun-java6-jre

to install Java 6 from Sun.

Setting Charset on an Apache Server

How to demand from an Apache Server to serve pages with a specific encoding?

You will ask this question sooner or later if you are making web sites. Probably sooner if you are dealing with non-ASCII characters.

There are a lot of charsets in the world, from ones that support most languages to ones that are very specific. In web development business UTF-8 is a standard, specially on non-english web sites. It's not that big of a problem on HTML sites, but it can be a hell of a mess if you have server with first encoding, PHP with second encoding and MySQL with completely different encoding.

In this post, I will show you how to tell Apache to use specific charset (in our examples I'll use UTF-8).

The easiest way to set encoding is by following meta tag in HTML:

<meta http-equiv="content-type" content="text/html; charset=UTF-8">

But this is not always enough. So to be sure you need to specify charset in the Apache configuration file or in .htaccess.

You can use one of the following methods:

<FilesMatch "\\.(htm|html|css|js|php)$">
ForceType 'text/html; charset=UTF-8'
</FilesMatch>

or

AddType 'text/html; charset=UTF-8' html

or

AddCharset UTF-8 .html

or

AddDefaultCharset UTF-8

I recommend last option because I had no problems what so ever with it so far. It always works fine. (Other should too.)