Friday, April 29, 2011

Notes: Rsync syntax SSH to another port

Rsync to another port (port:555) using SSH:
$> rsync -avh --progress --rsh='ssh -p555' /my/path/ username@192.168.1.1:/my/path/

Thursday, April 28, 2011

Installing MySQL 5.5.11 Generic Tar Binaries in Ubuntu

I decided to install MySQL from binary in my fresh Ubuntu 10.10 Maverick machine. I installed from binary since I do not want to depend on the packages from Ubuntu again and its nice to have a the newest version of MySQL.

1) Download the binary package from mysql.com. I chose to download  mysql-5.5.11-linux2.6-x86_64.tar.gz since that matches my system:
$> cd /usr/local/src/
$> wget http://dev.mysql.com/get/Downloads/MySQL-5.5/mysql-5.5.11-linux2.6-x86_64.tar.gz/from/http://mysql.he.net/
$> mv index.html mysql5.5.11.tar.gz
$> tar -xzvf mysql5.5.11.tar.gz 

Note: that when i did wget it named the file "index.html" for some reason so i had to rename it back to "mysql5.5.11.tar.gz" then untar.
Tip: if you dont know your system version I did: $>uname -m

2.) Copy the files to /usr/local
$> cp -R mysql5.5.11 /usr/local/mysql

NOTE: At first. I wanted to copy to /usr/local/lib/mysql but for some reason when I try to do the install command, I got some file pointer problems such as not finding mysqld_safe file so I just decided to stick to /usr/local/mysql directory which is the recommended installation directory according to mysql docs. I think some install codes may have hard coded paths to "/usr/local/mysql" hence the reason for not working.

2b.)Optional. If you really want to copy to /usr/local/lib/mysql or some other directory for example besides the usual /usr/local/mysql directory you need to copy the mysql directory to /usr/local/lib first then soft link to it but notice that I am still using the /usr/local/mysql destination. This is much better setup if you want to upgrade mysql later, you just change the soft link!:
$> cp -R mysql5.5.11 /usr/local/lib/mysql5.5.11
$> cd /usr/local
$> sudo ln -s lib/mysql-5.5.11/ mysql

3) Install other libraries:
$> sudo apt-get install libaio

Notes: I installed a couple more libraries that i ended up not using since I chose to install Mysql binary. No need to cmake/make/make install.

4) Make sure mysql user exists so create "mysql" as a group and user. I have to because of my new computer.
$> groupadd mysql
$> useradd -R -g mysql mysql

5) Copy bin files to /usr/local/bin from /usr/local/mysql/bin. I prefer to soft link instead.
$> ln -s /usr/local/mysql/bin/* /usr/local/bin/

6) Create the socket directory if it does not exist:
$> mkdir /var/run/mysqld
$> chown -R mysql:mysql /var/run/mysqld

7) Copy the sample MySql Configuration file to /etc/ and edit it.
$> cd /usr/local/mysql/support-files/
$> cp my-large.cnf /etc/my.cnf

Note: you can also copy it to /etc/mysql/my.cnf if you prefer. Now edit my.cnf:
$> vim /etc/my.cnf

user = mysql
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
tmpdir = /tmp
log_error = /var/log/mysql/error.log
bind-address = xxx.xxx.xx.xx

Note: I chose to bind mine to a different IP since I plan to make this box as an exclusive db server. You dont have to do this.
Warning: Make sure you get the paths correctly or MySql will complain upon running the install script. Trust me on this s double check!

8) Copy the MySql Server startup script to /etc/init.d
$> cd /usr/local/mysql/support-files
$> cp mysql.server /etc/init.d/mysql
$> sudo chmod +x /etc/init.d/mysql


Now this will install it as a startup:$> update-rc.d mysql defaults


9) Change owner in /usr/local/mysql to "mysql" then install.
$> cd /usr/local/mysql
$> sudo chown -R mysql:mysql ./data

Now run the install script. Dont forget to add --user=mysql
$> sudo ./scripts/mysql_install_db --user=mysql --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data

Note: I need to add --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data in order for my install to work so just to make sure. Notice too that I only did this if I don't have the grants table yet because this is a fresh install so you may not need to do this if upgrading.

10) Start the server daemon. Hopefully everything works if not check the folder permissions in /data.
$> sudo mysqld_safe --user=mysql &

Also try to run $>mysqld --user=mysql & if you are having problems. If mysqld does not work try running as a regular user, not sudo.

Notes: Notice the "&" . If you don't add the ampersand sign you'll get stuck in a blank command line. It means run the script in the background.

Most of my problem with MySql not starting up is the /var/run/mysqld socket not generating so you may try running as below if MySql is being stubborn:
$> sudo /etc/init.d/mysql start

11) Do some test to make sure everything is fine.
$> ps aux | grep mysql

Should output something like:

mysql    19272  0.0  1.0 592836 85036 pts/0    Sl   18:04   0:00 /usr/local/mysql/bin/mysqld --basedir=/usr/local/mysql --datadir=/usr/local/mysql/data --plugin-dir=/usr/local/mysql/lib/plugin --user=mysql --log-error=/var/log/mysql/error.log --pid-file=/usr/local/mysql/data/Franz-Ubuntu-DB.pid --socket=/var/run/mysqld/mysqld.sock --port=3306

Then do some test like below and it should produce some result:
$> mysqladmin version
$> mysqladmin variables

Shutdown the server and start it again. You should not have any problems if you did it right
$> sudo mysqladmin -u root shutdown
$> sudo mysqld_safe --user=mysql &

Test also mysql startup
$> sudo /etc/init.d/mysql stop
$> sudo /etc/init.d/mysql start

12) If everything is good then change root's password. If you are not sure about the hostname below, skip this step and edit it in phpmyadmin if you have it installed. Much easier there.
$> mysqladmin -u root password "newpassword"
$> mysqladmin -u root -h host_name password "newpassword"

NOTE: if you are unfamiliar with what host you are it is usually "localhost" or 127.0.0.1
I also wrote a blog here http://kelmadics.blogspot.com/2011/06/notes-target-mysql-hosts.html if you want to create specific db for certain users and host.

13) Optional, create a new user that you can use to access from outside the localhost
$> mysql -u root -p

Now in mysql command line:
mysql> CREATE USER 'new-user-name'@'%' IDENTIFIED BY 'new-user-password';

GRANT ALL PRIVILEGES ON * . * TO 'new-user-name'@'%' IDENTIFIED BY 'new-user-password' WITH GRANT OPTION MAX_QUERIES_PER_HOUR 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0 MAX_USER_CONNECTIONS 0 ;


14) Optional. To take it further if you have PhpMyAdmin, edit its config.inc.php and add this to be able to easily administer MySql:
$i++
$cfg['Servers'][$i]['verbose'] = 'New Mysql Server';
$cfg['Servers'][$i]['host'] = 'xxx.xx.xx.xx';
$cfg['Servers'][$i]['port'] = '';
$cfg['Servers'][$i]['socket'] = '';
$cfg['Servers'][$i]['connect_type'] = 'tcp';
$cfg['Servers'][$i]['extension'] = 'mysqli';
$cfg['Servers'][$i]['auth_type'] = 'cookie';
$cfg['Servers'][$i]['user'] = 'new-mysql-user';
$cfg['Servers'][$i]['password'] = '';
$cfg['Servers'][$i]['AllowNoPassword'] = false;


Thats it!

Wednesday, April 27, 2011

TIP: ssh to another port

$ ssh -p 500 192.xxx.xx.xx

Wednesday, April 20, 2011

TIP: PHP array_map with multiple arguments

If you want to use array_map with a native php function that has multiple arguments, do the following.

Our task is to remove the hyphen "-" from each array and replace it with " ". Instead of using a foreach lets use
PHP's array_map():

        $hyphenated = array('fat-dog', 'small-dog', 'thin-dog');
        $container = array_map('str_replace',
                               array_fill(0, count($hyphenated), '-'),
                               array_fill(0, count($hyphenated), ' '), $hyphenated);
Result:
array('fat dog', 'small dog', 'thin dog');   

A thing to note here. The second argument and so forth of array_map reflects exactly the required arguments of str_replace(mixed $search , mixed $replace , mixed $subject) in order so for the example above str_replace $search argument goes in argument number 2 of array_map which is "array_fill(0, count($hyphenated)" (notice how i fill the array to match the subject array's count?), same for the second and the third is the $subject itself which is $hyphenated.

In addition, array_map will only accept an array type on its second and consecutive arguments and also needs to match the count number of subject array or the remaining array values will not be populated at all.

 Now if you are using PHP 5.3 it is better to use Closures or Lambda function:

array_map(function ($content) {
  $content = str_replace('-', ' ', $content);
  $content = ucwords($content);
  return $content;
}, array('fat-dog', 'small-dog', 'thin-dog'));


Now that is neat!!



Tuesday, April 19, 2011

NOTES: soft linking

I tend to forget and get confused with $> ln syntax not being used for awhile.
To use it say you want to link /usr/local/lib/php cli to /usr/bin

$>    cd /usr/bin
                TARGET           LINK NAME
$>    ln -s /usr/local/lib/php php

Monday, April 18, 2011

PHP extension_dir not working?

From my last log post. I installed PHP from source and installed it in another directory. When I tried to install a PECL extension namely APC I was dumb founded that the extension_dir=PATH in my php.ini was not working!!

The reason this was happening was because I installed PECL before already so when I do
$> sudo pecl install apc

The old pecl command was the one that is executing so therefore will not install the extension appropriately. In addition my PHP.ini will not recognize my extension_dir whatever I do.

So the solution is use the binary command that came with your php install. Say I installed mine at /usr/local/php5.3/ then i would execute:
$> sudo /usr/local/php5.3/bin/pecl install apc

The above code will create an extension dir in your /usr/local/php5.3/lib/extension which should hold the apc.so file then your pecl module should work now.

Friday, April 15, 2011

Ubuntu PHP 5.3.6 installation from source in a system with existing PHP 5.2 and Apache 2

I have been contemplating on installing PHP 5.3 on my Ubuntu box since it takes awhile for Ubuntu to upgrade their APT packages but Im afraid of messing around with the install cause i got important live web files in my server and don't want to ruin the perfectly working PHP 5.2 version.

In either case, I finally summoned the guts to install PHP 5.3 from source and bit the bullet. I did it from source because I dont want to depend on the Ubuntu packages on PHP anymore and want the ability to upgrade easily in the next major PHP release.

1.) So I downloaded PHP 5.3.6 from php.net and did all the untarring.
$> wget http://www.php.net/get/php-5.3.6.tar.gz/from/a/mirror
$> tar -xzvf php-5.3.6.tar.gz

2.) Then I have a bunch of library dependencies that I need to install. Here apt-get works a charm.
$> apt-get install libxml2-dev
$> apt-get install libbz2-dev
$> apt-get install libcurl4-openssl-dev
$> apt-get install libjpeg-dev
$> apt-get install libpng12-dev
$> apt-get install libXpm-dev
$> apt-get install libfont-freetype-perl
$> apt-get install libfreetype6-dev
$> apt-get install libxslt-dev

I got a little bit more than the above library dependencies but their pretty easy to install with apt-get so I didn't bother to document them.

*For those who are new to compiling, note that the above packages are depending upon your build. I have to install the libraries above because if you look at my build configure below, I enabled this feature in PHP. If you do not know what libraries you are required to have then just run configure with your build options, then the script will halt if it cannot find the libraries. From there you can see your requirements although this takes time especially if you are enabling a bunch of feature. There also maybe much more efficient way in doing this though.

Another problem I cannot figure out was the --with-snmp, I keep getting the
error: SNMP sanity check failed.
My solution was to remove it from the build for now since I do not use. Feel free to suggest a solution though.

UPDATE: I solved this problem by installing package libsnmp-dev.
Look here in my blog for details: http://kelmadics.blogspot.com/2011/06/php-install-problems-with-snmp.html

TIP: One command I use if apt cannot find the package is search for it, for example:
apt-cache search freetype

Also, since I installed Apache differently before, the --with-apxs2=PATH will not work. I have to remove the threaded version and install Apache prefork:
$> sudo apt-get remove apache2-threaded-dev
$> sudo apt-get install apache2-prefork-dev

3.) Build time! Here is what I used to build:
$> cd php-5.3.6
$> ./configure \
--prefix=/usr/local/lib/php5.3.6 \
--with-apxs2=/usr/bin/apxs2 \
--with-mysql=/usr/local/mysql \
--with-mysqli=/usr/local/mysql/bin/mysql_config \
--with-pear=/usr/local/lib/php5.3.6/lib/php \
--with-tidy \
--with-curl \
--with-curlwrappers \
--with-openssl-dir \
--with-xpm-dir \
--with-pdo-mysql \
--with-xsl \
--with-ldap \
--with-xmlrpc \
--with-iconv-dir \
--with-bz2 \
--with-mcrypt \
--with-jpeg-dir \
--with-png-dir \
--with-zlib-dir \
--with-freetype-dir \
--with-gd \
--with-snmp \
--enable-mbstring \
--enable-zip \
--enable-exif \
--enable-calendar


A thing to note here. I used --prefex=PATH to install it from a different location since I already have an existing PHP 5.2 and don't want to overwrite anything.

4.) Ok, finally done with the build, time for "make & make install".
$> make
$> make install

Hit a minor snag though. I got this error:
chmod 755 /usr/lib/apache2/modules/libphp5.so
apxs:Error: Activation failed for custom /etc/apache2/httpd.conf
file..
apxs:Error: At least one `LoadModule' directive already has to exist..
make: *** [install-sapi] Error 1 

Quick googling and the fix is just adding a dummy place holder on whichever file it is looking for. In the case above it is httpd.conf so just add or append in httpd.conf the following code:
# Dummy LoadModule directive to aid module installations
#LoadModule dummy_module /usr/lib/apache2/modules/mod_dummy.so


Take note of the "#". That is included!

Reference: Here's where I got the solution from for the above problem: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=231134

Please note that doing "make install" will overwrite your php5 module in /usr/lib/apache2/modules/libphp5.so so if you want to have a backup do it BEFORE you do "make install"

After adding the dummy text, do make install again.
$> make install

5.) Optional. After that I copied my existing php.ini from /etc/apache2/php5 to where ever I installed my php5.3 library. In my case this is /usr/local/lib/php5.3.6/lib
$> cp /etc/apache2/php5/php.ini /usr/local/lib/php5.3.6/php.ini

6.) Restart apache and you have php5.3 installed. I didn't even have down time!
$> sudo apache2ctl restart

IMPORTANT: One more final note since we placed a dummy module line in httpd.conf, this caused
the install to write a line: LoadModule php5_module /usr/lib/apache2/modules/libphp5.so
in httpd.conf but since you may already have this line in /etc/apache2/mods-available/php5.load if you  have Apache installed already through aptitude, restarting apache will warn that you have a duplicate module!

Solution is to simply remove the extra generated line from httpd.conf and everything should work!!!

Tuesday, April 5, 2011

Tip: MySql ORDER BY FIELD()

Say you have a rowset of:

Categories table:
id     |     Category
1      |    Books
2      |    Electronics
3      |    Other
4      |    Autos
5      |    Furniture


And you want to order in a way that you would put "Other" category as the last element and also order the other elements in an ascending way.

Simply use ORDER BY FIELD():
SELECT *  FROM `categories` ORDER BY field(Category, 'Other') ASC, Category ASC

Notice the placement of FIELD() before "name ASC". You need to do this so that it will also reorder the remaining rows in ascending.

Result
id     |     Category
5      |    Autos
1      |    Books
2      |    Electronics
5      |    Furniture
3      |    Other