Most of my development and technical experience has been in the Microsoft world. Last year I decided to develop a web application using an open source stack. I wrote the app using CakePHP. As I added Solr/Lucene search and started work on deploying to a production environment, server configuration turned out to be as big a job as development.
These are my build and deploy procedures, which include instructions for:
- Installing a Linux server OS and LAMP environment
- Installing and configuring the necessary services
- Installing Tomcat and Solr
- Setting up a CakePHP website
- Installing and configuring Solr/Lucene in Tomcat
I am indebted to the hundreds of blog posts, forum posts, articles, and help docs that got me through every single setup step. In a few cases there are references to sources but practically all of these instructions are based on the work of others. It did take (me) a lot of time to find and collate all the information and I hope that sharing these procedures will help save you considerable time and effort.
Windows is my regular client OS but occasionally I use Mac or Linux (desktop) also. The client OS makes no difference to the Linux server configuration. But the build process can be easier with the use of OS specific tools like WinSCP.
These instructions apply equally to building a Linux web server on bare metal or a virtual machine. I used them for my production server which is dedicated hardware hosted by 1&1 and also for setting up test and development environments hosted on VirtualBox running in Windows or OSX.
A few conventions:
When the instructions say “edit” they mean: use your preferred Linux text editor. I use nano, so to create and/or edit a file called solr.xml located outside my home directory, I would enter: sudo nano solr.xml
My home network uses a typical NetGear router on a 192.168.1.x network. I reference this IP range throughout the instructions but this address should be replaced with the appropriate IP address for your environment.
The procedures are specifically for my site so when it comes to naming things like the server, be sure to substitute a name appropriate to your requirements.
If using VirtualBox to host the Linux server, install VirtualBox and reboot the host machine. In VirtualBox, create a Linux, Ubuntu, 64-bit virtual machine. Before starting the virtual machine, change network setting to bridged adapter.
Download the Ubuntu 12.04.1 LTS server amd-64 iso file to the host machine and mount or reference it so that VirtualBox uses it to install the OS.
The terminal provided by VirtualBox is lacking. On a Windows or Linux host I use Putty; on Mac I use the included Terminal app. To transfer files between a Windows host and the Linux target environment, WinSCP is very convenient.
Install Ubuntu 12.04.1 LTS server amd-64. Depending on the hosting environment, download and mount the iso, burn it to a CD and pop it in the drawer, or install from an image provided by the hosting company.
- Hostname: mysite
- Partitioning method: “Guided–use entire disk” and default guidance.
- Choose not to encrypt home directory files
- No proxy
- Automatic security updates
- Manual package selection
- Choose Grub boot loader to master record
- Create root user password
sudo apt-get update
sudo apt-get install openssh-server
ifconfig will show the IP address of the Linux server for connecting with an SSH terminal and WinSCP or for adding to your hosts file.
sudo apt-get install apache2
sudo apt-get install php5
sudo apt-get install mysql-server
- Choose auto-configure Apache2 web server
sudo apt-get install php5-mysql
sudo apt-get install tomcat6
sudo apt-get install zip
Install the Java JDK
Downloading the Java JDK requires clicking “yes” in a web form to agree to T&C. So download the file to a Windows client and then push to Linux using WinSCP.
- Download from: http://www.oracle.com/technetwork/java/javase/downloads/java-se-jdk-7-download-432154.html
- Windows may strip the “.tar” from the filename, so restore to [FILENAME].tar.gz prior to pushing to linux.
- On Linux server, to unpack xxx.tar.gz file: tar –xzf xxx.tar.gz
- cd jdk1.7.0_10/
- sudo rm -f src.zip
- cd ..
- sudo cp -rf jdk1.7.0_10/ /opt/jdk
- edit /etc/environment and add: JAVA_HOME=/opt/jdk
- sudo mv /usr/bin/java /usr/bin/java1
- sudo ln -s /opt/jdk/bin/java /usr/bin/java
- Either refresh the environment file or restart the machine
- The command java -version should return the version of Java just installed.
sudo apt-get install phpmyadmin
Login to phpmyadmin:
- In a browser navigate to http://192.168.1.x/phpmyadmin
- Login using root credentials
Default webroot directory is /var/www
mv cakephp-cakephp-850a33d cakeroot
mv cakeroot /usr/share/cakephp/cakeroot
Change owner of the cakephp files so that the web server process can modify
- cd /usr/share/cakephp
- ls –l command shows current owner and permissions. Note owner of cakeroot.
- sudo chown -R www-data.www-data /usr/share/cakephp/cakeroot
- ls –l to check change of ownership of cakeroot directory.
CONFIGURE APACHE2 TO RUN CAKEPHP WEBSITE
Create the website under apache.
- cd /etc/apache2/sites-available
- sudo nano mysite
- Paste the following lines into the file and save.
<VirtualHost *:80> ServerAdmin webmaster@localhost ServerName mysite.com ServerAlias www.mysite.com *.mysite.com DocumentRoot /usr/share/cakephp/cakeroot <Directory /> Options FollowSymLinks AllowOverride All </Directory> <Directory /usr/share/cakephp/cakeroot/> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all </Directory> ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/ <Directory "/usr/lib/cgi-bin"> AllowOverride None Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch Order allow,deny Allow from all </Directory> ErrorLog /var/log/apache2/mysite_error.log # Possible values include: debug, info, notice, warn, error, crit, # alert, emerg. LogLevel warn CustomLog /var/log/apache2/mysite_access.log combined Alias /doc/ "/usr/share/doc/" <Directory "/usr/share/doc/"> Options Indexes MultiViews FollowSymLinks AllowOverride None Order deny,allow Deny from all Allow from 127.0.0.0/255.0.0.0 ::1/128 </Directory> # Deny access to phpmyadmin # <Directory /usr/share/phpmyadmin> # Order Deny,Allow # Deny from all # </Directory> </VirtualHost>
- Enable or disable phpmyadmin access within this file by commenting or uncommenting lines at the bottom of the file.
sudo a2dissite default
sudo a2ensite mysite
sudo a2enmod rewrite
Edit the apache config file to add a custom environmental value (http://bakery.cakephp.org/articles/stevena0/2010/08/29/use-different-configs-for-different-environments):
- sudo nano /etc/apache2/apache2.conf
- at bottom of file add: SetEnv CAKE_ENV production
sudo /etc/init.d/apache2 restart
For testing non-production machine, modify the hosts file on your local Windows client:
- Open notepad as administrator and edit: C:\Windows\System32\drivers\etc\hosts (on Linux client machine the file is: /etc/hosts).
- 192.168.1.x mysite.com
- 192.168.1.x www.mysite.com
- Or, for local testing, use 127.0.0.1
- CakePHP website: 192.168.1.x OR mysite.com
- Tomcat: 192.168.1.x:8080/
- phpmyadmin: 192.168.1.x/phpmyadmin
- Solr admin: 192.168.1.x:8080/solr/admin (not installed yet)
DEPLOY CakePHP APPLICATION
In phpmyadmin, create the database for your custom CakePHP application and import a backup (.sql file) from your development environment. The backup should include permissions for the web app user. But in case you need to re-create the user:
- username: [your application username]
- password: [corresponding password]
- Use default permissions, which grant permission to all standard db objects, but not grant privileges
For non-production, you may want to modify your production data, for example to prevent sending unintended emails: update users set email = ‘firstname.lastname@example.org’
rm the cakeroot /app folder and replace it with your application’s app folder. Use WinSCP to push the CakePHP app directory from your Windows environment to your home directory. Then cp or mv the directory to /usr/share/cakephp/cakeroot/app.
sudo chown -R www-data.www-data /usr/share/cakephp/cakeroot
Website should work fully except search functions.
INSTALL SOLR 3.6.2 UNDER TOMCAT6
- Download and unpack apache-solr-3.6.2.tgz.
- wget http://apache.mirrors.pair.com/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
- tar –zxf apache-solr-3.6.2.tgz
- Create /opt/solr directory. This will be $SOLR_HOME.
- Copy the example/solr directory from the source to $SOLR_HOME. Copy the .war file dist/apache-solr-*.war into $SOLR_HOME as solr.war.
- cp /opt/solr/conf/lang/stopwords_en.txt /opt/solr/conf/stopwords_en.txt
- Create $SOLR_HOME/data. chown -R tomcat6.tomcat6 data
- The configuration file $SOLR_HOME/conf/solrconfig.xml in the example sets dataDir for the index to be ./solr/data relative to the current directory – which is true for running the Jetty server provided with the example, but incorrect for Tomcat running as a service. Modify the dataDir to specify the full path to $SOLR_HOME/data:
- Create a Tomcat Context fragment named solr.xml to point docBase to the $SOLR_HOME/solr.war file and solr/home to $SOLR_HOME. Symlink or place the file in $CATALINA_HOME/conf/(/etc/tomcat6/)Catalina/localhost/solr.xml where Tomcat will automatically pick it up. Tomcat deletes the file on undeploy (which happens automatically if the configuration is invalid).
<?xml version="1.0" encoding="utf-8"?> <Context docBase="/opt/solr/solr.war" debug="0" crossContext="true"> <Environment name="solr/home" type="java.lang.String" value="/opt/solr" override="true"/> </Context>
7. To set up Solr logging in its own file (use /var/log/solr), follow these instructions: (http://skybert.wordpress.com/2009/07/22/how-to-get-solr-to-log-to-a-log-file/)
8. Modify /etc/tomcat6/logging.properties
9. Add the following to the end of the existing comma-delimited list handlers:
10. Below that, add:
6localhost.org.apache.juli.FileHandler.level = FINE 6localhost.org.apache.juli.FileHandler.directory = /var/log/solr 6localhost.org.apache.juli.FileHandler.prefix = solr. org.apache.solr.level=INFO org.apache.solr.handlers=6localhost.org.apache.juli.FileHandler
11. sudo mkdir /var/log/solr
12. sudo chown -R tomcat6.tomcat6 /var/log/solr
13. Overwrite the schema.xml file /$SOLR_HOME/conf with your application’s solr schema file.
14. Delete everything inside the /opt/solr/data/ directory.
15. Restart tomcat service
Test search feature of mysite.com application. Should get no results and no errors.
Create Solr index. In my CakePHP app I added an admin URL that URL that triggers indexing or re-indexing everything in the database.
Search should now return accurate results.