Using 'find', 'exec' and 'mv' - A More Complex Example

This example follows on from my previous example of using find in conjunction with the nifty -exec command. It builds on my previous example as it adds a bit of complexity through the use of the 'mv' command and parameters AFTER the '{}'. The commands are listed in bold below along with the results.

First up I run the 'find' command to get a listing of all the '*.tar' files within the current directory structure.

The second bold command shows how I use the results of the listing from the first command to feed into the 'mv; command. What I'm doing here is basically saying;
- Find all the files in the current directory tree that end in *.tar.
- For each file name thats found we get the path to it.
- then we call 'exec' and pass in the file path to the 'mv' command.

So the end result is to find all of the *.tar files and then to move them to the directory of my choice (temp_burn in this case).

mkopka@random:~/Sandbox$ find . -name '*.tar'
./lisasoft/DefaultProject.tar
./terrapages/TerraLoad.tar
./terrapages/TPLoc-Demo.tar
./terrapages/TPAPI.tar
./terrapages/OpenGIS-API.tar
./terrapages/TerraDirectory.tar
./terrapages/TerraAddressHelper.tar
./terrapages/FreeMaps.tar
./sa.deh.tar
./telstra/WebFOOT/lib/ldapSchemaUpdate.tar
./telstra-WEBFOOT.tar
./sa.epa/eelf.tar

mkopka@random:~/Sandbox$ find . -name '*.tar' -exec mv {} ~/temp_burn/ \;

As you can see above, the location to 'mv' the files to is given after the '{}'. So, in this situation you can think of the '{}' as simply a placeholder for the path from the command above it. This means that you could have the '{}' used as a parameter anywhere within whatever command you are using.

Using 'find' and 'exec'

I needed to go through all of the files in my project workspace just recently and in particular I was looking for certain string. Now in Windoze world I would've done a GUI search for text in file. How do I do the same in Linux world. Well this is one way:

find . -exec grep "BLABLABLA" '{}' \; -print

What this does is as follows:

1. First find all files starting at '.', ie: at my current location
2. For every file that I find execute the command grep "BLABLABLA" '{}', where '{}' contains the results of the find command (in this case a listing of every file path.
3. 'print' out any resulting files that have this text in it.

Sweet. All done and the results I get are something along the lines of:

mkopka@random:~/$ find . -exec grep "year.development" '{}' \; -print
<center>Copyright © @year.development@ TerraPages International</center>
./src/html/common/footer.jsp.in
<filter token="year.development" value="2006, 2007" />
./build.xml


Which shows me that there are two files 'footer.jsp.in' and build.xml which have the text that I am looking for, and it also gives me the path to each file and the context of the text Im searching for is printed out.

Some links to more info:
- http://en.wikipedia.org/wiki/Find
- http://www.ibm.com/developerworks/aix/library/au-unix-find.html

Installing Tomcat5.5 On Ubuntu 8.10 (Intrepid Ibex)

So whats next in my intrepid (he he he ... get it ..intrepid...Intrepid Ibex, ubuntu 8.10....ahhh never mind :-) ) adventures in trying to get a *nix development environment going. Well. I got Java happening in my last post. A web server would be good now. So tomcat here we come.

I want to install Tomcat 5.5 as our SOE at work is still Java 5 and Tomcat 5.5 so I went to the following URL at the tomcat website and downloaded three packages as tar.gz files:

http://tomcat.apache.org/download-55.cgi

which where called:

mkopka@random:~$ ls -l martys_stuff/
total 9892
-rw-r--r-- 1 mkopka mkopka 2401560 2009-04-20 15:33 apache-tomcat-5.5.27-admin.tar.gz
-rw-r--r-- 1 mkopka mkopka  855053 2009-04-20 15:32 apache-tomcat-5.5.27-deployer.tar.gz
-rw-r--r-- 1 mkopka mkopka 6478912 2009-04-20 15:32 apache-tomcat-5.5.27.tar.gz

Next i setup some directories where I wanted my setup to go. Im used to the following so this is what I used but your not limited to this:

  /home/mkopka/tools/apache/tomcat-5.5.27

Then i unzip and extracted the contents of each of the tar.gz files into the aforementioned directory. Now I thought, hooray Im done. I went to the bin directory and tried to run:

  ./catalina.sh -version

...and rather than a happy version number proving all is good, I was rewarded with the following:

  mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ ./catalina.sh version 
      Neither the JAVA_HOME nor the JRE_HOME environment variable is defined
      At least one of these environment variable is needed to run this program

Well, heres me thinking Java was all happy after my last post, but obviously somethings not quite right so I did the following:

mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ echo $JAVA_HOME

mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ java -version 
java version "1.5.0_16"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-b02)
Java HotSpot(TM) Server VM (build 1.5.0_16-b02, mixed mode)
mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ which java
/usr/bin/java
mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ ls -l /usr/bin/java
lrwxrwxrwx 1 root root 22 2009-04-02 13:42 /usr/bin/java -> /etc/alternatives/java
mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ ls -l /usr/bin/java
lrwxrwxrwx 1 root root 22 2009-04-02 13:42 /usr/bin/java -> /etc/alternatives/java
mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ ls -la /etc/alternatives/java
lrwxrwxrwx 1 root root 40 2009-04-14 18:23 /etc/alternatives/java -> /usr/lib/jvm/java-1.5.0-sun/jre/bin/java
mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ ls -la /usr/lib/jvm/java-1.5.0-sun/jre/bin/java
-rwxr-xr-x 1 root root 65116 2008-05-28 18:30 /usr/lib/jvm/java-1.5.0-sun/jre/bin/java

So as you can see above, yep, no JAVA_HOME setup. And by doing some ls,-eing I was able to find where my java was installed. So time to set a JAVA_HOME and JRE_HOME:

    /usr/lib/jvm/java-1.5.0-sun/   <--- thats the location of my Java 5

   export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun/
 export JRE_HOME=/usr/lib/jvm/java-1.5.0-sun/jre

I added these to my ~/.bashrc so that I would have these everytime i started a shell. Now running the same ./catalina.sh version command i get:

mkopka@random:~/tools/apache/tomcat-5.5.27/bin$ ./catalina.sh version
Using CATALINA_BASE:   /home/mkopka/tools/apache/tomcat-5.5.27
Using CATALINA_HOME:   /home/mkopka/tools/apache/tomcat-5.5.27
Using CATALINA_TMPDIR: /home/mkopka/tools/apache/tomcat-5.5.27/temp
Using JRE_HOME:       /usr/lib/jvm/java-1.5.0-sun/jre
Server version: Apache Tomcat/5.5.27
Server built:   Aug 28 2008 10:08:26
Server number:  5.5.27.0
OS Name:        Linux
OS Version:     2.6.27-11-generic
Architecture:   i386
JVM Version:    1.5.0_16-b02
JVM Vendor:     Sun Microsystems Inc.

Woohoo!!!! It works I have the version number which is always a good start. So next, its time to cross fingers and try to start up the sever.

  ./catalina.sh run

and I was happy to see a line saying:

   INFO: Server startup in 1651ms

...and most importantly going to : http://localhost:8080/ shows me the tomcat startup page! We have lift off!!!

Okay, okay, okay, lets not get too excited yet. time to kill the server and setup some security stuff.

Setup Security
In order to use the tomcat manager and admin components you need to setup a username and password. There is a tomcat-users.xml file that can be used for this purpose:

     $CATALINA_HOME/conf/tomcat-users.xml

is where the file is located in my install. Within it you'll find a number of users. You need to create a new admin user entry:

   <user username="admin" password="admin" roles="admin,manager">

this is the user name and password to use for the admin and manager console and along with this you need the roles created as well so that the tomcat-users.xml file now looks like:

<tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="role1"/>
  <role rolename="admin"/>
  <role rolename="manager"/>
  <user username="tomcat" password="tomcat" roles="tomcat"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="role1" password="tomcat" roles="role1"/>
  <user username="admin" password="admin" roles="admin,manager"/>
</tomcat-users/>

Library Driver Setup
Tomcat needs to have the DB drivers added to it for Oracle and PostgreSQL. If you don't add them you will get the error messages as shown here:

Cannot load JDBC driver class 'oracle.jdbc.driver.OracleDriver'
Put the Oracle JDBC jar into Tomcats lib directory: $CATALINA_HOME/lib (not into the applications libs directory!).
Note: if you use Tomcat 5.5.x: use $CATALINA_HOME/common/lib (and not $CATALINA_HOME/server/lib)!

Cannot load JDBC driver class 'org.postgis.DriverWrapper'
The full message is:

org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot load JDBC driver class 'org.postgis.DriverWrapper'


Same as above, requires the Postgis class added to the tomcat libs directory.

So you need to add the driver libraries. The libraries are added to:

  $CATALINA_HOME/common/endorsed  (or common/lib directory - not sure which one, try one then other if fails still)

and the libraries that need to be added are:

  ojdbc14.jar
  postgis_1.1.0.jar
  postgresql-8.1-407.jdbc.jar

Testing of Port Setup
At this stage you should have a Tomcat install of the web server, the admin and the manager consoles. First check:

  $CATALINA_HOME/conf/server.xml

and confirm that the port number you are using is 8080. This is done by checking that the following block is available within the server.xml:

  <service name="Catalina">
      <connector port="8080" redirectport="8443" minsparethreads="25" connectiontimeout="20000" maxsparethreads="75" maxthreads="150">
      <connector port="8009" redirectport="8443" protocol="AJP/1.3">
      <engine defaulthost="localhost" name="Catalina">
          <realm classname="org.apache.catalina.realm.UserDatabaseRealm">
            <host appbase="webapps" name="localhost"></host>
          </realm>
      </engine>
      </connector>
      </connector>
  </service>

Startup Of Server
Then you should be able to start the server by going to the bin/ directory and running

   ./catalina.sh run | tee ../logs/standard_out.log 

which runs Tomcat in the same window and also feeds the logs to the log file listed.

Testing of Server
You should now be able to go to :

 http://localhost:8080/

which should bring up the tomcat start page. On the left side should be the options to go to the Tomcat manager and Administration. Test that you can go to both of these. The user name and password that were used in tomcat-users.xml is what is required to log in to both.

Deployment of Web Applications
Using the Tomcat Manager you can deploy a war file by selecting it with the browse button and then deploying it through the Tomcat Manager Deploy Tool. This should cause it to eventually appear in the Tomcat Manager list of applications. Then you should be able to either click on the application or type in:

  http://localhost:8080/

(ie: /directory, /opengis-xls, etc etc, depending on the web app you deployed.)

Changing / Selecting Java Versions in Ubuntu / XUbuntu

Okay, so I had Java6 installed by default when I installed XUbuntu. Now at work we primarily still use Java5 due to the backwards comparability etc etc, and so I needed to run that as my default JVM. Not a problem I thought.

(BTW: If I dont make it to the end of this article/post its the hiccups man!!!!I've got a mo*&&*ther f&*^king case of hiccups that just wont freaking go away. they been holding on to me for the last freaking hour and Im stuck as a fuck tryign to get rid of them. My chest is starting to hurt and I juast cant make them stop. Every freakin 10 seconds I hiccup!!! SHIIIIIIT!!!!....anyways bacak to Java Versions ey...although if this post stops half way through then you've been warne and all i could manage to do was to despreately push the publich button).

So I used Synaptic Package Manager and I happily installed Java 5. Then I went ahead and I type:

mkopka@random:~/Sandbox$ java -version
java version "1.6.0_10"
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) Server VM (build 11.0-b15, mixed mode)


Ok, well thats no good. I want my Java 5 that I just installed. So what to do. A bit of research led me to a couple of things. First of all:

  mkopka@random:~/Sandbox$ which java 
  /usr/bin/java
  mkopka@random:~/Sandbox$ ls -l /usr/bin/java
  lrwxrwxrwx 1 root root 22 2009-04-02 13:42 /usr/bin/java -> /etc/alternatives/java
  mkopka@random:~/Sandbox$ ls -la /etc/alternatives/java
  lrwxrwxrwx 1 root root 36 2009-04-11 18:04 /etc/alternatives/java -> /usr/lib/jvm/java-6-sun/jre/bin/java
 mkopka@random:~/Sandbox$ ls -la /usr/lib/jvm/java-6-sun/bin/java
 lrwxrwxrwx 1 root root 15 2009-04-02 13:39 /usr/lib/jvm/java-6-sun/bin/java -> ../jre/bin/java

Okay, so that led me through a bunch of soft links from /usr/bin to /etc/alternatives to /usr/lib/jvm/java-6 and then back up a directory to the jre/bin. Interesting. Whats more insteresting was this:

   mkopka@random:~/Sandbox$ ls -l /usr/lib/jvm/
   total 8
   lrwxrwxrwx  1 root root   23 2009-04-11 18:03 java-1.5.0-sun -> java-1.5.0-sun-1.5.0.16
   drwxr-xr-x 10 root root 4096 2009-04-11 18:04 java-1.5.0-sun-1.5.0.16
   lrwxrwxrwx  1 root root   19 2009-04-02 13:39 java-6-sun -> java-6-sun-1.6.0.10
   drwxr-xr-x  8 root root 4096 2009-04-11 18:04 java-6-sun-1.6.0.10

So it seems like we got sim links galore occurring. Which made me think, this is gonna be a total shit trying to re-simlink all of that mess.Surely theres got to be another way.......and guess what? There is!!! Thank god almighty for XUbuntu!!!

Okay so this is what I did:

mkopka@random:~/Sandbox$ sudo update-alternatives --config java
[sudo] password for mkopka:

There are 2 alternatives which provide `java'.

Selection    Alternative
-----------------------------------------------
*+        1    /usr/lib/jvm/java-6-sun/jre/bin/java
        2    /usr/lib/jvm/java-1.5.0-sun/jre/bin/java

Press enter to keep the default[*], or type selection number: 2
Using '/usr/lib/jvm/java-1.5.0-sun/jre/bin/java' to provide 'java'.
mkopka@random:~/Sandbox$ java -version
java version "1.5.0_16"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_16-b02)
Java HotSpot(TM) Server VM (build 1.5.0_16-b02, mixed mode)

The line above in bold is all i needed to be provideed with a nice menu for configuring my default java versioon. And then a quick java -version and looky looky we now are running java 5 as the default. Damn it I luuurve linux/XUbunutu more and more the more I use it!.

So then to switch the javva version and the javac version that is being used these are used:

    sudo update-alternatives --config java ---> is used to choose the java version
    sudo update-alternatives --config javac --> is used to change the javac version

Setting up Dual Boot Ubuntu / Windows OS-

Setting up Dual Boot

Im setting up a dual boot OS at work. This is a mixture of Windows XP and Ubuntu .... or to be more precise Xubuntu as its Ubuntu with an XFCE4.4 Windows Manger. What follows is some of the stuff that Ive learnt as Ive gone along.

Partitioning / Mounting / Un-mounting
After installing everythign one of the things I needed to do was set up some mounts for the partitions. I ended up with the following partitions on a 300Gig Hard Drive:

1. 50 GB Windows NTFS Partition - Windows XP OS installed on this. Having 50GB meant that not only could I put the OS there but Id have a good 45GB left over to install various apps there (ie: development tools for my job)
   
2. 48 GB Linux / XUbuntu ext3 Partition - this was the XUbunut OS install partiton which was done with the same ideas in mind as the Windows one above. ie: enough to instal lthe OS and then have another 45GB of free space left over to put all my bits and bobs in.
3. 2GB Linus Swap Partition - The reasons I used the odd 48GB above and not 50GB was so tyhat I could assing 2GB to swap space (this was on the advice of my works MIS department so who am I to argue with people who do this for a living :-) )
4. 200GB NTFS Partition - So what was left was 200GB of space which I formatted as NTFS filesystem and am going to use this as my data storage area. ie: an area that will be accessible by both Windows and Linux and will be where I store data in. Applications will be run from the OS partition but storage will occur here and hence the largish 200GB partition.
   

So I now had to set this up. The actual partitions where set up during the installation of Ubuntu but as I logged in I noticed that all I could see was the 48GB Linux partition. Hmmm....I used a tool called GParted:

sudo apt-get install gparted

which is able to provide a graphical view of the partition and allow you to mount/unmount etc. On start up this tool complained that it couldn't mount two partitions called 50GB Volume and new Volume respectively.

So first I had a look at the /etc/fstab and /etc/mtab files. These show the volumes to auto mount on startup and the volumes currently mounted an in these cases neither one had my missing partitions.

Next I attempted to mount the storage data partition figuring that since it was empty I was less likely to cause any damage if I did something wrong. I wanted to see what my partitions were called and any other info about them so had a look at :

ls -la /dev/disk/by-uuid/
ls -la /dev/disk/by-id/
ls -la /dev/disk/by-label/
ls -la /dev/disk/by-path/


which allows me to see what the partitions are named that are available. Well it turns out my partitions are at:

/dev/sda1
/dev/sda2
/dev/sda3
/dev/sda4


with the sda1 and 2 being my windows and data partitions. Okay. So next i attempted to do:

mkdir /mnt/windows
mount /dev/sda1 /mnt/windows

to show myself that yes, I can actually mount the directory. At this stage I was ready to update the /etc/fstab file so that in the future my partitions where auto-mounted. My addition to the fstab file was as follows (the following is a cat /etc/fstab with the last few lines being the new ones):


# /etc/fstab: static file system information.
#
#
proc /proc proc defaults 0 0
# /dev/sda4
UUID=fffe5c7d-2046-4254-a5b5-5433508d4732 / ext3 relatime,errors=remount-ro 0 1
# /dev/sda3
UUID=725ec9c9-88a9-45f3-a7b9-01a96db36c64 none swap sw 0 0
/dev/scd0 /media/cdrom0 udf,iso9660 user,noauto,exec,utf8 0 0
#
# /dev/sda1 and /dev/sda2 are both ntfs FS's with them being windows boot partition and 200Gig data store respectively
# - could also use LABEL=DATA cos ive set the LABEL to DATA for /dev/sda2 using ntfslabel, ie: ntfslabel LABEL DATA
# - blksize=4096 is added to the end by default when mount -a is run
# - sda1 uses type ntfs and ro so that the windows partiton can be read but not written to (for safety there should be no reason to write to it)
# - sda2 uses type ntfs-3g and rw so that the data partition can be written to and it also becomes fuseblk on mount -a (which is default and normal)
# - Note: the idea is windows and linux partitions can write to the /mnt/data partition BUT not to each other, hence ro and rw options
# - not sure about relatime use (perhaps atime with its performance hit would be better)
#
/dev/sda1 /mnt/windows ntfs auto,nodev,exec,suid,ro,user,uid=mkopka,gid=mkopka,allow_other,utf8 0 0
/dev/sda2 /mnt/data ntfs-3g auto,nodev,exec,suid,rw,user,uid=mkopka,gid=mkopka,allow_other,relatime,utf8 0 0
#


with the last two lines being the important ones that perform the mount. The comments provide info on what the options mean and why I selected them/

And there you go, I have now got auto mounted partitions. By running:


sudo mount -a


I was able to now mount the two new partitions.