IMOD - compiling

From NoskeWiki
Jump to navigation Jump to search

About

NOTE: This page is a daughter page of: IMOD


IMOD is a powerful, free and open source suite of software for reconstructing, segmenting and analyzing 3D image data - particularly data acquired via 3D electron tomography, and other electron microscopy techniques. On this page I've documented the process of compiling the IMOD code on a Mac computer. Compiling the code requires the installation/setup of several libraries and environmental variables. This page was last updated Sep 2011, IMOD version 4.2.23 and as of that date the libraries you'll need compiled are:

  • gfortran -IMOD is a very old program and so much of it was created in legacy fortran.
  • jpeg-8b - allows IMOD to save out jpeg files.
  • tiff-3.9.4 - allows IMOD to read and write tiff files.... (for example users often want to take a stack of .tif files and covert it to IMOD's preferred 3D image format: "MRC").
  • zlib-1.2.5 - luckily this library is already pre-installed on most Mac machines, so we don't have to worry about it.
  • Qt-4.6.3 - Qt is used for most of the GUI - it's a powerful library for creating cross-platform GUIs using C++.

Certain programs in IMOD also use Java (most notably "eTomo"), but this is something you hopefully don't have to worry about during compilation if you're using a platform where Java is already installed and setup. Setting up all the necessarily environmental variables and compiling all these libraries correctly gets very tricky, but luckily the primary author of IMOD, Dr. David Mastronarde from the University of Colorado Boulder, is a very helpful chap and has help me set it up twice now - the first time way back in 2007 (see here). If you intend to build IMOD I highly suggest to get into contact with David first, and he'll give you the latest instructions in his elusive "BUILDING" document, since this is not on the IMOD website. David's BUILDING document touches on how to install on all operating systems - here I only talk about installing on a Mac. If you get stuck I highly recommend a tool like Skype's Screen Share to help troubleshoot - worked well for me, but this only a last last resort. On this page I've merely listed out all the commands I typed, mostly as a record for myself, so don't hold me accountable if something goes wrong! Think of these instructions as a "path of least resistance" and keep in mind it will only work if you have a 64-bit Mac (any Mac after 2006) with OS X 10.5 or later - for any other platform you'll need the building document.


Installing IMOD on OS X

(1) Setup all your environmental variables

Setting up the relevant environmental variables is something you have to do before compiling Qt and then compiling IMOD. Rather than setting up these variables one at a time, I prefer to setup all these variables at once as the very first step. To setup environmental variables you'll need to open a Terminal window (Applications/Utilities/Terminal), and now check what type of SHELL you have running by typing "echo $SHELL". By default it is probably in bash, but we want to use csh, so the first two commands to type are: 

csh
pico ~/.cshrc

The first command changes you to the csh shell, the second opens up the file ".cshrc" in your home directory (eg: "/Users/anoske/.cshrc") for editing and odds are this file doesn't exist yet, so it will appear as empty. Paste this code at the top: 

# SETUP THE IMPORTANT ENVIRONMENTAL VARIABLES:
setenv QTDIR /usr/local/Trolltech/Qt-4.6.3
setenv PATH $QTDIR/bin:$PATH
setenv QMAKESPEC macx-g++
if ($?DYLD_LIBRARY_PATH) then
    setenv DYLD_LIBRARY_PATH $QTDIR/lib:$DYLD_LIBRARY_PATH
else
    setenv DYLD_LIBRARY_PATH $QTDIR/lib
endif
if ($?DYLD_FRAMEWORK_PATH) then
    setenv DYLD_FRAMEWORK_PATH $QTDIR/lib:$DYLD_FRAMEWORK_PATH
else
    setenv DYLD_FRAMEWORK_PATH $QTDIR/lib
endif
 
# SETUP THE DIRECTORY WHERE YOU WANT TO OPEN IMOD FROM: 
setenv IMOD_DIR /Users/anoske/Documents/MACMOD
if (-e $IMOD_DIR/dist/IMOD-mac.csh) source $IMOD_DIR/dist/IMOD-mac.csh

To save this hit (ctrl+o) then (ctrl+x) to exit. The most important variables here are: "QTDIR" and "PATH" (helps your compiler know where to find Qt and "qmake" for compilation), "QMAKESPEC" (defines the compiler we want to compile Qt code).... and then "DYLD_LIBRARY_PATH" and "DYLD_FRAMEWORK_PATH" point us to where Qt's library is. The next couple of lines tell us where we want to load IMOD from. Commands in .cshrc are only applied when csh starts, so to apply these changes you'll either have to close and reopen Terminal, or (as an easier method) exit then re-enter csh by entering into the command line: 

exit
csh

Another thing you should never forget is that these environmental variables won't work outside csh. If you type: "echo $IMOD_DIR" in csh it should now return "/Users/anoske/Documents/MACMOD" (and yes, make sure you change these paths to match your username!). If, however, you type "echo $IMOD_DIR" in the bash shell it will probably return either an empty string or, if you have a released version of IMOD installed on your machine, it will echo "/Applications/IMOD". In this way you can use "3dmod" to quickly launch either a released version of IMOD or your "experimental" version IMOD depending on wether you are in bash or csh.


(2) Install libtif + jpeg

Before IMOD compiles you'll need to download and install a few different library packages, so start by creating a folder on your desktop and call it "IMOD_LIBRARIES" (and yes, this folder is temporary so you may delete it the end).

To install libtif and jpeg David has helped by providing the correct versions in a file called "TiffJpeg.tar.gz" which you can download to your IMOD_LIBRARIES folder from: http://bio3d.colorado.edu/ftp/Software/ . After you uncompresses this file (use "gunzip" in command line or "Archive Utility") you'll notice three folders: "jpeg-8b", "tiff-3.9.6" and "zlib-1.2.5". As mentioned above, zlib should already be installed on most Macs which have the development environment, so you won't need it. You should start by compiling jpeg first: 

csh
cd /User/anoske/Desktop/IMOD_LIBRARIES/TiffJpegZlib/jpeg-8b
setenv CFLAGS "-g -02 -m64"
setenv CXXFLAGS "-g -02 -m64" 
./configure
make 
sudo make install

Setting the environmental variables CFLAGS and CXXFLAGS should only be necessary for OSX 10.5, but not 10.6. Next you can compile the tif library: 

cd /User/anoske/Desktop/IMOD_LIBRARIES/
./configure --with-jpeg-include-dir=/usr/local/include --with-jpeg-lib-dir=/usr/local/lib
make
sudo make install

As an alternative: David has already run ./configure on both these libraries, so if you open the makefile and do a find replace with "/User/mast/" to "/User/anoske/Desktop/IMOD_LIBRARIES/" you should just be able to run "sudo make" on each (jpeg first) and I found this option worked fine for me.


(3) Install gfortan

Gfortran is relatively easy to install by downloading the appropriate .dmz files from: http://r.research.att.com/tools/. I downloaded and ran "gfortran-4.2.3.dmg " and also "gfortran-42-5664.pkg (for XCode)" and gfortran seems to work fine. You can check gfortran is installed and the correct version by typing "gfortran" and you should see: 

[stratus:~] andrew_noske% gfortran
i686-apple-darwin10-gfortran-4.2.1: no input files


(4) Install Qt 4.6.3

Something very important to note is that certain features of IMOD won't yet work with version Qt4.7 . The safest version for a 64 bit Mac is Qt version 4.6.3. To install this package download qt-everywhere-opensource-src-4.6.3.tar.gz from the Qt site at ftp://ftp.trolltech.com/qt/source/ and then unzip it (use the "tar -xzf file.tar.gz" command or a application like "stuffit expander"). From memory, I tried installing the ".dmg" file "qt-mac-opensource-4.6.3.dmg" but didn't have any luck (maybe you will) and even the ".zip" version may have had some strange problem for my machine. Here's I'll assume you have unzipped the "qt-everywhere-opensource-src-4.6.3.tar.gz" version. In theory you can install Qt from anywhere and it should install to "/usr/local/Trolltech/Qt-4.6.3/" (as specified by "QTDIR"), but to play it safe you should first move the whole directory you just unzipped to this location using: 

cd usr/local/
mkdir Trolltech/
sudo mv /User/anoske/Desktop/IMOD_LIBRARIES/qt-everywhere-opensource-src-4.6.3 Trolltech/Qt-4.6.3

Next you will want to have Qt configure and then build itself, which you can achieve using: 

./configure -prefix /usr/local/Trolltech/Qt-4.6.3 -fast -no-exceptions \
-no-phonon -no-phonon-backend -no-webkit -no-rpath -no-libtiff -qt-libpng \
-qt-libmng -qt-libjpeg -continue -cocoa -arch x86_64 -release -nomake demos \
-nomake examples -opensource -confirm-license
make

Note here that you can *possibly* just get away with typing ./configure && make, but the lines above make sure you're building using 86 bit and don't bother installing the qt examples and demos which would otherwise take up extra space. Even without building these extra files, expect the two steps above to take a long time (over half and hour) to complete. Make sure you don't run make install or install multiple times, as that can give issues.

  • TIP: If you are having permission issues, or don't like typing sudo every time you might try changing to the directory you want then running "sudo chown anoske .".
  • WARNING: I had a problem at one stage which said #error "You are building a 64-bit application, but using a 32-bit version of Qt. Check your build configuration.", which was a pain to fix, but if you have to just delete Qt-4.6.3 completely and start again.


(5) Use Mercurial to copy IMOD source code from the SIMBA server in Boulder

You are now ready to compile IMOD. To get a copy of the IMOD source code, the easiest option is to request david email you zipped file and unzip this it to a directory such as "/Users/anoske/Documents/MACMOD". If, however you would like to contribute code to IMOD you should think about requesting access to the Mercurial system they have setup... allowing you to check code in and out. Keep in mind this may not be granted to everyone, since there is a chance of stuffing up his code. David used to manage code using cvs, but they switched to Mercurial in sep 2011.

First you'll need to install Mercurial, which is pretty straightforward.... go to: http://mercurial.selenic.com/downloads/ and download the latest Mercurial (I installed 1.9.2) and it's straightforward to install. Next you should restart a Terminal window and the "hg" command show show the list of Mercurial commands.

  • To use Mercurial to transfer a fresh copy of the latest code you'll need to execute these commands:
cd ~/Documents/
hg clone ssh://imodtemp@simba.colorado.edu//home/hg/IMOD MACMOD

This command creates a new directory called "MACMOD" at "/Users/anoske/Documents/MACMOD" and imports all the latest code from Boulder. Other Mercurial commands you may want to use are as follows:

  • To update with latest (changed) code from SIMBA:
cd ~/Documents/MACMOD/
hg incoming
hg pull -u
  (then enter password)
  • To copy back files to the SIMBA server:
cd /Users/a.noske/Documents/MACMOD/plugs/drawingtools/
hg add drawingtools.h
hg add drawingtools.cpp
hg add Makefile
hg commit -m "testing 123" [filename]
hg push

If it asks for a username the best means to set one up is by creating a .hgrc file in your root directory (eg: /Users/andrew_noske/) and instructions on that are here: Mercurial. Some other tips are to run "hg incoming" and "hg outgoing" to see what changes you're about to pull or push, respectively.... and be warned that if you add any folder all files underneath will be added recursively. When pushing files, David suggests this order: 

#1) MANUAL BACKUP!!!   (in case something goes wrong)
#2) commit everything you know you've changed.
#3) hg fetch
#4) hg status
#5) hg push

If you have "merge" issues appear (two files where the same lines have been changed either side), you may have to run commands like "hg diff X", "hg resolve -m X" and "hg commit X" to resolve these problems (see instructions here).


(6) Setup and build IMOD

Assuming you now have a copy of the latest imod code you'll now need to compile it. To setup and compile code: 

cd ~/Documents/MACMOD/
./setup -c gcc
make -j2
make install 
./installqtlib

And now you should be read to add to the code and recompile.

Creating a new "special plugin"

At this stage IMOD should be compiling, but you now want instructions on how to add your own plugin to IMOD. The best place to get started is to read the HTML help page which should now exist under "../MACMOD/html/libdoc/3dmodplug.html"... but here I give some tips on how to go about writing and compiling your first plugin:

  • Go to the "plugs" source directory (".../MACMOD/plugs/") and copy the whole folder "beadfix" to "mynewplugin" (or whatever special module you want - lets use my example "drawingtools") in the same directory.
  • Rename everything in the new folder from "beadfixer", "beadfix" and "Bead Fixer" to "drawingtools" - including the entries in "Makefile"
  • NOTE: I use Xcode for editing code on a Mac, but anything will do
  • Edit the Makefile in the plugs folder (up a directory) -> add lines to compile the new folder
  • To compile just this new plugin:
cd /Users/a.noske/Documents/MACMOD/plugs/drawingtools/ 
make clean && make install
  • Or if I want to install and test:
make install && 3dmod -C 0 -E P,1,2 /Users/a.noske/Desktop/test.rec /Users/a.noske/Desktop/test.mod
  • This will compile the plugin into a dynamic library drawingtools.dylib which will be opened when 3dmod is run
  • Unlike a "free standing plugin", which is part of the program (compiled with the rest), these "special plugins" don't include complete structures - just empty definitions which they call from the dynamic libraries.
  • Using the -E optional argument you can open most dialogs and David has added the feature to open every "true plugin" under IMOD_PLUGIN_DIR if you add "-E P". He also added "1" to start in model mode and "2" to suppress query to save on exit.
  • In the new directory is "imodplug.h", "model.h", "imodview.h" and several other useful files which are included... (i.e. plugin has its own version). There were a few functions and constants/defines in cont.h which I needed which were not in "model.h", but adding them was not particularly difficult.


Using your special plugin on other computers

  • To copy your plugin simply go to the lib directory and copy the dynamic library (".../MACMOD/lib/imodplug/drawingtools.dylib"), and copy it to the same directory on another computer.
  • For this to work both computers must be the same platform (same operating system, both 32 or 62-bit and both similar chipset).
  • NOTE: You should also make sure the same version of IMOD is installed on both computer (or else certain function calls might be missing).


Links


Acknowledgements: David Mastronarde from the University of Colorado Boulder for all his patience and help! Thanks also to Brad Marsh for encouraging me to put my tools into IMOD
Retrieved from "http://andrewnoske.com/w/index.php?title=IMOD_-_compiling&oldid=424"