p3d README
Version information:
This is the release version 1.1 of p3d. All parts of the tool work, although
some parts require precise operation in order to work as intended.
$Id: README 159 2010-04-16 14:26:46Z christersandin $
Table of Contents:
------------------
1.0) Introduction
* 1.1 About p3d
* 1.2 Installing p3d
2.0) Contact
* 2.1 Funding acknowledgments
* 2.2 Collaborators
* 2.3 Reporting Bugs
* 2.4 Feature Requests
3.0) Software Platform
* 3.1 Coding Philosophy
4.0) Supported Instruments
5.0) Features
6.0) Recompiling the binary metafile for the IDL Virtual Machine
1.0) Introduction:
------------------
1.1) About p3d:
---------------
1.2) Installing p3d:
--------------------
Installing p3d requires a few steps depending on if you only intend to use it
with the IDL Virtual Machine (see 3.0) or if you want to use it and develop it
using your own IDL license.
In any case you need to have IDL, version>=6.2, installed on your system.
IDL can be downloaded from the following web page: http://www.ittvis.com.
See the relevant IDL documents on how to install IDL, and on how to acquire
and setup an IDL license (if you require one).
i) Acquiring the p3d software
You can either download one of the prepared releases of p3d from the program
files area (https://sourceforge.net/projects/p3d/files/) or you can download
the latest bleeding edge version using subversion. The latter approach is
preferred if you want new functionality, which has not yet been included in
the released versions. Although note that there is no guarantee that the code
of subversion will be bug free (it is being corrected as time permits).
In order to download p3d from the subversion repository at sourceforge.net the
first time you first open a terminal and change directory to where you would
like to have the software. Thereafter you type the following
svn checkout https://p3d.svn.sourceforge.net/svnroot/p3d/trunk p3d
whereby all files are placed in the directory 'p3d/' (you can of course use
any name on this directory).
In order to update p3d you should at any time change directory to p3d/ (or
whatever name you have used) and type 'svn update'. For any additional
functionality of subversion please see its web page
http://subversion.tigris.org/.
ii) Using the IDL Virtual Machine
There is a (bash-)script provided to run p3d using the IDL Virtual Machine on
all UNIX-type platforms. Although, before you can use the script you have to
prepare your system in a few steps.
After you have installed IDL you have to setup your environment to tell
the system where it can find IDL. The environmental variable IDL_DIR must be
set to the name of the directory where your IDL installation resides.
Depending on your system shell you can add the following line to your
~/.bashrc (bash shell):
export IDL_DIR=/path/to/IDL/
or ~/.cshrc (C shell):
setenv IDL_DIR /path/to/IDL/
where the tilde character (~) denotes your home-directory.
If you have python installed (with the name 'python') the p3d startup script
will check if you have the right version of IDL installed (>=6.2).
The environmental variable p3d_path must be set to the name of the directory
where you have copied the p3d distribution. Additionally you might want to
set the environmental variable p3d_data_path to the name of the (base)
directory where you keep the data that you will work with. You may also want
to add the vm/ directory to you path variable so that you don't have to enter
the full path to where p3d resides every time you want to run it. Depending on
your system shell you can add the following lines to a suitable place in your
~/.bashrc file (bash shell; Bourne Shell type environment):
export p3d_path=/path/to/p3d/
export p3d_data_path=/path/to/IFU/data/ # Optional
export PATH=$PATH:${p3d_path}/vm # Optional
or ~/.cshrc (C shell):
setenv p3d_path /path/to/p3d/
setenv p3d_data_path /path/to/IFU/data/ # Optional
setenv PATH $PATH:${p3d_path}/vm # Optional
In order to start p3d you thereafter enter the following in a terminal
(if you added ${p3d_dir}/vm to your system path):
p3d_vm.sh
or, if you didn't add ${p3d_dir}/vm to your path:
${p3d_dir}/vm/p3d_vm.sh
Note! The bash-script p3d_vm.sh checks if the required IDL binary meta file
p3d.sav is present in the ${p3d_path}/vm/ directory. If it exists IDL
is started using that binary file.
Note! If you have an IDL license, and want to recompile the binary for use
with the IDL Virtual Machine you can use the provided script
${p3d_dir}/vm/p3d_tool_makevm.sh. See below (6.0) for further
instructions.
iii) Using an IDL license
In order to run p3d using a license it is necessary to tell IDL where it can
find all required source files. In addition to all files of the p3d
distribution the files of the following sources are required:
Astro-lib (http://idlastro.gsfc.nasa.gov/homepage.html)
All files (actually a subset, but downloading all is easier [for us]).
Mpfit (http://purl.com/net/mpfit)
mpfit.pro and mpcurvefit.pro
These files can be put anywhere, but if you would like to use the provided
scripts when generating a binary metafile for use with the IDL Virtual
Machine (if you made any changes to the code) it is recommended to download
these files to ${p3d_path}/contrib/astro-lib (for the files of Astro-lib) and
${p3d_path}/contrib/ (all other contributing files). In order to download the
files to these directories automatically use the provided script:
${p3d_path}/vm/p3d_acquire_xtools.sh
When all files are downloaded, or if they are already present on the system,
it is necessary to setup IDL to find these files. The most straightforward
way is to set the environmental variable IDL_STARTUP to the name of an IDL
program where the necessary information is defined. The following lines can
be used to do this (the filename IDL_STARTUP.PRO can of course be chosen
freely):
For a Bourne Shell type environment, add these lines to your ~/.bashrc
export IDL_STARTUP=/path/to/IDL_STARTUP.PRO # Modify to fit your system
export p3d_data_path=/path/to/IFU/data/ # Optional
For a C Shell type environment, add these lines to your ~/.cshrc
setenv IDL_STARTUP /path/to/IDL_STARTUP.PRO # Modify to fit your system
setenv p3d_data_path /path/to/IFU/data/ # Optional
Thereafter add the following lines to IDL_STARTUP.PRO:
astrolib_path='/path/to/astrolib/root/directory' # Modify to fit your sys.
markwardt_path='/path/to/markwardt/files/directory' # Modify...
p3d_path='/path/to/p3d' # Modify to fit your sys.
!path=markwardt_path+':'+astrolib_path+'/pro:'+!path
!path=p3d_path+':'+p3d_path+'/routines:'+!path
If you chose to put the additionally required files under the root directory
of p3d, as we recommended above, you could add the following lines to
IDLSTARTUP.PRO instead:
p3d_path='/path/to/p3d' # Modify to fit your sys.
!path=p3d_path+'/contrib:'+p3d_path+'/contrib/astro-lib/pro:'+!path
!path=p3d_path+':'+p3d_path+'/routines:'+!path
Note! The exact order that paths are specified in is important. Also, observe
that no extra directory separator ('/') should be used in the final
position of the specification of the paths (i.e. '/path/to/p3d' not
'/path/to/p3d/').
p3d can now be started from the idl prompt using the command
IDL> p3d
If you'd like to know the available options you could just have a look in
the program file (using your favorite editor, or e.g. more or less), or type:
IDL> p3d,/help
2.0) Contact:
-------------
All collaborators, unless noted otherwise, have the following affiliation:
Astrophysikalisches Institut Potsdam (AIP)
Instrumentation
An der Sternwarte 16
DE-14482 Potsdam
GERMANY
Tel: +49-(0)331-74990
http://www.aip.de
p3d project web site:
http://p3d.sourceforge.net
2.1) Funding acknowledgments:
-----------------------------
This project has been made possible through funds from the following sources:
BMBF: 03Z2A51, 05A08BA1
2.2) Collaborators:
-------------------
Lead Programmer:
Christer Sandin o If you have comments of bugs or feature requests, which
are directly related to p3d, and that you cannot add to
the tracker at the project web site (see below for
instructions) then please send them to:
Christer Sandin <CSandin@aip.de>
Additional Programming:
Joris Gerssen o Improved the treatment of the wavelength calibration in
order to make it more intuitive than it used to be.
o Included support for reading and writing output in the
e3d-format.
o Started the work to write code to fit profiles for the
optimal extraction.
Peter Weilbacher o Included an option to calibrate the dispersion mask
against sky emission lines.
Testing of Software:
Ana Monreal-Ibero (ESO)
PMAS/LARR
Original Programmers/Authors:
Thomas Becker o Thomas wrote the first version of many parts of p3d for
a large number of instruments. This public release
version is based upon the earlier variant of p3d that
was called p3d_online.
Petra Böhm o Maintained all previous versions of p3d (including
p3d_online, ppak_online, virus_online, p3d, etc.), she
also fixed bugs and wrote various supporting documents.
Martin Matthias Roth
o Martin worked with software design of early versions of
p3d.
Additional acknowledgments:
Luzma Cairós o Tested the dispersion mask creation tool thoroughly and
provided many tips on its improvement.
Ole Streicher o Has provided p3d with a first version of the tutorials
at the project web site.
Stefano Ciroi (Padova)
o Figured out how to setup a basic configuration of p3d to
be used with the BTA/MPFS (3kx2k CCD) IFU.
2.3) Reporting Bugs:
--------------------
In order to report any kind of bug we urge you to use the bug tracker that is
available at the p3d project web site (Develop/Tracker/Bugs). This way your
bug will be seen by all users and programmers - and it won't be forgotten.
At the bug tracker page you can search previously reported bugs, to see if
your bug has already been reported. Or you can open your own bug description
(by clicking on "Add new"). Doing this, please make sure you are using the
latest version on the subversion trunk of p3d (or specify the version of the
release you are using). Thereafter specify a category for the bug, and provide
a short summary of the problem, and most importantly, describe the problem with
enough detail that somebody who does not use your setup and data files is able
to follow your arguments. Bug reports without a useful description will be
assigned a low priority, or even be deleted. If you think it would help us,
attach all relevant data files to your report (please compress fits-files using
tar/gzip or zip to save space). Additionally, please specify a reasonable
priority to your bug. While our intention is to correct serious bugs, which
prevent p3d from producing spectra, as soon as possible, we will assign GUI-
related issues a lower priority.
Important note. When reporting a bug, turn on the debugging mode (/DEBUG) of
p3d and paste the resulting output from the IDL terminal/window (once the bug
has been reached) into your bug description. The reason we ask you to do this
is that the information that is returned by the error handler when the debug
keyword is unset only provides the name of the file where the problem occurred,
and an IDL-specific number. Such sparse information is much more difficult to
interpret than a full output.
2.4) Feature requests:
----------------------
If p3d is missing some functionality that you would very much like to have
included then you can make a feature request using the tracker on the p3d
project web site (Develop/Tracker/Feature request). Following the description
on how to add a new bug description we ask you, also in this case, to be as
detailed as possible. Please understand that we are a team with limited
resources. If your request concerns a huge modification, which requires many
days, or even months, to code it might not happen at all. On the other hand, we
are highly interested in taking on project members who are interested in
contributing to p3d with their own time and coding. Please drop us a note if
you think this could apply to you.
3.0) Software Platform:
-----------------------
p3d is fully based on the language of the proprietary tool Interactive Data
Language (IDL; cf. http://www.ittvis.com) and cannot be used if IDL is not
installed on the system. Although it is necessary to have an IDL license in
order to modify code, p3d can be run without an IDL-license using the supplied
binary metafile ('vm/p3d.sav'). In this case the so-called IDL Virtual Machine
is used to run p3d. p3d has been implemented in such a way that it can be used
with full functionality with the Virtual Machine.
Since p3d is based fully on IDL it can be used on any platform that is
supported by IDL. Note, however, that the installation procedure differs
for an installation on a Windows machine compared to that on the other
platforms supported (the windows version of the installation instructions is
not yet implemented).
3.1) Coding Philosophy:
-----------------------
p3d is a large, and growing, code. In order to keep it manageable we have
applied rules of coding practice; use of this practice has so far made
debugging easy as bugs are found quickly.
These are the rules, which we would ask any project programming member to
follow:
o All files must have filenames which conform to: p3d_task_name.pro
o Code layout
+ All files must have a fully explained header.
+ All routines use the options: compile_opt hidden,IDL2
+ The maximum number of character columns per line is 79.
+ All blocks are indented by 2 characters.
+ Comment lines begin with ';;'
+ Integers are always specified as long with L, e.g. '0L'.
Exceptions: VERBOSE and ERROR variables use integers.
+ All commands and statements are written with small characters.
o Common blocks are only allowed for variables which obviously
will never ever change, i.e. pretty much never.
o An error handler is setup for every routine, unless the user keyword DEBUG
is set.
o Logging comment statements must always use 'p3d_misc_logger'.
o Input parameters should be checked for consistency in terms
of type, size, value and range, as far as possible.
o All command keywords are for clarity specified in full, i.e.
no abbreviations (for example, /XSTYLE not /XST).
o Text widgets must include error checking to ensure that the
entered values are of the right type and range.
o All widgets in GUI tools with a status line must include the
UNAME field that specifies what the widget in questions does.
Issues related to the subversion repository of p3d:
o No file on the trunk should be committed before the modifications are
confirmed to work as intended.
o All commits to the repository should include a describing message:
Example: svn ci --message "Corrected/modified this and that." <file>
o Larger modifications should use a branch, and not the trunk.
o The idea is to copy the trunk to branch/ in preparations for a
release. Then we only make bug fixes on that branch before we
copy the final release version to tags/.
4.0) Supported Instruments:
---------------------------
We have developed p3d to make it a general tool. General in the sense that
it can be used with any Integral Field Unit (IFU). However, before p3d can be
used with any instrument it must be properly setup. p3d has been setup, tested,
and found to work well with the following instruments:
---The following needs editing!---
PMAS - is mounted on the Calar Alto 3.5m telescope
PMAS :: http://www.aip.de/groups/opti/pmas/OptI_pmas.html
Calar Alto :: http://www.caha.es
LARR (Lens Array) :: 16x16 square-shaped elements that is used with either
the 0.5"x0.5" or the 1.0"x1.0" magnification settings.
PPAK () :: 331 circular elements, each of 2.8" diameter.
Since October 2009 PMAS has been equipped with a new 4kx4k CCD, replacing
the previous 2kx4k chip. p3d supports data of both CCDs. In order to use
either setup you can select them in the menu upon startup of p3d, or
use the input parameters: /larr2k, /larr4k, /ppak2k, or /ppak4k.
VIRUS-P - is mounted on the Harlan J. Smith 2.7m telescope
McDonald Observatory ::
http://www.as.utexas.edu/mcdonald/facilities/2.7m/2.7.html
VIRUS-P is a prototype for the VIRUS instrument that is made for the
Hobby Eberly Telescope.
VIRUS-P (bundle 2) :: has 246 circular spatial elements
VIRUS-P (bundle 1) :: has 247 circular spatial elements
SPIRAL
VIMOS
FLAMES/ARGUS
IMACS
GMOS
...
5.0) Features
-------------
6.0) Recompiling the binary metafile for the IDL Virtual Machine:
-----------------------------------------------------------------
The simplest way to (re-)compile the binary metafile that is used with the IDL
Virtual Machine is to use the provided scripts in ${p3d_path}/vm/. Note that in
order to recompile p3d you must have an IDL license. You could use the
following steps from a terminal:
i) Set the p3d root directory and IDL_DIR environmental variables. In a Bourne
Shell type environment (e.g. bash) type:
export p3d_path=/path/to/p3d/
export IDL_DIR=/path/to/IDL/ # IDL version >=6.2 is required!
In a C Shell type environment (e.g. csh) type:
setenv p3d_path /path/to/p3d/
setenv IDL_DIR /path/to/IDL/ # IDL version >=6.2 is required!
ii) Download the required auxiliary files. You could, most conveniently, use
the provided script:
${p3d_path}/vm/p3d_acquire_xtools.sh
Note! The binary metafile generation script requires the additional files
to be present in ${p3d_path}/contrib/astro-lib (with all .pro-files
in ${p3d_path}/contrib/astro-lib/pro/) and ${p3d_path}/contrib/.
iii) Change directory to ${p3d_path} and generate the binary metafile
${p3d_path}/vm/p3d.sav. You could, most conveniently, use the provided
script:
pushd ${p3d_path} > /dev/null
${p3d_path}/vm/p3d_tool_makevm.sh
popd > /dev/null
Note! The binary metafile generation script must be executed from the p3d
directory ${p3d_path}!
Note! If there is any problem in the binary metafile generation process please
check the output of the scripts to find out what the problem is.