The forum is locked.
The Ocean Color Forum has transitioned over to the Earthdata Forum (https://forum.earthdata.nasa.gov/). The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.
Topic Frequently Asked Questions / SeaDAS 6 FAQ / Is there an example of a Unix shell script for automating processing? (locked)
Is there an example of a Unix shell script for automating processing?
In general, IDL is not required for data processing and Unix shell scripts can be more efficient and flexible. Using shell scripts also gives you *direct* access to the SeaDAS processing binaries so you will have access to all command-line arguments for the program, which isn't always the case with our IDL wrappers. Also for programs like l2gen, accessing the binary directly will also avoid any potential bugs in our IDL code (l2gen has so many options there is potential for bugs when writing a wrapper for it). If access to IDL functionality is desired at any point in the processing, lines can be easily inserted in the script to call your IDL scripts like so:
seadas -b idl_batch_script.txt
Two example scripts for MODIS and SeaWiFS are provided below...
=========================================
The following is an example Unix shell script that automates MODIS L1A->L2 processing including subscening the MODIS files and auto ancillary retrieval. To create the script paste the text below into a Unix text file, and then make the file executable (c_h_m_o_d u+x script_file). The script can then be run directly on the Unix command line by typing "./script_file" in the directory where the script and data reside. (Normally you would put all your scripts in a directory such as ~/bin/ and add this directory to your path.)
#!/bin/sh
# first define the region to be extracted from your MODIS L1A files
# NOTE! change these for your own region of interest
SWLON=-77
SWLAT=37
NELON=-74.5
NELAT=39.5
for FILE in *L1A_LAC
do
# The line below assumes an extension, and creates a base name without that extension
BASE=`e_c_h_o $FILE |awk -F. '{ print $1 }'`
GEOFILE=${BASE}.GEO
L1ASUB=${BASE}_sub.L1A_LAC
GEOSUB=${BASE}_sub.GEO
L1BFILE=${BASE}.L1B
L2FILE=${BASE}.L2
# process the L1A file to GEO
modis_L1A_to_GEO.csh $FILE -o $GEOFILE
# extract a subscene from the MODIS file
modis_L1A_extract.csh $FILE $GEOFILE $SWLON $SWLAT $NELON $NELAT $L1ASUB $GEOSUB
# process the L1A/GEO subscene files to L1B
modis_L1A_to_L1B.csh $L1ASUB $GEOSUB -o $L1BFILE -delete-hkm -delete-qkm
# determine ancillary data
getanc $L1BFILE
# the above command creates a file in l2gen's par file format called L1BFILE.anc
# process the L1B subscene to L2
e_c_h_o "Processing $L1BFILE to Level 2.."
# NOTE! customize the l2gen parameters here
l2gen ifile=$L1BFILE geofile=$GEOSUB ofile1=$L2FILE \
l2prod1='chlor_a,K_490,nLw_412,nLw_551,l2_flags' \
par=${L1BFILE}.anc >$BASE.log
done
=========================================
The following is an example Unix shell script that automates SeaWiFS L1A->L2 processing including subscening the SeaWiFS files and auto ancillary retrieval. To create the script paste the text below into a Unix text file, and then make the file executable (c_h_m_o_d u+x script_file). The script can then be run directly on the Unix command line by typing "./script_file" in the directory where the script and data reside. (Normally you would put all your scripts in a directory such as ~/bin/ and add this directory to your path.)
#!/bin/sh
# Unix shell script to subscene a Level-1A file and process it to Level-2
# first define the region to be extracted from your MODIS L1A files
# NOTE! change these for your own region of interest
slat=37
elat=39.5
slon=-77
elon=-74.5
for FILE in S*.L1A_MLAC
do
# The line below assumes an extension, and creates a base name without that extension
BASE=`e_c_h_o $FILE |awk -F. '{ print $1 }'`
L1ASUB=$BASE.L1A_sub
L2FILE=$BASE.L2
e_c_h_o "Commencing L1A extraction and Level-2 processing for: $FILE"
e_c_h_o "Converting lat/lon to pix/line.."
e_c_h_o "lonlat2pixline $FILE $slon $slat $elon $elat"
lonlat2pixline $FILE $slon $slat $elon $elat >tmp.txt
spixl=`grep spixl= tmp.txt |cut -c7-`
epixl=`grep epixl= tmp.txt |cut -c7-`
sline=`grep sline= tmp.txt |cut -c7-`
eline=`grep eline= tmp.txt |cut -c7-`
e_c_h_o "Extracting the L1A file..."
e_c_h_o "l1aextract_seawifs $FILE $spixl $epixl $sline $eline 1 1 $L1ASUB"
l1aextract_seawifs $FILE $spixl $epixl $sline $eline 1 1 $L1ASUB
e_c_h_o "Determining ancillary data to be used for Level-2 processing.."
e_c_h_o "getanc $L1ASUB"
getanc $L1ASUB
# the above command creates a file in l2gen's par file format called L1ASUB.anc
e_c_h_o "Processing $L1ASUB to Level 2.."; e_c_h_o
# NOTE! customize the l2gen parameters here
l2gen ifile=$L1ASUB ofile1=$L2FILE \
l2prod1='chlor_a,K_490,nLw_412,nLw_551,l2_flags' \
par=${L1ASUB}.anc |tee $BASE.log
rm tmp.txt
done
In general, IDL is not required for data processing and Unix shell scripts can be more efficient and flexible. Using shell scripts also gives you *direct* access to the SeaDAS processing binaries so you will have access to all command-line arguments for the program, which isn't always the case with our IDL wrappers. Also for programs like l2gen, accessing the binary directly will also avoid any potential bugs in our IDL code (l2gen has so many options there is potential for bugs when writing a wrapper for it). If access to IDL functionality is desired at any point in the processing, lines can be easily inserted in the script to call your IDL scripts like so:
seadas -b idl_batch_script.txt
Two example scripts for MODIS and SeaWiFS are provided below...
=========================================
The following is an example Unix shell script that automates MODIS L1A->L2 processing including subscening the MODIS files and auto ancillary retrieval. To create the script paste the text below into a Unix text file, and then make the file executable (c_h_m_o_d u+x script_file). The script can then be run directly on the Unix command line by typing "./script_file" in the directory where the script and data reside. (Normally you would put all your scripts in a directory such as ~/bin/ and add this directory to your path.)
#!/bin/sh
# first define the region to be extracted from your MODIS L1A files
# NOTE! change these for your own region of interest
SWLON=-77
SWLAT=37
NELON=-74.5
NELAT=39.5
for FILE in *L1A_LAC
do
# The line below assumes an extension, and creates a base name without that extension
BASE=`e_c_h_o $FILE |awk -F. '{ print $1 }'`
GEOFILE=${BASE}.GEO
L1ASUB=${BASE}_sub.L1A_LAC
GEOSUB=${BASE}_sub.GEO
L1BFILE=${BASE}.L1B
L2FILE=${BASE}.L2
# process the L1A file to GEO
modis_L1A_to_GEO.csh $FILE -o $GEOFILE
# extract a subscene from the MODIS file
modis_L1A_extract.csh $FILE $GEOFILE $SWLON $SWLAT $NELON $NELAT $L1ASUB $GEOSUB
# process the L1A/GEO subscene files to L1B
modis_L1A_to_L1B.csh $L1ASUB $GEOSUB -o $L1BFILE -delete-hkm -delete-qkm
# determine ancillary data
getanc $L1BFILE
# the above command creates a file in l2gen's par file format called L1BFILE.anc
# process the L1B subscene to L2
e_c_h_o "Processing $L1BFILE to Level 2.."
# NOTE! customize the l2gen parameters here
l2gen ifile=$L1BFILE geofile=$GEOSUB ofile1=$L2FILE \
l2prod1='chlor_a,K_490,nLw_412,nLw_551,l2_flags' \
par=${L1BFILE}.anc >$BASE.log
done
=========================================
The following is an example Unix shell script that automates SeaWiFS L1A->L2 processing including subscening the SeaWiFS files and auto ancillary retrieval. To create the script paste the text below into a Unix text file, and then make the file executable (c_h_m_o_d u+x script_file). The script can then be run directly on the Unix command line by typing "./script_file" in the directory where the script and data reside. (Normally you would put all your scripts in a directory such as ~/bin/ and add this directory to your path.)
#!/bin/sh
# Unix shell script to subscene a Level-1A file and process it to Level-2
# first define the region to be extracted from your MODIS L1A files
# NOTE! change these for your own region of interest
slat=37
elat=39.5
slon=-77
elon=-74.5
for FILE in S*.L1A_MLAC
do
# The line below assumes an extension, and creates a base name without that extension
BASE=`e_c_h_o $FILE |awk -F. '{ print $1 }'`
L1ASUB=$BASE.L1A_sub
L2FILE=$BASE.L2
e_c_h_o "Commencing L1A extraction and Level-2 processing for: $FILE"
e_c_h_o "Converting lat/lon to pix/line.."
e_c_h_o "lonlat2pixline $FILE $slon $slat $elon $elat"
lonlat2pixline $FILE $slon $slat $elon $elat >tmp.txt
spixl=`grep spixl= tmp.txt |cut -c7-`
epixl=`grep epixl= tmp.txt |cut -c7-`
sline=`grep sline= tmp.txt |cut -c7-`
eline=`grep eline= tmp.txt |cut -c7-`
e_c_h_o "Extracting the L1A file..."
e_c_h_o "l1aextract_seawifs $FILE $spixl $epixl $sline $eline 1 1 $L1ASUB"
l1aextract_seawifs $FILE $spixl $epixl $sline $eline 1 1 $L1ASUB
e_c_h_o "Determining ancillary data to be used for Level-2 processing.."
e_c_h_o "getanc $L1ASUB"
getanc $L1ASUB
# the above command creates a file in l2gen's par file format called L1ASUB.anc
e_c_h_o "Processing $L1ASUB to Level 2.."; e_c_h_o
# NOTE! customize the l2gen parameters here
l2gen ifile=$L1ASUB ofile1=$L2FILE \
l2prod1='chlor_a,K_490,nLw_412,nLw_551,l2_flags' \
par=${L1ASUB}.anc |tee $BASE.log
rm tmp.txt
done
Topic Frequently Asked Questions / SeaDAS 6 FAQ / Is there an example of a Unix shell script for automating processing? (locked)