The forum is locked.
The Ocean Color Forum has transitioned over to the Earthdata Forum (https://forum.earthdata.nasa.gov/). The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.
Hello,
I am using modis_GEO.py with L1A MODIS file to get a GEO file. For many files I get the following error:
Percent valid data (0.00) is less than threshold (95.00)
ERROR: MODIS geolocation processing failed.
I am trying to find out how 'valid data' are defined.
Thank you.
Amita
I am using modis_GEO.py with L1A MODIS file to get a GEO file. For many files I get the following error:
Percent valid data (0.00) is less than threshold (95.00)
ERROR: MODIS geolocation processing failed.
I am trying to find out how 'valid data' are defined.
Thank you.
Amita
Amita,
When the MODIS geolocation processing fails, it leaves behind several log files (with filenames that begin with "Log"). These can be cryptic, but are informative.
If you can't decipher them, and don't see a similar post on this forum explaining the issue, attach the log files (or snippets that seem interesting) and someone may be able to help you.
Sean
When the MODIS geolocation processing fails, it leaves behind several log files (with filenames that begin with "Log"). These can be cryptic, but are informative.
If you can't decipher them, and don't see a similar post on this forum explaining the issue, attach the log files (or snippets that seem interesting) and someone may be able to help you.
Sean
Dear sir,
Please guide how to solve the solution this error
I am try to generate L1B data from L1A data of MODIS aqua, followed by command ::::: modis_GEO.py filename
Traceback (most recent call last):
File "/home/umamahesh/ocssw/scripts/modis_GEO.py", line 7, in <module>
import modules.anc_utils as anc_utils
File "/home/umamahesh/ocssw/scripts/modules/anc_utils.py", line 9, in <module>
import modules.ProcUtils as ProcUtils
File "/home/umamahesh/ocssw/scripts/modules/ProcUtils.py", line 16, in <module>
import requests
ImportError: No module named requests
Please guide how to solve the solution this error
I am try to generate L1B data from L1A data of MODIS aqua, followed by command ::::: modis_GEO.py filename
Traceback (most recent call last):
File "/home/umamahesh/ocssw/scripts/modis_GEO.py", line 7, in <module>
import modules.anc_utils as anc_utils
File "/home/umamahesh/ocssw/scripts/modules/anc_utils.py", line 9, in <module>
import modules.ProcUtils as ProcUtils
File "/home/umamahesh/ocssw/scripts/modules/ProcUtils.py", line 16, in <module>
import requests
ImportError: No module named requests
Dear Sir,
Good morning, when running the command: modis_GEO.py filename
*** ERROR: Authentication Failue retrieving:
*** oceandata.sci.gsfc.nasa.gov/ob/getfile/PM1ATTNR.P2012062.0600.003
*** Please check that your ~/.netrc file is setup correctly and has proper permissions.
***
*** see: https://oceancolor.gsfc.nasa.gov/data/download_methods/
***
Cannot create geolocation from A2012062073500.L1A_LAC; exiting.
Sir, Please provide the solution
Good morning, when running the command: modis_GEO.py filename
*** ERROR: Authentication Failue retrieving:
*** oceandata.sci.gsfc.nasa.gov/ob/getfile/PM1ATTNR.P2012062.0600.003
*** Please check that your ~/.netrc file is setup correctly and has proper permissions.
***
*** see: https://oceancolor.gsfc.nasa.gov/data/download_methods/
***
Cannot create geolocation from A2012062073500.L1A_LAC; exiting.
Sir, Please provide the solution
Have you verified the contents and permissions of your
At one time, I was getting login failures from
If the permissions aren't correct, you may be able to correct them using:
Another potential source of failures can be problems with Windows line editing or passwords that contain non-ASCII characters with look-alike Unicode characters (dash versus minus versus em-dash, etc.). Working in Canada, I encountered problems with a system that switched to a French keyboard when I tried to enter a password (there was a little icon that changed but it took me a while to notice).
~/.netrc? You should have:$ cat ~/.netrc
machine urs.earthdata.nasa.gov
login <earthdata_user>
password <earthdata_password>
$ ls -l ~/.netrc
-rw-------+ 1 <linux_user> <linux_group> 72 Jan 21 2020 /home/<linux_user>/.netrc
At one time, I was getting login failures from
oceandata.sci.gsfc.nasa.gov. Adding a second entry with the same login and password lines, but machine oceandata.sci.gsfc.nasa.gov was needed to work around the failures. I have not needed the second machine entry recently.If the permissions aren't correct, you may be able to correct them using:
$ chmod 600 ~/.netrc
Another potential source of failures can be problems with Windows line editing or passwords that contain non-ASCII characters with look-alike Unicode characters (dash versus minus versus em-dash, etc.). Working in Canada, I encountered problems with a system that switched to a French keyboard when I tried to enter a password (there was a little icon that changed but it took me a while to notice).
Dear Sir,
Good evening, thank you for your great sir, followed your commands, as followed show this type of error. Please guide me on how to solve the problem.
(base) umamahesh@uma:~/Seadas_data_processing$ cat ~/.netrc
cat: /home/umamahesh/.netrc: No such file or directory
(base) umamahesh@uma:~/Seadas_data_processing$ ls -l ~/.netrc
ls: cannot access '/home/umamahesh/.netrc': No such file or directory
(base) umamahesh@uma:~/Seadas_data_processing$ chmod 600 ~/.netrc
chmod: cannot access '/home/umamahesh/.netrc': No such file or directory
Good evening, thank you for your great sir, followed your commands, as followed show this type of error. Please guide me on how to solve the problem.
(base) umamahesh@uma:~/Seadas_data_processing$ cat ~/.netrc
cat: /home/umamahesh/.netrc: No such file or directory
(base) umamahesh@uma:~/Seadas_data_processing$ ls -l ~/.netrc
ls: cannot access '/home/umamahesh/.netrc': No such file or directory
(base) umamahesh@uma:~/Seadas_data_processing$ chmod 600 ~/.netrc
chmod: cannot access '/home/umamahesh/.netrc': No such file or directory
See: NASA Earthdata: How to access data ... (the instructions are for wget and curl, but the OCSSW software requires the same
~/.netrc file).
Dear Sir,
Good morning, I followed your instructions as followed by link, but my problem was not solved. Please guide me where I am doing mistakes.
I am using Ubuntu 20.04 LTS in Window10 with 12 GB RAM laptop.
https://wiki.earthdata.nasa.gov/display/EL/How+To+Access+Data+With+cURL+And+Wget
errors were
(base) umamahesh@uma:~/Seadas_data_processing$ modis_GEO.py A2012062073500.L1A_LAC
*** ERROR: Authentication Failue retrieving:
*** oceandata.sci.gsfc.nasa.gov/ob/getfile/PM1ATTNR.P2012062.0600.003
*** Please check that your ~/.netrc file is setup correctly and has proper permissions.
***
*** see: https://oceancolor.gsfc.nasa.gov/data/download_methods/
***
Cannot create geolocation from A2012062073500.L1A_LAC; exiting.
Good morning, I followed your instructions as followed by link, but my problem was not solved. Please guide me where I am doing mistakes.
I am using Ubuntu 20.04 LTS in Window10 with 12 GB RAM laptop.
https://wiki.earthdata.nasa.gov/display/EL/How+To+Access+Data+With+cURL+And+Wget
errors were
(base) umamahesh@uma:~/Seadas_data_processing$ modis_GEO.py A2012062073500.L1A_LAC
*** ERROR: Authentication Failue retrieving:
*** oceandata.sci.gsfc.nasa.gov/ob/getfile/PM1ATTNR.P2012062.0600.003
*** Please check that your ~/.netrc file is setup correctly and has proper permissions.
***
*** see: https://oceancolor.gsfc.nasa.gov/data/download_methods/
***
Cannot create geolocation from A2012062073500.L1A_LAC; exiting.
Did you review the checks in my previous post? I have used Ubuntu 20.04 LTS with WSL2 to run
modis_GEO.py using a ~/.netrc with the two machine entries. Character set problems I mentioned can be very hard to diagnose remotely (especially as your ~/.netrc c ontains secrets so can't be posted in a forum). One check is to log out from the forum and then log back in by pasting the username and password from your ~/.netrc. If this fails then you know there is an issue with the ~/.netrc.
Thank you, for your previous replay on L1A to L1B generation
My present Problem
I am generating Chlorophylll product from L1A different from MODIS providing Chlorophyll product, Please guide me how to generate same product as MODIS-Aqua
MODIS aqua process : l2gen par=/data4/sdpsoper/vdc/vpu7/workbuf/A2012062073500.L1B_LAC.param metafile=A2012062073500.L2_LAC_OC.nc.meta
Myself generation process : l2gen ifile=A2012062073500.L1B_LAC geofile=A2012062073500.GEO ofile=A2012062073500.L2 cloud_thresold_0.027 l2prod=chlor_a parfile=A2012062073500.L1B_LAC.anc north=17.475 cloud_thresh=0.027
Here, MODIS-A using A2012062073500.L1B_LAC.param ?? what is it ?? how we can generate it ?? Is it really possible generate like MODIS product ??
Please guide me Sir.
My present Problem
I am generating Chlorophylll product from L1A different from MODIS providing Chlorophyll product, Please guide me how to generate same product as MODIS-Aqua
MODIS aqua process : l2gen par=/data4/sdpsoper/vdc/vpu7/workbuf/A2012062073500.L1B_LAC.param metafile=A2012062073500.L2_LAC_OC.nc.meta
Myself generation process : l2gen ifile=A2012062073500.L1B_LAC geofile=A2012062073500.GEO ofile=A2012062073500.L2 cloud_thresold_0.027 l2prod=chlor_a parfile=A2012062073500.L1B_LAC.anc north=17.475 cloud_thresh=0.027
Here, MODIS-A using A2012062073500.L1B_LAC.param ?? what is it ?? how we can generate it ?? Is it really possible generate like MODIS product ??
Please guide me Sir.
The NetCDF4-CF metadata provides many of the details for the level-2 processing. You can view the metadata using "ncdump -h" in a terminal or using a SeaDAS or SNAP GUI, NASA Panoply, or many other programs.
When we process the data in our operational environment, we're using a parameter file so as to not have to put all the runtime parameters on the command line. That's what the "
What is likely different between your processing and ours is the use of the ancillary data. We defined them (in the .param file), but you did not so l2gen used climatologies for those inputs.
Use the getanc.py script to retrieve the ancillary data (the script will also generate a
You can verify the inputs by doing as George said and using
Regards,
Sean
.param" file is.What is likely different between your processing and ours is the use of the ancillary data. We defined them (in the .param file), but you did not so l2gen used climatologies for those inputs.
Use the getanc.py script to retrieve the ancillary data (the script will also generate a
.anc file that is analogous to the .param file, containing the ancillary parameters needed).You can verify the inputs by doing as George said and using
ncdump -h to see exactly how the file was generated. The processing_control group contains an input_parameters group with every single input parameter used. MOST are defaults and need not be passed in, but some, like the met#, ozone#, sstfile, etc. are specific to the file being processed and need to be provided by the calling process.Regards,
Sean
Dear Sir,
Thank your support, Please support me solve the below problem.
(base) umamahesh@uma:/mnt/d/Ubuntu/MODIS-A$ l2gen ifile=A2012062073500.L1B_LAC geofile=A2012062073500.GEO ofile=A2012062073500.L2 l2prod=chlor_a par=A2012062073500.L1B_LAC.anc north=17.518 west=81.297 south=16.614 east=84.434
Input file A2012062073500.L1B_LAC is MODIS Aqua Level-1B HDF-EOS product.
Loading default parameters from /home/umamahesh/ocssw/share/common/msl12_defaults.par
Loading default parameters for MODISA from /home/umamahesh/ocssw/share/modis/msl12_defaults.par
Loading default sub-sensor parameters for MODISA from /home/umamahesh/ocssw/share/modis/aqua/msl12_defaults.par
Loading parameters for suite OC from /home/umamahesh/ocssw/share/modis/msl12_defaults_OC.par
Loading command line parameters
Loading user parameters for MODISA
Loading characteristics for MODISA
Opening sensor information file /home/umamahesh/ocssw/share/modis/aqua/msl12_sensor_info.dat
Bnd Lam Fo Tau_r k_oz k_no2 t_co2 awhite aw bbw
0 412.000 172.912 3.099e-01 1.987e-03 5.814e-19 1.000e+00 1.000e+00 2.030e-02 2.884e-03
1 443.000 187.622 2.367e-01 3.189e-03 4.985e-19 1.000e+00 1.000e+00 1.110e-02 2.181e-03
2 469.000 205.878 1.914e-01 8.745e-03 3.938e-19 1.000e+00 1.000e+00 1.034e-02 1.749e-03
3 488.000 194.933 1.592e-01 2.032e-02 2.878e-19 1.000e+00 1.000e+00 1.676e-02 1.447e-03
4 531.000 185.747 1.126e-01 6.838e-02 1.525e-19 1.000e+00 1.000e+00 4.494e-02 1.014e-03
5 547.000 186.539 9.906e-02 8.622e-02 1.194e-19 1.000e+00 1.000e+00 5.491e-02 8.886e-04
6 555.000 183.869 9.432e-02 9.553e-02 9.445e-20 1.000e+00 1.000e+00 5.839e-02 8.443e-04
7 645.000 157.811 5.082e-02 7.382e-02 1.382e-20 1.000e+00 9.133e-01 3.458e-01 4.469e-04
8 667.000 152.255 4.443e-02 4.890e-02 7.065e-21 1.000e+00 8.921e-01 4.923e-01 3.900e-04
9 678.000 148.052 4.146e-02 3.787e-02 8.304e-21 1.000e+00 8.783e-01 5.116e-01 3.634e-04
10 748.000 128.065 2.849e-02 1.235e-02 2.157e-21 1.000e+00 7.832e-01 2.800e+00 2.470e-04
11 859.000 97.174 1.613e-02 2.347e-03 6.212e-23 1.000e+00 6.519e-01 4.396e+00 1.384e-04
12 869.000 95.824 1.540e-02 1.936e-03 7.872e-23 1.000e+00 6.403e-01 4.727e+00 1.320e-04
13 1240.000 45.467 3.617e-03 0.000e+00 0.000e+00 9.994e-01 0.000e+00 1.147e+02 3.021e-05
14 1640.000 23.977 1.219e-03 0.000e+00 0.000e+00 9.890e-01 0.000e+00 6.689e+02 1.000e-05
15 2130.000 9.885 4.286e-04 0.000e+00 0.000e+00 9.696e-01 0.000e+00 2.810e+03 3.465e-06
Bnd Lam
16 3750.000
17 3959.000
18 4050.000
19 6715.000
20 7325.000
21 8550.000
22 11000.000
23 12000.000
Internal data compression requested at compression level: 4
Opening filter file /home/umamahesh/ocssw/share/modis/msl12_filter.dat
Setting 7 x 5 straylight filter on HILT mask
Filter Kernel
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
Minimum fill set to 1 pixels
Setting 7 x 5 straylight filter on CLDICE mask
Filter Kernel
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
Minimum fill set to 1 pixels
Processing at 1000 meter resolution.
1000-meter file: A2012062073500.L1B_LAC
Allocated 2365502 bytes in L1 record.
Allocated 920720 bytes in L2 record.
Opening: A2012062073500.L2
The following products will be included in A2012062073500.L2.
0 chlor_a
1 l2_flags
Begin l2gen Version 9.5.0-V2019.3 Processing
Sensor is MODISA
Sensor ID is 7
Sensor has 16 reflective bands
Sensor has 8 emissive bands
Number of along-track detectors per band is 10
Number of input pixels per scan is 1354
Processing pixels 918 to 1160 by 1
Processing scans 1018 to 1172 by 1
Ocean processing enabled
Land processing enabled
Atmospheric correction enabled
Begin MSl12 processing at 2021035202858000
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Loading radiance to brightness temperature from /home/umamahesh/ocssw/share/modis/aqua/cal/bt_modisa.hdf
Loading land mask file from /home/umamahesh/ocssw/share/common/landmask_GMT15ARC.nc
Loading bathymetry mask file from /home/umamahesh/ocssw/share/common/watermask.dat
Loading ice mask file from /home/mohammed_suhail/SeaDAS/OCSSW/var/anc/2012/061/N201206100_SEAICE_NSIDC_24h.hdf
-E- /home/seadas/ocssw/src/l2gen/ice_mask.c line 223: File, "/home/mohammed_suhail/SeaDAS/OCSSW/var/anc/2012/061/N201206100_SEAICE_NSIDC_24h.hdf", doesn't follow naming convention described at http://nsidc.org/data/docs/daac/nsidc0051_gsfc_seaice.gd.html#namingconvention .
-E- /home/seadas/ocssw/src/l2gen/loadl1.c : Unable to initialize ice mask
Thank your support, Please support me solve the below problem.
(base) umamahesh@uma:/mnt/d/Ubuntu/MODIS-A$ l2gen ifile=A2012062073500.L1B_LAC geofile=A2012062073500.GEO ofile=A2012062073500.L2 l2prod=chlor_a par=A2012062073500.L1B_LAC.anc north=17.518 west=81.297 south=16.614 east=84.434
Input file A2012062073500.L1B_LAC is MODIS Aqua Level-1B HDF-EOS product.
Loading default parameters from /home/umamahesh/ocssw/share/common/msl12_defaults.par
Loading default parameters for MODISA from /home/umamahesh/ocssw/share/modis/msl12_defaults.par
Loading default sub-sensor parameters for MODISA from /home/umamahesh/ocssw/share/modis/aqua/msl12_defaults.par
Loading parameters for suite OC from /home/umamahesh/ocssw/share/modis/msl12_defaults_OC.par
Loading command line parameters
Loading user parameters for MODISA
Loading characteristics for MODISA
Opening sensor information file /home/umamahesh/ocssw/share/modis/aqua/msl12_sensor_info.dat
Bnd Lam Fo Tau_r k_oz k_no2 t_co2 awhite aw bbw
0 412.000 172.912 3.099e-01 1.987e-03 5.814e-19 1.000e+00 1.000e+00 2.030e-02 2.884e-03
1 443.000 187.622 2.367e-01 3.189e-03 4.985e-19 1.000e+00 1.000e+00 1.110e-02 2.181e-03
2 469.000 205.878 1.914e-01 8.745e-03 3.938e-19 1.000e+00 1.000e+00 1.034e-02 1.749e-03
3 488.000 194.933 1.592e-01 2.032e-02 2.878e-19 1.000e+00 1.000e+00 1.676e-02 1.447e-03
4 531.000 185.747 1.126e-01 6.838e-02 1.525e-19 1.000e+00 1.000e+00 4.494e-02 1.014e-03
5 547.000 186.539 9.906e-02 8.622e-02 1.194e-19 1.000e+00 1.000e+00 5.491e-02 8.886e-04
6 555.000 183.869 9.432e-02 9.553e-02 9.445e-20 1.000e+00 1.000e+00 5.839e-02 8.443e-04
7 645.000 157.811 5.082e-02 7.382e-02 1.382e-20 1.000e+00 9.133e-01 3.458e-01 4.469e-04
8 667.000 152.255 4.443e-02 4.890e-02 7.065e-21 1.000e+00 8.921e-01 4.923e-01 3.900e-04
9 678.000 148.052 4.146e-02 3.787e-02 8.304e-21 1.000e+00 8.783e-01 5.116e-01 3.634e-04
10 748.000 128.065 2.849e-02 1.235e-02 2.157e-21 1.000e+00 7.832e-01 2.800e+00 2.470e-04
11 859.000 97.174 1.613e-02 2.347e-03 6.212e-23 1.000e+00 6.519e-01 4.396e+00 1.384e-04
12 869.000 95.824 1.540e-02 1.936e-03 7.872e-23 1.000e+00 6.403e-01 4.727e+00 1.320e-04
13 1240.000 45.467 3.617e-03 0.000e+00 0.000e+00 9.994e-01 0.000e+00 1.147e+02 3.021e-05
14 1640.000 23.977 1.219e-03 0.000e+00 0.000e+00 9.890e-01 0.000e+00 6.689e+02 1.000e-05
15 2130.000 9.885 4.286e-04 0.000e+00 0.000e+00 9.696e-01 0.000e+00 2.810e+03 3.465e-06
Bnd Lam
16 3750.000
17 3959.000
18 4050.000
19 6715.000
20 7325.000
21 8550.000
22 11000.000
23 12000.000
Internal data compression requested at compression level: 4
Opening filter file /home/umamahesh/ocssw/share/modis/msl12_filter.dat
Setting 7 x 5 straylight filter on HILT mask
Filter Kernel
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
Minimum fill set to 1 pixels
Setting 7 x 5 straylight filter on CLDICE mask
Filter Kernel
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
1 1 1 1 1 1 1
Minimum fill set to 1 pixels
Processing at 1000 meter resolution.
1000-meter file: A2012062073500.L1B_LAC
Allocated 2365502 bytes in L1 record.
Allocated 920720 bytes in L2 record.
Opening: A2012062073500.L2
The following products will be included in A2012062073500.L2.
0 chlor_a
1 l2_flags
Begin l2gen Version 9.5.0-V2019.3 Processing
Sensor is MODISA
Sensor ID is 7
Sensor has 16 reflective bands
Sensor has 8 emissive bands
Number of along-track detectors per band is 10
Number of input pixels per scan is 1354
Processing pixels 918 to 1160 by 1
Processing scans 1018 to 1172 by 1
Ocean processing enabled
Land processing enabled
Atmospheric correction enabled
Begin MSl12 processing at 2021035202858000
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Allocated 2365502 bytes in L1 record.
Loading radiance to brightness temperature from /home/umamahesh/ocssw/share/modis/aqua/cal/bt_modisa.hdf
Loading land mask file from /home/umamahesh/ocssw/share/common/landmask_GMT15ARC.nc
Loading bathymetry mask file from /home/umamahesh/ocssw/share/common/watermask.dat
Loading ice mask file from /home/mohammed_suhail/SeaDAS/OCSSW/var/anc/2012/061/N201206100_SEAICE_NSIDC_24h.hdf
-E- /home/seadas/ocssw/src/l2gen/ice_mask.c line 223: File, "/home/mohammed_suhail/SeaDAS/OCSSW/var/anc/2012/061/N201206100_SEAICE_NSIDC_24h.hdf", doesn't follow naming convention described at http://nsidc.org/data/docs/daac/nsidc0051_gsfc_seaice.gd.html#namingconvention .
-E- /home/seadas/ocssw/src/l2gen/loadl1.c : Unable to initialize ice mask
make sure the N201206100_SEAICE_NSIDC_24h.hdf is a vaild HDF4 file. The Linux command "file" should show:
Sean
$ file N201206100_SEAICE_NSIDC_24h.hdf
N201206100_SEAICE_NSIDC_24h.hdf: Hierarchical Data Format (version 4) dataSean