The forum is locked.

The Ocean Color Forum has transitioned over to the Earthdata Forum (https://forum.earthdata.nasa.gov/). The information existing below will be retained for historical reference. Please sign into the Earthdata Forum for active user support.

Up Topic SeaDAS / SeaDAS 6.x Virtual Appliance for Windows / SeaDAS VA6.4 memory allocation and file processing limits (locked)
- By mimoyski Date 2013-02-20 16:02 Edited 2013-02-21 08:43
Hi... i'm running into some memory allocation issues, hope somebody could help, please?
i'm running a script that process l2binned, daily SeaWiFs and MODISA images to l3binned/mapped, then output to asc and png. regardless of total no. of input l2binned files, processing always stopped every 155 julian day has been processed to png. i re-run starting from 156 julian day where it left off and it stopped again after the 310 julian day. gives me this error everytime:
% Unable to allocate memory: to make array.
  Cannot allocate memory
[MAIN]: Error executing the command:
FOR jd=0,jyr-1 DO BEGIN.....

Since accdg to the error message it's a memory allocation issue, i've tried a few things that i thought could solve it. below are my system environment info and the things i tried:
System environment:
IntelCore i7-2600K, 16GB RAM, >1TB diskspace, 64-bit Win 7 Enterprise, VMWare player 5.01, SeaDASVA 6.4

What i tried:
1) Set SeaDAS memory to 1,2,8,10 GB --> same result, stopped at every 155 julian day
2) Emptied seadas_shared folder and processed directly from mounted external HDD -->same result
3) Attempted to clear memory and buffer cache --> no permission to do this

My questions:
1) Is there a way to increase memory allocation of SeaDAS-VA?
    -- maybe a line i can insert in the script that would free up allocatedor used memory during the processing?
2) Why is memory allocation not changing despite changing SeaDAS-VA memory in the .vmx file?
3) Why are there memory allocation errors in  SeaDAS-VA 6.4?
    -- if i remember right, i used to be able to process 1 year worth of L2 data set, doing l2bin, l3bin, map, output to asc   
        and png in one run without any errors, using SeaDAS 6.2 in a WinXP environment.

would really appreciate your help, please??? :-) am not a linux expert, but i can find my way around a little bit.
thanks....
- By seanbailey Date 2013-02-21 19:46
Without seeing your script, I'll assume you're running a "seadas batch" script (and thus an IDL script). 
IDL tends to not free up memory that it's no longer using, so you can get memory allocation errors if you don't
include a clear_up statement at the end of your loop.

Sean
- By mimoyski Date 2013-03-01 14:17
Thanks so much sean!!! yes, you're right, i'm running idl batch scripts. and 'clear_up' did work for the memory allocation :-) it still runs out of memory at some point but at least the allocations have been extended somehow to do more processing. also, i separated the batch script into several independent scripts (binning routines), so i can do clear_up memory used after each binning routine, without affecting input for the next routine.

sorry, i have an additional question :-/ just want to understand a little bit more about this so i can customize future scripts according to system limits/capabilities:
1) does clear_up work only for clearing loaded bands and display windows?
    --when i was testing clear_up and the parsed scripts, i monitored memory usage with 'free -m'. i noticed that even 
       with clear_up, used memory was still increasing (with the binning routines), and free memory doesn't go back to
       original value after a routine has finished. so i assumed that memory wasn't being freed up (?)

cheers,
- By seanbailey Date 2013-03-02 03:22
The clear_up function was written to release the memory allocated by global variables defined in the SeaDAS IDL environment.
It is quite probable that it isn't catching everything - and even more likely that some of the functions create arrays that are not
"global" - thus not known to clear_up.  One of a long list of reasons 6.x is no longer under development.

Regards,
Sean
Up Topic SeaDAS / SeaDAS 6.x Virtual Appliance for Windows / SeaDAS VA6.4 memory allocation and file processing limits (locked)