Importing netCDF-3 file

[petercreateParticipant]

I have a global air-temperature dataset of netCDF-3 format. When I import it into the DG either by directly dragging or by importing gear, I cannot get the full dataset. As you can see, there is not incomplete showing long, lati and time columns with different lengths, lacking air temperature data.

I have consulted a senior expert, but ze has no way at present. I guess there could be others who experienced this as I did, seeking wise from a broader pool.

Thanks.

[daniel leathersParticipant]

I have the same issue with climate data of any type I have tried.  I get long/lat and that is it (and not even all of these points).  Looking forward to anyone that might have any ideas.

[petercreateParticipant]

The help party is doing on this. We just wait for that coming out. 🙂

[davidModerator]

Just uploaded a new beta with improved netCDF import.

Before the only thing that DataGraph picked up were the 1D arrays and they were imported as columns.

The new version imports 2D and 3D arrays as “flattened” arrays. That is, a 3D array like the one you have is imported as four columns, the x, y, z coordinate followed by the value.  So a 73x144x12 array is imported as 4 columns with 126,144 numbers each.  This is a fairly small number of rows only about half a million numbers total.  The 1D arrays are still imported as columns.

Then you can draw this in the scalar field drawing command and use a mask to implement animations.

[daniel leathersParticipant]

Many thanks David!  Looking forward to trying the new beta.

Thanks for the work on this and thanks for leting us know it is available.

Take care!

Dan

 

[petercreateParticipant]

DG does not work well on chipped netCDF files. Here I tested two more files both were chipped from NASA or NCAR database previously.

 

———

Hi David, does it have to use so much CPU power when doing the animation?

——–

Thank you, David. It’s great!

• This reply was modified 4 years, 4 months ago by petercreate. Reason: New issue to this fix
• This reply was modified 4 years, 4 months ago by petercreate. Reason: New emerging issues after testing more files

[dgteamModerator]

Are these files we can get on-line?

We can profile them to see what is causing the slowness.

[petercreateParticipant]

Yes. For example, here is a usually-used data portal, https://psl.noaa.gov/data/composites/day/. You can try a small area and a few days as the input and quickly get a tiny-size netCDF chip on a global basis.

[petercreateParticipant]

Hi dgteam,

I hope you have followed up this topic. Appreciating what you have made. If you wish, I can pass on a file to you, and so you can push forward.

Best –

Peter

[dgteamModerator]

Hi Peter,

Thanks for the gentle nudge 🙂

We looked back at this some more and have some ideas.  First, try clicking the Integer button to the right of the Duration input of the animation, when you run the animation.

We think this might speed things up a lot.  Here is why… When the value of t is input into the variable ‘Month’, the ‘t’ will be rounded to the nearest index (integer) to find a match, However the calculations are still be performed every time t changes, which happens about 30 times a second.  So it is likely performing the needed calculations much more than you really need for this data. However, if ‘t’ is set to an integer the program will only calculate a new frame for the animation when t hits the next integer value.

I will be very curious if that helps!

Also, I tried to get a similar file but when I download from the site I only get one time value in the netCDF that is generated.  Are you appending the data or is there a way to get multiple days or months in one netCDF file?

A final thought, if you are happy with the animation, then you may want to go ahead and  create the movie to share as an mp4 file.  That will run at the speed you specify as it is not performing the calculations anymore.  So not a big hit to your CPU.

 

[Author]

Posts