Nesting bug with quilting I/O

Post here if you think you might have found any type of bug

Nesting bug with quilting I/O

Postby zaphiri » Mon Nov 02, 2015 12:32 pm

I fist came across what I thought was a bug in 2005, on a POWER cluster, and I reported it. I have also seen several posts on this subject, but it appears that no resolution or fix was applied. The problem is consistent on many systems to my knowledge. Does anybody know of any fix, especially now that folks run at higher resolutions where quilting is critical to I/O performance?

The problem :twisted: :evil: .
When running WRF (from versions 2.2 to 3.7) with a minimum of two nests using quilting with Netcdf, it appears that the outer nest (Domain1) wrfout* is consistent and correct. However, none of wrfout* files of the inner nests contain data that can be used for post-processing. It seems that the files have data, but dates and end-of-file markers are missing, making them practically useless.

The solution: :cry: :roll: ?
zaphiri
 
Posts: 2
Joined: Mon Nov 02, 2015 12:08 pm

Re: Nesting bug with quilting I/O

Postby kwthomas » Mon Nov 02, 2015 4:15 pm

Quilting doesn't seem to work on all systems, however, I can still make a guess.

Did you build WRF with the environmental variable "WRFIO_NCD_LARGE_FILE_SUPPORT" set to 1?

If not, and your netCDF files are >2gb, they *will* be corrupt.
Kevin W. Thomas
Center for Analysis and Prediction of Storms
University of Oklahoma
kwthomas
 
Posts: 168
Joined: Thu Aug 07, 2008 6:53 pm

Re: Nesting bug with quilting I/O

Postby zaphiri » Mon Nov 02, 2015 9:56 pm

Yes, WRFIO_NCD_LARGE_FILE_SUPPORT was set to 1, which is reflected by the fact that the outer nest has the right information, even though it's size is larger than 2GB.

The size of the wrfout netcdf file of the inner nest with quilting is identical to the size of the netcdf inner nest wrfout file without quilting. The difference is that the file does not show anything after the header right at the keyword data when you ncdump it, while a good netcdf file shows actual numerical values.
zaphiri
 
Posts: 2
Joined: Mon Nov 02, 2015 12:08 pm


Return to Bugs

Who is online

Users browsing this forum: No registered users and 1 guest