Interoperability for Climate and Atmospheric sciences 

This les­son intro­duces inter­op­er­abil­i­ty in cli­mate and atmos­pher­ic sci­ences as the abil­i­ty to make research data usable across dif­fer­ent tools, sys­tems, and work­flows with min­i­mal dataset-spe­cif­ic inter­ven­tion.  

Cli­mate and atmos­pher­ic research rou­tine­ly com­bines het­ero­ge­neous and often large datasets pro­duced by mod­els, satel­lites, radar sys­tems, sen­sors, and research infra­struc­tures. Mak­ing these data avail­able is there­fore not enough: researchers and soft­ware must also be able to deter­mine how the data are organ­ised, under­stand what they mean, and access them through pre­dictable tech­ni­cal mech­a­nisms. 

The course approach­es this chal­lenge through three com­ple­men­tary lay­ers of inter­op­er­abil­i­ty: struc­tur­al inter­op­er­abil­i­ty, which con­cerns how data are organ­ised and rep­re­sent­ed; seman­tic inter­op­er­abil­i­ty, which con­cerns how sci­en­tif­ic mean­ing is expressed and shared; and tech­ni­cal inter­op­er­abil­i­ty, which con­cerns how inde­pen­dent sys­tems access and exchange data and meta­da­ta. 

Using cli­mate and atmos­pher­ic data as the prac­ti­cal con­text, learn­ers work with com­mu­ni­ty for­mats and con­ven­tions such as NetCDF and the CF Con­ven­tions, inspect and sub­set remote datasets through DAP/OPeNDAP, inter­act with repos­i­to­ry meta­da­ta through Web APIs, and explore how Zarr and Ker­chunk sup­port selec­tive and scal­able access to large mul­ti­di­men­sion­al datasets.  

Togeth­er, these episodes show how inter­op­er­abil­i­ty depends on coor­di­nat­ed choic­es about data struc­ture, sci­en­tif­ic mean­ing, access mech­a­nisms, stor­age lay­outs, and repro­ducibil­i­ty rather than on any sin­gle for­mat or tech­nol­o­gy. 

    

Learning objectives

• Assess a cli­mate or atmos­pher­ic dataset in terms of struc­tur­al, seman­tic, and tech­ni­cal inter­op­er­abil­i­ty and iden­ti­fy bar­ri­ers to its reuse. 
• Analyse how the struc­ture of a sci­en­tif­ic dataset, par­tic­u­lar­ly the NetCDF data mod­el, enables soft­ware to iden­ti­fy and process its dimen­sions, vari­ables, coor­di­nates, attrib­ut­es, and rela­tion­ships. 
• Eval­u­ate whether sci­en­tif­ic vari­ables are described with suf­fi­cient shared, machine-action­able mean­ing for reli­able inter­pre­ta­tion and com­par­i­son, using seman­tic resources. 
• Use DAP/OPeNDAP with Python to inspect and sub­set remote NetCDF data while dis­tin­guish­ing remote meta­da­ta access from data trans­fer. 
• Use a Web API to pro­gram­mat­i­cal­ly query and retrieve research data and meta­da­ta and explain how APIs sup­port machine-to-machine inter­op­er­abil­i­ty. 
• Com­pare con­ven­tion­al NetCDF, Zarr, and Ker­chunk-based access mod­els to explain how cloud-native lay­outs sup­port selec­tive and scal­able access to large mul­ti­di­men­sion­al datasets. 
• Explain AI readi­ness as a task-depen­dent prop­er­ty of a data work­flow by con­nect­ing struc­tur­al, seman­tic, and tech­ni­cal inter­op­er­abil­i­ty with scal­able access and repro­ducibil­i­ty require­ments. 

    

Target audience

The work­shop is intend­ed for researchers in cli­mate and atmos­pher­ic sci­ences who work with, or are inter­est­ed in work­ing with, mul­ti­di­men­sion­al NetCDF datasets and want to improve the FAIR­ness, inter­op­er­abil­i­ty, and reusabil­i­ty of their research data. It is also intend­ed for sup­port staff that need capac­i­ty in those top­ics. 

Par­tic­i­pants should have: 

• Work­ing knowl­edge of Python, includ­ing the abil­i­ty to write and exe­cute short scripts. 

• Basic aware­ness of the NetCDF for­mat. 

• A lap­top for the hands-on exer­cis­es. 

If you are inter­est­ed in hav­ing this train­ing at your insti­tute, please get in con­tact with the our Skills train­er Leila Iñi­go de la Cruz.