3. Any feedback on RCS ICD 1.2, REGU 2.1, RDP 1.2 and SOC DPDD template 1.0
4. Next telecon
5. AOB (banish CDF Excel skeleton?)
Discussion items
Time
Item
Who
Notes
General
XB
See slides in roc-rcs-telecon14_roc-status-planning.pdf below for support.
ROC Pipelines
XB
XB says that the ROC pipelines design rework is still in progress. However, the CDF production part is not affected by the rework for now and, hence, he will not wait before producing new ROC-SGSE V3 CDF. This should be done before the end of April 2018. The teams should be ready to test the generation of preliminary ROC-SGSE L1R V3 CDF at this date.
ROC RCS-related planning
XB
Concerning the planning, it still has to be confirmed, but based on a Feb. 2020 launch scenario, RCS teams can expect to deliver the RCS "ready-for-flight" version around April 2019.
Although the ROC man-power issue is still not fixed, XB proposes to have an intermediate delivery step on December 2018. The exact content of this "preliminary ready-for-flight" version has to be detailled, but it might contain at least a first full data processing RCS with partial RCS ICD compliance (TBC).
ESAC DPDD document
XB
There is no feedback from teams concerning the DPDD template document from ESAC (see template here). XB assumes that there is no question or issue concerning this document and will send an email before the next telecon to propose a strategy for the writing of this document. In all cases, the idea is to avoid to many redundancies with data description already provided in existing documents (ESAC has indicated that references can be provided if required).
In practice, the description of RPW science data can be done in this DPDD, but the way the data are generated (including calibration) should be given in the RCS software user manual (SUM).
The case of the L1R data description must be clarified. ESAC indicates that the L1R data should be also described in the DPDD if the RPW team plans to archive it at ESAC. For the moment L1R is assumed to be an "internal" data level for RPW only, but there are possible options to archive them at ESAC too. See also the questions-answers between XB and Pedro Osuna from ESAC in the comments at the bottom of this page.
2. TDS status
DP, JS
DP indicates that creation and delivery of a first version of RCT files should be done before the end of the week. The delivery concerns RCT for a high frequency mode and magnetic component.
The generation of a first set of TDS L1R CDF is possible with the TDS_CALBA 0.8 version. TDS team should be thus ready to generate L1R CDF when ROC will publish ROC-SGSE L1 V3 CDF.
2. Bias status
EJ
First version of RCT files for Bias should be available before the end of March.
2. THR status
QN
QN has to discussed with Milan and Lorenzo about the RCT as well as documentation status for THR.
XB can organize a dedicated meeting with the THR team if required
3.
All
XB has received no feedback from teams concerning the new RCS ICD 1.2, REGU 2.1 and RDP 1.2 versions. It is assumed that there is no issue with this new versions, and the draft will be frozen and officialy released. Applicability of these new versions will be discussed at the next telecon, nevertheless teams are invited to start upgrading of their RCS and CDF skeletons to include new features. Any question can be sent to XB.
4.
All
Next ROC RCS telecon #15 is planned on Monday, 23th of April 2018.
5.
EJ suggest to not used anymore the Excel format to generate the CDF skeleton files (but used ASCII skeleton table files directly). He points out, among other, that the Excel format cannot permit to easily check if skeleton structure is ok and can be converted without error into master binary CDF. XB reminds that the Excel format was initially introduced to support teams, which were not familiar with CDF skeleton concept, and there is no problem from the ROC point of view to abandon this format if teams agree. But, it is admited that it is not an urgent task to do, and the priority should be done first on the achievement of the current open tasks (i.e., RCS CDF science and RCT skeletons, data generation and validation).
Below is the answers of Pedro Osuna from ESAC, about questions asked by Xavier Bonnin (ROC) concerning the content of the DPDD:
Dear Xavier,
please fin attached answers to your questions:
1) RPW uses some internal data processing levels (LZ, L1R, HK), which are not defined at SOC level. Especially, the L1R level is generated from the L1 CDF files and required for the production of the L2 CDF files. This intermediate level only contains L1 data + information to link science parameters with the calibration table files. It is used as an interface by our software to easily/automatically associate L1, L2 and calibration table files. Do this L1R data should be described somewhere in the DPDD? (For a full understanding of the RPW science data processing it should be interesting to have at least a description of this level, for instance in appendix.)
- LZ: According to your document ROC-PRO-DAT-NTT-00006-LES (RPW Data Products):
[…]There is only one RPW LZ data set containing the RPW TM raw binary packets, as returned by the Solar Orbiter Data Dissemination System (DDS) [RD7].[…]
it seems that your LZ data are the “TM” data that we are already storing in the archive, Telemetry packets coming from MOC. You can see what we currently have (from simulations) in the archive at:
Also, according to the same document you seem to store the TM packets in the L0 HDF5 file, right? So there does not seem to be a specific need to store your LZ data.
- HK: In my understanding, the HK data should be part of the L0. Please let us know if this would not suit you.
- L1R: if I understand properly, this is only an internal product:
[…]The L1R intermediary datasets will be for internal purpose only, and thus should be not distributed to teams outside the RPW consortium[…]
So in principle we would not store it, right? In case you would like us to store it, but not make it available to the whole community, we would need to discuss this case specifically to find an ad-hoc solution.
2) Does the section 3.3 (Data generation) of the DPDD shall describe in details the data processing, including the way to produce L2/L3 data (i.e., link to the calibration table, transfer function, equations, modeling, input/output etc.)? Or a short description with a reference to a more complete document is enough? (It is just to avoid as much as possible redundancies between SOC and RPW documents).
A short description with a reference to a more complete document would be fine, provided the document is made available.
3) In the section 4.1.X, can we describe data content using the same structure than in [RD.02] (i.e., table with list of zVariables and variable attributes, similar to CDF skeleton table format) or do you expect a structure such as in [AD.01].
This document should resemble the LLCDFDPDD as much as possible, so it is OK to use the [RD.02] as a reference (albeit for Science data instead of LL).
4) Do you have specific expectations/conventions concerning the data products name? For instance at RPW level, we have a dedicated convention for uniquely identified datasets generated by our pipeline (i.e., SOLO_[LEVEL]_[DESCRIPTOR], where [LEVEL] corresponds to the data processing level e.g., L1/L2/…, and [DESCRIPTOR] is the prefix part (i.e., before ">") of the Descriptor global attribute, e.g., RPW-TDS-SBM1-RSWF). Can we use this convention in the DPDD (as data product name and to link with other dataset/calibration table dependencies)?
As long as it obeys the [AD.01] (Metadata document) naming convention, the rest is OK with us. After checking with the ESDC (SOAR developers) the name you sent seems OK, and it also looks readable for the user, so you are fine with it.
5) We also plan to generate summary plots. Does the SOC plan to archive also this kind of data products at some point?
Of course, we will be happy to store your summary plots. This should be part of L3 data.
Let me know if you have any further questions or comments, and thanks for looking into this now.
1 Comment
Xavier Bonnin
Below is the answers of Pedro Osuna from ESAC, about questions asked by Xavier Bonnin (ROC) concerning the content of the DPDD:
Dear Xavier,
please fin attached answers to your questions:
- LZ: According to your document ROC-PRO-DAT-NTT-00006-LES (RPW Data Products):
[…]There is only one RPW LZ data set containing the RPW TM raw binary packets, as returned
by the Solar Orbiter Data Dissemination System (DDS) [RD7].[…]
it seems that your LZ data are the “TM” data that we are already storing in the archive, Telemetry packets coming from MOC. You can see what we currently have (from simulations) in the archive at:
http://soardev.esac.esa.int/soar-beta/#home
selecting “Telemetry” in the type of data.
Also, according to the same document you seem to store the TM packets in the L0 HDF5 file, right? So there does not seem to be a specific need to store your LZ data.
- HK: In my understanding, the HK data should be part of the L0. Please let us know if this would not suit you.
- L1R: if I understand properly, this is only an internal product:
[…]The L1R intermediary datasets will be for internal purpose only, and thus should be not
distributed to teams outside the RPW consortium[…]
So in principle we would not store it, right? In case you would like us to store it, but not make it available to the whole community, we would need to discuss this case specifically to find an ad-hoc solution.
A short description with a reference to a more complete document would be fine, provided the document is made available.
This document should resemble the LLCDFDPDD as much as possible, so it is OK to use the [RD.02] as a reference (albeit for Science data instead of LL).
As long as it obeys the [AD.01] (Metadata document) naming convention, the rest is OK with us. After checking with the ESDC (SOAR developers) the name you sent seems OK, and it also looks readable for the user, so you are fine with it.
Of course, we will be happy to store your summary plots. This should be part of L3 data.
Let me know if you have any further questions or comments, and thanks for looking into this now.
Cheers,
P.