Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

ItemWhoNotesAction-Items
WelcomeX.Bonnin
RPW ground segment validation plan overviewS.Lion
Information from ECSS on testsD.Bagot
Discussions on ROC validation activities: ROC fonctions validation, ROC software validation, ROC interface validation, ROC data validationAll
  • Validation procedure
    • REQ-ROC-SSS-0152 too long --> need to split RSSS requirement between retrieving and processing
  • Determine/write the validation procedures
    • Responsibilities: Sonny Lion (SL)
  • Action Xavier: split the REQ-ROC-SSS-0152 into two requirements
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-73
  • Action Sonny (with the support of Xavier): Finish the write of the "roc_validation_activities_v02.xls" Excel file
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-74
  • Action Sonny: Once the Excel file is completed, then update the Validation Plan in consequence
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-75

Discussion on RPW science data validation (ROC, THR, BIAS, LFR, SCM, TDS): description, task, workflow validation, human resources / team, responsibilities, constraints, anomalies, .. etc.

All
  • Data validation
    • Formal validation - How to?
      • ROC checks automatically the metadata and structure
      • Can also automate the checking of val_min/val_max (out of range values)?
      • When formal validation is done → global attribute "Validate = 1" (validate flag)
  • Science quality?
  • Idea:

    - Check if a value is lower than the sensitivity

    - Check if a value is constant

    - Check for discontinuities

    Teams are in charge of determining the quality flag

    If quality of the records is enough -> The ROC can set the validate flag to 2

    ROC is responsible for the bitmask

    Science quality verification procedure:

    - Add a piece of software in the RCS to compute the quality flag

    - in case of problem -> generate a report to be sent to the scientist in charge


    - SCM

    Problem : Only one quality flag for each x,y,z record (no quality flag by axis)

  • Reference frame

    • Add rotation matrix in the L1 data -> we need to process ancillary data

  • Action Milan: Send an email to Alexis Rouillard and the MADAWG concerning the qualit_flag level definition in the SOL-SGS-TN-0009_2_2 document.
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-76
  • Action RCS teams: Define criteria to determine the quality flag
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-77

  • Action RCS teams: Fill the Excel sheets provided by the ROC and send it to XB
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-78
  • Action Xavier : Send a document that allows RCS teams to list the expected values for the bitmask
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-79
  • Action RCS teams: Determinate the bitmask content required to define the science quality flag
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-80
  • Action Xavier: Determine the full procedures for each data product/team
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-81
  • Action SCM team: See how to set the quality flag for each X,Y,Z axis records
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-82
Discussion on RPW flight validation: description, task, workflow validation, human resources / team, responsibilities, constraints, anomalies, .. etc.All
  • roc_validation_workshop_flight_validation_intro_v01.pdf
  • ROC has prepared specific Excel sheets for the in-flight validation activities of RPW. The main idea is to list all the activities and related relevant information in order to prepare these tasks (to ensure nothing has been forgotten)
  • COMMISSIONING validation activity:
    • The reaction wheels filtering should be added somewhere (during interference campaign or antenna rolls?)
    • A 8th RPW activity should be asked to MOC to test the nominal working of RPW before the Cruise Phase (CP). Especially, all modes should be tested and run for enough long time (~24h) and nominal modes with other IS instruments should be run. If possible the SBM_DETECTION should be activated (but quick downlink of selective data is not garantuee for now during the NECP and CP phases)
    • What about the inter-instrument communication (IIC) validation? Done during NECP or CP? Functional part (checking TM exchanged via S20 should be possible, but it would be difficult for the detection part without event).
  • Action Xavier: Verify with MOC/SOC if IIC/S20 flight validation campaign is planned and when
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-83
  • Action Xavier: Prepare and submit to MOC the 8th activity for RPW
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-84
  • Action teams: submit to the ROC (Xavier) a preliminary version of the "<XXXXX>_flight-validation_V<YYY>.xls"
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-85


Splinter session RPW flight configurations

RPW lead CoI teams + Diane and Antonio
  • COMMISSIONING

——————————————

    • During commissioning HK at least every second (or even higher cadence if possible)
    • iBoom deployement : In order to synchronize snapshots between LFR and TDS, TDS will increase the time between two snapshots to 11s (instead of 10s) 
    • Antenna + I-Boom + interference campaign : all sub systems in SBM1+normal mode. Same configuration for LFR and TDS for each campaign.
    • LFR : temps entre 2 snapshots 22s
    • TDS : 1 snapshot chaque seconde + 1 triggered toutes les 11s —> un snapshot every 11s, one simultaneously with LFR, one without LFR
    • SCM agrees with LFR config
    • LFR wants to add during interference campaign a configuration to test the reaction wheels effect.
    • The LFR document for interference campaign has been transfered to the Bias team for filling its own doc.
  • LOW RATE

——————————————

    • The daily 10 min. of  SURVEY_BURST mode do not take too much telemetry rate (<5%) so we can let it. But the SBM1 mode has significant telemetry rate.
    • For LFR, the main goal is to have the higher cadence as possible for the BP.
    • TDS has no specific constraint
    • 2 configs for low rate : 

1)

  • LFR :
    • no SBM,
    • 10 min of SURVEY_BURST,
    • time between two snapshots 1800s;
    • time between two ASM : 3600s,
    • time between BP : 8s;
    • time between 2 products : 40s
  • TDS :
    • Time between 2 RSWF : 1800s;
    • Time between 2 TSWF : 7200s

—> cf calculator LOW RATE 1 (TODO : Put the ref)

2)

  • LFR :
    • SBM1 activated
    • 10 min of SURVEY_BURST,
    • time between two snapshots 3600s;
    • time between two ASM : 3600s,
    • time between BP : 16s;
    • time between 2 products : 60s

—> cf calculator LOW RATE 2 (TODO : Put the ref)


  • Update Calculator

—————————————————————
LFR calibration : snapshots every 22 s (change in LFR software in progress).This will have a slight impact on the data rate because currently the same configuration is kept in NORMAL-DEFAULT: 1 snapshot every 300s

  • All team will provide word files about antenna, I-Boom and Interference campaign before the end of next week
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCOPE-217
  • Action Antonio : Put on Git the two calculator sheets for the low-rate config.
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCOPE-218
  • Action Diane : Send the calculator sheets with the two low rate config to LFR and SCM (TDS already has them).
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCOPE-219

  • Action Antonio : Put on Confluence the page with all the nominal configs (as well as the fixed parameters) and send the url to the teams.
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCOPE-220
  • Action Diane : Add LFR calibration in the calculator sheet (because in low-rate we stay in default for the calibration)
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCOPE-221

Splinter session : ROC validation traceabilityCNES + Stephane Papais + Sonny Lion

1. Sonarqube -  Code Analysis

 Actions to be carried out by LESIA (Sonny Lion) with Stéphane Papais support :

- Finalize metrics according to CNES specifications document

- Installation of Sonarqube on the ROC Dev platform => November 2018

- Objective on Sonarqube: implement for the RSS3 validation campaign =>  End of December 2018

- Tests reports perimeter : based on RSS3 => Music (Faust / Figaro), ROC-SGSE, LLVM

 

Actions to be taken by Dominique Bagot:

- Realize the code analysis with Sonarqube on RSS3 => December 2018


2. Validation Plan

Actions to be carried out by LESIA (Sonny) :

  • Define the scope that will be part of the validation campaign and list the associated requirements (RSS3)
  • Complete the validation plan with the different test procedures (test case)
  • Complete the traceability matrix between the RSSS and the test plans => Sonny Lion with the support of Stéphane Papais
  • Make the links between the different unit tests / integration and the test procedures (test case) => Sonny Lion
  • Identify / distinguish the testers (technical or functional (beta tester)) => cf. example of the Taranis project
  • Referencing unit tests / integration
  • Action Sonny (with support of Stéphane):
    • Finalize the metrics according the CNES Quality requirements doc.
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-86
    • Install Sonarqube in the ROC dev. machine
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-87
    • Prepare Sonarqube to be used for the RSS3 validation "rehearsal" campaign
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-88
  • Action Dominique : Do an analysis of the ROC code with Sonarqube in prevision of the RSS3 release
    Jira
    showSummaryfalse
    serverjira-lesia.obspm.fr JIRA
    serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
    keyROCMAN-89
  • Action Sonny:
    • Define the perimeter for the RSS3 validation campaign (and associated requirements)
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-90
    • Complete the validation plan with the test procedures (test cases)
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-91
    • Complete the traceability matrux between RSSS and the test plans (with the support of Stéphane)
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-92
    • Link the unit/integration tests and the test cases
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-93
    • Identify the actors for the validation campaign
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-94
    • Make reference for the unit/integration tests
      Jira
      showSummaryfalse
      serverjira-lesia.obspm.fr JIRA
      serverIdcb08a004-0694-3237-bcb1-3125e3ed9bc0
      keyROCMAN-95


Conclusion

1) Schedule :

  • RSS3 campaign validation Schedule : November 2018 (to be confirmed)
  • PTF delivery of RSS3 to CNES : mid-December 2018
    This delivery will contain :
      • Software
      • Documentation (updated documents) :
        At least the delivered versions for the specifications  and validation workshops

      • The LESIA emphasizes that it is difficult to advance (due to the HR resources constraints / lack of a developer) to develop the tools and to write and to update documentation.


2) Actions to be taken into account following the validation workshop:

a) Reminder of the actions/issues from specifications workshop and its deadlines to be followed (and documents identified to be updated) => see meeting note Spécifications workshop

b) The validation workshop allowed to define a roadmap for ROC validation activities :

  • Each team (consortium member) have to complete validation activities in excel files. The LESIA will follow up this action  => deadline : mid-December 2018.
  • The LESIA complete  the validation plan from excel files  => deadline : end of december 2018

c) Question : is it necessary to organize monthly teleconf with extended participants (CNES, LESIA, LeadCoI, Instrument scientis) to follow up the previous point (=> point b)  => consistency of actions / roadmaps

d) Question : is it necessary to organize a next validation workshop ?

e)  For the next Consortium Meeting in Kiruna (March 2019), it is important and necessary to add the topics related to ground segment activities (requirements, development, validation, data products, sciences data and validation, tools, development progress, ESA interfaces, ..) : plenary sessions and  splinter sessions to be added in agenda.              

=> Xavier will discuss it with Milan

    

3) Validation plan final version : scheduled for RSS4VC (2019). The LESIA have to ensure the consistency between requirements, implementation, validation and tests.

 



...