(NEW) HD4DP v2 CSV Upload
Table of contents
- Additional field information
- Author group, Author, Coauthor
- Field name 'STATUS'
- NISS number
- Country
- Formatting of 'Date' and 'Date:Time'
- Formatting of repeatable fields
- Formatting of a multifple choice
- Diverse formatting
- Architecture
- Testing and Working
- Third-party libraries and frameworks
Additional field information
Author group, Author and Coauthor
When the Author Group, Author and Coauthor have been left out in the csv file, the default Author group, Author and Coauthor will be used automatically.
When the desired Author Group, Author and Coauthor are specified in the csv file, the headers TX_AUTHOR_GR, TX_AUTHOR and TX_COAUTHOR must be added to the csv file with their respective values.
Example:TX_AUTHOR_GR;TX_AUTHOR;TX_COAUTHORTest group;test@sciensano.be;test@sciensano.be
Note:
The Author group, Author and Coauthor must exist and are well configured at the back-end of the system. TX_AUTHOR_GR can be a string that identifies the Author group to which this Author belongs. Commonly, the first name and last name are used to identify the TX_AUTHOR_GROUP. Be sure to avoid leading and trailing spaces when entering the Author group value.
Field name 'STATUS'
Add the field name 'STATUS' in capitals in an additional column.
Add the value 'draft' for this field name in case a manual submission of the record is requested. If the value is not added, the record will be submitted without manual intervention.
NISS number
Adding separators to a NISS number
It is not necessary to add separators in a NISS number when uploading a file using CSV Upload. You can fill out the NISS number both with or without separators. E.g.: 85.04.02-169.32 or 85040216932.
Example:

Field 'Country'
Completion of the postal code if the country to be selected is not Belgium
Only for the registries that need completion of the field Country (CD_CNTRY_RES) and the country to be selected is NOT Belgium, then the postal code (CD_POSTCODE) needs to be left empty and thus the code “9999 – Woonplaats niet in België” should not be selected.
This is important to avoid blocking and not uploading of your CSV uploads.
Formatting of 'Date' and 'Date:Time'
The correct format for a field reporting the Date is dd/mm/yyyy.
The correct format for a field reporting the Date and time is dd/mm/yyyy: hh:mm:ss
Formatting of repeatable fields
In some DCDs, a complete block of fields is repeatable. In that case, the values have to be separated by a semi-colon (";")
Example:
- as presented in the DCD:

- as presented in the form:

Manually enter the repeatable fields as shown in the following template: field_category|<index>|field.
- Values:
Repeatable 1:
Transplants
transplant_status
Repeatable 2:
Transplants
transplant_status
- Formatting:
transplants|0|transplant_status;transplants|1|transplant_status
treating_physician|0|rizivinamicode_of_the_treating_physician;treating_physician|1|rizivinamicode_of_the_treating_physician
Formatting of a multiple choice
With multi codes there is only one column per field. So when a select box is set as multiple, values have to be separated by a "|". e.g.: 68452|68453|68454.
Example:
- as presented in the DCD:

- as presented in the form:

For reporting fields with a multiple choice formatting, the selected answers can be reported separatedly by means of the pipe symbol (|).
- Example:
cftr_modulating_therapy_1
1|2|3|4|5|6
Diverse formatting
- Boolean: true / false
- Codes: the value of the code (not the translation)
- CSV file name: HD_DCD_submcsv_HDBPnumber_HDBPabbreviation_versionnumber_versionreleasedate
Architecture
The CSV Uploader is located under hd-connect/csvuploader. It uses both hd-connect-csvuploader and hd-connect-proxy modules.
The general architecture of the CSVUploader is explained in the sequence diagram below.

Testing and working
- The CSVUploader creates a folder at root level (SFTP for end-user, or hd-all for developer) that contains a subfolder per existing organisation.
- The CSVUploader will poll with a delay of 1 min, process the csv file and then create 3 folders:
- ARCHIVE folder: contains the source csv file.
- RESULTS folder: contains the results of processing the csv file. This file contains the specified data, and the final status of the processing: Success or Error. If an error occurred, the error message is displayed. For multiple uploads, the result is added to the end of the result file each time.
- ERROR folder: this folder is created if the csv test file was not processed, due to an I/O error (file corrupted, not found etc.). So for now, only technical errors are caught and the source csv file is moved to that folder instead of the ARCHIVE folder. In principle, this folder should contain any result that is an error. The RESULT folder should only contain results that end with a SUCCESS status.
Third-party libraries and frameworks
- Apache Camel: https://camel.apache.org/
- Spring Boot
docs.healthdata.be