General Information
Studies can be uploaded to SEND Explorer through the SEND Explorer Dashboard interface by using the Upload Study Data or Upload to Watch Folder (Warehouse only) buttons. Only studies from Source Systems with an uploadable Source System Type can be uploaded through these methods.
Studies in SEND 3.x XPT or Dataset JSON format must be in a .zip file with no password protection. Studies in CSV, pipe-delimited, or tab-delimited format must also be zipped. In addition, all datasets must have at least one record and cannot have a size of 0 KB.
All primary dataset packages must include a TS domain with at least one record. This is used to establish the STUDYID in SEND Explorer. Loading a study without a TS domain can leave orphaned records in the database stage tables.
During the file upload process, datasets are scanned and if basic errors are identified with the source file format or the data, error messages will appear in the interface.
For files uploaded through the Upload Study Data button, error messages will appear directly on the screen and will prevent the study from being uploaded (Plus and Warehouse) or added to the Load Queue (Warehouse).
For files uploaded through the Upload to Watch Folder button, files processed successfully are moved to the “queued” folder and files that are not able to be processed are moved to the “failed” folder. Status of studies uploaded through Watch Folders can also be checked from Data Steward/Watch Folders Watch Folders Log button.
Any time an error is encountered while trying to load studies, always check the SEND Explorer error log to gather more details on the event. The error log can be accessed from the Dashboard page, bottom left, Log hyperlink (see screenshot below).
Common Loading Errors
The most common issues seen during the study loading process are included in the table below.
Description | Message | Resolution |
Zip files containing SAS XPT files for a study upload cannot be password protected | One or more errors occurred. (The archive entry was compressed using an unsupported compression method.) | Extract the zip file using the password and create a new zip file from the extracted folder without a password. Upload the newly created zip file to SEND Explorer. |
SEND required variable that is also required for upload to SEND Explorer is null | Integrity constraint SENDEXP.FK_SEND31_ LOAD_HISTORY_REF_ SEND31_STUDY_DETAIL) violated - parent key not found: cannot insert NULL into ("SEND_PROD"."SEND31_DM"."RFSTDTC") | Dataset must be updated to include data for all records for the null required variable.
Pinnacle Community validator can be run against the datasets to identify null required variables. |
Primary key or unique constraint violation | Unique constraint (SENDEXP.SEND31_LB_U01) violated | Dataset must be updated to remove duplicate records. The problem domain will appear in the name of the constraint that was violated.
Pinnacle Community validator can be run against the datasets to help identify domains with duplicate records. |
Datasets contain one or more XPT files with 0 length (do not contain any records). This can be confirmed by opening the zip file in Windows Explorer and reviewing the Size column. | One or more errors occurred. (Index and length must refer to a location within the string. (Parameter 'length')) Index and length must refer to a location within the string. (Parameter 'length') | Contact the dataset supplier to request updated datasets.
Alternatively, you can remove the dataset(s) that have length = 0 and reload the zip file to load all valid data. This may result in missing data for the study if the zero length datasets should contain data. Note the TS, TX and DM domains are required to upload a study to SEND Explorer. |
Column length is too long and violates SEND and SEND Explorer database requirements | Error(s) in array DML | Contact the dataset supplier to request updated datasets that correct any column length issues.
Pinnacle Community validator can be run against the datasets to identify variables with invalid column lengths. |
A spreadsheet that includes SEND Explorer database table constraints is included below.
Load History
Load History, available from the Data Steward menu, provides logging information related to SEND Explorer study loading. There are several rows related to each load, reflecting the various stages of the process including Load Stage, Load Reference Data, Load Master Data, Load Data Store and Load Mart. Following completion of a load cycle, there will be a Load Status value for each step to indicate if the step succeeded, failed or succeeded with rejects. If a step fails, an error message will be displayed in the Status Message column. Click on the Details column to see more information including complete logging entries and links to rejected records, if applicable.
Incomplete Loads
Rejected Records
The number of records rejected during the load process are displayed in the Rejected column on the Load History Detail page for any step with Load Status = Succeeded w/Rejects. Click on the Details link in the Details column to see more details and a link to a list of the rejected for the affected table. Click on the link in the Rejected column to display a table with the rejected records that includes an error message in the oraErrMesg column, see screenshot below.
Often rejected records are due to unique or integrity constraint violations in the data. Review of the records can help determine if the rejected records are duplicates and do not need to be loaded, or if the records do need to be loaded and the dataset corrected to eliminate issues. Contact Certara report if for assistance troubleshooting rejected record issues.
Domains Available to Load
SEND Explorer will try to load all data from all domains during initial loading. When not all domains make it through the full upload process, the Load Status for all steps in Load History may be "Succeeded," but the Load Status for the study in the Dashboard once the load is finished will be "Domains Available to Load." Click on the link in the Details column in the Dashboard for the study to determine what domains did not load successfully. Domains that did not load will have 0 in the Data Store Count column, see screenshot below.
Often domains do not load if there are issues with non-matching USUBJID/POOLID values between findings domains and the DM or POOLDEF domains, or there are records referenced in related domains (--SUPP, --RELREC) that are not present in the parent domain. If you are unable to identify why a domain is not loading for a study, contact Certara support for assistance.