Complete technical environment for operating a health data warehouse in compliance with EDS and SNDS standards. Concepción is the foundation on which all Clinityx EDS platforms are built, also available as SaaS for third parties.
The Concepción architecture is based on strict segmentation into 4 VLANs, from raw SNDS data ingestion through to analysis environments for internal and external researchers.
Each component has been designed to ensure security, traceability, and regulatory compliance at every stage of data processing.
3-step ETL pipeline: conversion of raw SNDS data to Parquet format, application of documented cleaning rules (WikiPipeline), extraction by business entities. Developed in Spark/Scala. Produced entities: CCAM, ICD-10, LPP, Drugs (with DDD), UM, Patients, Consultations, Biological Acts, ALD, GHM. V2: medico-economic pipeline for care pathway valuation.
Spark/Scala · ETL · SNDSSuite of 3 complementary tools: Filters (extraction of analysis datasets from the datalake, 11 specialised Spark/Scala jobs covering care pathways, hospitalisations, drugs, DDD exposure, polypharmacy, resource consumption and costs), Rosetta (registry/cohort data processing in Python, pre-quality assessment and data-management library), DataMerger (cross-referencing SNDS and registry sources). 100% test coverage on Filters.
Filters · Rosetta · DataMergerLibrary of statistical micro-services in Python to capitalise on analysis code. 8 packages: data_management (imputation, MICE propensity scores), Description_table_generator (descriptive analyses in Excel/HTML), Epidemio_indicators (incidence, prevalence), stats (Kaplan-Meier, Cox, Fine & Gray, logistic regression, Poisson), Criteria_filters (population creation), Table_merger, Description_graph_generator and Preliminary_report_generator. Collaborative architecture with Git versioning and peer review.
Python · Micro-services · StatisticsPatient identifier pseudonymisation module. Irreversible hashing compliant with CNIL requirements for the protection of personal data.
PseudonymisationSecure object storage in three layers: raw data, structured SNDS data, and health data (Health Data). AES-256 encryption at rest.
Storage · EncryptionIsolated storage space per research project. Each study has its own directory with granular role-based access control.
Project isolationConcepción implements a role-based access control model (RBAC). Each profile only accesses the data and tools required for its mission.
Management of ingestion flows, VLAN configuration, infrastructure supervision, and security auditing.
Operation of the Victoria pipeline (Spark/Scala), management of SNDS structuring ETL jobs, Data Minimization Toolbox configuration, preparation of project datasets.
Access to Jupyter and RStudio environments, use of Atlas packages (data_management, Description_table_generator, Epidemio_indicators, stats), DAG workflows via Jupyter, execution of statistical analyses and ML models.
Read access to the project NFS and aggregated results. No access to individual data or the datalake.
Concepción is the infrastructure on which all health data warehouses operated by Clinityx are built, whether managed on its own behalf or on behalf of third parties (learned societies, university hospitals, industry).
Each third-party client benefits from their own isolated VLAN with their own access and retention rules.
Clinityx handles the complete regulatory pathway: CESREES, CNIL, MR-004, DPO, and processing register.
Third-party data can be linked to the SNDS within the Concepción infrastructure to enrich patient pathways.
R, Python, and Jupyter available. Researchers work in a secure environment without ever exporting individual data.
Looking to deploy a health data warehouse compliant with CNIL and SNDS standards? Contact our team to discuss your project, in dedicated or SaaS mode.