Recoding and migrating your code

Migration from SAS to Python of the SCES data processing pipeline

HAS is a public authority of a scientific nature whose objective is to assess quality in the healthcare, social and medico-social fields.

We were able to support the Healthcare Facility Certification Department (SCES) in transitioning these table and indicator creation processes from SAS to Python. While initially this was intended to be a simple recoding from SAS to Python, the teams ultimately took the opportunity - rightly so - presented by the rollout of a new Data Warehouse with a PostgreSQL database to overhaul the entire data storage and processing pipeline.

We therefore set up a Python working environment (using poetry) enabling:

1. querying SQL data in the Data Warehouse
2. creating the full set of tables, variables and indicators,
3. diagnostic and data validation testing (against the old SAS data and Pandera data schemas)
4. saving the tables in Parquet format to object storage

The SCES teams were then able to reuse these tables directly in their PowerBI visualisations.

Throughout this project, we also trained staff in Git (Gitlab platform) and Python so that they could take ownership of the project and be self-sufficient once the assignment was complete.

• Date: janvier - août 2025
• Client: HAS

Recoding programs from SAS to R

Injep is an institute attached to the Ministry of National Education and Youth, producing statistical indicators in the field of Education and Youth.

As part of the gradual move away from SAS, Statoscop carried out the recoding in R of certain scripts used to calculate these indicators, and also supported study officers in getting to grips with R: using automated reports with RMarkdown, optimising the execution speed of certain scripts, etc.

• Date: février - septembre 2023
• Client: INJEP

Recoding programs from SAS into an R package

HAS is a public authority of a scientific nature whose objective is to assess quality in the healthcare, social and medico-social fields.

Like other public administrations we have supported, HAS is faced with the challenge of gradually moving away from SAS for its analyses in favour of open-source, higher-performing options. As part of this, we supported the Department for the Evaluation and Tools for the Quality and Safety of Care (SEvOQSS), which wanted to move its SAS scripts and macros to R.

We therefore coded a documented R package that replicates the full set of features of the old SAS macros. In addition, the project was placed on a Gitlab repository to encourage the team to start using version control. We also supported the team on other ongoing recoding work to optimise their scripts in terms of execution time and management of particularly large databases. To do this, we made use of the Parquet format and the features of Apache Arrow.

• Date: juin - novembre 2024
• Client: HAS

Recoding Prometheus in R

The Prometheus microsimulation model is used to simulate individual household energy consumption, covering both transport modes and housing. It can therefore be used to assess the impact of schemes designed to help pay energy bills.

Statoscop supported the energy and solidarity transition office, in charge of developing Prometheus, in its wish to move from SAS to R. We first defined a methodology for recoding the model in R, before implementing it while making sure not to affect the model's performance.

• Date: janvier - mai 2023
• Client: CGDD - Bureau de la Transition énergétique et solidaire

Recoding the Ines microsimulation model

The Ines microsimulation model is jointly managed by Drees (the statistical department of the Ministry of Health), Insee and Cnaf. It is used to simulate the budgetary and redistributive effects of tax and welfare policy reforms.

After working on this model for 4 years, including one year as head of the microsimulation unit at Drees, Antoine initiated the project to recode the model, proposing a method (choice of programming language and libraries to use) and a projected timeline. He then recoded half of the model (around 20,000 lines of code) from SAS to R, making sure that the model's results remained strictly identical and that performance (the model's execution time) was not affected by this change.

• Date: octobre 2018 - mai 2020
• Client: Drees