The AI Clinician

AI Clinician: Reinforcement Learning in Intensive Care

Code for a reinforcement learning model applied to the management of intravenous fluids and vasopressors in patients with sepsis in intensive care.

Code release for publication:

https://www.nature.com/articles/s41591-018-0213-5

Corresponding Authors: A.A. Faisal & A Gordon

Code Development Lead: Dr Matthieu Komorowski, Imperial College London - m.komorowski14@imperial.ac.uk

The 2 datasets (MIMIC&eICU) used in the research are available here and as per license agreements with the data providers cannot be shared by us and instead the respective data owners should be contacted to obtain permission and access to the data:

• MIMIC-III : https://mimic.physionet.org/
• eICU-RI: not publicly available in full, subset available here: http://eicu-crd.mit.edu/

Cohort definition: all adult patients fulfilling the sepsis-3 definition: http://jamanetwork.com/journals/jama/fullarticle/2492881

The unique identifiers for these patients in both datasets are provided (patientIDs_MIMIC3.csv and patientIDs_eRI.csv), along with a detailed desciption of the datasets (Dataset description Komorowski 111118.xlsx).

https://gitlab.doc.ic.ac.uk/AIClinician/AIClinician

This repository contains the Matlab code (AIClinician_core_111118.m) to:

1. build 500 different discrete state and action MDP models from the MIMIC-III training dataset;
2. select the best policy from off-policy evaluation on the MIMIC-III validation set;
3. test this optimal policy on the eICU-RI dataset;
4. compute the main results and key figures.

External files and toolboxes used:

• MDP toolbox (some functions modified): https://uk.mathworks.com/matlabcentral/fileexchange/25786-markov-decision-processes--mdp--toolbox
• Fastknnsearch : https://uk.mathworks.com/matlabcentral/fileexchange/19345-efficient-k-nearest-neighbor-search-using-jit