Skip to content

jupyterhealth/jupyterhealth-exchange

BranchesTags

Repository files navigation

JupyterHealth Exchange

JupyterHealth Exchange is a Django web application that facilitates sharing patient-consented medical data with authorized users via a web UI and REST, MCP, and FHIR APIs.

Typical User Flow

Researchers create studies and recruit patients, who consent and submit observations via client applications, and the data is then stored in JupyterHealth Exchange and queried by researchers using Jupyter Notebooks or other systems.

Typical Data Flow

Users manage the system via the Web UI, and data producers receive invitation credentials by email, manage consents through the Admin API, and upload data to JupyterHealth Exchange using the FHIR API. Data consumers such as Jupyter Notebooks or other systems then query and download the data through REST and MCP APIs.

Features include:

Limitations & Status

This project is currently in a Proof of Concept stage, the project can be viewed on GitHub at the following URL:

https://github.com/orgs/the-commons-project/projects/8

Contributing

See CONTRIBUTING.md for test requirements, coding standards, and PR checklist.

Testing

See doc/TESTING.md for the full testing guide — unit tests, integration tests, smoke tests, and CI pipelines.

Upgrade / Migration Guides

  • v0.0.8 → v0.0.9 — DB-backed settings, multi-client support, breaking API changes

Getting Started

  1. Set up your Python environment and install dependencies from Pipfile - this project uses Django version 5.2 which requires python 3.12

    • NB: If using pipenv it is recommended to run pipenv sync against the lock file to match package versions

  2. Create a new Postgres DB (currently only Postgres is supported)

  3. Copy dot_env_example.txt to .env and update the DB_* parameters to match (2) above.

    • Optionally you can add a Django SECRET_KEY by running the command below or you can leave this for now to use a randomly generated value at runtime (this will not work with more than one worker) $ openssl rand -base64 32

  4. Ensure the .env is loaded into your Python environment, eg for pipenv run $ pipenv shell

  5. Run the Django migration $ python manage.py migrate to create the database tables.

  6. Seed the database by running the Django management command $ python manage.py seed

  7. Start the server with $ python manage.py runserver

  8. Browse to http://localhost:8000/admin and enter the credentials sam@example.com Jhe1234!

  9. Under Django OAuth Toolkit > Applications you should see the seeded OAuth2 application named JHE Admin UI with a Redirect URI for http://localhost:8000/auth/callback - this is used for the Web UI OAuth 2.0 login.

  10. Click the LOG OUT button at the top

  11. Finally, we need to create an RS256 Private Key for signing the JWT

    • Run openssl genrsa -out oidc.key 4096

    • Run awk '{printf "%s%s", (NR==1?"":"\\n"), $0}' oidc.key to remove line breaks Note: some python environments and OS combinations do not handle the "\n" so you may need to include line breaks in the .env file.

    • Return to the .env file and update the OIDC_RSA_PRIVATE_KEY

    • Keep the oidc.key somewhere safe

  12. Browse to http://localhost:8000/ and log in with the credentials mary@example.com Jhe1234! and you should be directed to the /portal/organizations path with some example Organizations is the dropdown

  13. New users can be signed up from the base URL (eg http://localhost:8000/) with the default invitation code "jhe" which is set from the REGISTRATION_INVITE_CODE in .env

Note

Due to browser security restrictions and the oidc-client-ts used for authentication, the web app must be accessed over HTTPS for any hostname other than localhost - see Running in Production below.

Remote debugging tips

  1. Control log verbosity – the backend reads the DJANGO_LOG_LEVEL env with a default if INFO for both the root logger and the Django logger.

  2. SPA debug banner – temporarily enabling Django DEBUG surfaces server-side tracebacks in the portal so you can correlate HTTP logs with richer context. Turn it off afterward to avoid exposing stack traces.

Understanding the different Entities

Entities are based on the HL7 FHIR model.

Patients & Practitioners

  • Any user accessing the Web UI is a Practitioner (data consumer) by default.
  • Patient users (data producers) are registered by Practitioners and sent a link to authenticate and upload data.
  • The same OAuth 2.0 strategy is used for both Practitioners and Patients, the only difference being that the authorization code is provided out-of-band for Patients (ie invitation links) and has a longer expiration date (2 weeks).

Organizations

  • An Organization is a group of Practitioners, Patients and Studies (FHIR Groups).
  • An Organization is typically hierarchical with sub-Organizations eg Institution, Department, Lab etc.
  • A Practitioner belongs to one or more Organization.
  • A Patient belongs to one or more Organization.
  • A Study belongs to one single Organization.

Studies

  • A Study is a Group of Patients and belongs to a single Organization.
  • A Study has one or more Clients, one or more matching Data Sources and one or more Scope Requests.
  • When a Patient is added to a Study, they must explicitly consent to sharing the requested Scopes before any personal data (Observations) can be uploaded or shared.

Observations

  • An Observation is Patient data and belongs to a single Patient.
  • An Observation must reference a Patient ID as the subject and a Data Source ID as the device.
  • Personal device data is expected to be in the Open mHealth (JSON) format however the system can be easily extended to support any binary data attachments or discrete Observation records.
  • Observation data is stored as a valueAttachment in Base 64 encoded JSON binary.
  • Authorization to view Observations depends on the relationship of Organization, Study and Consents as described above.

Data Sources

  • A Data Source is anything that produces Observations (typically a device app eg iHealth).
  • A Data Source supports one or more Scopes (types) of Observations (eg Blood Glucose).
  • An Observation references a Data Source ID in the device field.
  • A Study has one or more associated Data Sources.

Clients

  • Clients are apps that talk to JupyterHealth Exchange.
  • Each Client has its own OAuth 2.0 Client ID.
  • A Client has one or more associated Data Sources.
  • In some cases a single app may be both a Data Source and a Client, in which case a record is created for each and both are added to the Study.
  • A Study has one or more associated Clients.

Use Case Example

  1. Sign up as a new user from the Web UI
  2. Create a new Organization (your user is automatically added to the Organization with a Manager role)
  3. Create a new Study for the Organization (View Organization > Studies+)
  4. Create a new Patient for the Organization using a different email than (1) (Patients > Add Patient)
  5. Add Data Sources and Scopes to the Study (View Study > Data Sources+, Scope Requests+)
  6. Add the Patient to the Study (Patients > check box > Add Patient(s) to Study)
  7. Create an Invitation Link for the Patient (View Patient > Generate Invitation Link)
  8. Use the code in the invitation link with the Auth API to swap it for an access token
  9. Upload Observations using the FHIR API and access token
  10. View the Observations from the web UI

Role Based Access Control

Whether or not a Practitioner user can view particular Patients, Studies and Observations depends on membership of the Organization that the Patients/Studies/Observations belong to.

Whether or not a Practitioner user can edit the Data Sources, Organization, Organization's Patients or Studies depends on the role they are assigned to for that Organization at the time of being added. When a user create a new Organization they are automatically added as a Manager of the Organization. Permissions for roles are outlined in the table below.

Permission Super User Manager Member Viewer
Edit Data Sources ✅
Edit Top Level Organizations ✅
Edit Organization ✅ ✅
Edit Patients ✅ ✅ ✅
Edit Studies ✅ ✅ ✅
View All ✅ ✅ ✅ ✅

Open mHealth Schemas

If Observations are sent with data attached in the Open mHealth format (eg Observation.code=omh:blood-glucose:4.0) JSON Schema validation is used to check the payload. Example values can be found at data/omh/examples/data-points and the JSON schemas can be found at data/omh/json-schemas. See Open mHealth for more information.

Test Data

Database Seed

  • The initial Database is seeded with a minimal set of records to provide an example of the different entity relationships, see the diagram below.
---
config:
  flowchart:
    nodeSpacing: 50
    rankSpacing: 60
---
flowchart TD
    %% node definitions
    sam("SuperUser Sam<br/><small>sam@example.com</small>")
    eri("Organization:<br/>Example Research Institute")
    esds("Organization:<br/>Example School of Data Science")
    eglab("Organization:<br/>Example Lab")
    victor("ViewerVictor<br/><small>victor@example.com</small>")
    megan("MemberMegan<br/><small>megan@example.com</small>")
    mary("ManagerMary<br/><small>mary@example.com</small>")
    tom("ThreeOrgTom<br/><small>tom@example.com</small>")
    exStudyOnBPHR("Example Study on BP & HR<br/><small>Blood Pressure<br/>Heart Rate</small>")
    exStudyOnBP("Example Study on BP<br/><small>Blood Pressure</small>")
    peter("LabPatientPeter<br/><small>peter@example.com</small>")
    pamela("LabPatientPamela<br/><small>pamela@example.com</small>")
    eguni("Organization:<br/>Example University")
    med("Organization:<br/>Example Department")
    cardio("Organization:<br/>Heart Research Division")
    mosl("Organization:<br/>Example Lab Alpha")
    beta("Organization:<br/>Example Lab Beta")
    mark("ManagerMark<br/><small>mark@example.com</small>")
    exStudyOnRR("Example Study on RR<br/><small>Respiratory rate</small>")
    exStudyOnBT("Example Study on BT<br/><small>Body Temperature</small>")
    exStudyOnO2("Example Study on O2<br/><small>Oxygen Saturation</small>")
    percy("AlphaPatientPercy<br/><small>percy@example.com</small>")
    paul("BetaPatientPaul<br/><small>paul@example.com</small>")
    pat("HeartBetaPatientPat<br/><small>pat@example.com</small>")

    %% styles
    style sam fill:#CFC
    style mary fill:#CFC
    style megan fill:#CFC
    style victor fill:#CFC
    style tom fill:#CFC
    style mark fill:#CFC
    style exStudyOnBPHR fill:#CFF
    style exStudyOnBP fill:#CFF
    style exStudyOnRR fill:#CFF
    style exStudyOnBT fill:#CFF
    style exStudyOnO2 fill:#CFF
    style peter fill:#FCC
    style pamela fill:#FCC
    style percy fill:#FCC
    style paul fill:#FCC
    style pat fill:#FCC

    %% example institute org hierarchy
    eri --> esds
    esds --> eglab

    %% example institute user roles
    eri -- Manager --> mary
    esds -- Manager --> mary
    eglab -- Viewer --> victor
    eglab -- Member --> megan
    eglab -- Viewer --> tom
    eglab -- Manager --> mary

    %% example institute studies & patients
    eglab --> peter
    eglab --> pamela
    eglab --> exStudyOnBPHR
    eglab --> exStudyOnBP
    peter -- Consented --> exStudyOnBPHR
    peter -- Requested --> exStudyOnBP
    pamela -- Consented --> exStudyOnBPHR
    pamela -- Consented --> exStudyOnBP

    %% example university org hierarchy
    eguni --> med
    med --> cardio
    cardio --> mosl
    cardio --> beta

    %% example university user roles
    eguni -- Manager --> mark
    med -- Manager --> mark
    cardio -- Manager --> mark
    mosl -- Manager --> mark
    mosl -- Member --> tom
    beta -- Manager --> tom

    %% example university studies & patients
    cardio --> exStudyOnRR
    mosl --> exStudyOnBT
    beta --> exStudyOnO2
    mosl --> percy
    beta --> paul
    cardio --> pat
    beta --> pat
    percy -- Consented --> exStudyOnBT
    paul -- Consented --> exStudyOnO2
    pat -- Consented --> exStudyOnRR
    pat -- Consented --> exStudyOnO2
Loading

Iglu Test Data

  • Additional test data from the iglu project can be seeded by running the following command (please note this can take 10-20 minutes to run) $ python manage.py iglu
  • This creates a new study under the "Example Research Institute" Organization with 19 mock patients and 1745 real Observation data points

Working with APIs

Auth API

  • The OAuth 2.0 Authorization Code grant flow with PKCE is used to issue access and refresh and OIDC for ID tokens for both Practitioners (web login) and Patients (JHE Clients via secret invitation link)

  • Endpoints and configuration details can be discovered from the OIDC metadata endpoint: /o/.well-known/openid-configuration

  • Separate OAuth clients are created for the Web UI (Practitioner) and for individual JHE (Patient) Clients

  • In an effort to utilize the standard libraries (Django OAuth Toolkit) without too much modification, the JHE Client Authorization Code grant flow is divided into three parts:

    1. The first part of the flow used to generate the authorization code is instigated by the JHE server with no interaction from the client
    2. The authorization code is then shared with the client out-of-band (E-mail or SMS) as a secret invitation link
    3. The JHE Client at a later date then swaps the authorization code for an access token

  • Because the Patient authorization code is instigated by the server (Step 1 above) rather than a web browser, the PKCE code challenge and code verifier must be static values and travel with the authorization code. The Patient JHE Client then sends this code_verifier along with the authorization_code to obtain tokens. The redirect_uri serves no purpose (as the initial authorization code has already been issued) but is required per OAuth spec so is defined as a static path (see below).

  • Bringing this altogether, the JHE Client requires 4 values to the invitation code used in the link

    1. host_with_port - The host name (and optionally :port) of the JHE to connect with, eg example.com
    2. client_id - The OAuth 2.0 Client ID
    3. authorization_code - The OAuth 2.0 authorization code
    4. code_verifier - The OAuth 2.0 code verifier
    https://app.tcp.org/invitation/jhe.tcp.org~G7lkfTooTemBHzfya2wpGOZZSIbbtPH6joiVZvCF~psJUkrTx9ZJ2KHyG5Iz9F2NzrKdCL3~7sMHvAWzSEKIj2tSifAIFTruBmfLYriljxVBI5NyrQ

invitation_code = "jhe.tcp.org~G7lkfTooTemBHzfya2wpGOZZSIbbtPH6joiVZvCF~psJUkrTx9ZJ2KHyG5Iz9F2NzrKdCL3~7sMHvAWzSEKIj2tSifAIFTruBmfLYriljxVBI5NyrQ"

host_with_port, client_id, authorization_code, code_verifier = invitation_code.split("~")

post_url = parse(f"https://{host_with_port}/o/.well-known/openid-configuration").token_endpoint
redirect_url = f"https://{host_with_port}/auth/callback" # Doesn't do anything but required by spec

POST https://jhe.tcp.org/o/token/
Content-Type: application/x-www-form-urlencoded
grant_type=authorization_code&redirect_uri=https%3A%2F%2Fjhe.tcp.org%2Fauth%2Fcallback&client_id=G7lkfTooTemBHzfya2wpGOZZSIbbtPH6joiVZvCF&code=psJUkrTx9ZJ2KHyG5Iz9F2NzrKdCL3&code_verifier=7sMHvAWzSEKIj2tSifAIFTruBmfLYriljxVBI5NyrQ

RESPONSE
{
  "access_token":"1CxKTwNOK5vr0gnO6LY0ZfJF70prfy",
  "expires_in": 1209600,
  "token_type": "Bearer",
  "scope": "openid",
  "refresh_token": "DkZRFWWHB9Qndbv1WQn4UVOvKZbOz",
  "id_token": "eyJ0e..."
}

  • The returned access_token should be included in the Authorization header for all subsequent API requests with the prefix Bearer

Note

It is understood using static values for PKCE defeats the intended purpose but because the JHE Client authorization code generation is instigated by the server and shared out of band (rather than a web browser) dynamic PKCE can not be used.

Single Sign-On (SSO) with SAML2

The django-saml2-auth library is included to support SSO with SAML2.

Example SAML2 Flow with mocksaml.com

Modify the .env to match below

# Default: SSO disabled. Change to 1 to enable.
SAML2_ENABLED=1

# Comma-separated list matches email domains permitted to sign in via SSO
SSO_VALID_DOMAINS=example.com,example.org

# MockSAML metadata URL (used to auto-configure IdP endpoints & certificate)
IDENTITY_PROVIDER_METADATA_URL=https://mocksaml.com/api/saml/metadata
Temporarily Switch on Debug for logging

Add the below to ./settings.py

# settings.py (demo only)
DEBUG = True

Warning

Use Debug for testing only, switch off Debug for Production.

When DEBUG is enabled the SPA debug page now summarizes server errors (including HTML tracebacks), auto-scrolls the banner into view, and auto-hides after a few seconds so developers can quickly see the actionable message.

Test Flow

  1. Visit https://mocksaml.com/saml/login and enter the fields below:

    • ACS URL: http://localhost:8000/sso/acs/ (or substitute your hostname) End with a trailing slash

    • Audience: http://localhost:8000/sso/acs/

  2. Enter any email name @example.com

  3. Enter any password

  4. Click Sign in

  5. The JHE portal should be displayed with the user in the matching user name in the bottom left hand corner

Notes
  • The JHE app reads IdP configuration from IDENTITY_PROVIDER_METADATA_URL
  • Only users with email addresses on the SSO_VALID_DOMAINS are permitted

Admin REST API

  • The Admin API is used by the Web UI SPA for Practitioner/Patient/Organization/Study management and Patient data provider apps/clients to manage Patient consents.

Profile

  • The profile endpoint returns the current user details.
// GET /api/v1/users/profile
{
    "id": 10001,
    "email": "peter@example.com",
    "firstName": "Peter",
    "lastName": "ThePatient",
    "patient": {
        "id": 40001,
      	...
    }
}

Patient Consents

  • The consents endpoint returns the studies that are pending and consented for the specified Patient. In this example, the Patient has been invited to Demo Study 2 and has already consented to sharing blood glucose data with Demo Study 1.
// GET /api/v1/patients/40001/consents
{
    "patient": {
        "id": 40001,
				//...
    },
    "consolidatedConsentedScopes": [
        {
            "id": 50002,
            "codingSystem": "https://w3id.org/openmhealth",
            "codingCode": "omh:blood-pressure:4.0",
            "text": "Blood pressure"
        }
    ],
    "studiesPendingConsent": [
        {
            "id": 30002,
            "name": "Demo Study 2",
            "organization": { ... }
            "dataSources": [ ... ],
            "pendingScopeConsents": [
                {
                    "code": {
                        "id": 50002,
                        "codingSystem": "https://w3id.org/openmhealth",
          							"codingCode": "omh:blood-pressure:4.0",
                        "text": "Blood pressure"
                    },
                    "consented": null
                }
            ]
        }
    ],
    "studies": [
        {
            "id": 30001,
            "name": "Demo Study 1",
            "organization": { ... },
            "dataSources": [ ... ],
            "scopeConsents": [
                {
                    "code": {
                        "id": 50001,
                        "codingSystem": "https://w3id.org/openmhealth",
                        "codingCode": "omh:blood-glucose:4.0",
                        "text": "Blood glucose"
                    },
                    "consented": true
                }
            ]
        }
    ]
}
  • To respond to requested consents, a POST is sent to the same consents endpoint with the scope and the consented boolean.
// POST /api/v1/patients/40001/consents
{
  "studyScopeConsents": [
    {
      "studyId": 30002,
      "scopeConsents": [
        {
            "codingSystem": "https://w3id.org/openmhealth",
            "codingCode": "omh:blood-pressure:4.0",
            "consented": true
        }
      ]
    }
  ]
}
  • A PATCH request can be sent with the same payload to update an existing Consent
  • A DELETE request can be sen with the same payload excluding scopeConsents.consented to delete the Consent

FHIR REST API

Patients

  • The FHIR Patient endpoint returns a list of Patients as a FHIR Bundle for a given Study ID passed as query parameter_has:Group:member:_id or alternatively a single Patient matching the query parameter identifier=<system>|<value>
Query Parameter Example Description
_has:Group:member:_id 30001 Filter by Patients that are in the Study with ID 30001
identifier `http://ehr.example.com abc123` 
// GET /fhir/r5/Patient?_has:Group:member:_id=30001
{
    "resourceType": "Bundle",
    "type": "searchset",
    "entry": [
        {
            "resource": {
                "resourceType": "Patient",
                "id": "40001",
                "meta": {
                    "lastUpdated": "2024-10-23T12:35:25.142027+00:00"
                },
                "identifier": [
                    {
                        "value": "fhir-1234",
                        "system": "http://ehr.example.com"
                    }
                ],
                "name": [
                    {
                        "given": [
                            "Peter"
                        ],
                        "family": "ThePatient"
                    }
                ],
                "birthDate": "1980-01-01",
                "telecom": [
                    {
                        "value": "peter@example.com",
                        "system": "email"
                    },
                    {
                        "value": "347-111-1111",
                        "system": "phone"
                    }
                ]
            }
        },
        ...

Observations

  • The FHIR Observation endpoint returns a list of Observations as a FHIR Bundle
  • At least one of Study ID, passed as patient._has:Group:member:_id or Patient ID, passed as patient or Patient Identifier passed as patient.identifier=<system>|<value> query parameters are required
  • subject.reference references a Patient ID
  • device.reference references a Data Source ID
  • valueAttachment is Base 64 Encoded Binary JSON
Query Parameter Example Description
patient._has:Group:member:_id 30001 Filter by Patients that are in the Study with ID 30001
patient 40001 Filter by single Patient with ID 40001
patient.identifier `http://ehr.example.com abc123`
code `https://w3id.org/openmhealth omh:blood-pressure:4.0` 
// GET /fhir/r5/Observation?patient._has:Group:member:_id=30001&patient=40001&code=https://w3id.org/openmhealth|omh:blood-pressure:4.0
{
    "resourceType": "Bundle",
    "type": "searchset",
    "entry": [
        {
            "resource": {
                "resourceType": "Observation",
                "id": "63416",
                "meta": {
                    "lastUpdated": "2024-10-25T21:14:02.871132+00:00"
                },
                "identifier": [
                    {
                        "value": "6e3db887-4a20-3222-9998-2972af6fb091",
                        "system": "https://ehr.example.com"
                    }
                ],
                "status": "final",
                "subject": {
                    "reference": "Patient/40001"
                },
                "device": {
                  "reference": "Device/70001"
                },
                "code": {
                    "coding": [
                        {
                            "code": "omh:blood-pressure:4.0",
                            "system": "https://w3id.org/openmhealth"
                        }
                    ]
                },
                "valueAttachment": {
                    "data": "eyJib2R5IjogeyJlZmZlY3RpdmVfdGltZV9mcmFtZSI6IHsiZGF0ZV90aW1lIjogIjIwMjQtMDUt\nMDJUMDc6MjE6MDAtMDc6MDAifSwgInN5c3RvbGljX2Jsb29kX3ByZXNzdXJlIjogeyJ1bml0Ijog\nIm1tSGciLCAidmFsdWUiOiAxMjJ9LCAiZGlhc3RvbGljX2Jsb29kX3ByZXNzdXJlIjogeyJ1bml0\nIjogIm1tSGciLCAidmFsdWUiOiA3N319LCAiaGVhZGVyIjogeyJ1dWlkIjogIjZlM2RiODg3LTRh\nMjAtMzIyMi05OTk4LTI5NzJhZjZmYjA5MSIsICJtb2RhbGl0eSI6ICJzZW5zZWQiLCAic2NoZW1h\nX2lkIjogeyJuYW1lIjogImJsb29kLXByZXNzdXJlIiwgInZlcnNpb24iOiAiMy4xIiwgIm5hbWVz\ncGFjZSI6ICJvbWgifSwgImNyZWF0aW9uX2RhdGVfdGltZSI6ICIyMDI0LTEwLTI1VDIxOjEzOjMx\nLjQzOFoiLCAiZXh0ZXJuYWxfZGF0YXNoZWV0cyI6IFt7ImRhdGFzaGVldF90eXBlIjogIm1hbnVm\nYWN0dXJlciIsICJkYXRhc2hlZXRfcmVmZXJlbmNlIjogImh0dHBzOi8vaWhlYWx0aGxhYnMuY29t\nL3Byb2R1Y3RzIn1dLCAic291cmNlX2RhdGFfcG9pbnRfaWQiOiAiZTZjMTliMDQyOGM4NWJiYjdj\nMTk4MGNiOTRkZDE3N2YiLCAic291cmNlX2NyZWF0aW9uX2RhdGVfdGltZSI6ICIyMDI0LTA1LTAy\nVDA3OjIxOjAwLTA3OjAwIn19",
                    "contentType": "application/json"
                }
            }
        },
        ...
  • Observations are uploaded as FHIR Batch bundles sent as a POST to the root endpoint
// POST /fhir/r5/
{
  "resourceType": "Bundle",
  "type": "batch",
  "entry": [
    {
      "resource": {
        "resourceType": "Observation",
        "status": "final",
        "code": {
          "coding": [
            {
              "system": "https://w3id.org/openmhealth",
              "code": "omh:blood-pressure:4.0"
            }
          ]
        },
        "subject": {
          "reference": "Patient/40001"
        },
        "device": {
          "reference": "Device/70001"
        },
        "identifier": [
            {
                "value": "6e3db887-4a20-3222-9998-2972af6fb091",
                "system": "https://ehr.example.com"
            }
        ],
        "valueAttachment": {
          "contentType": "application/json",
          "data": "eyJzeXN0b2xpY19ibG9vZF9wcmVzc3VyZSI6eyJ2YWx1ZSI6MTQyLCJ1bml0IjoibW1IZyJ9LCJkaWFzdG9saWNfYmxvb2RfcHJlc3N1cmUiOnsidmFsdWUiOjg5LCJ1bml0IjoibW1IZyJ9LCJlZmZlY3RpdmVfdGltZV9mcmFtZSI6eyJkYXRlX3RpbWUiOiIyMDIxLTAzLTE0VDA5OjI1OjAwLTA3OjAwIn19"
        }
      },
      "request": {
        "method": "POST",
        "url": "Observation"
      }
    },
    ...

Practitioner Script: Upload Observations for Authorized Patients

Prerequisites

  • The database must already be seeded
  • The Django server must be running, i.e., https://jhe.fly.dev or http://localhost:
  • The target Practitioner, Organization, and Study records already exist

Test Procedure

  1. Select Parent Organization Choose Example Research Institute.

  2. Open Sub-Organization Navigate to Example School of Data Science, then click View for Example Lab.

  3. Create a New Study

    • Under Example Lab, create a study.
    • Add the iHealth data source.
    • Set the data scope to blood glucose.

  4. Record the Study ID Open the newly created study and copy its Study ID (e.g. 30006).

  5. Run the Practitioner Upload Script Replace <study_id> with the ID from step 4:

    python resources/practitioner_fhir_obs_upload.py \
  --email mary@example.com \
  --study-id <study_id>

Usage

python resources/practitioner_fhir_obs_upload.py \
  [--email <practitioner_email>] \
  [--password <practitioner_password>] \
  [--org-id <organization_id>] \
  [--study-id <study_id>] \
  [--patient-email <patient_email>]

Arguments & Defaults

Flag Description Default
--email Practitioner login email obs-upload@example.com
--password Practitioner password Jhe1234!
--org-id Target organization ID 20003
--study-id Target study ID (uses iHealth data source + blood-glucose scope) 30006
--patient-email Patient email (lookup / create / enroll) obs-upload-pat1@example.com

If omitted, the defaults will be used. You may override any or all flags.

Examples

With all defaults:

python practitioner_fhir_obs_upload.py

Specifying all arguments:

python practitioner_fhir_obs_upload.py \
  --email practitioner@example.com \
  --password "Sup3r$ecret!" \
  --org-id 42 \
  --study-id 99 \
  --patient-email patient2@example.com

Running in Production

Django Server

The Django development server should not be used for production - more information is available at the official Django Deployment docs. One option included with JHE is the gunicorn server with WhiteNoise for static files, which can be run with the commands below.

$ python manage.py collectstatic --no-input
$ gunicorn --bind :8000 --workers 2 jhe.wsgi

HTTPS

Due to browser security restrictions and the oidc-client-ts used for authentication, the web app must be accessed over HTTPS for any hostname other than localhost. Below is an example of serving the app over HTTPS using an NGINX reverse proxy.

Install nginx and certbot
$ sudo apt update
$ sudo apt install -y nginx certbot

For servers that are publicly reachable

1. Configure nginx
$ DOMAIN=YOUR_DOMAIN
$ sudo mkdir -p /var/www/certbot

$ sudo tee /etc/nginx/sites-available/jhe.conf >/dev/null <<NGINX
upstream jhe_app { server 127.0.0.1:8000; keepalive 16; }

server {
  listen 80;
  server_name ${DOMAIN};

  # ACME HTTP-01
  location ^~ /.well-known/acme-challenge/ {
    root /var/www/certbot;
  }

  # everything else HTTPS
  location / { return 301 https://\$host\$request_uri; }
}

server {
  listen 443 ssl http2;
  server_name ${DOMAIN};

  # temp self-signed while we fetch LE cert (optional if you already have none)
  ssl_certificate     /etc/ssl/certs/ssl-cert-snakeoil.pem;
  ssl_certificate_key /etc/ssl/private/ssl-cert-snakeoil.key;

  client_max_body_size 25m;

  location / {
    proxy_pass http://jhe_app;
    proxy_http_version 1.1;
    proxy_set_header Host               \$host;
    proxy_set_header X-Real-IP          \$remote_addr;
    proxy_set_header X-Forwarded-For    \$proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto  https;
    proxy_read_timeout 300s;
    proxy_send_timeout 300s;
  }
}
NGINX

$ sudo ln -sf /etc/nginx/sites-available/jhe.conf /etc/nginx/sites-enabled/jhe.conf
$ sudo rm -f /etc/nginx/sites-enabled/default
$ sudo nginx -t && (sudo nginx -s reload || sudo nginx)
2. Get the certificate
$ sudo certbot certonly --webroot -w /var/www/certbot -d "$DOMAIN" --agree-tos -m admin@"$DOMAIN" --no-eff-email
3. Update NGINX config to point to the new certificate and reload
$ sudo sed -i "s#ssl-cert-snakeoil.pem#/etc/letsencrypt/live/$DOMAIN/fullchain.pem#g; s#ssl-cert-snakeoil.key#/etc/letsencrypt/live/$DOMAIN/privkey.pem#g" /etc/nginx/sites-available/jhe.conf
sudo nginx -t && sudo nginx -s reload
4. Auto-renew (no systemd needed)
echo '0 3 * * * root certbot renew --quiet --post-hook "nginx -s reload"' | sudo tee /etc/cron.d/certbot_renew >/dev/null

For servers that are not publicly reachable (DNS-01)

1. Get the certificate
$ DOMAIN=YOUR_DOMAIN

# issue with manual DNS challenge
$ sudo certbot certonly --manual --preferred-challenges dns -d "$DOMAIN" --agree-tos -m admin@"$DOMAIN" --no-eff-email
# follow prompts to add the _acme-challenge TXT record, wait for DNS to propagate, then continue
2. Configure NGINX and reload
$ sudo tee /etc/nginx/sites-available/jhe.conf >/dev/null <<NGINX
upstream jhe_app { server 127.0.0.1:8000; keepalive 16; }

server {
  listen 80;
  server_name ${DOMAIN};
  return 301 https://\$host\$request_uri;
}

server {
  listen 443 ssl http2;
  server_name ${DOMAIN};

  ssl_certificate     /etc/letsencrypt/live/${DOMAIN}/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/${DOMAIN}/privkey.pem;

  client_max_body_size 25m;

  location / {
    proxy_pass http://jhe_app;
    proxy_http_version 1.1;
    proxy_set_header Host               \$host;
    proxy_set_header X-Real-IP          \$remote_addr;
    proxy_set_header X-Forwarded-For    \$proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto  https;
    proxy_read_timeout 300s;
    proxy_send_timeout 300s;
  }
}
NGINX

$ sudo ln -sf /etc/nginx/sites-available/jhe.conf /etc/nginx/sites-enabled/jhe.conf
$ sudo rm -f /etc/nginx/sites-enabled/default
$ sudo nginx -t && (sudo nginx -s reload || sudo nginx)
3. Renewals

For hands-off renewals with DNS-01, use a Certbot DNS plugin for your provider (e.g., python3-certbot-dns-cloudflare, python3-certbot-dns-route53) and issue with that plugin once then renewals become automatic. An example for Cloudflare is below.

$ sudo apt install -y python3-certbot-dns-cloudflare
$ sudo tee /root/cf.ini >/dev/null <<EOF
dns_cloudflare_api_token = <TOKEN_WITH_DNS_EDIT>
EOF

$ sudo chmod 600 /root/cf.ini
$ sudo certbot certonly --dns-cloudflare --dns-cloudflare-credentials /root/cf.ini -d "$DOMAIN"

Architecture

Django

Django is a mature and well-supported web framework but was specifically chosen due to resourcing requirements. There are a few accommodations that had to be made for Django to support FHIR as described below.

camelCase

  • FHIR uses camelCase whereas Django uses snake_case.
  • The djangorestframework-camel-case library is used to support camelCase but the conversion happens downstream whereas the schema validation happens upstream, so manually calling humps is also required in parts.

DRF Serializers and Pydantic

  • The Django Rest Framework uses the concept of Serializers to validate schemas, whereas the FHIR validator uses Pydantic.

  • It is not reasonable to re-write the entire validation in the Serializer, so instead a combination of the two are used:

    • Top-level fields (most importantly the id of a record) are managed by the Serializer.
    • Nested fields (for example code{}.coding[].system above) are configured as a JSON field in the Serializer (so the top level field is this example is code) and then Pydantic is used to validate the whole schema including nested JSON.

  • There is a library that may allow Pydantic to be used as a Serializer but this needs to be explored further

JSON Responses

  • Postgres has rich JSON support allowing responses to be built directly from a raw Django SQL queries rather than using another layer of transforming logic.

Single Page App (SPA) Web UI

A hard requirement was to avoid additional servers and frameworks (eg npm, react, etc) for the front end Web UI. Django supports traditional server-side templating but a modern Single Page App is better suited to this use case of interacting with the Admin REST API. For these reasons, a simple Vanilla JS SPA has been developed using handlebars to render client side views from static HTML served using Django templates. The only other additional dependencies are oidc-clinet-ts for auth and bootstrap for styling.

Data Model

erDiagram
  "JheUser (FHIR Person)" {
    int id PK
    string email UK
    boolean email_is_verified
    string identifier
    string user_type
  }

  "Organization (FHIR Organization)" {
    int id PK
    string name
    string type
    int part_of_id FK
  }

  "Practitioner (FHIR Practitioner)" {
    int id PK
    int jhe_user_id FK
    string identifier
    string name_family
    string name_given
    date birth_date
    string telecom_phone
    datetime last_updated
  }

  "Patient (FHIR Patient)" {
    int id PK
    int jhe_user_id FK
    string identifier
    string name_family
    string name_given
    date birth_date
    string telecom_phone
    datetime last_updated
  }

  PractitionerOrganization {
    int id PK
    int practitioner_id FK
    int organization_id FK
    string role
  }

  PatientOrganization {
    int id PK
    int patient_id FK
    int organization_id FK
  }

  "CodeableConcept (FHIR CodeableConcept)" {
    int id PK
    string coding_system
    string coding_code
    string text
  }

  "Study (FHIR Group)" {
    int id PK
    string name
    string description
    int organization_id FK
    string icon_url
  }

  StudyPatient {
    int id PK
    int study_id FK
    int patient_id FK
  }

  StudyPatientScopeConsent {
    int id PK
    int study_patient_id FK
    string scope_actions
    int scope_code_id FK
    boolean consented
    datetime consented_time
  }

  StudyScopeRequest {
    int id PK
    int study_id FK
    string scope_actions
    int scope_code_id FK
  }

  DataSource {
    int id PK
    string name
    string type
  }

  DataSourceSupportedScope {
    int id PK
    int data_source_id FK
    int scope_code_id FK
  }

  StudyDataSource {
    int id PK
    int study_id FK
    int data_source_id FK
  }

  "Observation (FHIR Observation)" {
    int id PK
    int subject_patient_id FK
    int codeable_concept_id FK
    int data_source_id FK
    string value_attachment_data
    datetime last_updated
    string status
  }

  ObservationIdentifier {
    int id PK
    int observation_id FK
    string system
    string value
  }

  "Organization (FHIR Organization)" ||--o{ "Organization (FHIR Organization)" : _
  "JheUser (FHIR Person)" ||--|| "Practitioner (FHIR Practitioner)" : _
  "JheUser (FHIR Person)" ||--|| "Patient (FHIR Patient)" : _

  "Practitioner (FHIR Practitioner)" ||--o{ PractitionerOrganization : _
  "Organization (FHIR Organization)" ||--o{ PractitionerOrganization : _
  "Patient (FHIR Patient)" ||--o{ PatientOrganization : _
  "Organization (FHIR Organization)" ||--o{ PatientOrganization : _

  "Organization (FHIR Organization)" ||--o{ "Study (FHIR Group)" : _
  "Study (FHIR Group)" ||--o{ StudyPatient : _
  "Patient (FHIR Patient)" ||--o{ StudyPatient : _

  StudyPatient ||--o{ StudyPatientScopeConsent : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ StudyPatientScopeConsent : _

  "Study (FHIR Group)" ||--o{ StudyScopeRequest : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ StudyScopeRequest : _

  DataSource ||--o{ DataSourceSupportedScope : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ DataSourceSupportedScope : _

  "Study (FHIR Group)" ||--o{ StudyDataSource : _
  DataSource ||--o{ StudyDataSource : _

  "Patient (FHIR Patient)" ||--o{ "Observation (FHIR Observation)" : _
  "CodeableConcept (FHIR CodeableConcept)" ||--o{ "Observation (FHIR Observation)" : _
  DataSource ||--o{ "Observation (FHIR Observation)" : _

  "Observation (FHIR Observation)" ||--o{ ObservationIdentifier : _
Loading

Deployment

For deployment options and a comprehensive guide take a look at the official Django Deployment docs

Deploying with the published container image

An example Dockerfile is included to deploy the app using gunicorn and WhiteNoise for static files. This image is published to ghcr.io/jupyterhealth/jupyterhealth-exchange.

  1. Create a new empty Postgres database
  2. Copy dot_env_example.txt to .env and update the DB_* parameters from (1) and generate a new value for SECRET_KEY, e.g. with openssl rand -base64 32.
  3. start a container with the image, mounting your .env file, with 
    TAG=sha-abc1234
docker run --rm -it -v$PWD/.env:/code/.env ghcr.io/jupyterhealth/jupyterhealth-exchange:$TAG bash
  4. Migrate the DB by running python manage.py migrate
  5. Seed the database by running the Django management command python manage.py seed_db
  6. exit your setup container and launch a new container running the JupyterHealth Exchange 
    docker run -v$PWD/.env:/code/.env -p8000:8000 ghcr.io/jupyterhealth/jupyterhealth-exchange:$TAG

Development

Git

  1. Before the each commit always make sure to execute pre-commit run --all-files to make sure the PEP8 standards.
  2. Git hook for the pre-commit can also be installed pre-commit install to automate the process.
  3. If a hook fails, fix the issues, stage the changes, and commit again — the commit only succeeds when hooks pass.

Front-end configuration tips

  • The vanilla JavaScript SPA pulls runtime settings via core/templates/client/client_settings.js, which exposes a global CONSTANTS object.
  • CONSTANTS is populated by the Django context processor core/context_processors.py, so any new value you expose there becomes available both to the template and to core/static/js/client.js.
  • Keep secrets and URLs on the Django side (settings or database-backed values) and let the context processor synthesize them, to prevent duplication and to keep the SPA reusable across environments.

About

Collect and Share Health Data over FHIR

jupyterhealth.org

Topics

django fhir identity-provider fhir-server clinical-data-management medical-research

Resources

Readme

License

MIT license

Contributing

Contributing
Activity
Custom properties

Stars

9 stars

Watchers

4 watching

Forks

11 forks
Report repository

Releases

9 tags

Packages

 
 
 

Contributors

Languages