Release Notes

All notable changes to this project will be documented in this file.

This project adheres to Semantic Versioning.

0.33.0

“Stable” API changes

(Won’t truly be stable until after a 1.0.0 release)

Installation changes

  • To install pymedphys with all of its user dependencies now the following needs to be run:
pip install pymedphys[user]
  • When pip install pymedphys is called, PyMedPhys will now be installed with minimal/no dependencies.

Beta API changes

Nil

Experimental API changes

Breaking changes

  • Removed a range of unmaintained experimental package APIs; film, collimation, sinogram, and Profile. The underlying code has not been removed, but they are no longer exposed through the APIs.

Bug fixes

  • Made it so that import pymedphys.experimental does not raise an ImportError when an optional dependency has not been installed.
  • Fixed a bug where pseudonymisation wouldn’t work when in cases of identifying sequences.

New Features

Internal Changes

0.32.0

“Stable” API changes

Bug fixes

  • Fixed bug in the PyMedPhys trf decoding logic where leaf pairs 77, 78, 79, and 80 on the Y2 bank were decoded into having the wrong sign.
    • See issue #968 and pull request #970 for more details.

Breaking changes

  • config.toml has undergone a few breaking changes.
    • See the example for a working config file.
    • See below for a comparison highlighting the key differences.
# Previous version
[[site]]
name = "rccc"
escan_directory = '\\pdc\Shared\Scanned Documents\RT\PhysChecks\Logfile PDFs'

    [[site.linac]]
    name = "2619"
    icom_live_directory = '\\rccc-physicssvr\iComLogFiles\live\192.168.100.200'


# New version
[[site]]
name = "rccc"

    [site.export-directories]
    escan = '\\pdc\Shared\Scanned Documents\RT\PhysChecks\Logfile PDFs'
    anonymised_monaco = 'S:\DataExchange\anonymised-monaco'
    icom_live = '\\rccc-physicssvr\iComLogFiles\live'

    [[site.linac]]
    name = "2619"
    ip = '192.168.100.200'

New Features

  • Two new optional keywords were added to pymedphys.dicom.anonymise. These are replacement_strategy and identifying_keywords. This was designed to support alternative anonymisation methods. The API to the anonymise function is being flagged for a rework and simplification for which a breaking change is likely to occur in the near future.
  • Added ability to configure logging via config.toml.

Data file changes

Refers to the data files accessible via pymedphys.data_path, pymedphys.zip_data_paths, and pymedphys.zenodo_data_paths.

  • The data file pinnacle_test_data_no_image.zip was removed and its contents were moved into pinnacle_test_data.zip.
  • Data files treatment-record-anonymisation.zip, negative-mu-density.trf, and trf-references-and-baselines.zip were added.

Beta API changes

Nil

Experimental API changes

New Features

  • Added pseudonymisation as an experimental extension of anonymise.
    • This API is undergoing refinement, however in its current form it is accessible via pymedphys.experimental.pseudonymisation.pseudonymisation_dispatch and pymedphys.experimental.pseudonymisation.get_default_pseudonymisation_keywords. These are designed to be passed to the new keywords replacement_strategy and identifying_keywords within pymedphys.dicom.anonymise.
    • The pseudonymisation strategy uses SHA3_256 hashing for text and UIDs, date shifting for dates, and jittering for Age. The intent is to enable sets of data that are correlated to remain correlated, and to prevent uncorrelated patient/study/series from clashing.
  • Added experimental pseudonymisation CLI. Callable via pymedphys experimental dicom anonymise --pseudo path/to/dicom.dcm
  • Added pymedphys experimental gui. This is a testing ground for new GUIs that are intended to appear within pymedphys gui in the future. The GUIs exposed under this experimental scope are minimally tested.
    • At this point in time, the new GUIs include a GUI index, an electron insert factor prediction tool, and a Monaco anonymisation tool.

Bug Fixes

  • Pinnacle Export Tool now allows for the trial to be set using the CLI. See issue #973 and pull request #995 for more details.
  • Fixed bug where the dose grid in the Pinnacle Export Tool was only correct when patients were in HFS. See #929 for more details.

0.31.0

“Stable” API changes

Critical bug fixes

  • Fixed bug where pymedphys dicom anonymise and pymedphys.dicom.anonymise would not anonymise nested tags. Thanks sjswerdloff for finding and fixing #920.

Breaking changes

  • Removed the --publish option from CLI pymedphys dev docs.
  • Moved pymedphys logfile orchestration to pymedphys trf orchestrate

New features

  • pymedphys.zenodo_data_paths has a new optional parameter filenames that can be used to only download some files.
  • pymedphys.data_path has a new optional parameter hash_filepath which can be used to provide a custom hash record.
  • Added usage warning to the MU Density GUI.

Deprecations

  • pymedphys.read_trf has been replaced with pymedphys.trf.read. The old API is still available, but will be removed in a future version.

Bug fixes

  • Cache data downloads now also retry when a ConnectionResetError occurs.

Beta API changes

New features

  • A new pymedphys.beta module created. This is intended to allow a section of the API to be exposed and iterated on but having breaking changes not induce a major version bump (when PyMedPhys goes to v1.0.0+)
  • Added pymedphys.beta.trf.identify to allow the usage of Mosaiq to identify a trf logfile.

Experimental API changes

Breaking changes

  • Instances of labs has been changed to experimental. This affects all imports from the labs and the CLI usage.

Bug fixes

  • Fixed issue with Pinnacle Export Tool crashing when an image is missing from the archive.

0.30.0

Breaking changes

  • Removed the proof of concept pymedphys bundle CLI as well all of its associated code.
  • Removed a range of unused files from the pymedphys.data_path API.
  • The previous install options pip install pymedphys[pytest] and pip install pymedphys[pylint] have been removed and replaced with pip install pymedphys[tests].

New Features

  • Added a new toolbox for retrieving PTW Quickcheck measurement data and write it to a csv file. pymedphys labs quickcheck to-csv your.quickcheck.ip path/to/output.csv
  • Added pymedphys dev tests to the CLI.
    • Moved all of tests into the pymedphys repo itself. Now the automated testing suite is able to be run from a pypi install.
    • This CLI has options such as --run-only-pydicom, --run-only-slow, and --run-only-pylinac so that upstream tools can run tests on this downstream project.
    • These extra options are directly passed through to pytest. To achieve this, made the pymedphys CLI be able to optionally handle arbitrary commands.
  • Made the Zenodo download tool retry up to four times should the download fail.
  • Added DICOM helpers functionality and updated the Mosaiq helpers as a part of the UTHSCSA TPS/OIS comparison project. Not yet exposed as part of the API. See _mosaiq/helpers.py#L353-L482
  • Added more debugging strings to the iCOM CLI. See these outputs by running pymedphys --debug icom listen external.nss.ip.address your/output/directory
    • These were added to support remotely debugging the iCOM listen software. To see the conversation around debugging that tool see the PyMedPhys forum discussion
  • Format of MU in logging display now rounded to one decimal.

Dependency changes

  • Now depending on pylibjpeg-libjpeg in order to decode lossless-jpeg files.
  • m2r is no longer used to build the docs.
  • No longer using tox for tests.

Bug fixes

  • Fixed an issue where the iCOM listener could not handle Machine IDs that were not entirely an integer.
  • Fixed a case where on some Windows environments pymedphys dev docs would not run.
  • Fixed a case where on some Windows environments pymedphys gui would not run.
  • Fixed issue where the pymedphys logfile orchestration CLI would not be able to create an index.json, or a range of the needed directories on its first run.

Documentation updates

  • Fixed an issue where the displayed CSV files for configuring pymedphys logfile orchestration would actually cause an error due to excess spaces used for display purposes.

Development changes

  • Removed any file that was larger than 300 kB from the git history bring down clone times to a manageable state.
    • The pre-commit tool now does not allow commits greater than 300 kB.
    • All testing files that were larger than 300 kB have been moved to Zenodo.
  • All tests have been moved from /tests into /pymedphys/tests, running these tests can now be undergone by calling pymedphys dev tests
    • No longer using tox.

0.29.1

Bug fixes

  • Fix issue in some Windows environments where running pymedphys gui would not find the streamlit installation. _gui/init.py

0.29.0

Breaking changes

  • Changed the patient_directories icom parameter to accept a list of paths instead of a single path within the pymedphys config.toml. config.toml#L67-L72
  • Changed pymedphys gui iCOM path resolution logic to instead search over a list of paths instead of just one path as before. mudensity-compare.py#L668-L670

0.28.0

Overview

This release primarily focused on changes regarding the iCOM listener and the PyMedPhys GUI that utilises these iCOM records.

Breaking changes

  • Removed the pymedphys icom archive CLI command, this archiving is now built directly into the listener itself.

New Features

  • The pymedphys icom listener CLI command now will collect the icom stream into beam delivery batches and index them by patient name. This functionality used to be undergone within the pymedphys icom archive CLI, but this functionality has now been merged into the listener. listener.py#L79
  • Should an error occur within pymedphys icom listener CLI it will now pause for 15 minutes and then reattempt a connection.
  • Add in extra sanity checks within the iCOM patient indexing tooling.
  • Added a --debug and --verbose flag to the PyMedPhys CLI which allows users to set the logging level. These logging levels are currently only utilised within the pymedphys icom listen CLI. cli/main.py#L51-L70

Bug fixes

  • Reduced the buffer size of the iCOM listener. listener.py#L9
  • If either the listener is turned off and then on again, or it is interrupted the next time an iCOM stream socket is opened the Linac appears to send a larger batch containing prior irradiations. The listener code was adjusted to handle these extra bursts. listener.py#L57-L83
  • Made PyMedPhys GUI skip name formatting attempt if the original patient name format was not as expected. mudensity-compare.py#L733-L738

0.27.0

New Features

  • Added an optional --structures flag to pymedphys dicom merge-contours. This allows you to only compute the merge for those structures named.

0.26.0

New Features

0.25.1

Dependency Changes

  • Now included psutil as an optional dependency.

Quality of life improvements

0.25.0

New Features

  • Created the command line tool pymedphys gui which boots the GUI for PyMedPhys within your browser. GUI at this stage is quite minimal.
  • Created a tool to handle a PyMedPhys config file, by default stored within ~/.pymedphys/.config.toml. That config file can have a redirect field to allow configuration to be stored in a different location such as within a git repo, or a network drive.
  • pymedphys.zip_data_paths now has a new optional parameter extract_directory. When this parameter is passed the contents of the zip downloaded zip data will be extracted to the provided directory. For example now the following is possible:
import pathlib

import pymedphys

CWD = pathlib.Path.cwd()
pymedphys.zip_data_paths("mu-density-gui-e2e-data.zip", extract_directory=CWD)
  • pymedphys.Delivery.from_dicom() now supports step and shoot and 3DCRT DICOM plan files.
  • Work on pymedphys.Delivery.from_monaco() was undergone with an attempt to support step and shoot plans. This work was preliminary.
  • Created a utility to pretty print patient names
  • Added ground work for e2e testing of pymedphys gui with the cypress tool.

0.24.3

Bug Fixes

  • Within the bundle created by pymedphys bundle fixed a bug where the streamlit server will not start due stdout not flushing.

0.24.2

Bug Fixes

  • Within the bundle created by pymedphys bundle fixed a bug where sometimes the streamlit server would not start should a stdout race condition occur.

0.24.1

Bug Fixes

  • Include matplotlib within streamlit bundle. Streamlit requires this but has not labeled it as a dependency.
  • Call yarn from os.system, for some reason on Windows subprocess.check_call could not find yarn on the path, although on Linux this worked fine.

0.24.0

Breaking Changes

  • If pymedphys.mosaiq.connect is passed a list of length one, it will now return a cursor within a list of length 1 instead of just returning a cursor by itself.

New Features

  • Added a pymedphys bundle cli function which creates an electron streamlit installation bundle.
  • Added the ‘all’ fractions option to Delivery.from_dicom which can be used as pymedphys.Delivery.from_dicom(dicom_file, fraction_number='all')
  • Made the iCOM patient archiving only save the data if MU was delivered.
  • Added wlutz mock image generation functions
  • Handle more Monaco tel.1 cases within Delivery.from_monaco
  • get_patient_name added to pymedphys._mosaiq.helpers

Algorithm Adjustments

  • Wlutz bb finding cost function adjusted
    • Note, wlutz algorithm still not ready for the prime time

0.23.0

Breaking Changes

  • Removed jupyter, bundle, and app sub commands from the CLI.
  • Removed the gui and jupyter optional extra installation commands.
  • In order to support Python 3.8, the pymssql dependency needed to be removed for that Python version. All tools that make SQL calls to Mosaiq will not currently work on Python 3.8.

New Features

  • PyMedPhys now is able to be installed on Python 3.8.

Dependency Changes

  • No longer depend upon pymssql for Python 3.8.

Bug Fixes

0.22.0

New Features

0.21.0

Dependency Changes

  • Once again made shapely a default dependency with the aim to make installation be “batteries included”.
  • Pinned pydicom due to a currently unknown issue with a new version breaking a pymedphys test.

0.20.0

New Features

  • Expose some portions of the Winston Lutz API.
  • Add iCom listener CLI.

0.19.0

Breaking Changes

  • Made shapely an optional dependency once more. No longer depending on shapely-helper.
    • Shapely can be installed by running pip install pymedphys[difficult]==0.19.0
    • This fixes an issue where pip refuses to install due to the shapely-helper workaround.

0.18.0

Breaking Changes

  • Removed the optional extras tags of library, labs, and difficult. All of these now install by default. For example PyMedPhys can no longer be installed with pip install pymedphys[library].

Quality of life improvements

  • Installation of PyMedPhys has been reverted to including all of its primary dependencies. This was done to make the default install less confusing. Nevertheless, these dependencies are mostly optional and if you wish you can install with pip install pymedphys --no-deps to have a minimal installation.
  • Made a shapely-helpers package which automatically handles installation of shapely on Windows. PyMedPhys now depends on shapely-helpers instead of shapely.

0.17.1

Quality of life improvements

  • Made wlutz determination less fussy.

0.17.0

New Features

  • Initial alpha release of an experimental JupyterLab application bundler. Run with pymedphys bundle in a directory that contains a notebooks dir and a requirements.txt file.

0.16.3

Bug Fixes

  • Gracefully reject ipython inspection for optional modules by returning None for ‘file’ attribute requests for modules that are not currently installed.

0.16.2

Bug Fixes

  • Fixed bug with optional dependency logic within apipkg. Occurred whenever an optional submodule was called, for example scipy.interpolate.

0.16.1

Aesthetic Changes

  • Updated the badges reported within the README.

0.16.0

Package changes

  • The license of the package has changed from AGPL-3.0-or-later to Apache-2.0.

New Features

  • Expose pymedphys.electronfactors.plot_model as part of the public API.

0.15.0

New Features

  • Experimental support for Elekta Unity trf log file decoding.

0.14.3

Package changes

  • Updated wheel to correctly handle optional dependencies.

0.14.2

Bug Fixes

  • Vendored in apipkg due to PyPI installation issues.

0.14.1

Bug Fixes

0.14.0

Breaking Changes

  • Moved pymedphys pinnacle cli command to be nested under pymedphys labs pinnacle

Dependency Changes

  • Made the greater majority of the pymedphys dependencies optional. Should a dependency be required during usage an error is raised informing the user to install the package. To install all pymedphys dependencies as before now run pip install pymedphys[library,labs]==0.14.0.

0.13.2

Bug Fix

  • Fixed issue where pymedphys.mosaiq.connect would not work for just one hostname.

0.13.1

Bug Fix

  • Fixed issue where pymedphys.mosaiq.connect would not work for just one hostname.

0.13.0

New Feature

  • Made pymedphys.mosaiq.execute a part of the API.

0.12.2

Package changes

  • Fixed version number within package.

0.12.1

Package changes

  • Re-added the license classifier to the PyPI upload.

0.12.0

Breaking Changes

  • The API has undergone a complete redesign. Expect most code to be broken with this release.

0.11.0

Breaking Changes

  • Within dose_from_dataset the reshape parameter has been removed.
  • Removed the following functions:
    • load_dicom_data
    • axes_and_dose_from_dicom
    • extract_depth_dose
    • extract_profiles

New Features

  • Added functions pymedphys.dicom.depth_dose and pymedphys.dicom.profiles.
  • Exposed the trf2pandas function via pymedphys.fileformats.trf2pandas.

Improvements

  • Made the resolution detection of pymedphys.plt.pcolormesh_grid more robust.

0.10.0

New Features

  • Re-exposed convert2_ratio_perim_area and create_transformed_mesh from pymedphys.electronfactors.
  • Pinnacle module providing a tool to export raw Pinnacle data to DICOM objects.

0.9.0 – 2019/06/06

New Features

  • Re-exposed multi_mosaiq_connect, multi_fetch_and_verify_mosaiq, get_qcls_by_date, and get_staff_name from pymedphys.msq.

0.8.4 – 2019/06/04

Package changes

  • Made xlwings not install by default if system is Linux within setup.py
  • Removed unreleased jupyter based GUI

0.8.3 – 2019/06/04

Package changes

  • Updated MANIFEST file within pymedphys_fileformats to appropriately include LICENSE files.

0.8.2 – 2019/06/01

Package changes

  • Included license files within the subpackage distributions

0.8.1 – 2019/06/01

Dependency changes

  • Removed numpy version upper-limit

0.8.0 – 2019/06/01

Breaking Changes

  • DeliveryData has been renamed to Delivery and is now importable by running from pymedphys import Delivery
    • A range of functions that used to use DeliveryData are now instead accessible as methods on the Delivery object.
  • A large number of functions that were previously exposed have now been made private in preparation for eventually stabilising the API. No function that was within the documentation has been removed. If there is a function that you were using that you would like to be exposed via import again, please let us know by opening an issue on GitHub and we will happily re-expose it! However, please bear in mind that the entire API that is currently exposed will likely change before a 1.0.0 release.
  • anonymise_dicom_dataset() has been renamed to anonymise_dataset() to remove redundant labelling.
  • mu_density_from_delivery_data moved from the msq module to the mudensity module.
  • compare_mosaiq_fields moved from the msq module into the plancompare module.
  • pymedphys.dicom.get_structure_aligned_cube has had its x0 parameter changed from required to optional. It is no longer the first parameter passed to the function. By default x0 is now determined using the min/max bounds of the structure.
  • The DICOM coordinate extraction functions - extract_dicom_patient_xyz(), extract_iec_patient_xyz() and extract_iec_fixed_xyz() - have been combined into a single function called xyz_from_dataset(). The x, y, z axes can still be returned in either the DICOM, IEC fixed or IEC patient coordinate systems by passing the following case-insensitive strings to the coord_system= parameter of xyz_from_dataset():
    • DICOM: 'd' or 'DICOM'
    • IEC fixed: 'f', 'fixed' or 'IEC fixed'
    • IEC patient: 'p', 'patient' or 'IEC patient'
  • gamma_dicom now take datasets as opposed to filenames

New Features

  • A DICOM anonymisation CLI! See the DICOM Files CLI docs.
  • anonymise_file() and anonymise_directory():
    • two new DICOM anonymisation wrapper functions that take a DICOM file and a directory as respective arguments.
  • is_anonymised_dataset(), is_anonymised_file() and is_anonymised_directory():
    • three new functions that check whether a pydicom dataset, a DICOM file or all files within a directory have been anonymised, respectively.
  • coords_from_xyz_axes() is a previously internal function that has now been exposed in the API. It converts x, y, z axes returned by xyz_from_dataset() into a full grid of coordinate triplets that correspond to the original grid (pixel array or dose grid).

0.7.2 – 2019/04/05

Dependency changes

  • Removed numpy version upper-limit

0.7.1 – 2019/04/05

Performance Improvements

  • reduced PyPI package size by removing unnecessary development testing files.

0.7.0 – 2019/04/05

Breaking Changes

  • anonymise_dicom has been renamed to anonymise_dicom_dataset
  • The CLI interface trf2csv has been replaced with pymedphys trf to-csv. This has the same usage, just a changed name to come in line with the rest of the CLI interfaces exposed by PyMedPhys.

New Features

  • Implementing a suite of Dicom objects, currently a work in progress:
    • DicomBase, a base DICOM class that wraps pydicom’s Dataset object. This class includes additions such as an anonymisation method.
    • DicomImage, designed to hold a single DICOM image slice. Might someday contain methods such as resample and the like.
    • DicomSeries, a series of DicomImage objects creating a CT dataset.
    • DicomStructure, designed to house DICOM structure datasets.
    • DicomPlan, a class that holds RT plan DICOM datasets.
    • DicomDose, a class that to hold RT DICOM dose datasets. It has helper functions and parameters such as coordinate transforms built into it.
    • DicomStudy, a class designed to hold an interrelated set of DicomDose, DicomPlan, DicomStructure, and DicomSeries. Not every type is required to create a DicomStudy. Certain methods will be available on DicomStudy depending what is housed within it. For example having both DicomDose and DicomStructure should enable DVH based methods.
    • DicomCollection, a class that can hold multiple studies, interrelated or not. A common use case that will likely be implemented is DicomCollection.from_directory(directory_path) which would pull all DICOM files nested within a directory and sort them into DicomStudy objects based on their header UIDs.
  • Added CLI commands for a WIP docker server, logfile orchestration, and DICOM editor tools.
  • Added a range of xlwings tools that allow the use of PyMedPhys functions within Excel
  • Added rudimentary code to pull profiles from Mephysto files.
  • The previously separate decodetrf library is now distributed within PyMedPhys. You can now simply install PyMedPhys and run pymedphys trf to-csv within the command line to convert .trf files into .csv files.

0.6.0 – 2019/03/15

Breaking Changes

  • All uses of “dcm” in directory names, module names, function names, etc. have been converted to “dicom”. Anything that makes use of this code will need to be adjusted accordingly. Required changes include:
    • pymedphys.dcm –> pymedphys.dicom
    • coords_and_dose_from_dcm() –> coords_and_dose_from_dicom()
    • dcmfromdict() –> dicom_dataset_from_dict()
    • gamma_dcm() –> gamma_dicom()
  • MU Density related functions are no longer available under the pymedphys.coll package, instead they are found within pymedphys.mudensity package.
  • The DICOM coordinate extraction functions now return simple tuples rather than Coords namedtuples:
    • extract_dicom_patient_xyz()
    • extract_iec_patient_xyz()
    • extract_iec_fixed_xyz()

New Features

  • DICOM anonymisation now permits replacing deidentified values with suitable “dummy” values. This helps to maintain compatibility with DICOM software that includes checks (beyond those specified in the DICOM Standard) of valid DICOM tag values. Replacing tags with dummy values upon anonymisation is now the default behaviour.
  • A set of 3D coordinate transformation functions, including rotations (passive or active) and translations. Transformations may be applied to a single coordinate triplet (an ndarray) or a list of arbitrarily many coordinate triplets (a 3 x n ndarray). NB: Documentation forthcoming.

Code Refactoring

  • All uses of dcm as a variable name for instances of PyDicom Datasets have been converted to ds to match PyDicom convention.

0.5.1 – 2019/01/05

New Features

  • Began keeping record of changes in changelog.md