Quick start with TemplateFlow Client API¶

This notebook showcases elementary functionality of the TemplateFlow API. We begin by importing the TemplateFlow Python client.

[1]:

import templateflow
import templateflow.api as tflow

print(f'Using TemplateFlow Client version {templateflow.__version__}')

Using TemplateFlow Client version 0.7.1

Finding templates¶

At the time of writing, there are 15 templates available in the TemplateFlow Archive. We can use the templates() function to obtain the unique resource identifiers for all currently available templates.

[2]:

templates = tflow.templates()

# Print template identifiers
ident_fmt = '\n'.join(templates)
print(f"{len(templates)} templates:\n-------------------\n{ident_fmt}")

16 templates:
-------------------
Fischer344
MNI152Lin
MNI152NLin2009cAsym
MNI152NLin2009cSym
MNI152NLin6Asym
MNI152NLin6Sym
MNIInfant
MNIPediatricAsym
NKI
OASIS30ANTs
PNC
RESILIENT
UNCInfant
WHS
fsLR
fsaverage

Accessing metadata¶

We can query metadata associated with individual data files (e.g., a volume or a surface) or general metadata of the template. For example, the method get_metadata(<template_id>) returns the general metadata as a dictionary:

[3]:

# Get the full, general metadata dictionary for the Fischer344 rodent template
tflow.get_metadata('Fischer344')

[3]:

{'Authors': ['Goerzen, D',
  'Fowler, C',
  'Devenyi, GA',
  'Germann, J',
  'Madularu, D',
  'Chakravarty, MM',
  'Near, J'],
 'BIDSVersion': '1.1.0',
 'Curators': ['MacNicol E', 'Esteban O'],
 'Identifier': 'Fischer344',
 'License': 'CC-by-nc-sa-4.0 International',
 'Name': 'MRI-Derived Neuroanatomical Atlas of the Fischer 344 Rat Brain',
 'ReferencesAndLinks': ['https://doi.org/10.5281/zenodo.3555555',
  'https://doi.org/10.1038/s41598-020-63965-x'],
 'TemplateFlowVersion': '1.0.0'}

Accordingly, to obtain the usage license for any template, we can select the relevant field from the template’s metadata dictionary.

[4]:

# Get the usage license for the OASIS30ANTs template
tflow.get_metadata('OASIS30ANTs')['License']

[4]:

'CC BY 4.0'

Similarly, we can compile desired metadata from all templates. Below, we retrieve for each template its full name, the list of contributing authors, and the usage license.

[5]:

import pandas as pd

metadata_fields = ['Name', 'Authors', 'License']
pd.DataFrame({col: [tflow.get_metadata(tpl)[col] for tpl in templates]
              for col in metadata_fields}, index=templates)

[5]:

	Name	Authors	License
Fischer344	MRI-Derived Neuroanatomical Atlas of the Fisch...	[Goerzen, D, Fowler, C, Devenyi, GA, Germann, ...	CC-by-nc-sa-4.0 International
MNI152Lin	Linear ICBM Average Brain (ICBM152) Stereotaxi...	[Evans AC, Fox PT, Lancaster J, Zilles K, Wood...	See LICENSE file
MNI152NLin2009cAsym	ICBM 152 Nonlinear Asymmetrical template versi...	[Fonov V, Evans AC, Botteron K, Almli CR, McKi...	See LICENSE file
MNI152NLin2009cSym	ICBM 152 Nonlinear Symmetrical template versio...	[Fonov V, Evans AC, Botteron K, Almli CR, McKi...	See LICENSE file
MNI152NLin6Asym	FSL's MNI ICBM 152 non-linear 6th Generation A...	[Janke AL]	See LICENSE file
MNI152NLin6Sym	ICBM 152 non-linear 6th Generation Symmetric A...	[Grabner G, Janke AL, Budge MM, Smith D, Prues...	See LICENSE file
MNIInfant	MNI Unbiased nonlinear Infant Atlases 0-4.5yr.	[Fonov V, Evans AC, Botteron K, Almli CR, McKi...	See LICENSE file
MNIPediatricAsym	MNI's unbiased standard MRI template for pedia...	[Fonov V, Evans AC, Botteron K, Almli CR, McKi...	MIT-derived. See LICENSE file
NKI	ANTs' NKI-1 template	[Avants BB, Tustison NJ, Song G, Cook PA, Klei...	CC BY 4.0
OASIS30ANTs	OASIS30 - Generated with ANTs for MICCAI 2016	[Avants BB, Tustison NJ, Song G, Cook PA, Klei...	CC BY 4.0
PNC	Philadelphia Neurodevelopmental Cohort template	[Satterthwaite TD, Connolly JJ, Ruparel K, Cal...	CC BY 4.0
RESILIENT	Age-specific adult rat brain MRI templates and...	[MacNicol, E, Wright, P, Kim, E, Brusini, I, E...	CC-BY-4.0 International
UNCInfant	UNC Infant 0-1-2 Atlases	[Feng Shi, Pew-Thian Yap, Guorong Wu, Hongjun ...	CC-BY
WHS	Waxholm Space atlas of the Sprague Dawley rat ...	[Papp EA, Leergaard TB, Calabrese E, Johnson G...	CC-by-nc-sa-4.0 International
fsLR	HCP Pipelines' fsLR template	[Tim Coalson, Matt Glasser, Mike Harms, David ...	See LICENSE file
fsaverage	FreeSurfer's fsaverage template	[]	See LICENSE file

The get_citations method allows for easy retrieval and formatting of references. By default, the method gives DOIs for each citation, but it can also be configured to return citations in BibTeX format. (This additionally requires the doi2bib package.)

[6]:

print(tflow.get_citations('UNCInfant', bibtex=True)[0])

@article{2011,
        doi = {10.1371/journal.pone.0018746},
        url = {https://doi.org/10.1371%2uncinfant1.pone.0018746},
        year = 2011,
        month = {apr},
        publisher = {Public Library of Science ({PLoS})},
        volume = {6},
        number = {4},
        pages = {e18746},
        author = {Feng Shi and Pew-Thian Yap and Guorong Wu and Hongjun Jia and John H. Gilmore and Weili Lin and Dinggang Shen},
        editor = {Hitoshi Okazawa},
        title = {Infant Brain Atlases from Neonates to 1- and 2-Year-Olds}
}

Accessing data¶

To retrieve the template resources themselves, we use the get method. This method accepts as arguments the unique template identifier together with any BIDS-like entities: key-value pairs specifying the resource to retrieve. It returns a path to the requested resource, which can (for example) be used with nibabel to read the resource into memory. The table below details valid entities for the API.

Data entity	API query example	Description
Template	`"MNI152Lin"`	The template dataset to which an image or other data file belongs.
Resolution	`resolution=1`	The image resolution. Each resolution is assigned a key, which is defined in the `res` field of `template_description.json`.
Mask	`desc="brain", suffix="mask"`	Indicates that the image is a binary-valued annotation, where voxels labelled 1 are part of the mask.
Discrete segmentation	`desc="malf", suffix="dseg"`	Indicates that the image is an integer-valued annotation. Each segmentation image file (`.nii.gz` format) is paired with a dictionary of segment names (`.tsv` format).
Probabilistic segmentation	`label="CSF", suffix="probseg"`	Indicates that the image is a probabilistic annotation, wherein the value of each voxel indicates the probability of that voxel belonging to the specified label.
Atlas	`atlas="Schaefer", desc="7Network"`	The atlas to which a segmentation file belongs.
Transformation	`from="MNI152Lin", suffix="xfm"`	File containing a mapping between 2 stereotaxic coordinate spaces. The source space is defined in the `from` field, while the target space is defined in the `tpl` field.
Image modality	`suffix="T1w"`	For non-annotation brain images, the suffix indicates whether the image is T1-weighted (`T1w`), T2-weighted (`T2w`), proton density-weighted (`PD`), or T2*-weighted (`T2star`).
Template cohort	`cohort=1`	Subsample of a dataset used to generate an average template.

TemplateFlow uses a lazy loading scheme for resource retrieval: at installation, the local copy of the archive tree contains only pointers to cloud-based resources. As data resources are requested (for instance using the get method), they are synced from the cloud to the local file system. If the data have already been downloaded, they can be retrieved from the local system cache without repeating the download.

As an example, we can use the get method to retrieve a path to the 100-node Schaefer atlas.

[7]:

print(tflow.get(
    'MNI152NLin2009cAsym',
    resolution=1,
    atlas='Schaefer2018',
    desc='100Parcels7Networks',
))

/Users/rastko/.cache/templateflow/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_atlas-Schaefer2018_desc-100Parcels7Networks_dseg.nii.gz

…or the T1-weighted MNI152NLin6Asym template.

[8]:

print(tflow.get(
    "MNI152NLin6Asym",
    desc=None,
    resolution=1,
    suffix="T1w"
))

/Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_T1w.nii.gz

By omitting arguments, we retrieve any data that match the remaining (specified) arguments.

[9]:

tflow.get("MNI152NLin6Asym", resolution=1, suffix='probseg')

[9]:

[PosixPath('/Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_atlas-HOCPA_probseg.nii.gz'),
 PosixPath('/Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_atlas-HOCPAL_probseg.nii.gz'),
 PosixPath('/Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_atlas-HOSPA_probseg.nii.gz')]

Integrating TemplateFlow into processing workflows¶

Motivation¶

TemplateFlow maximizes the accessibility and reuse potential of templates and atlases. For example, let’s reuse the base configuration file for FSL FEAT we proposed in our paper (Esteban et al., 2019). The design file design.fsf specifies a simple preprocessing workflow with FSL tools. The simplified code listing below shows that, just to make non-default templates available to FSL using the graphical user interface (GUI), at least five steps are necessary:

# 1. User determines two nondefault templates they want to spatially normalize into
# 2. User manually downloads templates, extracts the required files from packages
$ curl -sSL <url> | tar zxv --no-same-owner -C /data/templates/
# 3. User opens FSL's GUI, edits the target template box content pointing to the appropriate files
# 4. User generates FSL configuration files to permit batch execution on the command line
# 5. For the default and the two nondefault templates, execute FSL's feat:
$ feat design_<template>.fsf

The outputs of each feat design_<template>.fsf call will follow the pre-specified patterns of FSL, with whatever customization the user has introduced into the design file. The user, therefore, must then adapt the downstream analysis tools to correctly interpret the derived dataset, in each standard space, or reformat the output dataset according to the expectations of the analysis tools.

The user is also responsible for all aspects of provenance tracking and adequately reporting them in their communications. Information such as version of the template (or download date), citations to relevant papers, and other metadata (e.g., RRIDs) must be accounted for manually throughout the research process.

TemplateFlow integration with `fMRIPrep`¶

In contrast, tools using TemplateFlow dramatically simplify the whole process (note that MNI152NLin2009cAsym and OASIS30Ants are the two templates not found within the FSL distribution, and MNI152NLin6Asym denotes FSL’s MNI space (i.e., the default FSL template):

$ fmriprep /data /derivatives participant --output-spaces MNI152NLin2009cAsym MNI152NLin6Asym OASIS30Ants

fMRIPrep generates the results with BIDS-Derivatives organization for the three templates. The tool also leverages TemplateFlow to generate a boilerplate citation text that includes the full names, versions and references to credit the template’s authors for each of the templates involved.

fMRIPrep internally stages one spatial normalization workflow for each of the output spaces. Each of these normalization sub-workflows uses a simple line of Python code to retrieve the necessary resources from TemplateFlow using the TemplateFlow Client interface:

[10]:

tpl_ref_file = tflow.get("MNI152NLin6Asym", desc=None, resolution=1, suffix="T1w")
print(tpl_ref_file)

/Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_T1w.nii.gz

Obtaining templates for registration¶

Outside of fMRIPrep, we can also use TemplateFlow to streamline workflows that make use of standard spatial references, including (for instance) registration and segmentation workflows. For example, we can use an inline call to the Python client to retrieve the template (or, as necessary, a mask or tissue probability map) to build a call to SyN-based ANTs registration in a short bash script:)

[11]:

%%bash

templates=(
    MNI152NLin2009cAsym
    MNI152NLin6Asym
    OASIS30ANTs
)

for template in ${templates[@]}
    do
    template_path=$(
        python -c "
import templateflow.api as tflow
print(tflow.get('${template}', desc=None, resolution=1, suffix='T1w'))
        "
    )
    echo antsRegistrationSyN.sh \
        -d 3 \
        -f $template_path \
        -m /data/ds-001/sub-001/ses-001/sub-001_ses-001_T1w.nii.gz \
        -o /tmp/sub-001_ses-001_space-${template}_T1w
    echo
done

antsRegistrationSyN.sh -d 3 -f /Users/rastko/.cache/templateflow/tpl-MNI152NLin2009cAsym/tpl-MNI152NLin2009cAsym_res-01_T1w.nii.gz -m /data/ds-001/sub-001/ses-001/sub-001_ses-001_T1w.nii.gz -o /tmp/sub-001_ses-001_space-MNI152NLin2009cAsym_T1w

antsRegistrationSyN.sh -d 3 -f /Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_T1w.nii.gz -m /data/ds-001/sub-001/ses-001/sub-001_ses-001_T1w.nii.gz -o /tmp/sub-001_ses-001_space-MNI152NLin6Asym_T1w

antsRegistrationSyN.sh -d 3 -f /Users/rastko/.cache/templateflow/tpl-OASIS30ANTs/tpl-OASIS30ANTs_res-01_T1w.nii.gz -m /data/ds-001/sub-001/ses-001/sub-001_ses-001_T1w.nii.gz -o /tmp/sub-001_ses-001_space-OASIS30ANTs_T1w

Masks¶

One detail glossed over in the FSL example above is that, for a robust spatial normalization process, a precise binary mask of the brain is generally used. While FSL would require the user to manually set this mask up in the GUI, in the case of TemplateFlow, it requires a second minimal call:

[12]:

msk_ref_file = tflow.get("MNI152NLin6Asym", desc="brain", resolution=1, suffix="mask")
print(msk_ref_file)

/Users/rastko/.cache/templateflow/tpl-MNI152NLin6Asym/tpl-MNI152NLin6Asym_res-01_desc-brain_mask.nii.gz

These examples are extreme simplifications of what a pipeline developer can automate and make more robust by integrating TemplateFlow in their workflows.

Accessing file versions¶

TemplateFlow automatically tracks versions of all archived datasets and resources through DataLad. Currently, to access the version hashes of template resources, you must configure TemplateFlow to use DataLad for downloading files. A simple way to do this is by including export TEMPLATEFLOW_USE_DATALAD=1 in your .bash_profile. Under the hood, DataLad uses git-annex, an extension of Git designed for use with large files. Suppose we want to get the hash for the current version of the PNC template. Because DataLad uses git-annex, we can simply use git’s native logging functionality to obtain this.

[13]:

%%bash
pushd ${TEMPLATEFLOW_HOME}/tpl-PNC >>/dev/null
git log -n 1 --format=oneline
popd >>/dev/null

d24b6ed7a425574e21f8b3a96bd07674a518741a maint: use templateflow/gha-workflow-superdataset script

The hash is printed at left, and a description of the most recent change synchronised locally is printed at right. Similarly, we can also retrieve the hash of the dataset version in which any particular file was most recently changed…

[14]:

%%bash
pushd ${TEMPLATEFLOW_HOME}/tpl-PNC >>/dev/null
git log -n 1 --format=oneline template_description.json
popd >>/dev/null

00af2f16107b7834ff49fda3019c8531c62ed833 fix: bring template description back to git

…or the recent version history of the dataset (or any of its files).

[15]:

%%bash
pushd ${TEMPLATEFLOW_HOME}/tpl-PNC >>/dev/null
git log -n 20 --format=oneline
popd >>/dev/null

d24b6ed7a425574e21f8b3a96bd07674a518741a maint: use templateflow/gha-workflow-superdataset script
9be4f088d62b8dbc4721da619eb4ff940ba590cb [DATALAD] Recorded changes
87525f0a0cd52305c8bf41e2a4ada5a2f1a51ef2 enh: delete annexed GH action script
1b3db15dc2f270103408c6bf1f1c7d68e90ca757 fix: update the GH actions
bd0d1fecc5577ba431ab8618ff611a48d96fbb13 chore(gh-action): add update-superdataset post-hook
2c1263afb333c3dfce5835cddbbcdee42c295c8d [DATALAD] removed content
00af2f16107b7834ff49fda3019c8531c62ed833 fix: bring template description back to git
9e3eb802134a110d07c05bdf1b56a8164777f7e7 delete: links to template descriptions
d714dac28171b971c331bca95433af9d8bb4eaf0 fix: bring template description back to git
09e5cade904f2fdf75c77442ad9a6d3a395f4cd4 fix: bring template description back to git
98abf71886e3211299bc1a2a9ef2b6a5225da31a fix: bring template descrition back to git
84ad77f765354d3d007c77982b6e1aac2c33c809 fix: amend list of Curators
a8c86be1694c1e3b8f772e0cfb1980f61a75c6c2 fix: roll all 'ReferencesAndLinks' fields of the description back to lists
4ea613d7ef73d31e8a96945dfa9e49f7a899aa56 fix: roll all 'ReferencesAndLinks' fields of the description back to lists
c9b410f748830ca7d732f922d096e2be942c9509 [DATALAD] added content
e56f490ba3c6d9dd66bf13021541fee70c440bbb Create LICENSE
216c9a058deadd68560b9c4b7aa9c6a03f2525f1 update template_description.json fields to be more BIDS-like
a354a2310fd5fb5aca2117c5f6f02d30fa90765d remove readme and fix version number
d8cfe43eaae14020d0f2c15a361eca4ea66d8f57 metadata and changelog
2b6ae7d2c79b976bd0ca828957ddfeeeb45879b7 [DATALAD] add files from URLs

Each hash unambiguously references a specific version of the dataset, so it can be used to ensure that collaborators use identical versions of template resources. We can use these hashes to view the dataset as it appeared at any point in its history. See DataLad’s documentation for more details.

Summary¶

TemplateFlow makes many aspects of working with spatial standard datasets easier. Please check out the TemplateFlow homepage and the detailed documentation hub for TemplateFlow for further information about using the client or contributing new template resources. If you still have questions, you can ask on NeuroStars with the templateflow tag.