Skip to content
GitLab
Commit fb3645c5 authored by Dmitry ERSHOV's avatar Dmitry ERSHOV
Browse files

Make more comments and add TransferNotes

parent 1bcf71dc
Expand all Hide whitespace changes
Inline Side-by-side
README.md
View file @ fb3645c5
Implementation of Stardist for Trackmate
- Generate pairs image+labels out of the raw tif stacks (100 frames) and xml files (coordinates)
- Use these images to train Stardist models
- Detect particles with the model of choice.
Implementation of Stardist for detection of small particles.
- Raw data: synthetic time-lapses of different particles (microtubules, vesicles) in different SNR and corresponding ground truth coordinates of the particles stored in xml.
- Step 1: generate training data: Image + labeled image. Image will be same as those in the raw dataset, and the labeled image will be generated out of coordinates stored in corresponding xml files.
- Step 2: use the pairs image+labeled image to train Stardist models
- Step 3: test the models.
Read more in ./docs/TransferNotes.md
docs/TransferNotes.md 0 → 100644
View file @ fb3645c5
Implementation of Stardist for Particle Detection.
- Raw data: synthetic time-lapses of different particles (microtubules, vesicles) in different SNR and corresponding ground truth coordinates of the particles stored in xml.
- Step 1: generate training data: Image + labeled image. Image will be same as those in the raw dataset, and the labeled image will be generated out of coordinates stored in corresponding xml files.
- Step 2: use the pairs image+labeled image to train Stardist models
- Step 3: test the models.
Organization:
./data/1_raw_movies_and_xmls/
Original tifs and xmls.
./data/2_generated/
Training data generated from original tifs and xmls.
./docs/
All related documentation
./models
Contains trained models
./scripts
All related notebooks and scripts
./stardist
Contains the original Stardist iimplementation
1. Notebook "0_generate_training_data":
- This notebook is used to generate the training data.
- It is well commented, short recap: read tif+xml, convert xml to labeld image, save original tif + labeled.
2. Notebook "1_data-test":
- The idea of this notebook is identical to the original stardist notebook "stardist/examples/2D/1_data.ipynb"; read this notebook for more detaiils. Part of the code in our implementation is ommited, since our data is more straightforward than theirs.
- In this notebook they are looking at the labels and study if they can be approximated well with 8,16,32 ray casting technique.
- We simply plug in our data (generated with the notebook above) and do exactly what they did with their nuclei labels.
3. Notebooks with prefix 2,3,4:
- Training and testing the models. The implementation is exactly the same as they have for original Stardist in "stardist/examples/2D/2_training.ipynb"; read this notebook for more details. Part of the code in our implementation is ommited, since our data is more straightforward than theirs.
- There are several training regimes that we tested so far: training on the full dataset (mixing all data with different SNRs, different particle types, denstities), training only on Vesicles of all SNRs and all densities, traiining on Vesicles of SNR=7 and density=middle. Basically, these three notebooks differ only in the way the training data is loaded.
- For each of these 3 training there is also a corresponding testing notebook.
4. Notebook "5_generate_csvs":
- This notebook loads a model, detects particles in test images and generates a csv with predictions
- The format of this csv should match the TrackMate style formatting.
\ No newline at end of file
scripts/.ipynb_checkpoints/0_generate_training_data-checkpoint.ipynb
View file @ fb3645c5
......@@ -9,18 +9,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Made dir: /master/home/dershov/Development/DL/stardist-trackmate/data/2_generated/images\n",
"Made dir: /master/home/dershov/Development/DL/stardist-trackmate/data/2_generated/masks\n"
]
}
],
"outputs": [],
"source": [
"import os, sys\n",
"import xml.etree.ElementTree as ET\n",
......@@ -38,17 +29,6 @@
"IMAGES_DIR = os.path.join(DATA_GEN_DIR, 'images' )\n",
"MASKS_DIR = os.path.join(DATA_GEN_DIR, 'masks' )\n",
"\n",
"\n",
"\n",
"# TR_IMAGES_DIR = os.path.join(TRAINING_DIR, 'images' )\n",
"# TR_MASKS_DIR = os.path.join(TRAINING_DIR, 'masks' )\n",
"\n",
"# TEST_DIR = os.path.join(DATA_DIR, 'test' )\n",
"# TE_IMAGES_DIR = os.path.join(TEST_DIR, 'images' )\n",
"# TE_MASKS_DIR = os.path.join(TEST_DIR, 'masks' )\n",
"\n",
"\n",
"# for f in [TRAINING_DIR, TEST_DIR, TR_IMAGES_DIR, TR_MASKS_DIR, TE_IMAGES_DIR, TE_MASKS_DIR]:\n",
"for f in [IMAGES_DIR, MASKS_DIR]:\n",
" if not os.path.exists(f): \n",
" os.makedirs(f)\n",
......@@ -62,7 +42,17 @@
"outputs": [],
"source": [
"def read_xml_as_pands( xml_abs_path ):\n",
" \n",
" \"\"\"\n",
" Reads xml file and converts it to pandas DataFrame.\n",
" Input\n",
" -----\n",
" xml_abs_path : str\n",
" absolute path to xml file\n",
" Return\n",
" ------\n",
" df : pandas.DataFrame\n",
" Contains all required information: particle id, coordinates in space and time.\n",
" \"\"\"\n",
" tree = ET.parse( xml_abs_path )\n",
" root = tree.getroot()\n",
" particles = root[0]\n",
......@@ -94,7 +84,22 @@
"\n",
"\n",
"def df_to_lbl( img, df, radius ):\n",
" # df = ['pid', 'fr', 'x', 'y', 'z']\n",
" \"\"\"\n",
" Converts pandas DataFrame with particle coordinates in time and space to image stack.\n",
" Input\n",
" -----\n",
" img : numpy.ndarray\n",
" Array representing time-lapse; img.shape =(frame, y, x)\n",
" df : pandas.DataFrame\n",
" df = ['pid', 'fr', 'x', 'y', 'z']\n",
" radius : float\n",
" radius of the particle label. Each particle label will be represented by ciircle of this radius.\n",
" Return\n",
" ------\n",
" img_lbl : numpy.ndarray\n",
" Array representing time-lapse; img_lbl.shape = img.shape =(frame, y, x)\n",
" Contains all labels extracted from df. For each particle its label value is the same as its ID.\n",
" \"\"\"\n",
" \n",
" id_max = df['pid'].max()\n",
" assert id_max < 2**16-1, \"Label image is coded to be unsigned 16bit; you have %d particles.\"%id_max\n",
......@@ -115,6 +120,15 @@
" return img_lbl\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Find raw data \n",
"- tif, xml\n",
"- Name of xml and tif are exactly the same; just the extension is different."
]
},
{
"cell_type": "code",
"execution_count": 17,
......@@ -185,6 +199,17 @@
" print('%s \\t %s'%(x,t))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Generate label images\n",
"- read a pair of tif, xml\n",
"- convert xml to dataframe, dataframe to img_lbl. \n",
"- Store each slice in tif and img_lbl: tif slices go to Image folder and img_lbl slices go to Mask folder\n",
"- The data Image/Mask is used for training a stardist model.\n"
]
},
{
"cell_type": "code",
"execution_count": 20,
......@@ -238,8 +263,8 @@
}
],
"source": [
"# LAbel is a disk of radius:\n",
"R = 3\n",
"# frames_for_test_data = np.arange(0,99,10) # 10 out of 100\n",
"\n",
"for xml_file, tif_file in zip(xmls, tifs):\n",
"\n",
......@@ -247,12 +272,13 @@
" print(base_name, end='....')\n",
"\n",
" # read\n",
" df = read_xml_as_pands( os.path.join(DATA_RAW_DIR, xml_file) )\n",
" img = tifffile.imread( os.path.join(DATA_RAW_DIR, tif_file))\n",
"\n",
" df = read_xml_as_pands( os.path.join(DATA_RAW_DIR, xml_file) )\n",
" \n",
" # convert df to img_lbl\n",
" img_lbl = df_to_lbl( img, df, radius=R )\n",
" \n",
" # for each frame:\n",
" # store each slice.\n",
" for _fr, (_im, _lbl) in enumerate(zip(img, img_lbl)):\n",
" _fname = '%s_%05d.tif'%(base_name, _fr)\n",
" \n",
......@@ -269,177 +295,6 @@
"print('Done all.')"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(['VESICLE snr 7 density mid.xml'], ['VESICLE snr 7 density mid.tif'])"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#VESICLE snr 7 density mid\n",
"# target = 'VESICLE snr 7 density mid'\n",
"# xmls_chosen = [x for x in xmls if target in x]\n",
"# tifs_chosen = [x for x in tifs if target in x]\n",
"# xmls_chosen, tifs_chosen"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# xmls_chosen = xmls\n",
"# tifs_chosen = tifs"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"RECEPTOR snr 3 density low\n",
" done\n",
"MICROTUBULE snr 1 density low\n",
" done\n",
"RECEPTOR snr 7 density mid\n",
" done\n",
"VESICLE snr 4 density low\n",
" done\n",
"MICROTUBULE snr 4 density mid\n",
" done\n",
"VESICLE snr 1 density mid\n",
" done\n",
"RECEPTOR snr 2 density low\n",
" done\n",
"MICROTUBULE snr 7 density mid\n",
" done\n",
"RECEPTOR snr 1 density low\n",
" done\n",
"VESICLE snr 2 density mid\n",
" done\n",
"MICROTUBULE snr 2 density low\n",
" done\n",
"VESICLE snr 7 density low\n",
" done\n",
"RECEPTOR snr 4 density mid\n",
" done\n",
"RECEPTOR snr 2 density mid\n",
" done\n",
"VESICLE snr 1 density low\n",
" done\n",
"MICROTUBULE snr 4 density low\n",
" done\n",
"RECEPTOR snr 7 density high\n",
" done\n",
"VESICLE snr 2 density high\n",
" done\n",
"MICROTUBULE snr 4 density high\n",
" done\n",
"MICROTUBULE snr 7 density high\n",
" done\n",
"VESICLE snr 4 density mid\n",
" done\n",
"RECEPTOR snr 7 density low\n",
" done\n",
"VESICLE snr 1 density high\n",
" done\n",
"RECEPTOR snr 3 density mid\n",
" done\n",
"MICROTUBULE snr 1 density mid\n",
" done\n",
"RECEPTOR snr 4 density high\n",
" done\n",
"RECEPTOR snr 3 density high\n",
" done\n",
"RECEPTOR snr 4 density low\n",
" done\n",
"RECEPTOR snr 2 density high\n",
" done\n",
"VESICLE snr 7 density mid\n",
" done\n",
"VESICLE snr 7 density high\n",
" done\n",
"MICROTUBULE snr 2 density mid\n",
" done\n",
"MICROTUBULE snr 1 density high\n",
" done\n",
"MICROTUBULE snr 2 density high\n",
" done\n",
"VESICLE snr 4 density high\n",
" done\n",
"RECEPTOR snr 1 density high\n",
" done\n",
"VESICLE snr 2 density low\n",
" done\n",
"RECEPTOR snr 1 density mid\n",
" done\n",
"MICROTUBULE snr 7 density low\n",
" done\n",
"Done all.\n"
]
}
],
"source": [
"R = 3\n",
"frames_for_test_data = np.arange(0,99,10) # 10 out of 100\n",
"\n",
"for xml_file, tif_file in zip(xmls_chosen, tifs_chosen):\n",
" # each tif file is a stack; 100 frames\n",
"\n",
" base_name = os.path.splitext(xml_file)[0]\n",
" print(base_name)\n",
"\n",
" # read\n",
" df = read_xml_as_pands( os.path.join(DATA_DIR, xml_file) )\n",
" img = tifffile.imread( os.path.join(DATA_DIR, tif_file))\n",
"\n",
" img_lbl = df_to_lbl( img, df, radius=R )\n",
" \n",
" # for each frame:\n",
" for _fr, (_im, _lbl) in enumerate(zip(img, img_lbl)):\n",
" _fname = '%s_%03d.tif'%(base_name, _fr)\n",
" \n",
" if _fr in frames_for_test_data:\n",
" # save as TEST DATA\n",
" img_abs_path = os.path.join( TE_IMAGES_DIR, _fname )\n",
" lbl_abs_path = os.path.join( TE_MASKS_DIR, _fname )\n",
" else:\n",
" # save as TRAINING DATA\n",
" img_abs_path = os.path.join( TR_IMAGES_DIR, _fname )\n",
" lbl_abs_path = os.path.join( TR_MASKS_DIR, _fname )\n",
"\n",
" # print( ' ', img_abs_path )\n",
" # print( ' ', lbl_abs_path )\n",
" tifffile.imwrite(img_abs_path, data=_im )\n",
" tifffile.imwrite(lbl_abs_path, data=_lbl )\n",
" \n",
" print(' done')\n",
" \n",
"print('Done all.')"
]
},
{
"cell_type": "code",
"execution_count": 203,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
......@@ -464,7 +319,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.9"
"version": "3.5.2"
}
},
"nbformat": 4,
......
%% Cell type:markdown id: tags:
### This notebook is to generate training data from the original tifs and xmls
%% Cell type:code id: tags:
``` python
import os, sys
import xml.etree.ElementTree as ET
import pandas as pd
import tifffile
import matplotlib.pyplot as plt
import numpy as np
from skimage.morphology import dilation, selem
ROOT_DIR = os.path.abspath( '..' )
DATA_DIR = os.path.join( ROOT_DIR, 'data' )
DATA_RAW_DIR = os.path.join( DATA_DIR, '1_raw_movies_and_xmls' )
DATA_GEN_DIR = os.path.join(DATA_DIR, '2_generated' )
IMAGES_DIR = os.path.join(DATA_GEN_DIR, 'images' )
MASKS_DIR = os.path.join(DATA_GEN_DIR, 'masks' )
# TR_IMAGES_DIR = os.path.join(TRAINING_DIR, 'images' )
# TR_MASKS_DIR = os.path.join(TRAINING_DIR, 'masks' )
# TEST_DIR = os.path.join(DATA_DIR, 'test' )
# TE_IMAGES_DIR = os.path.join(TEST_DIR, 'images' )
# TE_MASKS_DIR = os.path.join(TEST_DIR, 'masks' )
# for f in [TRAINING_DIR, TEST_DIR, TR_IMAGES_DIR, TR_MASKS_DIR, TE_IMAGES_DIR, TE_MASKS_DIR]:
for f in [IMAGES_DIR, MASKS_DIR]:
if not os.path.exists(f):
os.makedirs(f)
print('Made dir: %s'%f)
```
%%%% Output: stream
Made dir: /master/home/dershov/Development/DL/stardist-trackmate/data/2_generated/images
Made dir: /master/home/dershov/Development/DL/stardist-trackmate/data/2_generated/masks
%% Cell type:code id: tags:
``` python
def read_xml_as_pands( xml_abs_path ):
"""
Reads xml file and converts it to pandas DataFrame.
Input
-----
xml_abs_path : str
absolute path to xml file
Return
------
df : pandas.DataFrame
Contains all required information: particle id, coordinates in space and time.
"""
tree = ET.parse( xml_abs_path )
root = tree.getroot()
particles = root[0]
# the fastes way is to first collect all the data and then feed it to pandas.
# if you create pandas in each loop step, it will be way longer
ii, tt, xx, yy, zz = [], [],[],[],[]
for pi, detections in enumerate(particles): #over particles
for d in detections: # over each p's detectoins
t,x,y,z = d.attrib['t'], d.attrib['x'], d.attrib['y'], d.attrib['z']
ii.append(pi)
tt.append(t)
xx.append(x)
yy.append(y)
zz.append(z)
# feed the collected data to pandas
df = pd.DataFrame(data={'pid':ii, 'fr':tt, 'x':xx, 'y':yy, 'z':zz} )
for n in ['x','y','z']:
df[n] = pd.to_numeric(df[n], downcast='float')
# it is all strings now; convert to numeric
df['fr'] = pd.to_numeric(df['fr'], downcast='integer')
df['pid'] = pd.to_numeric(df['pid'], downcast='integer')
return df
def df_to_lbl( img, df, radius ):
# df = ['pid', 'fr', 'x', 'y', 'z']
"""
Converts pandas DataFrame with particle coordinates in time and space to image stack.
Input
-----
img : numpy.ndarray
Array representing time-lapse; img.shape =(frame, y, x)
df : pandas.DataFrame
df = ['pid', 'fr', 'x', 'y', 'z']
radius : float
radius of the particle label. Each particle label will be represented by ciircle of this radius.
Return
------
img_lbl : numpy.ndarray
Array representing time-lapse; img_lbl.shape = img.shape =(frame, y, x)
Contains all labels extracted from df. For each particle its label value is the same as its ID.
"""
id_max = df['pid'].max()
assert id_max < 2**16-1, "Label image is coded to be unsigned 16bit; you have %d particles."%id_max
# dilate with:
se = selem.disk(radius)
frames_n, rows, cols = img.shape
img_lbl = np.zeros_like(img).astype(np.uint16)
for _fr in range(frames_n):
_df = df[df['fr']==_fr] # df for cur frame
for i,_d in _df.iterrows(): # df for cur particle in this frame
pid, c, r = int(_d['pid']), int(np.round(_d['x'])), int(np.round(_d['y']))
img_lbl[_fr, r, c] = pid # mark a pixel with particle id
img_lbl[_fr] = dilation( img_lbl[_fr], selem=se)
return img_lbl
```
%% Cell type:markdown id: tags:
# Find raw data
- tif, xml
- Name of xml and tif are exactly the same; just the extension is different.
%% Cell type:code id: tags:
``` python
files = os.listdir(DATA_RAW_DIR)
tifs = [f for f in files if f.endswith('.tif')]
xmls = [f for f in files if f.endswith('.xml')]
# sort the tifs according to the xmls file order:
xmls.sort()
tifs_sorted = []
for xmlname in xmls:
tifname = xmlname.replace('xml','tif')
if tifname in tifs:
tifs_sorted.append( tifname )
tifs = tifs_sorted
for x,t in zip( xmls, tifs):
print('%s \t %s'%(x,t))
```
%%%% Output: stream
MICROTUBULE snr 1 density high.xml MICROTUBULE snr 1 density high.tif
MICROTUBULE snr 1 density low.xml MICROTUBULE snr 1 density low.tif
MICROTUBULE snr 1 density mid.xml MICROTUBULE snr 1 density mid.tif
MICROTUBULE snr 2 density high.xml MICROTUBULE snr 2 density high.tif
MICROTUBULE snr 2 density low.xml MICROTUBULE snr 2 density low.tif
MICROTUBULE snr 2 density mid.xml MICROTUBULE snr 2 density mid.tif
MICROTUBULE snr 4 density high.xml MICROTUBULE snr 4 density high.tif
MICROTUBULE snr 4 density low.xml MICROTUBULE snr 4 density low.tif
MICROTUBULE snr 4 density mid.xml MICROTUBULE snr 4 density mid.tif
MICROTUBULE snr 7 density high.xml MICROTUBULE snr 7 density high.tif
MICROTUBULE snr 7 density low.xml MICROTUBULE snr 7 density low.tif
MICROTUBULE snr 7 density mid.xml MICROTUBULE snr 7 density mid.tif
RECEPTOR snr 1 density high.xml RECEPTOR snr 1 density high.tif
RECEPTOR snr 1 density low.xml RECEPTOR snr 1 density low.tif
RECEPTOR snr 1 density mid.xml RECEPTOR snr 1 density mid.tif
RECEPTOR snr 2 density high.xml RECEPTOR snr 2 density high.tif
RECEPTOR snr 2 density low.xml RECEPTOR snr 2 density low.tif
RECEPTOR snr 2 density mid.xml RECEPTOR snr 2 density mid.tif
RECEPTOR snr 3 density high.xml RECEPTOR snr 3 density high.tif
RECEPTOR snr 3 density low.xml RECEPTOR snr 3 density low.tif
RECEPTOR snr 3 density mid.xml RECEPTOR snr 3 density mid.tif
RECEPTOR snr 4 density high.xml RECEPTOR snr 4 density high.tif
RECEPTOR snr 4 density low.xml RECEPTOR snr 4 density low.tif
RECEPTOR snr 4 density mid.xml RECEPTOR snr 4 density mid.tif
RECEPTOR snr 7 density high.xml RECEPTOR snr 7 density high.tif
RECEPTOR snr 7 density low.xml RECEPTOR snr 7 density low.tif
RECEPTOR snr 7 density mid.xml RECEPTOR snr 7 density mid.tif
VESICLE snr 1 density high.xml VESICLE snr 1 density high.tif
VESICLE snr 1 density low.xml VESICLE snr 1 density low.tif
VESICLE snr 1 density mid.xml VESICLE snr 1 density mid.tif
VESICLE snr 2 density high.xml VESICLE snr 2 density high.tif
VESICLE snr 2 density low.xml VESICLE snr 2 density low.tif
VESICLE snr 2 density mid.xml VESICLE snr 2 density mid.tif
VESICLE snr 4 density high.xml VESICLE snr 4 density high.tif
VESICLE snr 4 density low.xml VESICLE snr 4 density low.tif
VESICLE snr 4 density mid.xml VESICLE snr 4 density mid.tif
VESICLE snr 7 density high.xml VESICLE snr 7 density high.tif
VESICLE snr 7 density low.xml VESICLE snr 7 density low.tif
VESICLE snr 7 density mid.xml VESICLE snr 7 density mid.tif
%% Cell type:markdown id: tags:
# Generate label images
- read a pair of tif, xml
- convert xml to dataframe, dataframe to img_lbl.
- Store each slice in tif and img_lbl: tif slices go to Image folder and img_lbl slices go to Mask folder
- The data Image/Mask is used for training a stardist model.
%% Cell type:code id: tags:
``` python
# LAbel is a disk of radius:
R = 3
# frames_for_test_data = np.arange(0,99,10) # 10 out of 100
for xml_file, tif_file in zip(xmls, tifs):
base_name = os.path.splitext(xml_file)[0]
print(base_name, end='....')
# read
df = read_xml_as_pands( os.path.join(DATA_RAW_DIR, xml_file) )
img = tifffile.imread( os.path.join(DATA_RAW_DIR, tif_file))
df = read_xml_as_pands( os.path.join(DATA_RAW_DIR, xml_file) )
# convert df to img_lbl
img_lbl = df_to_lbl( img, df, radius=R )
# for each frame:
# store each slice.
for _fr, (_im, _lbl) in enumerate(zip(img, img_lbl)):
_fname = '%s_%05d.tif'%(base_name, _fr)
img_abs_path = os.path.join( IMAGES_DIR, _fname )
lbl_abs_path = os.path.join( MASKS_DIR, _fname )
# print( ' ', img_abs_path )
# print( ' ', lbl_abs_path )
tifffile.imwrite(img_abs_path, data=_im )
tifffile.imwrite(lbl_abs_path, data=_lbl )
print(' DONE.')
print('Done all.')
```
%%%% Output: stream
MICROTUBULE snr 1 density high.... DONE.
MICROTUBULE snr 1 density low.... DONE.
MICROTUBULE snr 1 density mid.... DONE.
MICROTUBULE snr 2 density high.... DONE.
MICROTUBULE snr 2 density low.... DONE.
MICROTUBULE snr 2 density mid.... DONE.
MICROTUBULE snr 4 density high.... DONE.
MICROTUBULE snr 4 density low.... DONE.
MICROTUBULE snr 4 density mid.... DONE.
MICROTUBULE snr 7 density high.... DONE.
MICROTUBULE snr 7 density low.... DONE.
MICROTUBULE snr 7 density mid.... DONE.
RECEPTOR snr 1 density high.... DONE.
RECEPTOR snr 1 density low.... DONE.
RECEPTOR snr 1 density mid.... DONE.
RECEPTOR snr 2 density high.... DONE.
RECEPTOR snr 2 density low.... DONE.
RECEPTOR snr 2 density mid.... DONE.
RECEPTOR snr 3 density high.... DONE.
RECEPTOR snr 3 density low.... DONE.
RECEPTOR snr 3 density mid.... DONE.
RECEPTOR snr 4 density high.... DONE.
RECEPTOR snr 4 density low.... DONE.
RECEPTOR snr 4 density mid.... DONE.
RECEPTOR snr 7 density high.... DONE.
RECEPTOR snr 7 density low.... DONE.
RECEPTOR snr 7 density mid.... DONE.
VESICLE snr 1 density high.... DONE.
VESICLE snr 1 density low.... DONE.
VESICLE snr 1 density mid.... DONE.
VESICLE snr 2 density high.... DONE.
VESICLE snr 2 density low.... DONE.
VESICLE snr 2 density mid.... DONE.
VESICLE snr 4 density high.... DONE.
VESICLE snr 4 density low.... DONE.
VESICLE snr 4 density mid.... DONE.
VESICLE snr 7 density high.... DONE.
VESICLE snr 7 density low.... DONE.
VESICLE snr 7 density mid.... DONE.
Done all.
%% Cell type:code id: tags:
``` python
#VESICLE snr 7 density mid
# target = 'VESICLE snr 7 density mid'
# xmls_chosen = [x for x in xmls if target in x]
# tifs_chosen = [x for x in tifs if target in x]
# xmls_chosen, tifs_chosen
```
%%%% Output: execute_result
(['VESICLE snr 7 density mid.xml'], ['VESICLE snr 7 density mid.tif'])
%% Cell type:code id: tags:
``` python
# xmls_chosen = xmls
# tifs_chosen = tifs
```
%% Cell type:code id: tags:
``` python
R = 3
frames_for_test_data = np.arange(0,99,10) # 10 out of 100
for xml_file, tif_file in zip(xmls_chosen, tifs_chosen):
# each tif file is a stack; 100 frames
base_name = os.path.splitext(xml_file)[0]
print(base_name)
# read
df = read_xml_as_pands( os.path.join(DATA_DIR, xml_file) )
img = tifffile.imread( os.path.join(DATA_DIR, tif_file))
img_lbl = df_to_lbl( img, df, radius=R )
# for each frame:
for _fr, (_im, _lbl) in enumerate(zip(img, img_lbl)):
_fname = '%s_%03d.tif'%(base_name, _fr)
if _fr in frames_for_test_data:
# save as TEST DATA
img_abs_path = os.path.join( TE_IMAGES_DIR, _fname )
lbl_abs_path = os.path.join( TE_MASKS_DIR, _fname )
else:
# save as TRAINING DATA
img_abs_path = os.path.join( TR_IMAGES_DIR, _fname )
lbl_abs_path = os.path.join( TR_MASKS_DIR, _fname )
# print( ' ', img_abs_path )
# print( ' ', lbl_abs_path )
tifffile.imwrite(img_abs_path, data=_im )
tifffile.imwrite(lbl_abs_path, data=_lbl )
print(' done')
print('Done all.')
```
%%%% Output: stream
RECEPTOR snr 3 density low
done
MICROTUBULE snr 1 density low
done
RECEPTOR snr 7 density mid
done
VESICLE snr 4 density low
done
MICROTUBULE snr 4 density mid
done
VESICLE snr 1 density mid
done
RECEPTOR snr 2 density low
done
MICROTUBULE snr 7 density mid
done
RECEPTOR snr 1 density low
done
VESICLE snr 2 density mid
done
MICROTUBULE snr 2 density low
done
VESICLE snr 7 density low
done
RECEPTOR snr 4 density mid
done
RECEPTOR snr 2 density mid
done
VESICLE snr 1 density low
done
MICROTUBULE snr 4 density low