diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/README.md b/competitions/kaggle/Cryo-ET/1st_place_solution/README.md new file mode 100644 index 000000000..226b61be8 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/README.md @@ -0,0 +1,99 @@ +## Introduction + +This tutorial illustrates how to use MONAI for cryo electron tomography. The pipeline and models were partly used to win the [Cryo-ET competition on kaggle](https://www.kaggle.com/competitions/czii-cryo-et-object-identification/overview). The tutorial was tested with nvidia/pytorch:24.08-py3 docker container and a single A100 GPU. + +## What is Cryo-ET? + +If you ask ChatGPT: + +Cryo-ET (Cryo-Electron Tomography) is an advanced imaging technique that allows scientists to visualize biological structures in near-native states at high resolution. It combines cryogenic sample preservation with electron tomography to generate three-dimensional (3D) reconstructions of cellular structures, protein complexes, and organelles. + +### How It Works +1. Cryo-Fixation: The sample (e.g., a cell or a purified macromolecular complex) is rapidly frozen using liquid ethane or similar methods to prevent ice crystal formation, preserving its natural state. +2. Electron Microscopy: The frozen sample is placed under a transmission electron microscope (TEM), where images are taken from multiple angles by tilting the sample. +3. Tomographic Reconstruction: Computational algorithms combine these 2D images to create a detailed 3D model of the structure. + +### Applications +Studying cellular architecture at nanometer resolution. +Visualizing macromolecular complexes in their native environments. +Understanding interactions between viruses and host cells. +Investigating neurodegenerative diseases, cancer, and infectious diseases. +Cryo-ET is particularly powerful because it enables direct imaging of biological systems without the need for staining or chemical fixation, preserving their native conformation. + + + + +## Required Data + +This tutorial is build upon the official Cryo ET competition data. +It can be downloaded to a local ```DATA_FOLDER``` directly from kaggle (You will also need to follow the competition url and click "join competition" to accept the terms and conditions): https://www.kaggle.com/competitions/czii-cryo-et-object-identification/data . + +Alternativly it can be downloaded using the kaggle API (which can be installed via ```pip install kaggle```). If you decide to use the Kaggle API you need to create a Kaggle account and configure your token as described [here](https://github.com/Kaggle/kaggle-api/blob/main/docs/README.md#api-credentials) and then be allowed to download the data with the following command: + +```kaggle competitions download -c czii-cryo-et-object-identification -p DATA_FOLDER``` + +Unzip the competition dataset to DATA_FOLDER + +``` +cd DATA_FOLDER +unzip czii-cryo-et-object-identification.zip -d czii-cryo-et-object-identification/ +``` + + +## Environment: + +To have a common environment its suggested to use the basic pytorch docker container and add a few pip packages on top. This tutorial is designed to use a docker container with DATA_FOLDER mounted under "/mount/cryo/data/" + +1. This tutorial was tested with tag 24.08-py3, i.e. run the following command to pull/ start the container. + +```docker run nvcr.io/nvidia/pytorch:24.08-py3 -v DATA_FOLDER:/mount/cryo/data/``` + +2. Within the container clone this repository + +``` +git clone https://github.com/ProjectMONAI/tutorials +cd tutorials/competitions/kaggle/Cryo-ET/1st_place_solution/ +``` + + +3. And install necessary additional pip packages via + +```pip install -r requirements.txt``` + +If you dont want to mount the DATA_FOLDER, or don't want to use docker, you have to adjust path to the data in ```configs/common_config.py``` with specifying```cfg.data_folder``` + +## Training models + +For the competition we created a cross-validation scheme by simply simply splitting the 7 training tomographs into 7 folds. I.e. we train on 6 tomographs and use the 7th as validation. +For convenience we provide a file ```train_folded_v1.csv``` which contains the original training annotations and was also extended by a column containing fold_ids. + +We solve the competition with a 3D-segmentation approach leveraging [MONAI's FlexibleUNet](https://docs.monai.io/en/stable/networks.html#flexibleunet) architecture. Compared to the original implementation we adjusted the network to output more featuremap and enable deep-supervision. The following illustrates the resulting architecture at a high level: + +

+ figure of a Partly UNet + +We provide three different configurations which differ only in the used backbone and output feature maps. The configuration files are .py files and located under ```configs``` and share all other hyper-parameters. Each hyperparameter can be overwriten by adding a flag to the training command. To train a resnet34 version of our segmentation model simply run + +```python train.py -C cfg_resnet34 --output_dir WHATEVERISYOUROUTPUTDIR``` + +This will save checkpoints under the specified WHATEVERISYOUROUTPUTDIR when training is finished. +By default models are trained using bfloat16 which requires a GPU capable of that. Alternatively you can set ```cfg.bf16=False``` or overwrite as flag ```--bf16 False``` when running ```train.py ```. + +### Replicating 1st place solution (segmentation part) + +To train checkpoints necessary for replicating the segmentation part of the 1st place solution run training of 2x fullfits for each model. Thereby ```cfg.fold = -1``` results in training on all data, and using ```fold 0``` as validation. +``` +python train.py -C cfg_resnet34 --fold -1 +python train.py -C cfg_resnet34 --fold -1 +python train.py -C cfg_resnet34_ds --fold -1 +python train.py -C cfg_resnet34_ds --fold -1 +python train.py -C cfg_effnetb3 --fold -1 +python train.py -C cfg_effnetb3 --fold -1 +``` + +## Inference + +Inference after models are converted with torch jit is shown in our 1st place submission kaggle kernel. + +https://www.kaggle.com/code/christofhenkel/cryo-et-1st-place-solution?scriptVersionId=223259615 diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_effnetb3.py b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_effnetb3.py new file mode 100644 index 000000000..82d7ef8e0 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_effnetb3.py @@ -0,0 +1,34 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +from common_config import basic_cfg +import os +import pandas as pd +import numpy as np +import monai.transforms as mt + +cfg = basic_cfg + +cfg.name = os.path.basename(__file__).split(".")[0] +cfg.output_dir = f"/mount/cryo/models/{os.path.basename(__file__).split('.')[0]}" + +# model +cfg.backbone = "efficientnet-b3" +cfg.backbone_args = dict( + spatial_dims=3, + in_channels=cfg.in_channels, + out_channels=cfg.n_classes, + backbone=cfg.backbone, + pretrained=cfg.pretrained, +) +cfg.class_weights = np.array([64, 64, 64, 64, 64, 64, 1]) +cfg.lvl_weights = np.array([0, 0, 0, 1]) diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_resnet34.py b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_resnet34.py new file mode 100644 index 000000000..470d69087 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_resnet34.py @@ -0,0 +1,35 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from common_config import basic_cfg +import os +import pandas as pd +import numpy as np + +cfg = basic_cfg + +# paths +cfg.name = os.path.basename(__file__).split(".")[0] +cfg.output_dir = f"/mount/cryo/models/{os.path.basename(__file__).split('.')[0]}" + + +# model + +cfg.backbone = "resnet34" +cfg.backbone_args = dict( + spatial_dims=3, + in_channels=cfg.in_channels, + out_channels=cfg.n_classes, + backbone=cfg.backbone, + pretrained=cfg.pretrained, +) +cfg.class_weights = np.array([256, 256, 256, 256, 256, 256, 1]) +cfg.lvl_weights = np.array([0, 0, 0, 1]) diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_resnet34_ds.py b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_resnet34_ds.py new file mode 100644 index 000000000..6b177713a --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/cfg_resnet34_ds.py @@ -0,0 +1,32 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from common_config import basic_cfg +import os +import pandas as pd +import numpy as np + +cfg = basic_cfg + +# paths +cfg.name = os.path.basename(__file__).split(".")[0] +cfg.output_dir = f"/mount/cryo/models/{os.path.basename(__file__).split('.')[0]}" + +cfg.backbone = "resnet34" +cfg.backbone_args = dict( + spatial_dims=3, + in_channels=cfg.in_channels, + out_channels=cfg.n_classes, + backbone=cfg.backbone, + pretrained=cfg.pretrained, +) +cfg.class_weights = np.array([64, 64, 64, 64, 64, 64, 1]) +cfg.lvl_weights = np.array([0, 0, 1, 1]) diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/configs/common_config.py b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/common_config.py new file mode 100644 index 000000000..99aef862c --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/configs/common_config.py @@ -0,0 +1,209 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +from types import SimpleNamespace +from monai import transforms as mt + +cfg = SimpleNamespace(**{}) + +# stages +cfg.train = True +cfg.val = True +cfg.test = True +cfg.train_val = True + +# dataset +cfg.batch_size_val = None +cfg.use_custom_batch_sampler = False +cfg.val_df = None +cfg.test_df = None +cfg.val_data_folder = None +cfg.train_aug = None +cfg.val_aug = None +cfg.data_sample = -1 + +# model + +cfg.pretrained = False +cfg.pretrained_weights = None +cfg.pretrained_weights_strict = True +cfg.pop_weights = None +cfg.compile_model = False + +# training routine +cfg.fold = 0 +cfg.optimizer = "Adam" +cfg.sgd_momentum = 0 +cfg.sgd_nesterov = False +cfg.lr = 1e-4 +cfg.schedule = "cosine" +cfg.num_cycles = 0.5 +cfg.weight_decay = 0 +cfg.epochs = 10 +cfg.seed = -1 +cfg.resume_training = False +cfg.distributed = False +cfg.clip_grad = 0 +cfg.save_val_data = True +cfg.gradient_checkpointing = False +cfg.apex_ddp = False +cfg.synchronize_step = True + +# eval +cfg.eval_ddp = True +cfg.calc_metric = True +cfg.calc_metric_epochs = 1 +cfg.eval_steps = 0 +cfg.eval_epochs = 1 +cfg.save_pp_csv = True + + +# ressources +cfg.find_unused_parameters = False +cfg.grad_accumulation = 1 +cfg.syncbn = False +cfg.gpu = 0 +cfg.dp = False +cfg.num_workers = 8 +cfg.drop_last = True +cfg.save_checkpoint = True +cfg.save_only_last_ckpt = False +cfg.save_weights_only = False + +# logging, +cfg.neptune_project = None +cfg.neptune_connection_mode = "debug" +cfg.save_first_batch = False +cfg.save_first_batch_preds = False +cfg.clip_mode = "norm" +cfg.data_sample = -1 +cfg.track_grad_norm = True +cfg.grad_norm_type = 2.0 +cfg.track_weight_norm = True +cfg.norm_eps = 1e-4 +cfg.disable_tqdm = False + + +# paths + +cfg.data_folder = "/mount/cryo/data/czii-cryo-et-object-identification/train/static/ExperimentRuns/" +cfg.train_df = "train_folded_v1.csv" + + +# stages +cfg.test = False +cfg.train = True +cfg.train_val = False + +# logging +cfg.neptune_project = None +cfg.neptune_connection_mode = "async" + +# model +cfg.model = "mdl_1" +cfg.mixup_p = 1.0 +cfg.mixup_beta = 1.0 +cfg.in_channels = 1 +cfg.pretrained = False + +# data +cfg.dataset = "ds_1" +cfg.classes = ["apo-ferritin", "beta-amylase", "beta-galactosidase", "ribosome", "thyroglobulin", "virus-like-particle"] +cfg.n_classes = len(cfg.classes) + +cfg.post_process_pipeline = "pp_1" +cfg.metric = "metric_1" + + +cfg.particle_radi = { + "apo-ferritin": 60, + "beta-amylase": 65, + "beta-galactosidase": 90, + "ribosome": 150, + "thyroglobulin": 130, + "virus-like-particle": 135, +} + +cfg.voxel_spacing = 10.0 + + +# OPTIMIZATION & SCHEDULE + +cfg.fold = 0 +cfg.epochs = 10 + +cfg.lr = 1e-3 +cfg.optimizer = "Adam" +cfg.weight_decay = 0.0 +cfg.warmup = 0.0 +cfg.batch_size = 8 +cfg.batch_size_val = 16 +cfg.sub_batch_size = 4 +cfg.roi_size = [96, 96, 96] +cfg.train_sub_epochs = 1112 +cfg.val_sub_epochs = 1 +cfg.mixed_precision = False +cfg.bf16 = True +cfg.force_fp16 = True +cfg.pin_memory = False +cfg.grad_accumulation = 1.0 +cfg.num_workers = 8 + + +# Saving +cfg.save_weights_only = True +cfg.save_only_last_ckpt = False +cfg.save_val_data = False +cfg.save_checkpoint = True +cfg.save_pp_csv = False + + +cfg.static_transforms = static_transforms = mt.Compose( + [ + mt.EnsureChannelFirstd(keys=["image"], channel_dim="no_channel"), + mt.NormalizeIntensityd(keys="image"), + ] +) +cfg.train_aug = mt.Compose( + [ + mt.RandSpatialCropSamplesd(keys=["image", "label"], roi_size=cfg.roi_size, num_samples=cfg.sub_batch_size), + mt.RandFlipd( + keys=["image", "label"], + prob=0.5, + spatial_axis=0, + ), + mt.RandFlipd( + keys=["image", "label"], + prob=0.5, + spatial_axis=1, + ), + mt.RandFlipd( + keys=["image", "label"], + prob=0.5, + spatial_axis=2, + ), + mt.RandRotate90d( + keys=["image", "label"], + prob=0.75, + max_k=3, + spatial_axes=(0, 1), + ), + mt.RandRotated( + keys=["image", "label"], prob=0.5, range_x=0.78, range_y=0.0, range_z=0.0, padding_mode="reflection" + ), + ] +) + +cfg.val_aug = mt.Compose([mt.GridPatchd(keys=["image", "label"], patch_size=cfg.roi_size, pad_mode="reflect")]) + + +basic_cfg = cfg diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/data/ds_1.py b/competitions/kaggle/Cryo-ET/1st_place_solution/data/ds_1.py new file mode 100644 index 000000000..1c56f7563 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/data/ds_1.py @@ -0,0 +1,102 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +import os +import torch +import numpy as np +from torch.utils.data import Dataset, DataLoader +import zarr +from tqdm import tqdm + + +def batch_to_device(batch, device): + batch_dict = {key: batch[key].to(device) for key in batch} + return batch_dict + + +def collate_fn(batch): + + keys = batch[0].keys() + batch_dict = {key: torch.cat([b[key] for b in batch]) for key in keys} + return batch_dict + + +tr_collate_fn = collate_fn +val_collate_fn = collate_fn + +import monai.data as md +import monai.transforms as mt + + +class CustomDataset(Dataset): + def __init__(self, df, cfg, aug, mode="train"): + + self.cfg = cfg + self.mode = mode + self.df = df + self.experiment_df = self.df.drop_duplicates(subset="experiment").copy() + self.exp_dict = self.df.groupby("experiment") + self.class2id = {c: i for i, c in enumerate(cfg.classes)} + self.n_classes = len(cfg.classes) + self.data_folder = cfg.data_folder + + self.random_transforms = aug + data = [self.load_one(img_id) for img_id in tqdm(self.experiment_df["experiment"].values)] + data = md.CacheDataset(data=data, transform=cfg.static_transforms, cache_rate=1.0) + + if self.mode == "train": + self.monai_ds = md.Dataset(data=data, transform=self.random_transforms) + self.sub_epochs = cfg.train_sub_epochs + self.len = len(self.monai_ds) * self.sub_epochs + else: + self.monai_ds = md.CacheDataset(data=data, transform=self.random_transforms, cache_rate=1.0)[0] + self.sub_epochs = cfg.val_sub_epochs + self.len = len(self.monai_ds["image"]) + + def __getitem__(self, idx): + + if self.mode == "train": + monai_dict = self.monai_ds[idx // self.sub_epochs] + feature_dict = { + "input": torch.stack([item["image"] for item in monai_dict]), + "target": torch.stack([item["label"] for item in monai_dict]), + } + + else: + monai_dict = {k: self.monai_ds[k][idx] for k in self.monai_ds} + monai_dict["location"] = torch.from_numpy(self.monai_ds["image"].meta["location"][:, idx]) + feature_dict = { + "input": torch.stack([item["image"] for item in [monai_dict]]), + "location": torch.stack([item["location"] for item in [monai_dict]]), + "target": torch.stack([item["label"] for item in [monai_dict]]), + } + + return feature_dict + + def __len__(self): + return self.len + + def load_one(self, experiment_id): + + img_fp = f"{self.data_folder}{experiment_id}" + try: + with zarr.open(img_fp + "/VoxelSpacing10.000/denoised.zarr") as zf: + img = np.array(zf[0]).transpose(2, 1, 0) + # img = np.array(zarr.open(img_fp + '/VoxelSpacing10.000/denoised.zarr')[0]).transpose(2,1,0) + except Exception as e: + print(e) + + centers = self.exp_dict.get_group(experiment_id)[["x", "y", "z"]].values / 10 + classes = self.exp_dict.get_group(experiment_id)["particle_type"].map(self.class2id).values + mask = np.zeros((self.n_classes,) + img.shape[-3:]) + mask[classes, centers[:, 0].astype(int), centers[:, 1].astype(int), centers[:, 2].astype(int)] = 1 + return {"image": img, "label": mask} diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/metrics/metric_1.py b/competitions/kaggle/Cryo-ET/1st_place_solution/metrics/metric_1.py new file mode 100644 index 000000000..1d539d8ad --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/metrics/metric_1.py @@ -0,0 +1,201 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import numpy as np +import torch +from sklearn.metrics import roc_auc_score +import pandas as pd +from scipy.spatial import KDTree + + +class ParticipantVisibleError(Exception): + pass + + +def compute_metrics(reference_points, reference_radius, candidate_points): + num_reference_particles = len(reference_points) + num_candidate_particles = len(candidate_points) + + if len(reference_points) == 0: + return 0, num_candidate_particles, 0 + + if len(candidate_points) == 0: + return 0, 0, num_reference_particles + + ref_tree = KDTree(reference_points) + candidate_tree = KDTree(candidate_points) + raw_matches = candidate_tree.query_ball_tree(ref_tree, r=reference_radius) + matches_within_threshold = [] + for match in raw_matches: + matches_within_threshold.extend(match) + # Prevent submitting multiple matches per particle. + # This won't be be strictly correct in the (extremely rare) case where true particles + # are very close to each other. + matches_within_threshold = set(matches_within_threshold) + tp = int(len(matches_within_threshold)) + fp = int(num_candidate_particles - tp) + fn = int(num_reference_particles - tp) + return tp, fp, fn + + +def score( + solution: pd.DataFrame, + submission: pd.DataFrame, + row_id_column_name: str, + distance_multiplier: float, + beta: int, + weighted=True, +) -> float: + """ + F_beta + - a true positive occurs when + - (a) the predicted location is within a threshold of the particle radius, and + - (b) the correct `particle_type` is specified + - raw results (TP, FP, FN) are aggregated across all experiments for each particle type + - f_beta is calculated for each particle type + - individual f_beta scores are weighted by particle type for final score + """ + + particle_radius = { + "apo-ferritin": 60, + "beta-amylase": 65, + "beta-galactosidase": 90, + "ribosome": 150, + "thyroglobulin": 130, + "virus-like-particle": 135, + } + + weights = { + "apo-ferritin": 1, + "beta-amylase": 0, + "beta-galactosidase": 2, + "ribosome": 1, + "thyroglobulin": 2, + "virus-like-particle": 1, + } + + particle_radius = {k: v * distance_multiplier for k, v in particle_radius.items()} + + # Filter submission to only contain experiments found in the solution split + split_experiments = set(solution["experiment"].unique()) + submission = submission.loc[submission["experiment"].isin(split_experiments)] + + # Only allow known particle types + if not set(submission["particle_type"].unique()).issubset(set(weights.keys())): + raise ParticipantVisibleError("Unrecognized `particle_type`.") + + assert solution.duplicated(subset=["experiment", "x", "y", "z"]).sum() == 0 + assert particle_radius.keys() == weights.keys() + + results = {} + for particle_type in solution["particle_type"].unique(): + results[particle_type] = { + "total_tp": 0, + "total_fp": 0, + "total_fn": 0, + } + + for experiment in split_experiments: + for particle_type in solution["particle_type"].unique(): + reference_radius = particle_radius[particle_type] + select = (solution["experiment"] == experiment) & (solution["particle_type"] == particle_type) + reference_points = solution.loc[select, ["x", "y", "z"]].values + + select = (submission["experiment"] == experiment) & (submission["particle_type"] == particle_type) + candidate_points = submission.loc[select, ["x", "y", "z"]].values + + if len(reference_points) == 0: + reference_points = np.array([]) + reference_radius = 1 + + if len(candidate_points) == 0: + candidate_points = np.array([]) + + tp, fp, fn = compute_metrics(reference_points, reference_radius, candidate_points) + + results[particle_type]["total_tp"] += tp + results[particle_type]["total_fp"] += fp + results[particle_type]["total_fn"] += fn + + fbetas = [] + fbeta_weights = [] + particle_types = [] + for particle_type, totals in results.items(): + tp = totals["total_tp"] + fp = totals["total_fp"] + fn = totals["total_fn"] + + precision = tp / (tp + fp) if tp + fp > 0 else 0 + recall = tp / (tp + fn) if tp + fn > 0 else 0 + fbeta = ( + (1 + beta**2) * (precision * recall) / (beta**2 * precision + recall) if (precision + recall) > 0 else 0.0 + ) + fbetas += [fbeta] + fbeta_weights += [weights.get(particle_type, 1.0)] + particle_types += [particle_type] + + if weighted: + aggregate_fbeta = np.average(fbetas, weights=fbeta_weights) + else: + aggregate_fbeta = np.mean(fbetas) + + return aggregate_fbeta, dict(zip(particle_types, fbetas)) + + +def calc_metric(cfg, pp_out, val_df, pre="val"): + + particles = cfg.classes + pred_df = pp_out + + solution = val_df.copy() + solution["id"] = range(len(solution)) + + submission = pred_df.copy() + submission["experiment"] = solution["experiment"].unique()[0] + submission["id"] = range(len(submission)) + + best_ths = [] + for p in particles: + sol0a = solution[solution["particle_type"] == p].copy() + sub0a = submission[submission["particle_type"] == p].copy() + scores = [] + ths = np.arange(0, 0.5, 0.005) + for c in ths: + scores += [ + score( + sol0a.copy(), + sub0a[sub0a["conf"] > c].copy(), + row_id_column_name="id", + distance_multiplier=0.5, + beta=4, + weighted=False, + )[0] + ] + best_th = ths[np.argmax(scores)] + best_ths += [best_th] + + submission_pp = [] + for th, p in zip(best_ths, particles): + submission_pp += [submission[(submission["particle_type"] == p) & (submission["conf"] > th)].copy()] + submission_pp = pd.concat(submission_pp) + + score_pp, particle_scores = score( + solution[solution["particle_type"] != "beta-amylase"].copy(), + submission_pp.copy(), + row_id_column_name="id", + distance_multiplier=0.5, + beta=4, + ) + + result = {"score_" + k: v for k, v in particle_scores.items()} + result["score"] = score_pp + # print(result) + return result diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/models/mdl_1.py b/competitions/kaggle/Cryo-ET/1st_place_solution/models/mdl_1.py new file mode 100644 index 000000000..b8c871d4a --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/models/mdl_1.py @@ -0,0 +1,327 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import torch +import torch.nn.functional as F +from torch import nn +from torch.nn.parameter import Parameter +from torch.distributions import Beta +from monai.networks.nets.flexible_unet import SegmentationHead, UNetDecoder, FLEXUNET_BACKBONE + + +class PatchedUNetDecoder(UNetDecoder): + """add functionality to output all feature maps""" + + def forward(self, features: list[torch.Tensor], skip_connect: int = 4): + skips = features[:-1][::-1] + features = features[1:][::-1] + + out = [] + x = features[0] + out += [x] + for i, block in enumerate(self.blocks): + if i < skip_connect: + skip = skips[i] + else: + skip = None + x = block(x, skip) + out += [x] + return out + + +class FlexibleUNet(nn.Module): + """ + A flexible implementation of UNet-like encoder-decoder architecture. + + (Adjusted to support PatchDecoder and multi segmentation heads) + """ + + def __init__( + self, + in_channels: int, + out_channels: int, + backbone: str, + pretrained: bool = False, + decoder_channels: tuple = (256, 128, 64, 32, 16), + spatial_dims: int = 2, + norm: str | tuple = ("batch", {"eps": 1e-3, "momentum": 0.1}), + act: str | tuple = ("relu", {"inplace": True}), + dropout: float | tuple = 0.0, + decoder_bias: bool = False, + upsample: str = "nontrainable", + pre_conv: str = "default", + interp_mode: str = "nearest", + is_pad: bool = True, + ) -> None: + """ + A flexible implement of UNet, in which the backbone/encoder can be replaced with + any efficient or residual network. Currently the input must have a 2 or 3 spatial dimension + and the spatial size of each dimension must be a multiple of 32 if is_pad parameter + is False. + Please notice each output of backbone must be 2x downsample in spatial dimension + of last output. For example, if given a 512x256 2D image and a backbone with 4 outputs. + Spatial size of each encoder output should be 256x128, 128x64, 64x32 and 32x16. + + Args: + in_channels: number of input channels. + out_channels: number of output channels. + backbone: name of backbones to initialize, only support efficientnet and resnet right now, + can be from [efficientnet-b0, ..., efficientnet-b8, efficientnet-l2, resnet10, ..., resnet200]. + pretrained: whether to initialize pretrained weights. ImageNet weights are available for efficient networks + if spatial_dims=2 and batch norm is used. MedicalNet weights are available for residual networks + if spatial_dims=3 and in_channels=1. Default to False. + decoder_channels: number of output channels for all feature maps in decoder. + `len(decoder_channels)` should equal to `len(encoder_channels) - 1`,default + to (256, 128, 64, 32, 16). + spatial_dims: number of spatial dimensions, default to 2. + norm: normalization type and arguments, default to ("batch", {"eps": 1e-3, + "momentum": 0.1}). + act: activation type and arguments, default to ("relu", {"inplace": True}). + dropout: dropout ratio, default to 0.0. + decoder_bias: whether to have a bias term in decoder's convolution blocks. + upsample: upsampling mode, available options are``"deconv"``, ``"pixelshuffle"``, + ``"nontrainable"``. + pre_conv:a conv block applied before upsampling. Only used in the "nontrainable" or + "pixelshuffle" mode, default to `default`. + interp_mode: {``"nearest"``, ``"linear"``, ``"bilinear"``, ``"bicubic"``, ``"trilinear"``} + Only used in the "nontrainable" mode. + is_pad: whether to pad upsampling features to fit features from encoder. Default to True. + If this parameter is set to "True", the spatial dim of network input can be arbitrary + size, which is not supported by TensorRT. Otherwise, it must be a multiple of 32. + """ + super().__init__() + + if backbone not in FLEXUNET_BACKBONE.register_dict: + raise ValueError( + f"invalid model_name {backbone} found, must be one of {FLEXUNET_BACKBONE.register_dict.keys()}." + ) + + if spatial_dims not in (2, 3): + raise ValueError("spatial_dims can only be 2 or 3.") + + encoder = FLEXUNET_BACKBONE.register_dict[backbone] + self.backbone = backbone + self.spatial_dims = spatial_dims + encoder_parameters = encoder["parameter"] + if not ( + ("spatial_dims" in encoder_parameters) + and ("in_channels" in encoder_parameters) + and ("pretrained" in encoder_parameters) + ): + raise ValueError("The backbone init method must have spatial_dims, in_channels and pretrained parameters.") + encoder_feature_num = encoder["feature_number"] + if encoder_feature_num > 5: + raise ValueError("Flexible unet can only accept no more than 5 encoder feature maps.") + + decoder_channels = decoder_channels[:encoder_feature_num] + self.skip_connect = encoder_feature_num - 1 + encoder_parameters.update({"spatial_dims": spatial_dims, "in_channels": in_channels, "pretrained": pretrained}) + encoder_channels = tuple([in_channels] + list(encoder["feature_channel"])) + encoder_type = encoder["type"] + self.encoder = encoder_type(**encoder_parameters) + print(decoder_channels) + + self.decoder = PatchedUNetDecoder( + spatial_dims=spatial_dims, + encoder_channels=encoder_channels, + decoder_channels=decoder_channels, + act=act, + norm=norm, + dropout=dropout, + bias=decoder_bias, + upsample=upsample, + interp_mode=interp_mode, + pre_conv=pre_conv, + align_corners=None, + is_pad=is_pad, + ) + self.segmentation_heads = nn.ModuleList( + [ + SegmentationHead( + spatial_dims=spatial_dims, + in_channels=decoder_channel, + out_channels=out_channels + 1, + kernel_size=3, + act=None, + ) + for decoder_channel in decoder_channels[:-1] + ] + ) + + def forward(self, inputs: torch.Tensor): + + x = inputs + enc_out = self.encoder(x) + decoder_out = self.decoder(enc_out, self.skip_connect)[1:-1] + x_seg = [self.segmentation_heads[i](decoder_out[i]) for i in range(len(decoder_out))] + + return x_seg + + +def count_parameters(model): + return sum(p.numel() for p in model.parameters() if p.requires_grad) + + +def human_format(num): + num = float("{:.3g}".format(num)) + magnitude = 0 + while abs(num) >= 1000: + magnitude += 1 + num /= 1000.0 + return "{}{}".format("{:f}".format(num).rstrip("0").rstrip("."), ["", "K", "M", "B", "T"][magnitude]) + + +class DenseCrossEntropy(nn.Module): + def __init__(self, class_weights=None): + super(DenseCrossEntropy, self).__init__() + + self.class_weights = class_weights + + def forward(self, x, target): + x = x.float() + target = target.float() + logprobs = torch.nn.functional.log_softmax(x, dim=1, dtype=torch.float) + + loss = -logprobs * target + + class_losses = loss.mean((0, 2, 3, 4)) + if self.class_weights is not None: + loss = (class_losses * self.class_weights.to(class_losses.device)).sum() # / class_weights.sum() + else: + + loss = class_losses.sum() + return loss, class_losses + + +class Mixup(nn.Module): + def __init__(self, mix_beta, mixadd=False): + + super(Mixup, self).__init__() + self.beta_distribution = Beta(mix_beta, mix_beta) + self.mixadd = mixadd + + def forward(self, X, Y, Z=None): + + bs = X.shape[0] + n_dims = len(X.shape) + perm = torch.randperm(bs) + coeffs = self.beta_distribution.rsample(torch.Size((bs,))).to(X.device) + X_coeffs = coeffs.view((-1,) + (1,) * (X.ndim - 1)) + Y_coeffs = coeffs.view((-1,) + (1,) * (Y.ndim - 1)) + + X = X_coeffs * X + (1 - X_coeffs) * X[perm] + + if self.mixadd: + Y = (Y + Y[perm]).clip(0, 1) + else: + Y = Y_coeffs * Y + (1 - Y_coeffs) * Y[perm] + + if Z: + return X, Y, Z + + return X, Y + + +def to_ce_target(y): + # bs, c, h, w, d + y_bg = 1 - y.sum(1, keepdim=True).clamp(0, 1) + y = torch.cat([y, y_bg], 1) + y = y / y.sum(1, keepdim=True) + return y + + +class Net(nn.Module): + + def __init__(self, cfg): + super(Net, self).__init__() + + self.cfg = cfg + self.n_classes = cfg.n_classes + self.classes = cfg.classes + + self.backbone = FlexibleUNet(**cfg.backbone_args) + + self.mixup = Mixup(cfg.mixup_beta) + + print(f"Net parameters: {human_format(count_parameters(self))}") + self.lvl_weights = torch.from_numpy(cfg.lvl_weights) + self.loss_fn = DenseCrossEntropy(class_weights=torch.from_numpy(cfg.class_weights)) + + def forward(self, batch): + + x = batch["input"] + if "target" in batch.keys(): + y = batch["target"] + if self.training: + if torch.rand(1)[0] < self.cfg.mixup_p: + x, y = self.mixup(x, y) + out = self.backbone(x) + + outputs = {} + + if "target" in batch.keys(): + ys = [F.adaptive_max_pool3d(y, item.shape[-3:]) for item in out] + losses = torch.stack([self.loss_fn(out[i], to_ce_target(ys[i]))[0] for i in range(len(out))]) + lvl_weights = self.lvl_weights.to(losses.device) + loss = (losses * lvl_weights).sum() / lvl_weights.sum() + outputs["loss"] = loss + if not self.training: + outputs["logits"] = out[-1] + if "location" in batch: + outputs["location"] = batch["location"] + + return outputs + + +class TestNet(nn.Module): + + def __init__(self, **backbone_args): + super(TestNet, self).__init__() + + self.backbone = FlexibleUNet(**backbone_args) + + def forward(self, x): + # x shape is bs, c, h, w, d + out = self.backbone(x) + # out shape is bs, 7, h//2, w//2, d//2 + logits = out[-1] # for heatmap do softmax + reorder classes .softmax(1)[:,[0,2,3,4,5,1]] + return logits + + +# import torch +# import monai.transforms as mt +# import zarr +# import numpy as np + +# def preprocess_img(zarr_fn, transforms): +# img = np.array(zarr.open(zarr_fn)[0]).transpose(2,1,0) +# img = transforms({'image':img})['image'] +# return img + +# backbone_args = dict(spatial_dims=3, +# in_channels=1, +# out_channels=6, +# backbone='resnet34', +# pretrained=False) + +# net = TestNet(**backbone_args) +# sd = torch.load(CKPT_FILE)['model'] +# net.eval().cuda() +# net.load_state_dict(sd) + +# static_transforms = mt.Compose([mt.EnsureChannelFirstd(keys=["image"], channel_dim="no_channel"),mt.NormalizeIntensityd(keys="image"),]) +# zarr_fn = '/mount/cryo/data/czii-cryo-et-object-identification/train/static/ExperimentRuns/TS_5_4/VoxelSpacing10.000/denoised.zarr' +# img = preprocess_img(zarr_fn,static_transforms) # torch.Size([1, 630, 630, 184]) + +# patch = img[None, :, :96, :96, :96] # torch.Size([1, ,1 96, 96, 96]) + +# logits = net(patch.cuda()) +# proba_heatmap = logits.softmax(1)[:,[0,2,3,4,5,1]] diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/partly_Unet.png b/competitions/kaggle/Cryo-ET/1st_place_solution/partly_Unet.png new file mode 100644 index 000000000..7f4f72cdb Binary files /dev/null and b/competitions/kaggle/Cryo-ET/1st_place_solution/partly_Unet.png differ diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/postprocess/pp_1.py b/competitions/kaggle/Cryo-ET/1st_place_solution/postprocess/pp_1.py new file mode 100644 index 000000000..d0dee3e35 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/postprocess/pp_1.py @@ -0,0 +1,78 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import pandas as pd +import torch +from torch.nn import functional as F +from tqdm import tqdm + +import torch +from torch import nn + + +def simple_nms(scores, nms_radius: int): + """Fast Non-maximum suppression to remove nearby points""" + assert nms_radius >= 0 + + def max_pool(x): + return torch.nn.functional.max_pool3d(x, kernel_size=nms_radius * 2 + 1, stride=1, padding=nms_radius) + + zeros = torch.zeros_like(scores) + max_mask = scores == max_pool(scores) + return torch.where(max_mask, scores, zeros) + + +def reconstruct(img, locations, out_size, crop_size): + reconstructed_img = torch.zeros(out_size) + + for i in range(img.shape[0]): + reconstructed_img[ + :, + locations[0][i] : locations[0][i] + crop_size[0], + locations[1][i] : locations[1][i] + crop_size[1], + locations[2][i] : locations[2][i] + crop_size[2], + ] = img[i, :] + return reconstructed_img + + +def post_process_pipeline(cfg, val_data, val_df): + + img = val_data["logits"] + img = torch.nn.functional.interpolate( + img, size=(cfg.roi_size[0], cfg.roi_size[1], cfg.roi_size[2]), mode="trilinear", align_corners=False + ) + locations = val_data["location"] + out_size = [cfg.n_classes + 1] + [l.item() + r for l, r in zip(locations.max(0)[0], cfg.roi_size)] + rec_img = reconstruct(img, locations.permute(1, 0), out_size=out_size, crop_size=cfg.roi_size) + s = rec_img.shape[-3:] + rec_img = torch.nn.functional.interpolate( + rec_img[None], size=(s[0] // 2, s[1] // 2, s[2] // 2), mode="trilinear", align_corners=False + )[0] + preds = rec_img.softmax(0)[:-1] + + pred_df = [] + + for i, p in enumerate(cfg.classes): + p1 = preds[i][None,].cuda() + y = simple_nms(p1, nms_radius=int(0.5 * cfg.particle_radi[p] / 10)) + kps = torch.where(y > 0) + xyz = torch.stack(kps[1:], -1) * 10 * 2 + conf = y[kps] + pred_df_ = pd.DataFrame(xyz.cpu().numpy(), columns=["x", "y", "z"]) + pred_df_["particle_type"] = p + pred_df_["conf"] = conf.cpu().numpy() + pred_df += [pred_df_] + pred_df = pd.concat(pred_df) + pred_df = pred_df[ + (pred_df["x"] < 6300) & (pred_df["y"] < 6300) & (pred_df["z"] < 1840) & (pred_df["conf"] > 0.01) + ].copy() + pred_df.to_csv(f"{cfg.output_dir}/fold{cfg.fold}/val_pred_df_seed{cfg.seed}.csv", index=False) + return pred_df diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/requirements.txt b/competitions/kaggle/Cryo-ET/1st_place_solution/requirements.txt new file mode 100644 index 000000000..65331e998 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/requirements.txt @@ -0,0 +1,2 @@ +zarr +monai==1.4.0 diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/train.py b/competitions/kaggle/Cryo-ET/1st_place_solution/train.py new file mode 100644 index 000000000..c2e3589e2 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/train.py @@ -0,0 +1,438 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +import numpy as np +import pandas as pd +import importlib +import sys +from tqdm import tqdm +import gc +import argparse +import torch +import math + +try: + from torch.amp import GradScaler, autocast +except: + from torch.cuda.amp import GradScaler, autocast +from torch.nn.parallel import DistributedDataParallel as NativeDDP +from collections import defaultdict + +from utils import ( + sync_across_gpus, + set_seed, + get_model, + create_checkpoint, + load_checkpoint, + get_data, + get_dataset, + get_dataloader, + calc_grad_norm, + calc_weight_norm, +) +from utils import ( + get_optimizer, + get_scheduler, +) + + +from copy import copy +import os + +os.environ["OMP_NUM_THREADS"] = "1" +os.environ["MKL_NUM_THREADS"] = "1" +os.environ["OPENBLAS_NUM_THREADS"] = "1" +os.environ["VECLIB_MAXIMUM_THREADS"] = "1" +os.environ["NUMEXPR_NUM_THREADS"] = "1" +os.environ["TOKENIZERS_PARALLELISM"] = "false" + +try: + import cv2 + + cv2.setNumThreads(0) +except: + print("no cv2 installed, running without") + + +sys.path.append("configs") +sys.path.append("models") +sys.path.append("data") +sys.path.append("postprocess") +sys.path.append("metrics") + + +def run_eval(model, val_dataloader, cfg, pre="val", curr_epoch=0): + saved_images = False + model.eval() + torch.set_grad_enabled(False) + + # store information for evaluation + val_data = defaultdict(list) + val_score = 0 + for ind_, data in enumerate(tqdm(val_dataloader, disable=(cfg.local_rank != 0) | cfg.disable_tqdm)): + + batch = cfg.batch_to_device(data, cfg.device) + + if cfg.mixed_precision: + with autocast("cuda"): + output = model(batch) + else: + output = model(batch) + + if (cfg.local_rank == 0) and (cfg.calc_metric) and (((curr_epoch + 1) % cfg.calc_metric_epochs) == 0): + # per batch calculations + pass + + if (not saved_images) & (cfg.save_first_batch_preds): + save_first_batch_preds(batch, output, cfg) + saved_images = True + + for key, val in output.items(): + val_data[key] += [output[key]] + + for key, val in output.items(): + value = val_data[key] + if isinstance(value[0], list): + val_data[key] = [item for sublist in value for item in sublist] + + else: + if len(value[0].shape) == 0: + val_data[key] = torch.stack(value) + else: + val_data[key] = torch.cat(value, dim=0) + + if (cfg.local_rank == 0) and (cfg.calc_metric) and (((curr_epoch + 1) % cfg.calc_metric_epochs) == 0): + pass + + if cfg.distributed and cfg.eval_ddp: + for key, val in output.items(): + val_data[key] = sync_across_gpus(val_data[key], cfg.world_size) + + if cfg.local_rank == 0: + if cfg.save_val_data: + if cfg.distributed: + for k, v in val_data.items(): + val_data[k] = v[: len(val_dataloader.dataset)] + torch.save(val_data, f"{cfg.output_dir}/fold{cfg.fold}/{pre}_data_seed{cfg.seed}.pth") + + loss_names = [key for key in output if "loss" in key] + loss_names += [key for key in output if "score" in key] + for k in loss_names: + if cfg.local_rank == 0 and k in val_data: + losses = val_data[k].cpu().numpy() + loss = np.mean(losses) + + print(f"Mean {pre}_{k}", loss) + + if (cfg.local_rank == 0) and (cfg.calc_metric) and (((curr_epoch + 1) % cfg.calc_metric_epochs) == 0): + + val_df = val_dataloader.dataset.df + pp_out = cfg.post_process_pipeline(cfg, val_data, val_df) + val_score = cfg.calc_metric(cfg, pp_out, val_df, pre) + if type(val_score) != dict: + val_score = {f"score": val_score} + + for k, v in val_score.items(): + print(f"{pre}_{k}: {v:.3f}") + + if cfg.distributed: + torch.distributed.barrier() + + # print("EVAL FINISHED") + + return val_score + + +def train(cfg): + # set seed + if cfg.seed < 0: + cfg.seed = np.random.randint(1_000_000) + print("seed", cfg.seed) + + if cfg.distributed: + + cfg.local_rank = int(os.environ["LOCAL_RANK"]) + device = "cuda:%d" % cfg.local_rank + cfg.device = device + + torch.cuda.set_device(cfg.local_rank) + torch.distributed.init_process_group(backend="nccl", init_method="env://") + cfg.world_size = torch.distributed.get_world_size() + cfg.rank = torch.distributed.get_rank() + # print("Training in distributed mode with multiple processes, 1 GPU per process.") + print(f"Process {cfg.rank}, total {cfg.world_size}, local rank {cfg.local_rank}.") + cfg.group = torch.distributed.new_group(np.arange(cfg.world_size)) + # print("Group", cfg.group) + + # syncing the random seed + cfg.seed = int( + sync_across_gpus(torch.Tensor([cfg.seed]).to(device), cfg.world_size).detach().cpu().numpy()[0] + ) # + + print(f"LOCAL_RANK {cfg.local_rank}, device {device}, seed {cfg.seed}") + + else: + cfg.local_rank = 0 + cfg.world_size = 1 + cfg.rank = 0 # global rank + + device = "cuda:%d" % cfg.gpu + cfg.device = device + + set_seed(cfg.seed) + + train_df, val_df, test_df = get_data(cfg) + + train_dataset = get_dataset(train_df, cfg, mode="train") + train_dataloader = get_dataloader(train_dataset, cfg, mode="train") + + val_dataset = get_dataset(val_df, cfg, mode="val") + val_dataloader = get_dataloader(val_dataset, cfg, mode="val") + + if cfg.test: + test_dataset = get_dataset(test_df, cfg, mode="test") + test_dataloader = get_dataloader(test_dataset, cfg, mode="test") + + if cfg.train_val: + train_val_dataset = get_dataset(train_df, cfg, mode="val") + train_val_dataloader = get_dataloader(train_val_dataset, cfg, "val") + + model = get_model(cfg, train_dataset) + if cfg.compile_model: + print("compiling model") + model = torch.compile(model) + model.to(device) + + if cfg.distributed: + + if cfg.syncbn: + model = torch.nn.SyncBatchNorm.convert_sync_batchnorm(model) + + model = NativeDDP(model, device_ids=[cfg.local_rank], find_unused_parameters=cfg.find_unused_parameters) + + total_steps = len(train_dataset) + if train_dataloader.sampler is not None: + if "WeightedRandomSampler" in str(train_dataloader.sampler.__class__): + total_steps = train_dataloader.sampler.num_samples + + optimizer = get_optimizer(model, cfg) + scheduler = get_scheduler(cfg, optimizer, total_steps) + + if cfg.mixed_precision: + scaler = GradScaler() + else: + scaler = None + + cfg.curr_step = 0 + i = 0 + best_val_loss = np.inf + optimizer.zero_grad() + total_grad_norm = None + total_weight_norm = None + total_grad_norm_after_clip = None + + for epoch in range(cfg.epochs): + + set_seed(cfg.seed + epoch + cfg.local_rank) + + cfg.curr_epoch = epoch + if cfg.local_rank == 0: + print("EPOCH:", epoch) + + if cfg.distributed: + train_dataloader.sampler.set_epoch(epoch) + + progress_bar = tqdm(range(len(train_dataloader)), disable=cfg.disable_tqdm) + tr_it = iter(train_dataloader) + + losses = [] + + gc.collect() + + if cfg.train: + # ==== TRAIN LOOP + for itr in progress_bar: + i += 1 + + cfg.curr_step += cfg.batch_size * cfg.world_size + + try: + data = next(tr_it) + except Exception as e: + print(e) + print("DATA FETCH ERROR") + # continue + + model.train() + torch.set_grad_enabled(True) + + batch = cfg.batch_to_device(data, device) + + if cfg.mixed_precision: + with autocast("cuda"): + output_dict = model(batch) + else: + if cfg.bf16: + with autocast("cuda", dtype=torch.bfloat16): + output_dict = model(batch) + else: + output_dict = model(batch) + + loss = output_dict["loss"] + + losses.append(loss.item()) + + if cfg.grad_accumulation > 1: + loss /= cfg.grad_accumulation + + # Backward pass + + if cfg.mixed_precision: + scaler.scale(loss).backward() + + if i % cfg.grad_accumulation == 0: + if (cfg.track_grad_norm) or (cfg.clip_grad > 0): + scaler.unscale_(optimizer) + if cfg.track_grad_norm: + total_grad_norm = calc_grad_norm(model.parameters(), cfg.grad_norm_type) + if cfg.clip_grad > 0: + torch.nn.utils.clip_grad_norm_(model.parameters(), cfg.clip_grad) + if cfg.track_grad_norm: + total_grad_norm_after_clip = calc_grad_norm(model.parameters(), cfg.grad_norm_type) + scaler.step(optimizer) + scaler.update() + optimizer.zero_grad() + + else: + + loss.backward() + if i % cfg.grad_accumulation == 0: + if cfg.track_grad_norm: + total_grad_norm = calc_grad_norm(model.parameters()) + if cfg.clip_grad > 0: + torch.nn.utils.clip_grad_norm_(model.parameters(), cfg.clip_grad) + if cfg.track_grad_norm: + total_grad_norm_after_clip = calc_grad_norm(model.parameters(), cfg.grad_norm_type) + if cfg.track_weight_norm: + total_weight_norm = calc_weight_norm(model.parameters(), cfg.grad_norm_type) + optimizer.step() + optimizer.zero_grad() + # print(optimizer.state_dict()) + # break + + if cfg.distributed: + torch.cuda.synchronize() + + if scheduler is not None: + scheduler.step() + + if cfg.local_rank == 0 and cfg.curr_step % cfg.batch_size == 0: + + loss_names = [key for key in output_dict if "loss" in key] + + progress_bar.set_description(f"loss: {np.mean(losses[-10:]):.4f}") + + if cfg.eval_steps != 0: + if i % cfg.eval_steps == 0: + if cfg.distributed and cfg.eval_ddp: + val_loss = run_eval(model, val_dataloader, cfg, pre="val", curr_epoch=epoch) + else: + if cfg.local_rank == 0: + val_loss = run_eval(model, val_dataloader, cfg, pre="val", curr_epoch=epoch) + else: + val_score = 0 + + print(f"Mean train_loss {np.mean(losses):.4f}") + + if cfg.distributed: + torch.cuda.synchronize() + if cfg.force_fp16: + model = model.half().float() + if cfg.val: + + if (epoch + 1) % cfg.eval_epochs == 0 or (epoch + 1) == cfg.epochs: + if cfg.distributed and cfg.eval_ddp: + val_score = run_eval(model, val_dataloader, cfg, pre="val", curr_epoch=epoch) + else: + if cfg.local_rank == 0: + val_score = run_eval(model, val_dataloader, cfg, pre="val", curr_epoch=epoch) + else: + val_score = 0 + + if cfg.train_val == True: + if (epoch + 1) % cfg.eval_train_epochs == 0 or (epoch + 1) == cfg.epochs: + if cfg.distributed and cfg.eval_ddp: + _ = get_preds(model, train_val_dataloader, cfg, pre=cfg.pre_train_val) + + else: + if cfg.local_rank == 0: + _ = get_preds(model, train_val_dataloader, cfg, pre=cfg.pre_train_val) + + if cfg.distributed: + torch.distributed.barrier() + + if (cfg.local_rank == 0) and (cfg.epochs > 0) and (cfg.save_checkpoint): + if not cfg.save_only_last_ckpt: + checkpoint = create_checkpoint(cfg, model, optimizer, epoch, scheduler=scheduler, scaler=scaler) + + torch.save(checkpoint, f"{cfg.output_dir}/fold{cfg.fold}/checkpoint_last_seed{cfg.seed}.pth") + + if (cfg.local_rank == 0) and (cfg.epochs > 0) and (cfg.save_checkpoint): + checkpoint = create_checkpoint(cfg, model, optimizer, epoch, scheduler=scheduler, scaler=scaler) + + torch.save(checkpoint, f"{cfg.output_dir}/fold{cfg.fold}/checkpoint_last_seed{cfg.seed}.pth") + + if cfg.test: + run_eval(model, test_dataloader, test_df, cfg, pre="test") + + return val_score + + +if __name__ == "__main__": + + parser = argparse.ArgumentParser(description="") + + parser.add_argument("-C", "--config", help="config filename") + parser_args, other_args = parser.parse_known_args(sys.argv) + + cfg = copy(importlib.import_module(parser_args.config).cfg) + + # overwrite params in config with additional args + if len(other_args) > 1: + other_args = {k.replace("-", ""): v for k, v in zip(other_args[1::2], other_args[2::2])} + + for key in other_args: + if key in cfg.__dict__: + + print(f"overwriting cfg.{key}: {cfg.__dict__[key]} -> {other_args[key]}") + cfg_type = type(cfg.__dict__[key]) + if other_args[key] == "None": + cfg.__dict__[key] = None + elif cfg_type == bool: + cfg.__dict__[key] = other_args[key] == "True" + elif cfg_type == type(None): + cfg.__dict__[key] = other_args[key] + else: + cfg.__dict__[key] = cfg_type(other_args[key]) + + os.makedirs(str(cfg.output_dir + f"/fold{cfg.fold}/"), exist_ok=True) + + cfg.CustomDataset = importlib.import_module(cfg.dataset).CustomDataset + cfg.tr_collate_fn = importlib.import_module(cfg.dataset).tr_collate_fn + cfg.val_collate_fn = importlib.import_module(cfg.dataset).val_collate_fn + cfg.batch_to_device = importlib.import_module(cfg.dataset).batch_to_device + + cfg.post_process_pipeline = importlib.import_module(cfg.post_process_pipeline).post_process_pipeline + cfg.calc_metric = importlib.import_module(cfg.metric).calc_metric + + result = train(cfg) + print(result) diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/train_folded_v1.csv b/competitions/kaggle/Cryo-ET/1st_place_solution/train_folded_v1.csv new file mode 100644 index 000000000..3b4c53669 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/train_folded_v1.csv @@ -0,0 +1,1270 @@ +x,y,z,particle_type,experiment,fold +5417.091,1120.574,567.527,beta-amylase,TS_5_4,0 +4340.0,1220.0,365.0,beta-amylase,TS_5_4,0 +4908.973,1745.891,279.668,beta-amylase,TS_5_4,0 +5153.2,4595.2,420.8,beta-amylase,TS_5_4,0 +1500.0,4212.222,1016.667,beta-amylase,TS_5_4,0 +1068.16,3620.736,1033.129,beta-amylase,TS_5_4,0 +1025.185,2153.333,1215.556,beta-amylase,TS_5_4,0 +1750.0,2648.75,881.25,beta-amylase,TS_5_4,0 +2609.328,2513.109,1131.555,beta-amylase,TS_5_4,0 +3634.655,3226.691,1280.655,beta-amylase,TS_5_4,0 +1959.181,701.502,371.229,beta-galactosidase,TS_5_4,0 +2135.713,4459.195,936.655,beta-galactosidase,TS_5_4,0 +3449.782,5052.93,511.041,beta-galactosidase,TS_5_4,0 +4623.597,3652.095,1161.66,beta-galactosidase,TS_5_4,0 +1390.238,4121.667,570.595,beta-galactosidase,TS_5_4,0 +1317.617,2292.128,1089.064,beta-galactosidase,TS_5_4,0 +3540.625,1851.25,50.0,beta-galactosidase,TS_5_4,0 +5421.651,1158.868,563.302,beta-galactosidase,TS_5_4,0 +5132.863,4594.658,436.41,beta-galactosidase,TS_5_4,0 +455.241,2838.414,623.402,beta-galactosidase,TS_5_4,0 +766.848,1452.913,700.413,beta-galactosidase,TS_5_4,0 +4048.927,5220.881,1249.54,beta-galactosidase,TS_5_4,0 +4601.271,601.066,600.934,ribosome,TS_5_4,0 +4803.789,455.425,514.016,ribosome,TS_5_4,0 +4715.436,825.374,802.166,ribosome,TS_5_4,0 +5003.275,782.745,802.48,ribosome,TS_5_4,0 +710.459,3815.845,1405.435,ribosome,TS_5_4,0 +2421.852,1141.637,663.19,ribosome,TS_5_4,0 +691.64,897.383,288.35,ribosome,TS_5_4,0 +948.219,2155.294,372.13,ribosome,TS_5_4,0 +1910.761,755.539,1052.241,ribosome,TS_5_4,0 +1610.634,232.123,1068.46,ribosome,TS_5_4,0 +4217.488,4192.518,941.839,ribosome,TS_5_4,0 +4725.193,4090.105,1110.456,ribosome,TS_5_4,0 +2545.405,3069.551,1119.704,ribosome,TS_5_4,0 +4091.497,2159.453,874.047,ribosome,TS_5_4,0 +3986.554,1885.673,744.371,ribosome,TS_5_4,0 +3822.144,1883.156,1000.826,ribosome,TS_5_4,0 +2803.199,2701.106,336.099,ribosome,TS_5_4,0 +3189.676,2565.629,306.801,ribosome,TS_5_4,0 +3526.519,2317.862,1059.979,ribosome,TS_5_4,0 +4632.091,2990.234,244.138,ribosome,TS_5_4,0 +4620.799,2802.16,513.43,ribosome,TS_5_4,0 +4809.673,2925.589,730.053,ribosome,TS_5_4,0 +5072.909,2902.993,763.206,ribosome,TS_5_4,0 +4790.666,2517.002,489.436,ribosome,TS_5_4,0 +4805.633,2619.184,874.657,ribosome,TS_5_4,0 +5056.98,2532.435,877.002,ribosome,TS_5_4,0 +3397.301,4660.778,1309.286,ribosome,TS_5_4,0 +2843.556,6150.71,818.754,ribosome,TS_5_4,0 +4195.391,3961.637,866.076,ribosome,TS_5_4,0 +4142.395,2205.03,1126.4,ribosome,TS_5_4,0 +4356.021,3945.454,1156.437,ribosome,TS_5_4,0 +4527.706,221.468,279.908,thyroglobulin,TS_5_4,0 +5458.275,3743.418,332.075,thyroglobulin,TS_5_4,0 +5086.266,5378.977,411.051,thyroglobulin,TS_5_4,0 +2091.242,489.208,433.742,thyroglobulin,TS_5_4,0 +4338.644,1822.464,455.595,thyroglobulin,TS_5_4,0 +1310.154,4513.909,442.042,thyroglobulin,TS_5_4,0 +1585.642,3326.048,517.486,thyroglobulin,TS_5_4,0 +2293.097,1869.88,555.296,thyroglobulin,TS_5_4,0 +3469.929,3753.49,782.101,thyroglobulin,TS_5_4,0 +2439.53,3199.849,797.353,thyroglobulin,TS_5_4,0 +3088.296,2349.548,839.11,thyroglobulin,TS_5_4,0 +756.786,3307.765,923.717,thyroglobulin,TS_5_4,0 +3095.628,5843.591,947.596,thyroglobulin,TS_5_4,0 +2273.228,1492.983,949.712,thyroglobulin,TS_5_4,0 +5121.456,5468.983,1005.202,thyroglobulin,TS_5_4,0 +474.398,3091.932,1076.289,thyroglobulin,TS_5_4,0 +1541.559,5808.044,1379.853,thyroglobulin,TS_5_4,0 +441.346,2266.635,835.865,thyroglobulin,TS_5_4,0 +873.75,2086.875,1055.312,thyroglobulin,TS_5_4,0 +2405.172,6025.0,563.966,thyroglobulin,TS_5_4,0 +1990.0,5790.0,870.0,thyroglobulin,TS_5_4,0 +3087.172,5274.276,700.759,thyroglobulin,TS_5_4,0 +2917.805,3958.293,730.732,thyroglobulin,TS_5_4,0 +3419.647,2830.627,459.333,thyroglobulin,TS_5_4,0 +5090.0,573.478,122.174,thyroglobulin,TS_5_4,0 +2175.491,3100.549,452.861,thyroglobulin,TS_5_4,0 +2335.068,4957.432,703.716,thyroglobulin,TS_5_4,0 +2752.287,5760.574,881.249,thyroglobulin,TS_5_4,0 +2613.583,2253.208,884.042,thyroglobulin,TS_5_4,0 +534.129,1666.452,1038.0,thyroglobulin,TS_5_4,0 +626.933,1151.091,637.139,virus-like-particle,TS_5_4,0 +1480.552,1278.98,820.227,virus-like-particle,TS_5_4,0 +1251.324,5451.893,735.631,virus-like-particle,TS_5_4,0 +117.204,5393.268,923.733,virus-like-particle,TS_5_4,0 +672.259,3039.65,711.322,virus-like-particle,TS_5_4,0 +2620.699,3741.685,865.897,virus-like-particle,TS_5_4,0 +2636.539,4214.98,965.41,virus-like-particle,TS_5_4,0 +3137.396,3572.46,372.914,virus-like-particle,TS_5_4,0 +3294.133,3027.464,674.07,virus-like-particle,TS_5_4,0 +2997.686,4948.218,1169.375,virus-like-particle,TS_5_4,0 +3435.851,6177.824,1016.473,virus-like-particle,TS_5_4,0 +468.514,5915.906,604.167,apo-ferritin,TS_5_4,0 +5674.694,1114.354,565.068,apo-ferritin,TS_5_4,0 +5744.509,1049.172,653.712,apo-ferritin,TS_5_4,0 +5880.769,1125.348,579.56,apo-ferritin,TS_5_4,0 +4661.667,1269.497,810.409,apo-ferritin,TS_5_4,0 +4593.498,1195.874,877.982,apo-ferritin,TS_5_4,0 +4569.735,1249.967,735.033,apo-ferritin,TS_5_4,0 +4524.726,1359.207,763.445,apo-ferritin,TS_5_4,0 +1072.449,2963.265,384.014,apo-ferritin,TS_5_4,0 +1068.182,3024.848,515.628,apo-ferritin,TS_5_4,0 +995.294,3029.412,407.899,apo-ferritin,TS_5_4,0 +969.749,3086.695,557.95,apo-ferritin,TS_5_4,0 +1088.28,3115.16,609.8,apo-ferritin,TS_5_4,0 +1130.0,3130.0,795.0,apo-ferritin,TS_5_4,0 +891.818,2977.273,510.606,apo-ferritin,TS_5_4,0 +924.121,3210.767,736.134,apo-ferritin,TS_5_4,0 +1663.767,3311.767,1051.7,apo-ferritin,TS_5_4,0 +5870.268,5131.104,76.321,apo-ferritin,TS_5_4,0 +4807.948,4949.03,1039.478,apo-ferritin,TS_5_4,0 +4837.967,5049.35,1030.081,apo-ferritin,TS_5_4,0 +4714.452,5064.976,1077.238,apo-ferritin,TS_5_4,0 +5062.45,5082.947,1205.927,apo-ferritin,TS_5_4,0 +4829.392,4473.384,208.707,apo-ferritin,TS_5_4,0 +3684.954,4537.92,639.235,apo-ferritin,TS_5_4,0 +3738.736,4124.944,1182.974,apo-ferritin,TS_5_4,0 +2682.13,3630.154,642.963,apo-ferritin,TS_5_4,0 +2801.725,1630.415,204.473,apo-ferritin,TS_5_4,0 +2504.12,2311.852,267.454,apo-ferritin,TS_5_4,0 +2499.282,2197.394,298.989,apo-ferritin,TS_5_4,0 +2626.828,2295.534,254.337,apo-ferritin,TS_5_4,0 +2552.162,2711.815,416.023,apo-ferritin,TS_5_4,0 +609.824,4421.479,678.38,apo-ferritin,TS_5_4,0 +521.952,4275.286,650.429,apo-ferritin,TS_5_4,0 +612.529,4271.092,494.828,apo-ferritin,TS_5_4,0 +4657.21,5130.873,989.265,apo-ferritin,TS_5_4,0 +1151.918,3210.74,696.075,apo-ferritin,TS_5_4,0 +1038.494,3201.114,695.662,apo-ferritin,TS_5_4,0 +1059.84,3172.472,836.249,apo-ferritin,TS_5_4,0 +1066.853,3207.492,585.938,apo-ferritin,TS_5_4,0 +2340.306,2283.227,345.429,apo-ferritin,TS_5_4,0 +2349.924,2329.502,485.604,apo-ferritin,TS_5_4,0 +3779.642,4198.01,1296.612,apo-ferritin,TS_5_4,0 +611.797,4393.753,515.641,apo-ferritin,TS_5_4,0 +557.665,4367.989,605.753,apo-ferritin,TS_5_4,0 +5715.014,4998.216,115.143,apo-ferritin,TS_5_4,0 +5748.253,5115.846,108.033,apo-ferritin,TS_5_4,0 +320.256,5482.051,1088.205,beta-amylase,TS_86_3,1 +4290.0,2290.0,1050.0,beta-amylase,TS_86_3,1 +4268.588,1190.235,706.118,beta-amylase,TS_86_3,1 +3693.165,1512.658,849.62,beta-amylase,TS_86_3,1 +4384.167,445.833,1119.167,beta-amylase,TS_86_3,1 +5437.751,1527.337,824.201,beta-amylase,TS_86_3,1 +5509.412,4910.588,701.569,beta-amylase,TS_86_3,1 +2247.754,378.116,1050.217,beta-amylase,TS_86_3,1 +1188.393,2327.798,696.845,beta-amylase,TS_86_3,1 +348.342,3368.995,520.0,beta-galactosidase,TS_86_3,1 +3111.559,1404.395,597.264,beta-galactosidase,TS_86_3,1 +5128.266,4932.229,585.512,beta-galactosidase,TS_86_3,1 +3984.278,423.056,725.944,beta-galactosidase,TS_86_3,1 +2763.251,3038.958,855.062,beta-galactosidase,TS_86_3,1 +4073.83,3599.172,1070.207,beta-galactosidase,TS_86_3,1 +4463.183,1681.903,1096.54,beta-galactosidase,TS_86_3,1 +1304.049,2964.441,1180.928,beta-galactosidase,TS_86_3,1 +5695.682,831.818,827.273,beta-galactosidase,TS_86_3,1 +2369.583,5805.208,949.167,beta-galactosidase,TS_86_3,1 +1808.132,5982.125,1006.52,beta-galactosidase,TS_86_3,1 +1549.167,3946.437,574.713,beta-galactosidase,TS_86_3,1 +4913.909,2197.738,878.413,beta-galactosidase,TS_86_3,1 +565.0,1044.148,1032.481,beta-galactosidase,TS_86_3,1 +1908.833,229.833,948.167,beta-galactosidase,TS_86_3,1 +2474.59,4055.164,488.224,beta-galactosidase,TS_86_3,1 +4085.02,1689.352,644.332,beta-galactosidase,TS_86_3,1 +2511.429,480.0,807.619,beta-galactosidase,TS_86_3,1 +4450.0,1356.667,876.667,beta-galactosidase,TS_86_3,1 +2000.686,1232.08,923.473,beta-galactosidase,TS_86_3,1 +3552.92,3973.993,1034.795,beta-galactosidase,TS_86_3,1 +5021.325,3944.498,1080.0,beta-galactosidase,TS_86_3,1 +2745.385,872.86,1044.339,beta-galactosidase,TS_86_3,1 +4662.299,5907.258,945.084,ribosome,TS_86_3,1 +4504.116,5758.297,1102.158,ribosome,TS_86_3,1 +4462.021,6207.162,999.208,ribosome,TS_86_3,1 +4777.299,6156.435,1063.849,ribosome,TS_86_3,1 +4666.205,5215.513,747.842,ribosome,TS_86_3,1 +4438.999,4935.4,788.27,ribosome,TS_86_3,1 +5252.162,4295.573,849.171,ribosome,TS_86_3,1 +5689.451,4321.102,893.643,ribosome,TS_86_3,1 +6095.703,4218.054,560.215,ribosome,TS_86_3,1 +4798.895,4427.67,644.16,ribosome,TS_86_3,1 +4790.113,4162.789,708.444,ribosome,TS_86_3,1 +4327.26,4594.434,562.929,ribosome,TS_86_3,1 +4530.23,4362.667,507.066,ribosome,TS_86_3,1 +4685.378,4744.749,747.362,ribosome,TS_86_3,1 +4546.785,4603.197,1000.327,ribosome,TS_86_3,1 +5815.688,5910.474,1112.865,ribosome,TS_86_3,1 +3558.463,2113.569,1179.779,ribosome,TS_86_3,1 +2024.391,3063.731,968.932,ribosome,TS_86_3,1 +1917.914,2905.379,1174.505,ribosome,TS_86_3,1 +1649.253,2893.382,1100.787,ribosome,TS_86_3,1 +1665.413,3181.988,1083.413,ribosome,TS_86_3,1 +1563.059,6022.91,504.966,ribosome,TS_86_3,1 +1217.9,3972.649,1086.719,ribosome,TS_86_3,1 +1352.382,4356.122,751.612,ribosome,TS_86_3,1 +1213.789,4280.218,1050.137,ribosome,TS_86_3,1 +1444.336,4663.022,1049.317,ribosome,TS_86_3,1 +1974.574,4645.358,1227.115,ribosome,TS_86_3,1 +1764.31,4357.96,1082.894,ribosome,TS_86_3,1 +1677.802,4682.915,728.72,ribosome,TS_86_3,1 +615.248,5884.541,961.487,ribosome,TS_86_3,1 +253.933,5845.176,596.784,ribosome,TS_86_3,1 +238.655,5198.089,1175.451,ribosome,TS_86_3,1 +571.25,4918.854,1019.915,ribosome,TS_86_3,1 +536.338,5199.648,1025.404,ribosome,TS_86_3,1 +559.463,4480.427,1114.371,ribosome,TS_86_3,1 +420.204,4238.479,1147.024,ribosome,TS_86_3,1 +92.928,4298.622,1089.994,ribosome,TS_86_3,1 +5020.213,4244.872,791.973,ribosome,TS_86_3,1 +1452.097,5112.11,1022.865,ribosome,TS_86_3,1 +4438.741,4920.707,515.641,ribosome,TS_86_3,1 +4518.449,4937.775,1036.288,ribosome,TS_86_3,1 +4565.882,4348.095,775.964,ribosome,TS_86_3,1 +5572.189,4125.515,785.977,ribosome,TS_86_3,1 +5818.377,4480.871,856.064,ribosome,TS_86_3,1 +5938.142,4347.523,1016.263,ribosome,TS_86_3,1 +5416.05,4499.133,856.064,ribosome,TS_86_3,1 +5931.509,4108.851,866.076,ribosome,TS_86_3,1 +1959.59,4591.854,903.764,ribosome,TS_86_3,1 +1358.159,4893.146,936.164,ribosome,TS_86_3,1 +1285.497,4093.274,632.591,ribosome,TS_86_3,1 +1297.423,4036.911,866.076,ribosome,TS_86_3,1 +1633.782,4376.815,745.927,ribosome,TS_86_3,1 +616.559,5883.402,685.852,ribosome,TS_86_3,1 +513.851,5639.86,836.039,ribosome,TS_86_3,1 +4341.534,4578.56,816.014,ribosome,TS_86_3,1 +3491.458,1964.375,301.542,thyroglobulin,TS_86_3,1 +1059.895,338.01,466.754,thyroglobulin,TS_86_3,1 +5583.535,5055.373,522.266,thyroglobulin,TS_86_3,1 +3753.362,3195.014,497.106,thyroglobulin,TS_86_3,1 +4845.469,5984.182,512.044,thyroglobulin,TS_86_3,1 +3528.827,6079.586,566.737,thyroglobulin,TS_86_3,1 +4527.884,512.538,643.163,thyroglobulin,TS_86_3,1 +5107.291,5147.205,654.024,thyroglobulin,TS_86_3,1 +4062.524,5536.16,724.915,thyroglobulin,TS_86_3,1 +4426.324,5717.765,731.724,thyroglobulin,TS_86_3,1 +381.507,338.051,836.647,thyroglobulin,TS_86_3,1 +4184.942,666.409,809.653,thyroglobulin,TS_86_3,1 +1339.037,3068.829,802.446,thyroglobulin,TS_86_3,1 +2420.207,148.637,863.583,thyroglobulin,TS_86_3,1 +4547.914,2728.749,949.698,thyroglobulin,TS_86_3,1 +1358.067,798.14,896.974,thyroglobulin,TS_86_3,1 +5038.553,1656.659,993.759,thyroglobulin,TS_86_3,1 +3840.899,5011.353,972.844,thyroglobulin,TS_86_3,1 +4325.738,386.706,1027.049,thyroglobulin,TS_86_3,1 +3082.518,2718.95,1080.23,thyroglobulin,TS_86_3,1 +1856.208,5223.976,1135.201,thyroglobulin,TS_86_3,1 +1212.201,2091.175,1244.156,thyroglobulin,TS_86_3,1 +5353.462,5348.654,880.096,thyroglobulin,TS_86_3,1 +2431.176,5760.392,683.137,thyroglobulin,TS_86_3,1 +5853.226,2606.774,184.839,thyroglobulin,TS_86_3,1 +3131.439,1631.365,1285.867,thyroglobulin,TS_86_3,1 +4843.009,1838.761,1023.053,thyroglobulin,TS_86_3,1 +3472.222,325.556,845.556,thyroglobulin,TS_86_3,1 +3170.0,370.0,940.0,thyroglobulin,TS_86_3,1 +2042.034,923.559,1130.339,thyroglobulin,TS_86_3,1 +2640.654,2140.759,734.476,thyroglobulin,TS_86_3,1 +1979.6,2052.4,692.8,thyroglobulin,TS_86_3,1 +3782.069,3580.345,988.621,thyroglobulin,TS_86_3,1 +394.643,2931.786,1176.786,thyroglobulin,TS_86_3,1 +2799.672,5092.022,408.47,thyroglobulin,TS_86_3,1 +4646.594,1023.948,467.896,thyroglobulin,TS_86_3,1 +1756.142,3207.48,651.26,thyroglobulin,TS_86_3,1 +2860.196,2950.392,609.216,thyroglobulin,TS_86_3,1 +1544.041,3483.215,690.914,thyroglobulin,TS_86_3,1 +5901.04,600.347,761.965,thyroglobulin,TS_86_3,1 +4904.507,2199.128,875.641,thyroglobulin,TS_86_3,1 +3333.953,3082.093,905.233,thyroglobulin,TS_86_3,1 +3923.75,5790.0,952.5,thyroglobulin,TS_86_3,1 +3840.601,1873.09,970.687,thyroglobulin,TS_86_3,1 +5164.483,2126.724,1004.655,thyroglobulin,TS_86_3,1 +154.098,4542.461,671.862,virus-like-particle,TS_86_3,1 +5162.937,6250.567,783.617,virus-like-particle,TS_86_3,1 +5478.422,5660.856,1001.878,virus-like-particle,TS_86_3,1 +6024.374,5816.326,630.589,virus-like-particle,TS_86_3,1 +5670.599,3861.008,454.412,virus-like-particle,TS_86_3,1 +6097.154,3365.906,615.748,virus-like-particle,TS_86_3,1 +5623.065,2769.741,432.341,virus-like-particle,TS_86_3,1 +3895.295,4743.936,724.292,virus-like-particle,TS_86_3,1 +3921.092,4158.7,972.267,virus-like-particle,TS_86_3,1 +2720.586,4840.967,1057.297,virus-like-particle,TS_86_3,1 +984.028,881.454,770.454,virus-like-particle,TS_86_3,1 +796.253,1232.922,956.509,virus-like-particle,TS_86_3,1 +1212.736,396.454,913.84,virus-like-particle,TS_86_3,1 +1234.637,1416.69,758.009,virus-like-particle,TS_86_3,1 +713.243,3300.755,729.3,virus-like-particle,TS_86_3,1 +449.787,3348.949,936.699,virus-like-particle,TS_86_3,1 +1024.638,2983.1,649.246,virus-like-particle,TS_86_3,1 +640.315,2221.381,868.266,virus-like-particle,TS_86_3,1 +3443.658,858.443,609.676,virus-like-particle,TS_86_3,1 +2733.056,387.938,935.431,virus-like-particle,TS_86_3,1 +3993.256,185.537,625.067,virus-like-particle,TS_86_3,1 +2891.009,1717.577,755.086,virus-like-particle,TS_86_3,1 +2785.126,2361.186,1008.411,virus-like-particle,TS_86_3,1 +4638.201,1844.865,491.223,virus-like-particle,TS_86_3,1 +4683.103,1546.998,899.626,virus-like-particle,TS_86_3,1 +3563.17,2650.076,660.386,virus-like-particle,TS_86_3,1 +3994.606,2797.533,839.291,virus-like-particle,TS_86_3,1 +3829.926,2129.623,859.549,virus-like-particle,TS_86_3,1 +1905.062,2172.381,978.064,virus-like-particle,TS_86_3,1 +3870.343,4952.714,1261.6,apo-ferritin,TS_86_3,1 +4130.897,5422.292,501.86,apo-ferritin,TS_86_3,1 +2735.0,4668.447,520.291,apo-ferritin,TS_86_3,1 +2649.615,4690.615,600.923,apo-ferritin,TS_86_3,1 +2665.353,4810.641,612.019,apo-ferritin,TS_86_3,1 +2957.522,4630.03,489.701,apo-ferritin,TS_86_3,1 +2920.264,4718.238,659.559,apo-ferritin,TS_86_3,1 +2996.667,4730.0,436.667,apo-ferritin,TS_86_3,1 +2957.199,4845.319,394.858,apo-ferritin,TS_86_3,1 +2858.555,4515.665,523.699,apo-ferritin,TS_86_3,1 +2802.396,4464.345,937.859,apo-ferritin,TS_86_3,1 +3636.645,5702.839,646.419,apo-ferritin,TS_86_3,1 +349.225,4387.254,639.613,apo-ferritin,TS_86_3,1 +1421.215,4837.57,473.209,apo-ferritin,TS_86_3,1 +1516.687,4884.498,530.122,apo-ferritin,TS_86_3,1 +1344.405,5016.071,464.077,apo-ferritin,TS_86_3,1 +1615.455,5224.058,418.604,apo-ferritin,TS_86_3,1 +1529.628,5302.061,442.027,apo-ferritin,TS_86_3,1 +1744.252,5121.395,462.245,apo-ferritin,TS_86_3,1 +1289.961,4419.807,498.842,apo-ferritin,TS_86_3,1 +368.896,5304.785,611.472,apo-ferritin,TS_86_3,1 +349.415,5404.185,663.938,apo-ferritin,TS_86_3,1 +737.13,5657.546,451.944,apo-ferritin,TS_86_3,1 +1152.315,5759.259,463.056,apo-ferritin,TS_86_3,1 +1223.792,5738.917,536.083,apo-ferritin,TS_86_3,1 +1111.487,5848.885,503.234,apo-ferritin,TS_86_3,1 +2024.386,2430.912,928.211,apo-ferritin,TS_86_3,1 +1885.083,2451.761,899.07,apo-ferritin,TS_86_3,1 +1635.436,2052.785,702.483,apo-ferritin,TS_86_3,1 +764.053,3009.015,780.114,apo-ferritin,TS_86_3,1 +805.603,2963.582,867.943,apo-ferritin,TS_86_3,1 +812.748,562.691,574.079,apo-ferritin,TS_86_3,1 +2398.017,1136.609,1115.057,apo-ferritin,TS_86_3,1 +2339.747,1240.886,1132.057,apo-ferritin,TS_86_3,1 +2423.333,1373.903,1099.801,apo-ferritin,TS_86_3,1 +1882.737,1238.321,1279.672,apo-ferritin,TS_86_3,1 +2886.727,662.649,515.325,apo-ferritin,TS_86_3,1 +3243.811,189.939,1204.756,apo-ferritin,TS_86_3,1 +5632.418,4832.451,852.157,apo-ferritin,TS_86_3,1 +5571.34,4880.069,923.814,apo-ferritin,TS_86_3,1 +5786.293,5116.573,858.037,apo-ferritin,TS_86_3,1 +5775.152,4995.212,878.485,apo-ferritin,TS_86_3,1 +4335.3,3517.367,1224.1,apo-ferritin,TS_86_3,1 +4765.574,621.401,780.112,apo-ferritin,TS_86_3,1 +4728.462,597.308,964.615,apo-ferritin,TS_86_3,1 +4784.136,555.39,853.254,apo-ferritin,TS_86_3,1 +3028.839,2861.645,444.29,apo-ferritin,TS_86_3,1 +3098.086,3170.695,1146.425,apo-ferritin,TS_86_3,1 +3192.802,3247.743,1146.425,apo-ferritin,TS_86_3,1 +3123.334,3157.011,1266.574,apo-ferritin,TS_86_3,1 +3197.58,5525.735,1296.612,apo-ferritin,TS_86_3,1 +1912.78,3850.979,1236.537,apo-ferritin,TS_86_3,1 +3760.282,4956.3,1186.475,apo-ferritin,TS_86_3,1 +982.232,4714.162,475.591,apo-ferritin,TS_86_3,1 +996.927,4489.487,475.591,apo-ferritin,TS_86_3,1 +745.902,4018.275,615.765,apo-ferritin,TS_86_3,1 +375.196,4211.727,615.765,apo-ferritin,TS_86_3,1 +258.592,4183.515,645.803,apo-ferritin,TS_86_3,1 +2084.869,5220.131,435.541,apo-ferritin,TS_86_3,1 +718.028,4094.611,565.703,apo-ferritin,TS_86_3,1 +457.252,4137.12,505.628,apo-ferritin,TS_86_3,1 +1361.928,5703.278,435.541,apo-ferritin,TS_86_3,1 +1112.671,5943.359,455.566,apo-ferritin,TS_86_3,1 +69.928,5362.53,555.691,apo-ferritin,TS_86_3,1 +3824.768,5064.901,676.026,beta-amylase,TS_69_2,2 +5166.324,3905.441,924.118,beta-amylase,TS_69_2,2 +5123.125,5389.375,945.625,beta-amylase,TS_69_2,2 +4407.778,6266.944,1228.333,beta-amylase,TS_69_2,2 +350.0,1220.0,1210.0,beta-amylase,TS_69_2,2 +1567.122,2131.079,687.05,beta-amylase,TS_69_2,2 +1466.143,913.857,1258.714,beta-amylase,TS_69_2,2 +2791.339,678.571,1164.821,beta-amylase,TS_69_2,2 +4378.638,762.894,913.191,beta-amylase,TS_69_2,2 +3569.083,427.11,935.688,beta-amylase,TS_69_2,2 +4442.424,2286.136,669.773,beta-amylase,TS_69_2,2 +4263.574,3363.976,1001.928,beta-amylase,TS_69_2,2 +3124.091,2349.758,914.818,beta-galactosidase,TS_69_2,2 +3104.737,116.316,893.158,beta-galactosidase,TS_69_2,2 +1482.15,1976.86,809.963,beta-galactosidase,TS_69_2,2 +4736.857,1633.5,1281.857,beta-galactosidase,TS_69_2,2 +5440.842,2153.316,1065.263,beta-galactosidase,TS_69_2,2 +5829.167,3585.0,1234.167,beta-galactosidase,TS_69_2,2 +5934.359,4038.974,711.538,beta-galactosidase,TS_69_2,2 +5565.034,3118.993,770.604,beta-galactosidase,TS_69_2,2 +1786.416,1087.714,590.104,beta-galactosidase,TS_69_2,2 +5058.789,2709.421,670.105,beta-galactosidase,TS_69_2,2 +1884.097,2071.736,719.583,beta-galactosidase,TS_69_2,2 +5685.556,4460.606,717.172,beta-galactosidase,TS_69_2,2 +4895.506,2074.177,754.272,beta-galactosidase,TS_69_2,2 +3093.723,2528.936,923.617,beta-galactosidase,TS_69_2,2 +5015.0,1340.263,944.649,beta-galactosidase,TS_69_2,2 +5835.63,5542.991,1125.63,beta-galactosidase,TS_69_2,2 +6006.82,5328.856,808.633,ribosome,TS_69_2,2 +5957.767,5571.663,706.068,ribosome,TS_69_2,2 +5618.836,6161.76,693.695,ribosome,TS_69_2,2 +1127.794,187.938,537.361,ribosome,TS_69_2,2 +1767.82,3213.929,1001.62,ribosome,TS_69_2,2 +1916.347,3426.268,1005.688,ribosome,TS_69_2,2 +1171.606,3275.311,1023.63,ribosome,TS_69_2,2 +1475.204,3418.907,1062.102,ribosome,TS_69_2,2 +855.205,1905.764,988.053,ribosome,TS_69_2,2 +1041.483,1639.013,1025.697,ribosome,TS_69_2,2 +588.724,1717.082,887.573,ribosome,TS_69_2,2 +1158.38,2417.84,824.652,ribosome,TS_69_2,2 +3944.471,877.964,875.367,ribosome,TS_69_2,2 +3800.311,509.266,1148.402,ribosome,TS_69_2,2 +5080.617,763.376,705.302,ribosome,TS_69_2,2 +4661.842,201.026,1140.471,ribosome,TS_69_2,2 +4909.496,342.802,905.927,ribosome,TS_69_2,2 +3032.898,1801.65,980.107,ribosome,TS_69_2,2 +3281.41,1786.652,1039.956,ribosome,TS_69_2,2 +3263.628,3307.453,718.927,ribosome,TS_69_2,2 +3204.775,3028.787,664.15,ribosome,TS_69_2,2 +3411.387,3674.855,1015.965,ribosome,TS_69_2,2 +2957.875,3215.688,651.509,ribosome,TS_69_2,2 +2988.816,2839.513,747.089,ribosome,TS_69_2,2 +3759.918,3733.779,875.956,ribosome,TS_69_2,2 +4064.384,3855.712,1073.271,ribosome,TS_69_2,2 +4093.64,2806.907,992.895,ribosome,TS_69_2,2 +4553.26,2854.694,934.778,ribosome,TS_69_2,2 +4833.247,2785.558,1008.777,ribosome,TS_69_2,2 +4982.25,3012.384,1141.249,ribosome,TS_69_2,2 +4746.456,3247.044,989.765,ribosome,TS_69_2,2 +6205.712,3190.031,1166.83,ribosome,TS_69_2,2 +5678.07,3034.789,1201.75,ribosome,TS_69_2,2 +5473.934,5538.159,1008.782,ribosome,TS_69_2,2 +1863.411,1820.198,1074.291,ribosome,TS_69_2,2 +4310.582,2600.476,966.971,ribosome,TS_69_2,2 +3825.977,2568.575,1072.554,ribosome,TS_69_2,2 +1697.664,2484.89,553.359,thyroglobulin,TS_69_2,2 +1083.746,4336.916,660.712,thyroglobulin,TS_69_2,2 +2535.083,3739.817,626.862,thyroglobulin,TS_69_2,2 +719.11,2318.413,647.963,thyroglobulin,TS_69_2,2 +270.918,4900.032,677.824,thyroglobulin,TS_69_2,2 +5371.849,897.815,674.958,thyroglobulin,TS_69_2,2 +5539.777,1677.178,752.772,thyroglobulin,TS_69_2,2 +2021.719,797.706,864.533,thyroglobulin,TS_69_2,2 +2367.94,978.443,867.55,thyroglobulin,TS_69_2,2 +5605.799,1623.846,1155.562,thyroglobulin,TS_69_2,2 +843.974,3537.848,538.51,thyroglobulin,TS_69_2,2 +714.545,3129.091,673.636,thyroglobulin,TS_69_2,2 +1456.572,4357.201,783.836,thyroglobulin,TS_69_2,2 +4435.591,5401.685,874.624,thyroglobulin,TS_69_2,2 +4315.138,4204.558,650.083,thyroglobulin,TS_69_2,2 +3988.304,3924.261,623.783,thyroglobulin,TS_69_2,2 +5049.901,4123.515,1092.673,thyroglobulin,TS_69_2,2 +5901.79,5390.067,1034.295,thyroglobulin,TS_69_2,2 +4435.398,6184.513,718.702,thyroglobulin,TS_69_2,2 +4195.77,2991.516,666.993,thyroglobulin,TS_69_2,2 +5720.0,1579.247,825.146,thyroglobulin,TS_69_2,2 +3050.0,2350.0,810.0,thyroglobulin,TS_69_2,2 +2991.25,1755.417,616.25,thyroglobulin,TS_69_2,2 +3004.667,1165.667,938.333,thyroglobulin,TS_69_2,2 +2035.125,145.5,1198.875,thyroglobulin,TS_69_2,2 +2021.887,1799.321,698.34,thyroglobulin,TS_69_2,2 +763.441,469.624,885.591,thyroglobulin,TS_69_2,2 +6229.592,394.49,1286.327,thyroglobulin,TS_69_2,2 +1369.833,5924.057,828.807,thyroglobulin,TS_69_2,2 +785.587,2630.0,576.201,thyroglobulin,TS_69_2,2 +3362.941,695.882,620.588,thyroglobulin,TS_69_2,2 +3625.942,4047.846,729.154,thyroglobulin,TS_69_2,2 +2618.947,369.774,749.173,thyroglobulin,TS_69_2,2 +5679.653,4176.597,1108.889,thyroglobulin,TS_69_2,2 +235.918,2967.62,799.798,virus-like-particle,TS_69_2,2 +516.962,4021.185,878.484,virus-like-particle,TS_69_2,2 +2832.249,750.827,654.317,virus-like-particle,TS_69_2,2 +2587.613,1360.659,814.115,virus-like-particle,TS_69_2,2 +5063.363,1732.215,951.33,virus-like-particle,TS_69_2,2 +5930.927,978.082,911.592,virus-like-particle,TS_69_2,2 +3603.379,2385.445,924.877,virus-like-particle,TS_69_2,2 +5125.065,5681.883,888.948,virus-like-particle,TS_69_2,2 +5924.779,3598.073,941.761,virus-like-particle,TS_69_2,2 +770.625,1111.161,1088.795,apo-ferritin,TS_69_2,2 +828.291,1201.673,1153.745,apo-ferritin,TS_69_2,2 +668.986,1041.449,1102.246,apo-ferritin,TS_69_2,2 +834.049,592.958,698.099,apo-ferritin,TS_69_2,2 +81.893,2152.929,543.179,apo-ferritin,TS_69_2,2 +5194.716,2277.793,916.254,apo-ferritin,TS_69_2,2 +2720.996,975.578,759.681,apo-ferritin,TS_69_2,2 +409.224,5385.982,819.132,apo-ferritin,TS_69_2,2 +470.992,5471.736,826.116,apo-ferritin,TS_69_2,2 +1340.353,5860.929,518.237,apo-ferritin,TS_69_2,2 +6150.553,4248.018,598.848,apo-ferritin,TS_69_2,2 +4116.256,6243.555,550.142,apo-ferritin,TS_69_2,2 +1695.969,4701.783,576.473,apo-ferritin,TS_69_2,2 +1639.36,4850.438,1063.3,apo-ferritin,TS_69_2,2 +4775.286,5015.714,581.514,apo-ferritin,TS_69_2,2 +4844.739,4397.387,782.334,apo-ferritin,TS_69_2,2 +3192.063,3826.044,607.451,apo-ferritin,TS_69_2,2 +3809.442,4198.247,561.673,apo-ferritin,TS_69_2,2 +3583.403,4366.702,603.141,apo-ferritin,TS_69_2,2 +3476.717,4383.102,650.181,apo-ferritin,TS_69_2,2 +3346.316,4185.936,586.959,apo-ferritin,TS_69_2,2 +3356.477,4340.341,649.659,apo-ferritin,TS_69_2,2 +3288.077,4440.594,634.301,apo-ferritin,TS_69_2,2 +3227.491,4544.364,579.127,apo-ferritin,TS_69_2,2 +3986.778,4435.444,572.259,apo-ferritin,TS_69_2,2 +3838.352,3983.448,633.065,apo-ferritin,TS_69_2,2 +4414.178,3489.079,1138.289,apo-ferritin,TS_69_2,2 +2864.757,2420.825,1037.476,apo-ferritin,TS_69_2,2 +2965.475,2392.793,1081.285,apo-ferritin,TS_69_2,2 +2450.507,2722.905,999.426,apo-ferritin,TS_69_2,2 +3374.729,4537.176,585.728,apo-ferritin,TS_69_2,2 +3463.362,4759.63,585.728,apo-ferritin,TS_69_2,2 +3280.785,4270.741,665.828,apo-ferritin,TS_69_2,2 +3738.087,4294.279,665.828,apo-ferritin,TS_69_2,2 +3811.914,4381.302,616.461,apo-ferritin,TS_69_2,2 +2913.77,5921.278,867.604,beta-amylase,TS_99_9,3 +2806.057,5772.0,1138.4,beta-amylase,TS_99_9,3 +1676.875,5103.125,1180.0,beta-amylase,TS_99_9,3 +579.608,4457.059,1065.686,beta-amylase,TS_99_9,3 +920.0,5645.0,1530.0,beta-amylase,TS_99_9,3 +4033.412,3769.948,1173.543,beta-amylase,TS_99_9,3 +2777.874,3491.181,478.15,beta-amylase,TS_99_9,3 +1885.833,236.667,184.167,beta-amylase,TS_99_9,3 +1999.49,2273.469,310.663,beta-amylase,TS_99_9,3 +2345.73,2017.64,996.966,beta-amylase,TS_99_9,3 +2080.0,1815.0,311.275,beta-amylase,TS_99_9,3 +3379.651,1627.093,693.682,beta-amylase,TS_99_9,3 +2832.0,1580.0,670.0,beta-amylase,TS_99_9,3 +3680.78,2053.756,450.488,beta-amylase,TS_99_9,3 +1990.0,2810.0,730.0,beta-amylase,TS_99_9,3 +3614.741,2640.948,742.026,beta-amylase,TS_99_9,3 +1571.774,1085.222,661.613,beta-amylase,TS_99_9,3 +1440.667,1001.667,790.333,beta-amylase,TS_99_9,3 +2501.623,951.948,806.623,beta-amylase,TS_99_9,3 +730.0,1848.333,921.667,beta-amylase,TS_99_9,3 +4443.627,6262.353,945.882,beta-amylase,TS_99_9,3 +5628.926,4904.558,596.945,beta-galactosidase,TS_99_9,3 +5173.756,5124.78,558.683,beta-galactosidase,TS_99_9,3 +3510.155,195.58,743.106,beta-galactosidase,TS_99_9,3 +5368.571,3285.0,924.286,beta-galactosidase,TS_99_9,3 +5870.914,2555.995,402.796,beta-galactosidase,TS_99_9,3 +5692.178,363.77,822.506,beta-galactosidase,TS_99_9,3 +4330.594,227.707,232.155,beta-galactosidase,TS_99_9,3 +2962.0,1081.667,524.222,beta-galactosidase,TS_99_9,3 +2595.909,762.727,660.0,beta-galactosidase,TS_99_9,3 +2189.81,932.048,518.571,beta-galactosidase,TS_99_9,3 +2145.786,1449.929,283.357,beta-galactosidase,TS_99_9,3 +485.221,2212.794,529.044,beta-galactosidase,TS_99_9,3 +761.031,2527.937,656.812,beta-galactosidase,TS_99_9,3 +1503.711,5532.695,1127.148,beta-galactosidase,TS_99_9,3 +3130.0,3430.0,325.0,beta-galactosidase,TS_99_9,3 +1542.335,4102.275,970.0,beta-galactosidase,TS_99_9,3 +430.436,5762.282,972.349,beta-galactosidase,TS_99_9,3 +6028.531,1814.84,489.704,beta-galactosidase,TS_99_9,3 +5142.516,1120.644,610.598,beta-galactosidase,TS_99_9,3 +2907.5,3320.0,580.0,beta-galactosidase,TS_99_9,3 +3112.493,2771.804,800.133,beta-galactosidase,TS_99_9,3 +2058.911,3929.241,786.634,beta-galactosidase,TS_99_9,3 +807.718,4597.49,1060.788,beta-galactosidase,TS_99_9,3 +5390.233,5613.953,1080.233,beta-galactosidase,TS_99_9,3 +3319.52,5476.3,1014.511,ribosome,TS_99_9,3 +3106.079,5574.209,1186.11,ribosome,TS_99_9,3 +3676.997,5271.281,890.752,ribosome,TS_99_9,3 +3717.18,5455.125,1103.21,ribosome,TS_99_9,3 +3525.117,5614.423,1112.576,ribosome,TS_99_9,3 +3535.323,5008.014,1059.689,ribosome,TS_99_9,3 +2883.189,5382.516,1133.506,ribosome,TS_99_9,3 +2116.336,5189.631,1027.456,ribosome,TS_99_9,3 +2328.909,5130.186,1253.405,ribosome,TS_99_9,3 +1947.981,5391.709,1313.177,ribosome,TS_99_9,3 +2156.805,5609.771,1273.522,ribosome,TS_99_9,3 +2509.295,5370.443,1185.571,ribosome,TS_99_9,3 +2719.011,5096.74,1263.963,ribosome,TS_99_9,3 +3176.477,5049.588,1126.434,ribosome,TS_99_9,3 +4934.962,6071.261,947.376,ribosome,TS_99_9,3 +4999.826,5866.993,1122.244,ribosome,TS_99_9,3 +4927.254,5595.834,1145.519,ribosome,TS_99_9,3 +5615.95,4909.438,989.317,ribosome,TS_99_9,3 +5336.302,4963.521,1045.105,ribosome,TS_99_9,3 +5279.465,4564.465,417.07,ribosome,TS_99_9,3 +4916.58,5065.955,1120.492,ribosome,TS_99_9,3 +411.526,315.634,370.328,ribosome,TS_99_9,3 +430.518,110.647,133.701,ribosome,TS_99_9,3 +702.557,115.463,165.955,ribosome,TS_99_9,3 +1055.336,457.84,237.445,ribosome,TS_99_9,3 +1152.917,482.744,468.791,ribosome,TS_99_9,3 +1424.626,537.985,227.71,ribosome,TS_99_9,3 +852.272,670.775,246.253,ribosome,TS_99_9,3 +1135.8,1200.179,278.969,ribosome,TS_99_9,3 +833.018,1211.915,291.627,ribosome,TS_99_9,3 +521.382,991.567,232.765,ribosome,TS_99_9,3 +724.625,906.686,436.445,ribosome,TS_99_9,3 +971.899,924.657,189.731,ribosome,TS_99_9,3 +309.802,1409.819,659.974,ribosome,TS_99_9,3 +544.924,1197.59,831.877,ribosome,TS_99_9,3 +539.079,1402.459,906.531,ribosome,TS_99_9,3 +1131.172,171.919,184.836,ribosome,TS_99_9,3 +271.44,4207.943,1181.704,ribosome,TS_99_9,3 +642.942,3879.471,1147.333,ribosome,TS_99_9,3 +559.039,3663.243,1180.24,ribosome,TS_99_9,3 +807.849,4177.77,1283.998,ribosome,TS_99_9,3 +283.913,3819.565,1156.087,ribosome,TS_99_9,3 +256.686,5246.703,1066.913,ribosome,TS_99_9,3 +490.506,5275.121,1291.868,ribosome,TS_99_9,3 +882.72,5136.226,1305.376,ribosome,TS_99_9,3 +2003.082,3328.973,1001.08,ribosome,TS_99_9,3 +1873.277,3543.216,678.046,ribosome,TS_99_9,3 +1118.468,3619.591,1267.875,ribosome,TS_99_9,3 +956.16,2418.385,833.722,ribosome,TS_99_9,3 +842.784,2288.479,463.814,ribosome,TS_99_9,3 +1008.88,2721.708,938.224,ribosome,TS_99_9,3 +1534.252,2653.747,894.814,ribosome,TS_99_9,3 +1766.353,2563.641,1128.742,ribosome,TS_99_9,3 +1713.554,2870.877,1082.887,ribosome,TS_99_9,3 +2887.071,2920.025,1138.122,ribosome,TS_99_9,3 +5283.409,1249.497,917.141,ribosome,TS_99_9,3 +5762.274,1749.037,772.82,ribosome,TS_99_9,3 +5903.842,1565.63,977.371,ribosome,TS_99_9,3 +5482.753,1827.704,848.819,ribosome,TS_99_9,3 +4787.941,1452.551,904.813,ribosome,TS_99_9,3 +4561.989,1641.32,954.457,ribosome,TS_99_9,3 +4166.224,1356.023,1062.256,ribosome,TS_99_9,3 +4306.506,2049.676,1092.797,ribosome,TS_99_9,3 +258.14,5412.14,1270.491,ribosome,TS_99_9,3 +3763.052,5198.677,1156.437,ribosome,TS_99_9,3 +638.516,1499.883,279.57,thyroglobulin,TS_99_9,3 +2038.81,214.041,361.807,thyroglobulin,TS_99_9,3 +1504.54,3662.02,566.838,thyroglobulin,TS_99_9,3 +2166.658,2000.169,547.046,thyroglobulin,TS_99_9,3 +656.526,1988.951,597.846,thyroglobulin,TS_99_9,3 +3082.001,2240.364,645.634,thyroglobulin,TS_99_9,3 +2504.842,4565.01,628.011,thyroglobulin,TS_99_9,3 +5788.387,2978.892,692.567,thyroglobulin,TS_99_9,3 +5360.187,4157.772,727.216,thyroglobulin,TS_99_9,3 +2245.603,1789.767,785.667,thyroglobulin,TS_99_9,3 +1398.087,3775.91,1004.103,thyroglobulin,TS_99_9,3 +3817.136,1389.497,1017.085,thyroglobulin,TS_99_9,3 +1229.988,2017.224,1143.411,thyroglobulin,TS_99_9,3 +2169.349,329.172,1145.444,thyroglobulin,TS_99_9,3 +4890.0,4570.0,1210.0,thyroglobulin,TS_99_9,3 +4649.024,4763.902,624.268,thyroglobulin,TS_99_9,3 +5513.22,4635.311,759.492,thyroglobulin,TS_99_9,3 +4347.313,4142.09,1080.597,thyroglobulin,TS_99_9,3 +3747.657,4574.435,1079.079,thyroglobulin,TS_99_9,3 +5409.036,5614.819,878.072,thyroglobulin,TS_99_9,3 +5141.848,2562.391,982.609,thyroglobulin,TS_99_9,3 +4465.707,2667.488,1096.463,thyroglobulin,TS_99_9,3 +3957.731,2284.118,393.025,thyroglobulin,TS_99_9,3 +3382.5,2805.0,1050.0,thyroglobulin,TS_99_9,3 +3261.336,2442.708,1020.181,thyroglobulin,TS_99_9,3 +2357.384,3572.778,1085.972,thyroglobulin,TS_99_9,3 +1649.074,3786.667,624.074,thyroglobulin,TS_99_9,3 +1905.325,3728.049,1032.276,thyroglobulin,TS_99_9,3 +1498.586,4220.741,1323.401,thyroglobulin,TS_99_9,3 +2345.0,5647.5,895.0,thyroglobulin,TS_99_9,3 +2303.101,5998.481,899.873,thyroglobulin,TS_99_9,3 +3525.787,183.38,733.264,thyroglobulin,TS_99_9,3 +2731.186,1063.66,237.062,thyroglobulin,TS_99_9,3 +1054.579,3043.832,715.813,thyroglobulin,TS_99_9,3 +430.0,2690.0,930.0,thyroglobulin,TS_99_9,3 +1503.692,1698.051,288.667,thyroglobulin,TS_99_9,3 +676.591,834.773,70.909,thyroglobulin,TS_99_9,3 +492.857,798.312,686.104,thyroglobulin,TS_99_9,3 +452.931,1866.293,293.448,thyroglobulin,TS_99_9,3 +1604.444,695.079,1036.984,thyroglobulin,TS_99_9,3 +2620.97,1504.97,424.606,thyroglobulin,TS_99_9,3 +2576.97,774.646,642.323,thyroglobulin,TS_99_9,3 +2487.147,4256.221,704.01,thyroglobulin,TS_99_9,3 +2280.316,3380.752,791.189,thyroglobulin,TS_99_9,3 +2790.0,3945.0,790.0,thyroglobulin,TS_99_9,3 +1110.308,5734.615,862.154,thyroglobulin,TS_99_9,3 +3148.098,3234.683,952.634,thyroglobulin,TS_99_9,3 +1541.778,4082.63,989.996,thyroglobulin,TS_99_9,3 +1267.073,4858.78,1068.293,thyroglobulin,TS_99_9,3 +2766.389,185.542,510.129,virus-like-particle,TS_99_9,3 +3084.893,487.084,760.823,virus-like-particle,TS_99_9,3 +3693.973,2932.983,620.271,virus-like-particle,TS_99_9,3 +3010.666,2469.137,884.272,virus-like-particle,TS_99_9,3 +556.664,3167.112,664.183,virus-like-particle,TS_99_9,3 +289.388,4113.399,765.344,virus-like-particle,TS_99_9,3 +981.03,1910.541,780.365,virus-like-particle,TS_99_9,3 +1566.401,4710.311,791.778,virus-like-particle,TS_99_9,3 +2010.056,4752.618,1057.078,virus-like-particle,TS_99_9,3 +2244.068,4310.063,959.548,virus-like-particle,TS_99_9,3 +804.27,5817.135,579.493,virus-like-particle,TS_99_9,3 +4198.228,5534.578,858.169,virus-like-particle,TS_99_9,3 +5698.538,3331.427,806.002,virus-like-particle,TS_99_9,3 +6072.464,4038.0,715.679,apo-ferritin,TS_99_9,3 +5967.452,4228.213,1124.601,apo-ferritin,TS_99_9,3 +5847.622,5066.71,678.893,apo-ferritin,TS_99_9,3 +472.853,5632.618,580.676,apo-ferritin,TS_99_9,3 +564.507,5604.88,678.16,apo-ferritin,TS_99_9,3 +890.414,5615.759,467.69,apo-ferritin,TS_99_9,3 +5613.561,672.73,265.579,apo-ferritin,TS_99_9,3 +458.087,1972.013,765.906,apo-ferritin,TS_99_9,3 +1284.111,1752.411,250.87,apo-ferritin,TS_99_9,3 +1284.136,1713.797,367.525,apo-ferritin,TS_99_9,3 +1233.112,2053.458,251.902,apo-ferritin,TS_99_9,3 +1194.107,1911.285,267.179,apo-ferritin,TS_99_9,3 +5543.161,2879.598,408.42,apo-ferritin,TS_99_9,3 +5485.217,2774.62,411.413,apo-ferritin,TS_99_9,3 +5448.11,3486.46,985.533,apo-ferritin,TS_99_9,3 +3529.611,2919.728,380.389,apo-ferritin,TS_99_9,3 +3438.216,3010.29,397.925,apo-ferritin,TS_99_9,3 +4251.254,845.627,720.143,apo-ferritin,TS_99_9,3 +4592.044,1321.788,322.883,apo-ferritin,TS_99_9,3 +4250.071,1837.26,319.929,apo-ferritin,TS_99_9,3 +4395.13,1892.857,352.013,apo-ferritin,TS_99_9,3 +3670.295,1851.882,328.745,apo-ferritin,TS_99_9,3 +3287.917,1286.28,282.56,apo-ferritin,TS_99_9,3 +2967.331,1646.264,312.781,apo-ferritin,TS_99_9,3 +2987.912,1729.794,397.471,apo-ferritin,TS_99_9,3 +652.0,4574.214,416.571,apo-ferritin,TS_99_9,3 +1032.835,4506.417,1003.228,apo-ferritin,TS_99_9,3 +1100.435,4449.249,1078.617,apo-ferritin,TS_99_9,3 +929.932,4524.384,915.137,apo-ferritin,TS_99_9,3 +1189.237,4882.468,464.122,apo-ferritin,TS_99_9,3 +1649.873,4985.285,473.544,apo-ferritin,TS_99_9,3 +2165.042,4617.759,846.807,apo-ferritin,TS_99_9,3 +2215.952,4694.502,928.61,apo-ferritin,TS_99_9,3 +2276.394,4608.261,802.558,apo-ferritin,TS_99_9,3 +2137.354,3437.938,600.0,apo-ferritin,TS_99_9,3 +1685.81,3815.229,1354.985,apo-ferritin,TS_99_9,3 +1962.647,358.235,376.029,beta-amylase,TS_73_6,4 +2212.609,450.632,641.739,beta-amylase,TS_73_6,4 +2177.5,205.0,907.5,beta-amylase,TS_73_6,4 +943.718,1273.397,132.821,beta-amylase,TS_73_6,4 +2271.379,1911.034,546.293,beta-amylase,TS_73_6,4 +4376.951,2160.122,396.829,beta-amylase,TS_73_6,4 +3761.98,2860.0,1021.287,beta-amylase,TS_73_6,4 +2534.933,3468.622,646.267,beta-amylase,TS_73_6,4 +1948.482,3600.052,965.864,beta-amylase,TS_73_6,4 +1949.783,4223.261,241.304,beta-amylase,TS_73_6,4 +1838.74,5085.118,893.386,beta-amylase,TS_73_6,4 +270.861,5594.658,815.646,beta-amylase,TS_73_6,4 +2285.357,2137.571,596.214,beta-galactosidase,TS_73_6,4 +4008.603,383.309,798.529,beta-galactosidase,TS_73_6,4 +839.231,1602.692,1088.077,beta-galactosidase,TS_73_6,4 +188.815,2167.37,769.741,beta-galactosidase,TS_73_6,4 +742.788,2808.077,246.058,beta-galactosidase,TS_73_6,4 +3552.222,228.148,556.481,beta-galactosidase,TS_73_6,4 +4341.167,3756.5,532.667,beta-galactosidase,TS_73_6,4 +3670.099,2118.144,520.99,beta-galactosidase,TS_73_6,4 +2349.623,474.663,334.96,beta-galactosidase,TS_73_6,4 +3725.132,3520.212,537.937,beta-galactosidase,TS_73_6,4 +2132.535,2661.582,689.108,beta-galactosidase,TS_73_6,4 +388.433,3230.336,768.209,beta-galactosidase,TS_73_6,4 +4555.0,3188.75,868.75,beta-galactosidase,TS_73_6,4 +277.276,3258.78,897.378,beta-galactosidase,TS_73_6,4 +5812.21,954.718,134.827,ribosome,TS_73_6,4 +5598.248,890.665,230.051,ribosome,TS_73_6,4 +5869.463,488.783,285.901,ribosome,TS_73_6,4 +5850.904,575.293,735.879,ribosome,TS_73_6,4 +5940.845,736.687,114.293,ribosome,TS_73_6,4 +5600.385,584.159,361.892,ribosome,TS_73_6,4 +5755.45,2294.051,667.695,ribosome,TS_73_6,4 +6045.567,2430.597,715.623,ribosome,TS_73_6,4 +5923.825,2647.136,880.662,ribosome,TS_73_6,4 +5612.53,2504.846,757.101,ribosome,TS_73_6,4 +2975.042,5637.426,1086.291,ribosome,TS_73_6,4 +2783.415,5731.957,917.101,ribosome,TS_73_6,4 +3067.628,5861.795,977.738,ribosome,TS_73_6,4 +6067.043,3899.28,953.521,ribosome,TS_73_6,4 +4227.966,4408.917,1033.563,ribosome,TS_73_6,4 +4124.483,4142.615,1148.458,ribosome,TS_73_6,4 +3880.999,3552.474,1003.724,ribosome,TS_73_6,4 +4595.975,3413.592,1095.442,ribosome,TS_73_6,4 +2492.225,4304.511,735.979,ribosome,TS_73_6,4 +2141.774,4385.963,846.593,ribosome,TS_73_6,4 +2316.45,4391.133,1066.221,ribosome,TS_73_6,4 +2282.211,4170.642,807.039,ribosome,TS_73_6,4 +2155.564,4599.751,1226.02,ribosome,TS_73_6,4 +1845.342,4530.955,856.591,ribosome,TS_73_6,4 +1698.475,4467.553,1310.344,ribosome,TS_73_6,4 +2963.526,4115.076,947.772,ribosome,TS_73_6,4 +3230.411,3924.05,903.202,ribosome,TS_73_6,4 +3501.536,3769.701,1211.88,ribosome,TS_73_6,4 +4136.679,1185.915,793.199,ribosome,TS_73_6,4 +4244.615,1461.256,917.947,ribosome,TS_73_6,4 +4401.817,1002.721,835.005,ribosome,TS_73_6,4 +3898.451,1614.824,1008.257,ribosome,TS_73_6,4 +3506.524,1792.811,1036.299,ribosome,TS_73_6,4 +3056.445,2174.747,1036.756,ribosome,TS_73_6,4 +2259.0,1224.141,715.852,ribosome,TS_73_6,4 +2311.395,1129.953,423.085,ribosome,TS_73_6,4 +2652.943,1048.475,883.276,ribosome,TS_73_6,4 +2435.147,921.61,1049.207,ribosome,TS_73_6,4 +2677.436,868.552,342.294,ribosome,TS_73_6,4 +2587.987,1253.328,771.87,ribosome,TS_73_6,4 +2214.444,2556.385,1187.927,ribosome,TS_73_6,4 +2213.117,2918.397,1252.563,ribosome,TS_73_6,4 +1535.442,3177.381,1205.003,ribosome,TS_73_6,4 +1010.903,3442.274,1142.175,ribosome,TS_73_6,4 +3126.686,4057.299,1156.437,ribosome,TS_73_6,4 +2182.44,1225.034,1076.338,ribosome,TS_73_6,4 +1841.78,1057.592,258.848,thyroglobulin,TS_73_6,4 +2387.281,2326.345,331.932,thyroglobulin,TS_73_6,4 +3242.6,452.621,489.296,thyroglobulin,TS_73_6,4 +409.479,1796.38,525.534,thyroglobulin,TS_73_6,4 +4582.511,1738.01,569.479,thyroglobulin,TS_73_6,4 +1402.102,2384.13,834.912,thyroglobulin,TS_73_6,4 +728.817,4151.187,967.208,thyroglobulin,TS_73_6,4 +6183.204,996.602,946.99,thyroglobulin,TS_73_6,4 +2025.333,4083.449,1013.484,thyroglobulin,TS_73_6,4 +3103.444,4679.447,1043.352,thyroglobulin,TS_73_6,4 +2073.311,3496.993,1163.75,thyroglobulin,TS_73_6,4 +743.959,238.379,1177.853,thyroglobulin,TS_73_6,4 +524.99,5021.579,1300.106,thyroglobulin,TS_73_6,4 +2870.0,1590.0,215.0,thyroglobulin,TS_73_6,4 +4842.5,412.5,438.333,thyroglobulin,TS_73_6,4 +901.401,5325.541,774.427,thyroglobulin,TS_73_6,4 +1885.771,2380.697,668.159,thyroglobulin,TS_73_6,4 +1875.447,2614.342,921.947,thyroglobulin,TS_73_6,4 +608.235,2829.608,478.693,thyroglobulin,TS_73_6,4 +910.265,3239.47,907.848,thyroglobulin,TS_73_6,4 +550.896,1484.776,625.075,thyroglobulin,TS_73_6,4 +4329.907,3719.346,547.009,thyroglobulin,TS_73_6,4 +4524.646,1289.394,73.081,thyroglobulin,TS_73_6,4 +2349.196,3677.366,735.089,thyroglobulin,TS_73_6,4 +3302.378,1831.524,794.207,thyroglobulin,TS_73_6,4 +448.929,2528.214,811.667,thyroglobulin,TS_73_6,4 +2694.397,4488.369,818.369,thyroglobulin,TS_73_6,4 +3452.853,2717.301,962.577,thyroglobulin,TS_73_6,4 +3023.031,6098.949,542.343,virus-like-particle,TS_73_6,4 +2133.513,5544.092,839.866,virus-like-particle,TS_73_6,4 +5175.693,1206.756,215.022,virus-like-particle,TS_73_6,4 +4849.137,666.384,412.947,virus-like-particle,TS_73_6,4 +6161.707,2651.433,465.402,virus-like-particle,TS_73_6,4 +6042.997,3306.628,713.333,virus-like-particle,TS_73_6,4 +5654.074,2942.728,61.945,virus-like-particle,TS_73_6,4 +6013.231,1871.433,680.048,virus-like-particle,TS_73_6,4 +2836.001,455.2,275.173,virus-like-particle,TS_73_6,4 +2482.763,295.84,690.206,virus-like-particle,TS_73_6,4 +3530.063,771.996,413.662,virus-like-particle,TS_73_6,4 +2687.924,2725.328,632.393,virus-like-particle,TS_73_6,4 +2782.653,1819.135,1019.265,virus-like-particle,TS_73_6,4 +1002.78,2972.855,700.217,virus-like-particle,TS_73_6,4 +679.125,2770.252,855.0,virus-like-particle,TS_73_6,4 +1379.817,2702.33,783.111,virus-like-particle,TS_73_6,4 +985.214,2133.973,598.405,virus-like-particle,TS_73_6,4 +1436.036,1368.571,989.781,virus-like-particle,TS_73_6,4 +1636.918,3798.904,1006.075,virus-like-particle,TS_73_6,4 +3298.877,4560.393,515.641,virus-like-particle,TS_73_6,4 +3093.113,4729.102,645.803,virus-like-particle,TS_73_6,4 +3343.273,4722.883,735.915,virus-like-particle,TS_73_6,4 +268.662,4730.318,916.115,apo-ferritin,TS_73_6,4 +238.946,4853.061,909.898,apo-ferritin,TS_73_6,4 +83.114,5729.56,1219.524,apo-ferritin,TS_73_6,4 +582.143,2769.968,1076.364,apo-ferritin,TS_73_6,4 +510.389,2157.244,362.438,apo-ferritin,TS_73_6,4 +619.176,1991.61,520.449,apo-ferritin,TS_73_6,4 +1245.704,2002.993,743.873,apo-ferritin,TS_73_6,4 +1290.839,1518.942,797.372,apo-ferritin,TS_73_6,4 +1328.307,1547.677,653.425,apo-ferritin,TS_73_6,4 +1352.893,1445.597,725.912,apo-ferritin,TS_73_6,4 +993.954,1705.229,366.405,apo-ferritin,TS_73_6,4 +751.138,1426.892,458.677,apo-ferritin,TS_73_6,4 +2350.062,2564.704,216.978,apo-ferritin,TS_73_6,4 +1210.503,5687.487,1217.588,apo-ferritin,TS_73_6,4 +1134.771,5679.086,898.514,apo-ferritin,TS_73_6,4 +1385.017,5077.747,364.983,apo-ferritin,TS_73_6,4 +1501.315,5047.829,332.783,apo-ferritin,TS_73_6,4 +5980.404,4244.379,478.416,apo-ferritin,TS_73_6,4 +5388.235,4480.0,84.706,apo-ferritin,TS_73_6,4 +5377.5,4507.5,85.833,apo-ferritin,TS_73_6,4 +5395.425,4746.993,39.085,apo-ferritin,TS_73_6,4 +5415.078,4630.078,54.219,apo-ferritin,TS_73_6,4 +5269.381,4808.761,43.717,apo-ferritin,TS_73_6,4 +5704.167,3290.0,85.833,apo-ferritin,TS_73_6,4 +5543.821,3528.374,47.154,apo-ferritin,TS_73_6,4 +5303.962,3755.786,58.176,apo-ferritin,TS_73_6,4 +1643.592,547.51,582.98,apo-ferritin,TS_73_6,4 +1725.534,494.356,644.849,apo-ferritin,TS_73_6,4 +3173.276,4450.517,1039.138,apo-ferritin,TS_73_6,4 +3121.796,4434.709,264.563,apo-ferritin,TS_73_6,4 +3009.867,4384.053,285.76,apo-ferritin,TS_73_6,4 +2947.351,4494.183,306.881,apo-ferritin,TS_73_6,4 +3410.451,4713.045,295.94,apo-ferritin,TS_73_6,4 +2367.675,4872.213,269.412,apo-ferritin,TS_73_6,4 +2273.256,4937.781,298.732,apo-ferritin,TS_73_6,4 +2295.068,4704.11,382.363,apo-ferritin,TS_73_6,4 +2309.494,4590.127,344.557,apo-ferritin,TS_73_6,4 +2345.93,4470.807,329.193,apo-ferritin,TS_73_6,4 +2468.306,4779.707,273.485,apo-ferritin,TS_73_6,4 +2381.72,4060.32,1021.0,apo-ferritin,TS_73_6,4 +2428.114,4271.673,1335.552,apo-ferritin,TS_73_6,4 +4396.744,5082.692,132.436,apo-ferritin,TS_73_6,4 +4620.519,2441.364,105.357,apo-ferritin,TS_73_6,4 +4618.52,2364.605,202.105,apo-ferritin,TS_73_6,4 +4639.096,2192.289,61.386,apo-ferritin,TS_73_6,4 +4635.698,2262.0,151.66,apo-ferritin,TS_73_6,4 +4192.5,2611.806,149.769,apo-ferritin,TS_73_6,4 +4239.923,2540.192,172.222,apo-ferritin,TS_73_6,4 +4286.667,2535.149,605.498,apo-ferritin,TS_73_6,4 +4349.217,2580.569,697.082,apo-ferritin,TS_73_6,4 +4393.977,2435.568,552.273,apo-ferritin,TS_73_6,4 +5024.323,2631.095,824.726,apo-ferritin,TS_73_6,4 +3229.887,3158.038,1059.283,apo-ferritin,TS_73_6,4 +3335.99,3213.02,1097.624,apo-ferritin,TS_73_6,4 +3086.108,1940.784,475.514,apo-ferritin,TS_73_6,4 +2990.879,1990.733,521.832,apo-ferritin,TS_73_6,4 +3155.737,2509.436,149.875,apo-ferritin,TS_73_6,4 +3287.952,2245.186,149.096,apo-ferritin,TS_73_6,4 +3383.0,2800.036,194.679,apo-ferritin,TS_73_6,4 +3615.771,1045.806,224.552,apo-ferritin,TS_73_6,4 +3646.689,1782.23,88.311,apo-ferritin,TS_73_6,4 +3585.105,1694.895,99.247,apo-ferritin,TS_73_6,4 +3807.282,2263.107,561.197,apo-ferritin,TS_73_6,4 +4331.237,779.541,209.293,apo-ferritin,TS_73_6,4 +4393.259,885.599,236.462,apo-ferritin,TS_73_6,4 +3621.489,341.667,73.652,apo-ferritin,TS_73_6,4 +2798.99,1315.505,241.045,apo-ferritin,TS_73_6,4 +2536.024,3417.62,256.355,apo-ferritin,TS_73_6,4 +1334.486,3988.322,802.226,apo-ferritin,TS_73_6,4 +1085.929,5158.59,465.579,apo-ferritin,TS_73_6,4 +1177.73,5141.064,425.529,apo-ferritin,TS_73_6,4 +1178.346,5227.0,345.429,apo-ferritin,TS_73_6,4 +1504.843,5180.322,355.442,apo-ferritin,TS_73_6,4 +5058.617,2754.639,826.027,apo-ferritin,TS_73_6,4 +3167.563,2035.642,455.566,apo-ferritin,TS_73_6,4 +1573.409,390.881,495.616,apo-ferritin,TS_73_6,4 +1634.084,507.488,455.566,apo-ferritin,TS_73_6,4 +1545.135,468.754,595.74,apo-ferritin,TS_73_6,4 +1584.637,454.698,685.852,apo-ferritin,TS_73_6,4 +1779.966,580.292,485.604,apo-ferritin,TS_73_6,4 +1582.199,582.679,355.442,apo-ferritin,TS_73_6,4 +2192.89,171.392,495.616,apo-ferritin,TS_73_6,4 +2076.604,200.819,575.716,apo-ferritin,TS_73_6,4 +2189.662,117.874,595.74,apo-ferritin,TS_73_6,4 +1957.082,145.518,465.579,apo-ferritin,TS_73_6,4 +1944.65,161.916,585.728,apo-ferritin,TS_73_6,4 +3270.772,3235.528,1016.263,apo-ferritin,TS_73_6,4 +3421.573,1462.485,305.38,apo-ferritin,TS_73_6,4 +3221.982,2918.325,265.33,apo-ferritin,TS_73_6,4 +3391.454,2889.264,335.417,apo-ferritin,TS_73_6,4 +3927.906,4070.523,225.28,apo-ferritin,TS_73_6,4 +4263.185,4747.497,225.28,apo-ferritin,TS_73_6,4 +4122.371,4852.662,225.28,apo-ferritin,TS_73_6,4 +4236.513,2479.339,515.641,apo-ferritin,TS_73_6,4 +4309.621,2389.686,485.604,apo-ferritin,TS_73_6,4 +4806.667,980.0,211.111,beta-amylase,TS_6_4,5 +2405.9,2058.159,762.887,beta-amylase,TS_6_4,5 +1489.928,1645.29,797.246,beta-amylase,TS_6_4,5 +4400.989,5358.575,751.563,beta-amylase,TS_6_4,5 +3791.75,4560.917,811.0,beta-amylase,TS_6_4,5 +3495.477,4704.215,901.938,beta-amylase,TS_6_4,5 +3876.774,5060.387,1484.194,beta-amylase,TS_6_4,5 +4501.951,4290.139,1074.704,beta-amylase,TS_6_4,5 +1318.542,5045.417,1136.875,beta-amylase,TS_6_4,5 +804.615,1977.846,489.385,beta-galactosidase,TS_6_4,5 +2134.942,3647.023,657.082,beta-galactosidase,TS_6_4,5 +5361.881,5472.376,1032.772,beta-galactosidase,TS_6_4,5 +4130.0,3716.667,465.0,beta-galactosidase,TS_6_4,5 +3072.5,4610.0,1090.0,beta-galactosidase,TS_6_4,5 +2151.626,4674.959,1189.35,beta-galactosidase,TS_6_4,5 +2367.485,5196.81,587.178,beta-galactosidase,TS_6_4,5 +2680.0,2415.476,982.381,beta-galactosidase,TS_6_4,5 +250.577,1621.923,850.962,beta-galactosidase,TS_6_4,5 +1322.147,2720.516,952.283,beta-galactosidase,TS_6_4,5 +1514.91,1642.699,821.722,beta-galactosidase,TS_6_4,5 +4483.644,4570.968,1089.313,beta-galactosidase,TS_6_4,5 +5274.903,5288.121,619.798,ribosome,TS_6_4,5 +5493.057,5181.127,726.624,ribosome,TS_6_4,5 +5656.951,5168.655,421.572,ribosome,TS_6_4,5 +5559.324,5426.553,556.878,ribosome,TS_6_4,5 +5312.439,4964.156,602.574,ribosome,TS_6_4,5 +5130.837,5121.036,631.702,ribosome,TS_6_4,5 +5453.083,4696.154,504.224,ribosome,TS_6_4,5 +5635.855,4803.903,706.456,ribosome,TS_6_4,5 +5106.838,4835.263,619.225,ribosome,TS_6_4,5 +5734.44,5547.349,1265.455,ribosome,TS_6_4,5 +5571.792,5790.483,1296.079,ribosome,TS_6_4,5 +285.661,4791.067,779.819,ribosome,TS_6_4,5 +229.331,4316.582,690.387,ribosome,TS_6_4,5 +232.042,4432.901,1135.128,ribosome,TS_6_4,5 +348.532,4570.763,717.301,ribosome,TS_6_4,5 +97.34,4661.182,1144.298,ribosome,TS_6_4,5 +143.939,5029.347,732.139,ribosome,TS_6_4,5 +1273.648,4386.703,631.656,ribosome,TS_6_4,5 +1223.64,4186.768,819.895,ribosome,TS_6_4,5 +1236.034,4674.307,668.476,ribosome,TS_6_4,5 +650.107,3282.821,1144.294,ribosome,TS_6_4,5 +830.635,3537.878,963.135,ribosome,TS_6_4,5 +903.085,3391.971,1118.573,ribosome,TS_6_4,5 +630.784,3622.607,1171.482,ribosome,TS_6_4,5 +440.984,3866.177,643.857,ribosome,TS_6_4,5 +583.565,4051.159,828.723,ribosome,TS_6_4,5 +759.417,4295.623,734.899,ribosome,TS_6_4,5 +898.55,4141.046,883.996,ribosome,TS_6_4,5 +868.611,3893.736,743.916,ribosome,TS_6_4,5 +802.156,4625.141,708.375,ribosome,TS_6_4,5 +326.143,3537.823,937.014,ribosome,TS_6_4,5 +1504.444,3275.214,1257.063,ribosome,TS_6_4,5 +2644.576,4431.906,990.296,ribosome,TS_6_4,5 +2444.258,4567.69,1141.142,ribosome,TS_6_4,5 +2468.644,4101.445,1115.899,ribosome,TS_6_4,5 +3078.08,6072.239,1009.897,ribosome,TS_6_4,5 +3015.779,5943.612,1281.03,ribosome,TS_6_4,5 +3702.491,5596.93,1336.535,ribosome,TS_6_4,5 +1997.267,1411.194,1149.58,ribosome,TS_6_4,5 +884.666,2137.612,880.833,ribosome,TS_6_4,5 +663.969,2316.198,1203.572,ribosome,TS_6_4,5 +914.745,2431.255,1268.81,ribosome,TS_6_4,5 +394.061,658.244,750.429,ribosome,TS_6_4,5 +921.916,278.031,597.735,ribosome,TS_6_4,5 +1004.534,86.606,524.534,ribosome,TS_6_4,5 +1284.212,412.471,737.595,ribosome,TS_6_4,5 +1370.425,534.452,920.29,ribosome,TS_6_4,5 +619.064,850.866,358.126,ribosome,TS_6_4,5 +1015.725,640.198,622.387,ribosome,TS_6_4,5 +896.557,967.706,588.803,ribosome,TS_6_4,5 +1017.985,990.96,798.864,ribosome,TS_6_4,5 +898.732,759.425,1040.914,ribosome,TS_6_4,5 +1860.317,365.754,336.113,ribosome,TS_6_4,5 +1689.471,123.33,410.727,ribosome,TS_6_4,5 +886.465,1363.99,1164.549,ribosome,TS_6_4,5 +5007.198,2893.632,998.903,ribosome,TS_6_4,5 +4847.396,2618.105,908.936,ribosome,TS_6_4,5 +5318.939,2139.613,824.39,ribosome,TS_6_4,5 +5186.69,2325.778,977.738,ribosome,TS_6_4,5 +4891.054,1969.255,839.404,ribosome,TS_6_4,5 +4700.372,2111.558,1051.613,ribosome,TS_6_4,5 +4585.681,1839.145,1089.516,ribosome,TS_6_4,5 +4545.69,1538.897,425.389,ribosome,TS_6_4,5 +4576.092,1520.016,1009.884,ribosome,TS_6_4,5 +4654.602,1197.481,806.34,ribosome,TS_6_4,5 +2926.093,2721.038,955.726,ribosome,TS_6_4,5 +2928.412,2194.561,1051.608,ribosome,TS_6_4,5 +2930.904,1857.678,1081.158,ribosome,TS_6_4,5 +2971.267,127.642,534.313,ribosome,TS_6_4,5 +3098.923,2941.184,836.926,ribosome,TS_6_4,5 +3294.228,6090.415,1326.649,ribosome,TS_6_4,5 +5736.406,6034.086,1176.462,ribosome,TS_6_4,5 +503.142,393.241,485.604,ribosome,TS_6_4,5 +5336.84,2766.878,946.176,ribosome,TS_6_4,5 +5251.785,2090.452,490.516,thyroglobulin,TS_6_4,5 +5704.207,2059.378,481.002,thyroglobulin,TS_6_4,5 +5926.681,1787.182,496.498,thyroglobulin,TS_6_4,5 +4967.648,2249.978,522.505,thyroglobulin,TS_6_4,5 +1242.919,1464.644,581.353,thyroglobulin,TS_6_4,5 +3547.616,4770.646,595.253,thyroglobulin,TS_6_4,5 +4756.011,4447.041,581.386,thyroglobulin,TS_6_4,5 +3899.562,5206.682,624.795,thyroglobulin,TS_6_4,5 +4818.823,5953.736,667.288,thyroglobulin,TS_6_4,5 +910.287,2806.175,729.882,thyroglobulin,TS_6_4,5 +4305.698,3168.196,757.859,thyroglobulin,TS_6_4,5 +2447.604,3079.107,791.723,thyroglobulin,TS_6_4,5 +3238.774,4832.551,929.867,thyroglobulin,TS_6_4,5 +316.535,1807.724,1089.125,thyroglobulin,TS_6_4,5 +2022.746,4815.141,1054.014,thyroglobulin,TS_6_4,5 +6060.0,5590.0,150.0,thyroglobulin,TS_6_4,5 +1479.773,2517.727,424.545,thyroglobulin,TS_6_4,5 +1770.114,2095.398,713.92,thyroglobulin,TS_6_4,5 +1732.632,2841.579,1069.043,thyroglobulin,TS_6_4,5 +867.222,1503.333,658.333,thyroglobulin,TS_6_4,5 +1668.75,1150.0,758.75,thyroglobulin,TS_6_4,5 +1990.417,5721.667,858.333,thyroglobulin,TS_6_4,5 +2613.889,3310.0,727.778,thyroglobulin,TS_6_4,5 +6035.196,3635.735,1175.735,thyroglobulin,TS_6_4,5 +4056.946,3386.256,432.266,thyroglobulin,TS_6_4,5 +3753.041,5168.249,1048.157,thyroglobulin,TS_6_4,5 +6000.663,1943.878,948.52,thyroglobulin,TS_6_4,5 +3553.393,2129.821,561.696,thyroglobulin,TS_6_4,5 +4490.0,4330.0,747.5,thyroglobulin,TS_6_4,5 +5785.405,4142.342,1002.77,thyroglobulin,TS_6_4,5 +911.29,5638.402,671.21,virus-like-particle,TS_6_4,5 +122.332,5627.746,1066.367,virus-like-particle,TS_6_4,5 +1452.865,4795.545,1116.066,virus-like-particle,TS_6_4,5 +5580.108,1240.86,692.222,virus-like-particle,TS_6_4,5 +4765.58,3469.964,689.813,virus-like-particle,TS_6_4,5 +5088.704,4120.923,981.513,virus-like-particle,TS_6_4,5 +4268.076,2814.277,815.446,virus-like-particle,TS_6_4,5 +5211.319,5766.513,877.832,virus-like-particle,TS_6_4,5 +4509.57,5139.077,1161.95,virus-like-particle,TS_6_4,5 +6045.947,2340.359,745.927,virus-like-particle,TS_6_4,5 +616.51,2880.471,1294.039,apo-ferritin,TS_6_4,5 +1099.033,1820.423,371.571,apo-ferritin,TS_6_4,5 +1019.831,1859.831,400.424,apo-ferritin,TS_6_4,5 +959.708,1708.149,606.039,apo-ferritin,TS_6_4,5 +1010.329,1758.816,718.52,apo-ferritin,TS_6_4,5 +6116.31,2609.963,528.155,apo-ferritin,TS_6_4,5 +761.116,4901.459,1025.15,apo-ferritin,TS_6_4,5 +969.876,4757.709,1085.418,apo-ferritin,TS_6_4,5 +1648.828,4760.276,997.759,apo-ferritin,TS_6_4,5 +4984.373,5655.125,993.835,apo-ferritin,TS_6_4,5 +4320.826,5556.391,694.862,apo-ferritin,TS_6_4,5 +3683.627,5175.882,813.922,apo-ferritin,TS_6_4,5 +3448.962,5244.962,861.423,apo-ferritin,TS_6_4,5 +4215.917,4821.592,1046.194,apo-ferritin,TS_6_4,5 +2094.233,3235.89,953.497,apo-ferritin,TS_6_4,5 +4793.069,3457.834,1102.852,apo-ferritin,TS_6_4,5 +4804.171,3440.284,1208.91,apo-ferritin,TS_6_4,5 +4550.54,2147.492,218.857,apo-ferritin,TS_6_4,5 +4495.06,2052.976,296.726,apo-ferritin,TS_6_4,5 +4471.074,2131.577,383.456,apo-ferritin,TS_6_4,5 +4126.963,1933.063,190.969,apo-ferritin,TS_6_4,5 +3576.426,2612.49,304.859,apo-ferritin,TS_6_4,5 +3634.286,2431.136,572.418,apo-ferritin,TS_6_4,5 +3534.854,2407.485,425.439,apo-ferritin,TS_6_4,5 +4094.142,2567.337,295.148,apo-ferritin,TS_6_4,5 +4114.734,2446.095,331.538,apo-ferritin,TS_6_4,5 +3894.125,2456.271,258.185,apo-ferritin,TS_6_4,5 +3752.045,2505.17,411.989,apo-ferritin,TS_6_4,5 +3729.36,2566.163,517.093,apo-ferritin,TS_6_4,5 +3836.309,2673.691,474.128,apo-ferritin,TS_6_4,5 +3846.464,2570.493,545.246,apo-ferritin,TS_6_4,5 +3942.417,2660.695,413.609,apo-ferritin,TS_6_4,5 +3965.042,2591.765,574.37,apo-ferritin,TS_6_4,5 +4067.936,2609.929,620.107,apo-ferritin,TS_6_4,5 +4009.57,2947.043,412.634,apo-ferritin,TS_6_4,5 +4095.423,2885.423,469.801,apo-ferritin,TS_6_4,5 +4037.127,2899.179,280.336,apo-ferritin,TS_6_4,5 +4115.619,2817.426,332.054,apo-ferritin,TS_6_4,5 +4036.695,2773.898,490.254,apo-ferritin,TS_6_4,5 +4132.45,2712.15,541.35,apo-ferritin,TS_6_4,5 +3913.495,2803.827,494.974,apo-ferritin,TS_6_4,5 +4211.583,3077.606,520.463,apo-ferritin,TS_6_4,5 +4120.404,3128.081,582.424,apo-ferritin,TS_6_4,5 +4356.198,2834.341,298.892,apo-ferritin,TS_6_4,5 +4282.932,2913.759,318.195,apo-ferritin,TS_6_4,5 +4243.778,2985.682,420.398,apo-ferritin,TS_6_4,5 +4172.383,2955.235,626.173,apo-ferritin,TS_6_4,5 +4054.277,3025.843,547.47,apo-ferritin,TS_6_4,5 +4070.927,2541.192,935.695,apo-ferritin,TS_6_4,5 +3357.295,2074.139,225.328,apo-ferritin,TS_6_4,5 +2153.09,846.979,252.847,apo-ferritin,TS_6_4,5 +3596.958,2593.013,415.516,apo-ferritin,TS_6_4,5 +3486.946,2616.549,415.516,apo-ferritin,TS_6_4,5 +3631.179,2496.627,415.516,apo-ferritin,TS_6_4,5 +776.293,4893.666,1196.487,apo-ferritin,TS_6_4,5 +940.013,4882.972,1046.3,apo-ferritin,TS_6_4,5 +843.54,4841.021,1116.388,apo-ferritin,TS_6_4,5 +2326.455,5821.083,917.063,apo-ferritin,TS_6_4,5 +3850.0,1010.0,310.0,beta-amylase,TS_6_6,6 +4063.606,1651.538,239.038,beta-amylase,TS_6_6,6 +3229.286,1990.0,745.714,beta-amylase,TS_6_6,6 +3367.549,1766.986,676.789,beta-amylase,TS_6_6,6 +3241.333,2196.667,1108.667,beta-amylase,TS_6_6,6 +2695.942,1422.556,380.064,beta-amylase,TS_6_6,6 +2657.277,1564.851,477.787,beta-amylase,TS_6_6,6 +402.895,4085.0,1514.474,beta-amylase,TS_6_6,6 +1837.341,4133.064,835.607,beta-amylase,TS_6_6,6 +1485.96,3785.563,277.881,beta-amylase,TS_6_6,6 +2183.235,3039.118,670.588,beta-amylase,TS_6_6,6 +5169.63,5237.037,709.259,beta-amylase,TS_6_6,6 +4910.0,4665.714,738.857,beta-amylase,TS_6_6,6 +4580.0,5189.756,1065.528,beta-amylase,TS_6_6,6 +5524.088,3251.226,619.877,beta-galactosidase,TS_6_6,6 +5046.434,1586.382,988.516,beta-galactosidase,TS_6_6,6 +4640.0,1865.0,685.0,beta-galactosidase,TS_6_6,6 +1777.5,3650.417,600.0,beta-galactosidase,TS_6_6,6 +1343.732,5939.859,1039.93,beta-galactosidase,TS_6_6,6 +4634.583,4465.417,806.667,beta-galactosidase,TS_6_6,6 +4791.0,4911.6,473.6,beta-galactosidase,TS_6_6,6 +3827.744,5276.792,487.469,beta-galactosidase,TS_6_6,6 +4020.131,3614.629,916.419,beta-galactosidase,TS_6_6,6 +4002.222,5008.421,1137.719,beta-galactosidase,TS_6_6,6 +2440.282,2449.981,1140.433,beta-galactosidase,TS_6_6,6 +773.644,4975.664,1499.213,ribosome,TS_6_6,6 +1029.655,5119.687,1332.989,ribosome,TS_6_6,6 +2106.09,4846.302,1289.305,ribosome,TS_6_6,6 +2843.254,5123.492,1377.645,ribosome,TS_6_6,6 +3676.182,4058.335,1228.49,ribosome,TS_6_6,6 +5697.427,1950.562,972.48,ribosome,TS_6_6,6 +2452.111,3000.991,1108.533,ribosome,TS_6_6,6 +4305.75,4499.335,1214.481,ribosome,TS_6_6,6 +3740.366,4541.466,1101.761,ribosome,TS_6_6,6 +5405.801,2023.684,725.902,ribosome,TS_6_6,6 +5432.587,1978.296,1026.276,ribosome,TS_6_6,6 +5343.945,2419.647,826.027,ribosome,TS_6_6,6 +5182.757,2111.002,826.027,ribosome,TS_6_6,6 +5442.264,2473.504,1116.388,ribosome,TS_6_6,6 +5702.357,2282.812,916.139,ribosome,TS_6_6,6 +5156.231,2251.003,1036.288,ribosome,TS_6_6,6 +4984.874,4435.309,1086.35,ribosome,TS_6_6,6 +4816.361,4198.518,1076.338,ribosome,TS_6_6,6 +4333.457,4506.473,933.065,ribosome,TS_6_6,6 +3821.058,4739.232,976.213,ribosome,TS_6_6,6 +4248.584,4224.388,1176.462,ribosome,TS_6_6,6 +782.628,5388.786,1436.786,ribosome,TS_6_6,6 +585.528,5169.929,1436.786,ribosome,TS_6_6,6 +4709.027,5996.216,316.811,thyroglobulin,TS_6_6,6 +2822.785,2974.861,443.679,thyroglobulin,TS_6_6,6 +2974.63,5786.818,491.11,thyroglobulin,TS_6_6,6 +4622.227,4238.002,506.417,thyroglobulin,TS_6_6,6 +1743.59,605.843,529.632,thyroglobulin,TS_6_6,6 +4492.1,2145.4,584.3,thyroglobulin,TS_6_6,6 +2408.626,2515.879,632.912,thyroglobulin,TS_6_6,6 +6004.797,1528.455,670.081,thyroglobulin,TS_6_6,6 +6065.226,782.511,706.026,thyroglobulin,TS_6_6,6 +3444.054,1444.976,785.389,thyroglobulin,TS_6_6,6 +4327.106,2286.262,879.085,thyroglobulin,TS_6_6,6 +2092.183,1968.936,946.212,thyroglobulin,TS_6_6,6 +2141.26,4493.054,956.843,thyroglobulin,TS_6_6,6 +5348.617,5636.545,943.527,thyroglobulin,TS_6_6,6 +2505.834,1629.141,1118.847,thyroglobulin,TS_6_6,6 +760.567,4157.394,1146.652,thyroglobulin,TS_6_6,6 +228.897,2923.352,1286.104,thyroglobulin,TS_6_6,6 +524.918,2713.886,1291.522,thyroglobulin,TS_6_6,6 +3302.056,834.556,1336.967,thyroglobulin,TS_6_6,6 +586.238,3201.857,1507.786,thyroglobulin,TS_6_6,6 +3750.577,1852.692,950.385,thyroglobulin,TS_6_6,6 +3766.612,1578.415,588.798,thyroglobulin,TS_6_6,6 +5407.907,1216.163,407.558,thyroglobulin,TS_6_6,6 +3113.699,4522.603,960.342,thyroglobulin,TS_6_6,6 +3466.387,4875.21,408.571,thyroglobulin,TS_6_6,6 +1856.175,4545.519,1185.519,thyroglobulin,TS_6_6,6 +922.5,5110.0,1055.0,thyroglobulin,TS_6_6,6 +1184.054,5135.135,337.568,thyroglobulin,TS_6_6,6 +2511.073,1898.707,818.244,thyroglobulin,TS_6_6,6 +4817.354,4860.628,979.955,thyroglobulin,TS_6_6,6 +4461.316,3580.702,1067.544,thyroglobulin,TS_6_6,6 +2947.812,1297.5,1109.688,thyroglobulin,TS_6_6,6 +4719.918,1853.143,1229.265,thyroglobulin,TS_6_6,6 +1543.079,3649.778,1353.424,thyroglobulin,TS_6_6,6 +970.18,3473.559,1473.604,thyroglobulin,TS_6_6,6 +4113.927,1540.028,957.754,virus-like-particle,TS_6_6,6 +4205.011,514.238,377.245,virus-like-particle,TS_6_6,6 +3546.662,996.468,1199.592,virus-like-particle,TS_6_6,6 +238.569,3475.323,722.892,virus-like-particle,TS_6_6,6 +404.421,2707.26,970.408,virus-like-particle,TS_6_6,6 +5686.068,5829.641,364.486,virus-like-particle,TS_6_6,6 +5086.352,6157.563,696.444,virus-like-particle,TS_6_6,6 +4657.586,5875.914,627.971,virus-like-particle,TS_6_6,6 +5827.109,4997.341,854.623,virus-like-particle,TS_6_6,6 +5157.398,4441.079,683.717,virus-like-particle,TS_6_6,6 +5278.371,3038.921,406.17,virus-like-particle,TS_6_6,6 +3005.1,4116.745,533.11,virus-like-particle,TS_6_6,6 +2829.054,4175.449,828.66,virus-like-particle,TS_6_6,6 +3248.07,4513.06,582.863,virus-like-particle,TS_6_6,6 +2609.876,4569.876,1169.759,virus-like-particle,TS_6_6,6 +2213.287,4135.017,1286.851,virus-like-particle,TS_6_6,6 +3303.905,5697.825,789.744,virus-like-particle,TS_6_6,6 +1008.748,5949.213,1077.303,virus-like-particle,TS_6_6,6 +5749.052,3911.392,275.342,virus-like-particle,TS_6_6,6 +1916.83,3311.797,754.673,apo-ferritin,TS_6_6,6 +1996.861,3231.277,803.577,apo-ferritin,TS_6_6,6 +2206.512,2975.302,1179.674,apo-ferritin,TS_6_6,6 +285.292,1379.331,417.577,apo-ferritin,TS_6_6,6 +753.781,2633.219,973.094,apo-ferritin,TS_6_6,6 +726.176,2559.49,1473.314,apo-ferritin,TS_6_6,6 +747.829,2630.698,1549.302,apo-ferritin,TS_6_6,6 +1123.151,2698.296,725.852,apo-ferritin,TS_6_6,6 +1234.692,2994.751,920.469,apo-ferritin,TS_6_6,6 +332.707,3382.279,1241.909,apo-ferritin,TS_6_6,6 +766.772,3170.667,1543.544,apo-ferritin,TS_6_6,6 +1569.207,2394.552,1299.69,apo-ferritin,TS_6_6,6 +818.107,3625.562,671.657,apo-ferritin,TS_6_6,6 +1146.957,1508.462,679.398,apo-ferritin,TS_6_6,6 +4586.585,476.585,763.78,apo-ferritin,TS_6_6,6 +4528.663,551.489,773.374,apo-ferritin,TS_6_6,6 +4905.12,1090.843,266.175,apo-ferritin,TS_6_6,6 +5344.971,446.0,60.429,apo-ferritin,TS_6_6,6 +5250.109,3132.92,842.628,apo-ferritin,TS_6_6,6 +5066.127,1954.349,293.048,apo-ferritin,TS_6_6,6 +5671.396,2753.784,797.838,apo-ferritin,TS_6_6,6 +5794.353,2457.302,441.691,apo-ferritin,TS_6_6,6 +3355.743,2365.446,1138.614,apo-ferritin,TS_6_6,6 +3101.77,1278.918,264.197,apo-ferritin,TS_6_6,6 +2705.768,1889.625,1354.642,apo-ferritin,TS_6_6,6 +3109.837,1800.782,954.072,apo-ferritin,TS_6_6,6 +4212.422,2013.806,305.882,apo-ferritin,TS_6_6,6 +4057.61,5767.868,576.581,apo-ferritin,TS_6_6,6 +3153.077,4790.538,460.577,apo-ferritin,TS_6_6,6 +2977.845,4634.696,444.254,apo-ferritin,TS_6_6,6 +3332.697,5251.364,720.576,apo-ferritin,TS_6_6,6 +3442.903,5827.204,1080.502,apo-ferritin,TS_6_6,6 +318.185,5000.579,551.158,apo-ferritin,TS_6_6,6 +1776.173,4887.716,1019.877,apo-ferritin,TS_6_6,6 +3578.705,3480.026,1206.5,apo-ferritin,TS_6_6,6 +5680.786,2759.578,936.164,apo-ferritin,TS_6_6,6 +3432.444,1208.43,826.027,apo-ferritin,TS_6_6,6 +3074.149,1859.506,846.052,apo-ferritin,TS_6_6,6 +1949.78,3068.74,1156.437,apo-ferritin,TS_6_6,6 +3687.965,1751.484,1296.612,apo-ferritin,TS_6_6,6 +3684.71,1654.009,1306.624,apo-ferritin,TS_6_6,6 diff --git a/competitions/kaggle/Cryo-ET/1st_place_solution/utils.py b/competitions/kaggle/Cryo-ET/1st_place_solution/utils.py new file mode 100644 index 000000000..2f86c48e1 --- /dev/null +++ b/competitions/kaggle/Cryo-ET/1st_place_solution/utils.py @@ -0,0 +1,573 @@ +# Copyright (c) MONAI Consortium +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import random +import os +import numpy as np +import pandas as pd +import torch +from torch.utils.data import Sampler, RandomSampler, SequentialSampler, DataLoader, WeightedRandomSampler +from torch import nn, optim +from torch.optim.lr_scheduler import LambdaLR +from torch.optim import lr_scheduler +import importlib +import math + +import logging +import pickle + + +def get_linear_schedule_with_warmup(optimizer, num_warmup_steps, num_training_steps, last_epoch=-1): + """ + from https://github.com/huggingface/transformers/blob/main/src/transformers/optimization.py + Create a schedule with a learning rate that decreases linearly from the initial lr set in the optimizer to 0, after + a warmup period during which it increases linearly from 0 to the initial lr set in the optimizer. + Args: + optimizer ([`~torch.optim.Optimizer`]): + The optimizer for which to schedule the learning rate. + num_warmup_steps (`int`): + The number of steps for the warmup phase. + num_training_steps (`int`): + The total number of training steps. + last_epoch (`int`, *optional*, defaults to -1): + The index of the last epoch when resuming training. + Return: + `torch.optim.lr_scheduler.LambdaLR` with the appropriate schedule. + """ + + def lr_lambda(current_step: int): + if current_step < num_warmup_steps: + return float(current_step) / float(max(1, num_warmup_steps)) + return max(0.0, float(num_training_steps - current_step) / float(max(1, num_training_steps - num_warmup_steps))) + + return LambdaLR(optimizer, lr_lambda, last_epoch) + + +def get_cosine_schedule_with_warmup(optimizer, num_warmup_steps, num_training_steps, num_cycles=0.5, last_epoch=-1): + """ + from https://github.com/huggingface/transformers/blob/main/src/transformers/optimization.py + Create a schedule with a learning rate that decreases following the values of the cosine function between the + initial lr set in the optimizer to 0, after a warmup period during which it increases linearly between 0 and the + initial lr set in the optimizer. + Args: + optimizer ([`~torch.optim.Optimizer`]): + The optimizer for which to schedule the learning rate. + num_warmup_steps (`int`): + The number of steps for the warmup phase. + num_training_steps (`int`): + The total number of training steps. + num_cycles (`float`, *optional*, defaults to 0.5): + The number of waves in the cosine schedule (the defaults is to just decrease from the max value to 0 + following a half-cosine). + last_epoch (`int`, *optional*, defaults to -1): + The index of the last epoch when resuming training. + Return: + `torch.optim.lr_scheduler.LambdaLR` with the appropriate schedule. + """ + + def lr_lambda(current_step): + if current_step < num_warmup_steps: + return float(current_step) / float(max(1, num_warmup_steps)) + progress = float(current_step - num_warmup_steps) / float(max(1, num_training_steps - num_warmup_steps)) + return max(0.0, 0.5 * (1.0 + math.cos(math.pi * float(num_cycles) * 2.0 * progress))) + + return LambdaLR(optimizer, lr_lambda, last_epoch) + + +def calc_grad_norm(parameters, norm_type=2.0): + + if isinstance(parameters, torch.Tensor): + parameters = [parameters] + parameters = [p for p in parameters if p.grad is not None] + norm_type = float(norm_type) + if len(parameters) == 0: + return torch.tensor(0.0) + device = parameters[0].grad.device + total_norm = torch.norm( + torch.stack([torch.norm(p.grad.detach(), norm_type).to(device) for p in parameters]), norm_type + ) + if torch.logical_or(total_norm.isnan(), total_norm.isinf()): + total_norm = None + + return total_norm + + +def calc_weight_norm(parameters, norm_type=2.0): + + # l2_loss = 0 + # for param in parameters : + # l2_loss += 0.5 * torch.sum(param ** 2) + # return l2_loss + + if isinstance(parameters, torch.Tensor): + parameters = [parameters] + parameters = [p for p in parameters if p.grad is not None] + norm_type = float(norm_type) + if len(parameters) == 0: + return torch.tensor(0.0) + device = parameters[0].grad.device + + total_norm = torch.stack([torch.norm(p.detach(), norm_type).to(device) for p in parameters]).mean() + if torch.logical_or(total_norm.isnan(), total_norm.isinf()): + total_norm = None + + return total_norm + + +class OrderedDistributedSampler(Sampler): + def __init__(self, dataset, num_replicas=None, rank=None): + if num_replicas is None: + if not dist.is_available(): + raise RuntimeError("Requires distributed package to be available") + num_replicas = dist.get_world_size() + if rank is None: + if not dist.is_available(): + raise RuntimeError("Requires distributed package to be available") + rank = dist.get_rank() + self.dataset = dataset + self.num_replicas = num_replicas + self.rank = rank + self.num_samples = int(math.ceil(len(self.dataset) * 1.0 / self.num_replicas)) + self.total_size = self.num_samples * self.num_replicas + + print("TOTAL SIZE", self.total_size) + + def __iter__(self): + indices = list(range(len(self.dataset))) + + # add extra samples to make it evenly divisible + indices += indices[: (self.total_size - len(indices))] + assert len(indices) == self.total_size + + # subsample + indices = indices[self.rank * self.num_samples : self.rank * self.num_samples + self.num_samples] + print( + "SAMPLES", + self.rank * self.num_samples, + self.rank * self.num_samples + self.num_samples, + ) + assert len(indices) == self.num_samples + + return iter(indices) + + def __len__(self): + return self.num_samples + + +def sync_across_gpus(t, world_size): + torch.distributed.barrier() + gather_t_tensor = [torch.ones_like(t) for _ in range(world_size)] + torch.distributed.all_gather(gather_t_tensor, t) + return torch.cat(gather_t_tensor) + + +def set_seed(seed=1234): + random.seed(seed) + os.environ["PYTHONHASHSEED"] = str(seed) + np.random.seed(seed) + torch.manual_seed(seed) + torch.cuda.manual_seed(seed) + torch.backends.cudnn.deterministic = False + torch.backends.cudnn.benchmark = True + + +def worker_init_fn(worker_id): + np.random.seed(np.random.get_state()[1][0] + worker_id) + + +def get_model(cfg, ds): + Net = importlib.import_module(cfg.model).Net + net = Net(cfg) + if cfg.pretrained_weights is not None: + if type(cfg.pretrained_weights) == list: + cfg.pretrained_weights = cfg.pretrained_weights[cfg.fold] + print(f"{cfg.local_rank}: loading weights from", cfg.pretrained_weights) + state_dict = torch.load(cfg.pretrained_weights, map_location="cpu") + if "model" in state_dict.keys(): + state_dict = state_dict["model"] + state_dict = {key.replace("module.", ""): val for key, val in state_dict.items()} + if cfg.pop_weights is not None: + print(f"popping {cfg.pop_weights}") + to_pop = [] + for key in state_dict: + for item in cfg.pop_weights: + if item in key: + to_pop += [key] + for key in to_pop: + print(f"popping {key}") + state_dict.pop(key) + + net.load_state_dict(state_dict, strict=cfg.pretrained_weights_strict) + print(f"{cfg.local_rank}: weights loaded from", cfg.pretrained_weights) + + return net + + +def create_checkpoint(cfg, model, optimizer, epoch, scheduler=None, scaler=None): + + state_dict = model.state_dict() + if cfg.save_weights_only: + checkpoint = {"model": state_dict} + return checkpoint + + checkpoint = { + "model": state_dict, + "optimizer": optimizer.state_dict(), + "epoch": epoch, + } + + if scheduler is not None: + checkpoint["scheduler"] = scheduler.state_dict() + + if scaler is not None: + checkpoint["scaler"] = scaler.state_dict() + return checkpoint + + +def load_checkpoint(cfg, model, optimizer, scheduler=None, scaler=None): + + print(f"loading ckpt {cfg.resume_from}") + checkpoint = torch.load(cfg.resume_from, map_location="cpu") + model.load_state_dict(checkpoint["model"]) + optimizer.load_state_dict(checkpoint["optimizer"]) + scheduler_dict = checkpoint["scheduler"] + if scaler is not None: + scaler.load_state_dict(checkpoint["scaler"]) + + epoch = checkpoint["epoch"] + return model, optimizer, scheduler_dict, scaler, epoch + + +def get_dataset(df, cfg, mode="train"): + + # modes train, val, index + print(f"Loading {mode} dataset") + + if mode == "train": + dataset = get_train_dataset(df, cfg) + # elif mode == 'train_val': + # dataset = get_val_dataset(df, cfg) + elif mode == "val": + dataset = get_val_dataset(df, cfg) + elif mode == "test": + dataset = get_test_dataset(df, cfg) + else: + pass + return dataset + + +def get_dataloader(ds, cfg, mode="train"): + + if mode == "train": + dl = get_train_dataloader(ds, cfg) + elif mode == "val": + dl = get_val_dataloader(ds, cfg) + elif mode == "test": + dl = get_test_dataloader(ds, cfg) + return dl + + +def get_train_dataset(train_df, cfg): + + train_dataset = cfg.CustomDataset(train_df, cfg, aug=cfg.train_aug, mode="train") + if cfg.data_sample > 0: + train_dataset = torch.utils.data.Subset(train_dataset, np.arange(cfg.data_sample)) + return train_dataset + + +def get_train_dataloader(train_ds, cfg): + + if cfg.distributed: + sampler = torch.utils.data.distributed.DistributedSampler( + train_ds, num_replicas=cfg.world_size, rank=cfg.local_rank, shuffle=True, seed=cfg.seed + ) + else: + try: + if cfg.random_sampler_frac > 0: + + num_samples = int(len(train_ds) * cfg.random_sampler_frac) + sample_weights = train_ds.sample_weights + sampler = WeightedRandomSampler(sample_weights, num_samples=num_samples) + else: + sampler = None + except: + sampler = None + + if cfg.use_custom_batch_sampler: + sampler = RandomSampler(train_ds) + bsampler = CustomBatchSampler(sampler, batch_size=cfg.batch_size, drop_last=cfg.drop_last) + train_dataloader = DataLoader( + train_ds, + batch_sampler=bsampler, + # shuffle=(sampler is None), + # batch_size=cfg.batch_size, + num_workers=cfg.num_workers, + pin_memory=cfg.pin_memory, + collate_fn=cfg.tr_collate_fn, + # drop_last=cfg.drop_last, + worker_init_fn=worker_init_fn, + ) + else: + + train_dataloader = DataLoader( + train_ds, + sampler=sampler, + shuffle=(sampler is None), + batch_size=cfg.batch_size, + num_workers=cfg.num_workers, + pin_memory=cfg.pin_memory, + collate_fn=cfg.tr_collate_fn, + drop_last=cfg.drop_last, + worker_init_fn=worker_init_fn, + ) + print(f"train: dataset {len(train_ds)}, dataloader {len(train_dataloader)}") + return train_dataloader + + +def get_val_dataset(val_df, cfg, allowed_targets=None): + val_dataset = cfg.CustomDataset(val_df, cfg, aug=cfg.val_aug, mode="val") + return val_dataset + + +def get_val_dataloader(val_ds, cfg): + + if cfg.distributed and cfg.eval_ddp: + sampler = OrderedDistributedSampler(val_ds, num_replicas=cfg.world_size, rank=cfg.local_rank) + else: + sampler = SequentialSampler(val_ds) + + if cfg.batch_size_val is not None: + batch_size = cfg.batch_size_val + else: + batch_size = cfg.batch_size + val_dataloader = DataLoader( + val_ds, + sampler=sampler, + batch_size=batch_size, + num_workers=cfg.num_workers, + pin_memory=cfg.pin_memory, + collate_fn=cfg.val_collate_fn, + worker_init_fn=worker_init_fn, + ) + print(f"valid: dataset {len(val_ds)}, dataloader {len(val_dataloader)}") + return val_dataloader + + +def get_test_dataset(test_df, cfg): + test_dataset = cfg.CustomDataset(test_df, cfg, aug=cfg.val_aug, mode="test") + return test_dataset + + +def get_test_dataloader(test_ds, cfg): + + if cfg.distributed and cfg.eval_ddp: + sampler = OrderedDistributedSampler(test_ds, num_replicas=cfg.world_size, rank=cfg.local_rank) + else: + sampler = SequentialSampler(test_ds) + + if cfg.batch_size_val is not None: + batch_size = cfg.batch_size_val + else: + batch_size = cfg.batch_size + test_dataloader = DataLoader( + test_ds, + sampler=sampler, + batch_size=batch_size, + num_workers=cfg.num_workers, + pin_memory=cfg.pin_memory, + collate_fn=cfg.val_collate_fn, + worker_init_fn=worker_init_fn, + ) + print(f"test: dataset {len(test_ds)}, dataloader {len(test_dataloader)}") + return test_dataloader + + +def get_optimizer(model, cfg): + + params = model.parameters() + + if cfg.optimizer == "Adam": + optimizer = optim.Adam(params, lr=cfg.lr, weight_decay=cfg.weight_decay) + + elif cfg.optimizer == "AdamW_plus": + paras = list(model.named_parameters()) + no_decay = ["bias", "LayerNorm.bias"] + params = [ + { + "params": [param for name, param in paras if (not any(nd in name for nd in no_decay))], + "lr": cfg.lr, + "weight_decay": cfg.weight_decay, + }, + { + "params": [param for name, param in paras if (any(nd in name for nd in no_decay))], + "lr": cfg.lr, + "weight_decay": 0.0, + }, + ] + optimizer = optim.AdamW(params, lr=cfg.lr) + + elif cfg.optimizer == "AdamW": + optimizer = optim.AdamW(params, lr=cfg.lr, weight_decay=cfg.weight_decay) + + elif cfg.optimizer == "SGD": + optimizer = optim.SGD( + params, + lr=cfg.lr, + momentum=cfg.sgd_momentum, + nesterov=cfg.sgd_nesterov, + weight_decay=cfg.weight_decay, + ) + + return optimizer + + +def get_scheduler(cfg, optimizer, total_steps): + + if cfg.schedule == "steplr": + scheduler = optim.lr_scheduler.StepLR( + optimizer, + step_size=cfg.epochs_step * (total_steps // cfg.batch_size) // cfg.world_size, + gamma=0.5, + ) + elif cfg.schedule == "cosine": + scheduler = get_cosine_schedule_with_warmup( + optimizer, + num_warmup_steps=cfg.warmup * (total_steps // cfg.batch_size) // cfg.world_size, + num_training_steps=cfg.epochs * (total_steps // cfg.batch_size) // cfg.world_size, + num_cycles=cfg.num_cycles, + ) + elif cfg.schedule == "linear": + scheduler = get_linear_schedule_with_warmup( + optimizer, + num_warmup_steps=0, + num_training_steps=cfg.epochs * (total_steps // cfg.batch_size) // cfg.world_size, + ) + + elif cfg.schedule == "CosineAnnealingLR": + T_max = int(np.ceil(0.5 * total_steps)) + scheduler = lr_scheduler.CosineAnnealingLR(optimizer, T_max=T_max, eta_min=1e-8) + + # print("num_steps", (total_steps // cfg.batch_size) // cfg.world_size) + + else: + scheduler = None + + return scheduler + + +def read_df(fn): + + if "parquet" in fn: + df = pd.read_parquet(fn, engine="fastparquet") + else: + df = pd.read_csv(fn) + return df + + +def get_data(cfg): + + # setup dataset + if type(cfg.train_df) == list: + cfg.train_df = cfg.train_df[cfg.fold] + print(f"reading {cfg.train_df}") + df = read_df(cfg.train_df) + + if cfg.test: + test_df = read_df(cfg.test_df) + else: + test_df = None + + if cfg.val_df: + if type(cfg.val_df) == list: + cfg.val_df = cfg.val_df[cfg.fold] + val_df = read_df(cfg.val_df) + if cfg.fold > -1: + if "fold" in val_df.columns: + val_df = val_df[val_df["fold"] == cfg.fold] + train_df = df[df["fold"] != cfg.fold] + else: + train_df = df + else: + train_df = df + else: + if cfg.fold == -1: + val_df = df[df["fold"] == 0] + else: + val_df = df[df["fold"] == cfg.fold] + + train_df = df[df["fold"] != cfg.fold] + + return train_df, val_df, test_df + + +def upload_s3(cfg): + from boto3.session import Session + import boto3 + + BUCKET_NAME = cfg.s3_bucket_name + ACCESS_KEY = cfg.s3_access_key + SECRET_KEY = cfg.s3_secret_key + session = Session(aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY) + s3 = session.resource("s3") + + s3.Bucket(BUCKET_NAME).upload_file( + f"{cfg.output_dir}/fold{cfg.fold}/val_data_seed{cfg.seed}.pth", + f"output/{cfg.name}/fold{cfg.fold}/val_data_seed{cfg.seed}.pth", + ) + s3.Bucket(BUCKET_NAME).upload_file( + f"{cfg.output_dir}/fold{cfg.fold}/test_data_seed{cfg.seed}.pth", + f"output/{cfg.name}/fold{cfg.fold}/test_data_seed{cfg.seed}.pth", + ) + s3.Bucket(BUCKET_NAME).upload_file( + f"{cfg.output_dir}/fold{cfg.fold}/submission_seed{cfg.seed}.csv", + f"output/{cfg.name}/fold{cfg.fold}/submission_seed{cfg.seed}.csv", + ) + + +def flatten(t): + return [item for sublist in t for item in sublist] + + +def set_pandas_display(): + pd.set_option("display.max_columns", None) + pd.set_option("display.max_rows", 10000) + pd.set_option("display.width", 10000) + pd.set_option("display.float_format", lambda x: "%.3f" % x) + + +def dumpobj(file, obj): + with open(file, "wb") as handle: + pickle.dump(obj, handle, protocol=pickle.HIGHEST_PROTOCOL) + + +def loadobj(file): + with open(file, "rb") as handle: + return pickle.load(handle) + + +def get_level(level_str): + """get level""" + l_names = {logging.getLevelName(lvl).lower(): lvl for lvl in [10, 20, 30, 40, 50]} # noqa + return l_names.get(level_str.lower(), logging.INFO) + + +def get_logger(name, level_str): + """get logger""" + logger = logging.getLogger(name) + logger.setLevel(get_level(level_str)) + handler = logging.StreamHandler() + handler.setLevel(level_str) + handler.setFormatter( + logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") + ) # pylint: disable=C0301 # noqa + logger.addHandler(handler) + + return logger diff --git a/figures/partly_Unet.png b/figures/partly_Unet.png new file mode 100644 index 000000000..7f4f72cdb Binary files /dev/null and b/figures/partly_Unet.png differ