ACE-neuro: Miniscope pipeline tutorial¶

Step-by-step calcium imaging path: same project_path / data_path story as ephys, then MiniscopeDataManager → MiniscopePreprocessor → MiniscopeProcessor → MiniscopePostprocessor (same order as MiniscopePipeline.run).

Prerequisites¶

• ACE-neuro in an environment with CaImAn. Recommended: mamba env create -f linux_environment.yml (or conda), conda activate caiman, then pip install -e . --no-deps. See docs Getting started for Linux vs macOS/Windows notes.
• project_path with experiments.csv + analysis_parameters.csv.
• data_path where miniscope movies / metadata live for this line_num (UCLA V3 vs ONIX V4 is auto-detected from files — see Creating new data loaders).

Project layout (same two-path rule)¶

Path	Contents
project_path	experiments.csv, analysis_parameters.csv
data_path	Raw miniscope files (e.g. .avi, metaData.json, etc.)

analysis_parameters.csv supplies crop_coords when you pass crop_coords=None (or non-headless GUI flows).

In [ ]:

from pathlib import Path

project_path = Path("/path/to/your/project")
data_path = Path("/path/to/your/raw_data")
line_num = 96
filenames = ["0.avi"]  # match your session; CLI defaults to this pattern
headless = True  # True disables crop/curation GUIs (cluster / docs friendly)

for label, p in ("experiments.csv", project_path / "experiments.csv"), (
    "analysis_parameters.csv",
    project_path / "analysis_parameters.csv",
):
    if not p.is_file():
        raise FileNotFoundError(f"Missing {label} at {p}")
print("OK: project CSVs found.")

from pathlib import Path project_path = Path("/path/to/your/project") data_path = Path("/path/to/your/raw_data") line_num = 96 filenames = ["0.avi"] # match your session; CLI defaults to this pattern headless = True # True disables crop/curation GUIs (cluster / docs friendly) for label, p in ("experiments.csv", project_path / "experiments.csv"), ( "analysis_parameters.csv", project_path / "analysis_parameters.csv", ): if not p.is_file(): raise FileNotFoundError(f"Missing {label} at {p}") print("OK: project CSVs found.")

In [ ]:

from ace_neuro.shared.experiment_data_manager import ExperimentDataManager

edm = ExperimentDataManager(line_num, project_path=project_path, data_path=data_path)
print("calcium imaging dir:", edm.metadata.get("calcium imaging directory") if edm.metadata else None)
print("analysis params keys (sample):", list(edm.analysis_params.keys())[:12] if edm.analysis_params else [])

from ace_neuro.shared.experiment_data_manager import ExperimentDataManager edm = ExperimentDataManager(line_num, project_path=project_path, data_path=data_path) print("calcium imaging dir:", edm.metadata.get("calcium imaging directory") if edm.metadata else None) print("analysis params keys (sample):", list(edm.analysis_params.keys())[:12] if edm.analysis_params else [])

Table of contents¶

0. Paths + CSV validation
1. filenames and headless
2. MiniscopeDataManager.create
3. Preprocess (crop, detrend, ΔF/F)
4. Process (motion correction, CNMF-E)
5. Postprocess (events, phase, filter, spectrogram)
6. Inspect MiniscopeDataManager
7. One-shot MiniscopePipeline.run
8. Troubleshooting

Step 1 — filenames and headless¶

filenames must list movies to load (CLI default pattern: ["0.avi"]). headless=True skips interactive crop and curation GUIs — use analysis_parameters.csv for crop_coords when headless.

In [ ]:

print("filenames:", filenames, "headless:", headless)

print("filenames:", filenames, "headless:", headless)

Step 2 — Create data manager (MiniscopeDataManager.create)¶

ONIX vs UCLA-style acquisition is inferred from your data layout.

In [ ]:

from ace_neuro.miniscope.miniscope_data_manager import MiniscopeDataManager

dm = MiniscopeDataManager.create(
    line_num=line_num,
    project_path=project_path,
    data_path=data_path,
    filenames=filenames,
    auto_import_data=True,
)
print("frame rate:", getattr(dm, "fr", None))

from ace_neuro.miniscope.miniscope_data_manager import MiniscopeDataManager dm = MiniscopeDataManager.create( line_num=line_num, project_path=project_path, data_path=data_path, filenames=filenames, auto_import_data=True, ) print("frame rate:", getattr(dm, "fr", None))

Step 3 — Preprocess (MiniscopePreprocessor)¶

Crop coordinates come from analysis_parameters.csv / GUI when crop_coords is None. Below mirrors pipeline defaults you can tune.

In [ ]:

from ace_neuro.miniscope.miniscope_preprocessor import MiniscopePreprocessor
from ace_neuro.shared.misc_functions import get_coords_dict_from_analysis_params

coords_dict, crop_job_name = get_coords_dict_from_analysis_params(dm)
pre = MiniscopePreprocessor(dm)
dm = pre.preprocess_calcium_movie(
    coords_dict,
    crop=True,
    detrend_method="median",
    df_over_f=False,
    crop_job_name_for_file=crop_job_name,
    secs_window=5.0,
    quantile_min=8.0,
    df_over_f_method="delta_f_over_sqrt_f",
    headless=headless,
)

from ace_neuro.miniscope.miniscope_preprocessor import MiniscopePreprocessor from ace_neuro.shared.misc_functions import get_coords_dict_from_analysis_params coords_dict, crop_job_name = get_coords_dict_from_analysis_params(dm) pre = MiniscopePreprocessor(dm) dm = pre.preprocess_calcium_movie( coords_dict, crop=True, detrend_method="median", df_over_f=False, crop_job_name_for_file=crop_job_name, secs_window=5.0, quantile_min=8.0, df_over_f_method="delta_f_over_sqrt_f", headless=headless, )

Step 4 — Process (MiniscopeProcessor)¶

Heavy steps: motion correction (optional) and CNMF-E. For a quick notebook test, set run_CNMFE=False (you will not get spatial components until CNMF-E runs).

In [ ]:

from ace_neuro.miniscope.miniscope_processor import MiniscopeProcessor

proc = MiniscopeProcessor(dm)
dm = proc.process_calcium_movie(
    parallel=False,
    n_processes=4,
    apply_motion_correction=False,
    inspect_motion_correction=False,
    plot_params=False,
    run_CNMFE=True,
    save_estimates=True,
    save_CNMFE_estimates_filename="estimates.hdf5",
    save_CNMFE_params=False,
)

from ace_neuro.miniscope.miniscope_processor import MiniscopeProcessor proc = MiniscopeProcessor(dm) dm = proc.process_calcium_movie( parallel=False, n_processes=4, apply_motion_correction=False, inspect_motion_correction=False, plot_params=False, run_CNMFE=True, save_estimates=True, save_CNMFE_estimates_filename="estimates.hdf5", save_CNMFE_params=False, )

Step 5 — Postprocess (MiniscopePostprocessor)¶

Runs only if dm.CNMFE_obj is not None. With headless=True, component-removal GUI is skipped (see MiniscopePipeline.run).

In [ ]:

from ace_neuro.miniscope.miniscope_postprocessor import MiniscopePostprocessor

if dm.CNMFE_obj is None:
    print("Skip postprocess: CNMF-E object missing (run Step 4 with run_CNMFE=True).")
else:
    post = MiniscopePostprocessor(dm)
    dm = post.postprocess_calcium_movie(
        remove_components_with_gui=False,
        find_calcium_events=True,
        derivative_for_estimates="first",
        event_height=5.0,
        compute_miniscope_phase=True,
        filter_miniscope_data=True,
        n=2,
        cut=[0.1, 1.5],
        ftype="butter",
        btype="bandpass",
        inline=False,
        compute_miniscope_spectrogram=False,
        window_length=30.0,
        window_step=3.0,
        freq_lims=[0, 15],
        time_bandwidth=2.0,
    )

from ace_neuro.miniscope.miniscope_postprocessor import MiniscopePostprocessor if dm.CNMFE_obj is None: print("Skip postprocess: CNMF-E object missing (run Step 4 with run_CNMFE=True).") else: post = MiniscopePostprocessor(dm) dm = post.postprocess_calcium_movie( remove_components_with_gui=False, find_calcium_events=True, derivative_for_estimates="first", event_height=5.0, compute_miniscope_phase=True, filter_miniscope_data=True, n=2, cut=[0.1, 1.5], ftype="butter", btype="bandpass", inline=False, compute_miniscope_spectrogram=False, window_length=30.0, window_step=3.0, freq_lims=[0, 15], time_bandwidth=2.0, )

Step 6 — Inspect results¶

Common fields after a full run: Cn, ca_events_idx, miniscope_phases.

In [ ]:

import matplotlib.pyplot as plt

if getattr(dm, "Cn", None) is not None:
    plt.figure(figsize=(6, 6))
    plt.imshow(dm.Cn, cmap="gray")
    plt.title("Correlation image Cn")
    plt.show()
else:
    print("No Cn yet — complete CNMF-E and postprocessing.")

import matplotlib.pyplot as plt if getattr(dm, "Cn", None) is not None: plt.figure(figsize=(6, 6)) plt.imshow(dm.Cn, cmap="gray") plt.title("Correlation image Cn") plt.show() else: print("No Cn yet — complete CNMF-E and postprocessing.")

Step 7 — One-shot MiniscopePipeline.run¶

Equivalent to chaining the three stages with shared kwargs.

In [ ]:

from ace_neuro.pipelines.miniscope import MiniscopePipeline

pipe = MiniscopePipeline()
pipe.run(
    line_num=line_num,
    project_path=project_path,
    data_path=data_path,
    filenames=filenames,
    headless=headless,
)
dm2 = pipe.miniscope_data_manager
print("Cn:", getattr(dm2, "Cn", None) is not None)

from ace_neuro.pipelines.miniscope import MiniscopePipeline pipe = MiniscopePipeline() pipe.run( line_num=line_num, project_path=project_path, data_path=data_path, filenames=filenames, headless=headless, ) dm2 = pipe.miniscope_data_manager print("Cn:", getattr(dm2, "Cn", None) is not None)

Troubleshooting¶

• Empty filenames: Always pass movie names (e.g. ["0.avi"]); defaults in code are empty.
• Headless: GUIs for crop and curation are disabled — provide crop_coords or pre-filled analysis_parameters.csv.
• Memory: CNMF-E is heavy; reduce n_processes, crop harder, or shorten movies for learning runs.