ISC - Creating csv files within each ROI

Naturalistic-Data.org
1

Hi there,

I had achieved to run all ISC and IS-RSA tutorial without errors, and I am trying to run ISC with my pilot data. However, I couldn’t run the following code to create my own csv files. Sorry if I am missing out something, could you help me figure out how to get csv files?

Many thanks!

for scan in ['Part1', 'Part2']:
    file_list = glob.glob(os.path.join(data_dir, 'fmriprep', '*', 'func', f'*crop*{scan}*hdf5'))
    for f in file_list:
        sub = os.path.basename(f).split('_')[0]
        print(sub)
        data = Brain_Data(f)
        roi = data.extract_roi(mask)
        pd.DataFrame(roi.T).to_csv(os.path.join(os.path.dirname(f), f"{sub}_{scan}_Average_ROI_n50.csv" ), index=False)
2

Hi @ozgezer,

We will probably need a little more information about your error message to help you debug. From looking at that code snippet, I’m guessing there are 3 possible issues you might be running into.

  1. you are unable to load the data, this could be because the data_dir path is wrong, or you glob call isn’t finding any files

  2. you don’t have a mask loaded to apply to the data. This is probably my best guess. I think the extract_roi method will separately extract a different ROI if each ROI is coded as an integer. It’s possible the file you are loading is not an integer (i.e., 0,1,2,3) or is getting resampled when you load it to be float.

  3. the path to write out the file is wrong.

1 Like
3

Hi @ljchang ,

Many thanks for your reply! It was about loading the mask, now I got the csv files.

1 Like
4

Hi again,

I have a follow up question regarding to the csv file and thought makes more sense to ask there rather than a new topic. I am using the code below and my csv files include pretty high numbers that I could not figure out the cause for this yet. I checked the code step by step and copied them below as well. I will appreciate it a lot if you can at least point me out what I need to fix.

for scan in ['filtered_func_data']:
    file_list = glob.glob(os.path.join(data_dir, '*', 'func', f'*{scan}*hdf5'))
    for f in file_list:
        sub = os.path.basename(f).split('_')[0]
        print(sub)
        datam = Brain_Data(f)
        roi = datam.extract_roi(allmask)
        pd.DataFrame(roi.T).to_csv(os.path.join(os.path.dirname(f), f"{sub}_{scan}_Average_ROI_n50.csv" ), index=False)

roicsv
roicsv1069×634 231 KB 

datam = nltools.data.brain_data.Brain_Data(data=(420, 238955), Y=0, X=(0, 0), mask=MNI152_T1_2mm_brain_mask.nii.gz, output_file=[])
allmask = 
nltools.data.brain_data.Brain_Data(data=(420, 238955), Y=0, X=(0, 0), mask=MNI152_T1_2mm_brain_mask.nii.gz, output_file=[])

Many thanks in advance!

5

Apologies for the delay in responding. I’m not exactly sure what is going on, but here is my initial thought.

datam seems like it has 420 images. Why does your all_mask instance also have 420 images? I don’t believe the method extract_roi can accomodate a multimage mask. Rather, extract_roi expects a mask where each ROI is a unique integer. If you have a single binary image, then you should use apply_mask. Is it possible you might be loading the wrong mask for allmask?

6

Many thanks for your reply. I changed the mask, however I still get really high and weird numbers -as attached-.

I only adapted and followed the codes on the tutorial page, so I am not sure what I can change to fix this. I would appreciate it a lot if you let me know what I can provide to get a better understanding of this issue.

isc_jupyter
isc_jupyter662×605 220 KB

7

What is the shape of your roi variable? I can’t tell from just printing it. It should be 420 x 50 (time by roi) if the code is working correctly.

8

The shape of roi is (50,420), I copied the whole code I used (base_dir and data_dir are not added below so I wouldn’t give personal details). Does it mean I should have transposed a matrix that I am missing?

import os
import glob
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from nltools.stats import regress, zscore
from nltools.data import Brain_Data, Design_Matrix
from nltools.stats import find_spikes 
from nltools.mask import expand_mask

%autosave 5
mask = Brain_Data('http://neurovault.org/media/images/2099/Neurosynth%20Parcellation_0.nii.gz')

for scan in ['task-movie-moco_bold']:
    file_list = glob.glob(os.path.join(data_dir, '*', 'func', f'swau*{scan}*nii'))
    for f in file_list:
        sub = os.path.basename(f).split('_')[0]
        print(sub)
        data = Brain_Data(f)
        roi = data.extract_roi(mask)
        pd.DataFrame(roi.T).to_csv(os.path.join(os.path.dirname(f), f"{sub}_Average_ROI_n50.csv" ), index=False)

Many thanks in advance!

9

Great, looks like it is working then! For each subject you have a data matrix that has the average activity for each TR for each of 50 ROIs.

10

Thanks. Does it mean the numbers (around 3000s) I got in csv files are due to preprocessing?

11

Yes, the numbers are in arbitrary units depending on how you preprocessed the data. You can always zscore them if you would like them standardized.

1 Like
12

Thank you so much for that!