Querying Data¶

Query Setup¶

To define a query, first initialize a query object, using the following command:

q = OCPQuery;

Each query type requires setting various properties to specify the desired data to request from the server. To check to see if your query is complete, run the following command:

[valid, error] = q.validate;

To run a valid query against a specific server/token/channel, use the following command:

data = oo.query(q);

The following are the six most frequently used queries:

RAMONIdList¶

This returns a list of ids that meet some list of predicates (e.g. type equal to synapse and status equal to unprocessed)

q = OCPQuery;
q.setType(eOCPQueryType.RAMONIdList);
q.setResolution(1);
ids = oo.query(q);

imageDense¶

This returns a RAMONVolume containing a image voxel data in cutout format To get slice z only, zStart = z, zStop = z+1; .. code-block:: bash

q = OCPQuery; q.setType(eOCPQueryType.imageDense); % set xStart,xStop,yStart,yStop,zStart,zStop, resolution q.setCutoutArgs([4400,5424],[5440,6464],[1100,1120],1); im = oo.query(q);

annoDense¶

This returns a RAMONVolume containing a annotation voxel data in cutout format

q = OCPQuery;
q.setType(eOCPQueryType.annoDense);
% set xStart,xStop,yStart,yStop,zStart,zStop, resolution
q.setCutoutArgs([4400,5424],[5440,6464],[1100,1120],1);
anno = oo.query(q);

To visualize progress thus far, CAJAL has a viewer built in:

h = image(im); h.associate(anno);
%type the 'a' key to turn on annotations, and scroll with the arrow keys

RAMONDense¶

This returns a RAMON data type associated with a provided id with the voxel data in cutout form. If no voxel data a warning is given.

q = OCPQuery;
q.setType(eOCPQueryType.RAMONDense);
q.setId(42);
q.setResolution(1);
ramon1 = oo.query(q);

RAMONVoxelList¶

This returns a RAMON data type associated with a provided id with the voxel data in voxel list form. If no voxel data a warning is given.

q = OCPQuery;
q.setType(eOCPQueryType.RAMONVoxelList);
q.setId(42);
q.setResolution(1);
ramon2 = oo.query(q);

RAMONMetaOnly¶

This returns a RAMON data type associated with a provided id with only the metadata fields populated

q = OCPQuery;
q.setType(eOCPQueryType.RAMONMetaOnly);
q.setId(42);
q.setResolution(1);
ramon3 = oo.query(q);

Multiple RAMON Volumes can be retrieved at a time for any of the RAMON queries by passing in an array of IDs, instead of just one.

q = OCPQuery;
q.setType(eOCPQueryType.RAMONMetaOnly);
q.setId([42,45,1]);
q.setResolution([1]);
ramonMulti = oo.query(q);

Example Retrieving Centroids¶

To retrieve centroids that have been previously uploaded, users can combine ID queries with RAMONMetaOnly queries. There is an advanced mode to retrieve this information using OCPFields, which is not documented here. This example is in git as exampleCentroidRetrieval.m This example retrieves centroids for AC4 RAMONSynapses.

The data upload process is documented in the RAMONify page.

fileout = 'test.csv';
annoToken = 'test_ramonify_public';
annoChannel = 'synapse'
step = 100; %number of annotations to process at a time
oo = OCP;
oo.setAnnoToken(annoToken);
oo.setAnnoChannel(annoChannel);

q = OCPQuery;
q.setType(eOCPQueryType.RAMONIdList);
q.validate
id = oo.query(q);
q.setType(eOCPQueryType.RAMONMetaOnly);

cen = [];
for i = 1:step:length(id)
    i
    endId = min(i + step-1, length(id));
    q.setId(id(i:1:endId));
    c = oo.query(q);
    for j = 1:length(c)
        cen(end+1,:) = [c{j}.id,str2num(c{j}.dynamicMetadata('centroid'))];
    end
end

% IDs are returned in non-sorted order from OCP
[~,idx] = sort(cen(:,1),'ascend');
cen = cen(idx,:);

csvwrite(fileout,cen)

Example Counting Mitochondria¶

The data upload process is documented in the RAMONify page.

To count objects, users can proceed the classical way and download large data volumes, followed by a connected components process. However this may require extensive memory and a long period of time to compute! Instead, using the NeuroData infrastructure and RAMON, we can retrieve this information quickly via a simple ID query. Here we find 1053 mitochondria in about 0.1 of a second.

The mitochondria data we count here is part of the Kasthuri2015 Cell paper.

oo = OCP;
oo.setAnnoToken('kasthuri2015_ramon_v1');
oo.setAnnoChannel('mitochondria');
q = OCPQuery;
q.setType(eOCPQueryType.RAMONIdList);
q.setResolution(3);
tic
ids = oo.query(q);
sprintf('The total number of mitochondria are: %d!\n', length(ids))
toc