Re: [january-dev] Typed datasets in January

[<Peter.Chang@xxxxxxxxxxxxx>]

Hi Ian,

AxesMetadata can be found in January and some tests in LazyDynamicLoaderTest. Also further uses are in the scisoft-core repo of analysis.test: https://github.com/DawnScience/scisoft-core/tree/master/uk.ac.diamond.scisoft.analysis.test

Regards,
 Peter


-----Original Message-----
From: january-dev-bounces@xxxxxxxxxxx [mailto:january-dev-bounces@xxxxxxxxxxx] On Behalf Of Ian Mayo
Sent: 31 January 2017 09:24
To: january-dev@xxxxxxxxxxx
Subject: Re: [january-dev] Typed datasets in January

Hi there Jake,
that sounds like the Axis concept Mark referred to.   I don't suppose
you can point to an online example of it in use?  Is there one in the DAWN repo?

I can't find the AxesMetadata class in the DawnSci repo [1].

Cheers,
Ian

[1]. https://github.com/DawnScience/dawnsci


> Date: Mon, 30 Jan 2017 15:34:19 +0000
> From: <Jacob.Filik@xxxxxxxxxxxxx>
> To: <january-dev@xxxxxxxxxxx>
> Subject: Re: [january-dev] Typed datasets in January
> Message-ID: <91595EF9DACB9843AA623AB59EE84318A04F1AA0@exchmbx01>
> Content-Type: text/plain; charset=iso-8859-1
>
> Hi all,
>
> I'm another developer here at Diamond with Peter and Matt, I use January pretty heavily and am interested in the UoM and "datasets that only have meaning with other datasets".
>
> One feature I use pretty heavily is the Metadata interface.
>
> All IDatasets can carry Metadata, which might be axes (time, energy...), units, masks... anything that gives the data meaning beyond being just an NDarray.
>
> What makes this interface really useful is the @Sliceable annotation, which can be used to mark datasets in the metadata to be sliced when the main dataset is sliced. A lot of the data processing we do with January involves taking a very large ILazyDataset, backed by HDF5 or a stack of Tiff files, setting axes on the different dimensions of the dataset, then iterating through slices of the data. The @Sliceable metadata slices the metadata datasets to the correct size during the slicing of the main dataset, saving the large amounts of code we used to have to write to do this manually.
>
> I'm not sure if this is useful for your case, but if you took the altitude dataset, made an AxesMetadata object, inserted pressure, temperature etc as datasets in the AxesMetadata then set this in the altitude dataset, every slice you took from altitude would have the corresponding pressure and temperature values in.
>
> For units we have a very simple UnitMetadata object that allows us to associate units with a dataset (but not do much more). This is in DawnSci rather than January because it has extra dependencies, but it would be nice to improve the functionality of this area.
>
> Thanks,
>
> Jake
>
_______________________________________________
january-dev mailing list
january-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/january-dev

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom