IDA FAQ

Please read IDA instructions and FAQ before using the service.

 

1 General questions about IDA service

1.1 Who are entitled to use IDA?

IDA storage quota can be applied by researchers of Finnish higher education institutes and researchers funded by the Academy of Finland. IDA storage space is applied online in CSC's service platform, the Scientist's User Interface (SUI).  Authentication is done with CSC or Haka ID. If you don't have either, you can fill in a paper form. The instructions for applying IDA storage quota can be found at http://openscience.fi/how-to-apply-for-ida-storage.

1.2 What is a project and a PI of a CSC project?

IDA storage quota is applied by and granted for a project, like other CSC resources (https://research.csc.fi/accounts-and-projects). The applied IDA storage quota is not, however, tied to a research grant, except when applied from the quota of Academy of Finland on condition that the research is funded by the Academy of Finland. A single IDA storage quota can be applied for e.g. datasets produced by a series of research projects.

The project applying for IDA storage quota must have a PI of a CSC project as a responsible person. The person applying for quota will become the PI of the CSC and IDA project. The project PI has a certain number of responsibilities; he/she can e.g. add and remove project members. For this reason it is specifically asked in the application if the person applying can act as the project PI, i.e. he/she is a university professor, research director of a polytechnic, or a similar senior level researcher (https://www.csc.fi/en/prerequisites-for-a-principal-investigator). If you have any questions regarding this matter, please contact the IDA coordinator (idakoordinaattori (a) csc.fi).

1.3 How does one become an IDA user?

The project's Principal Investigator can apply for IDA storage space in Scientist's User Interface (SUI).

  1. Create a CSC username.
  2. Register your research project.
  3. Apply for IDA storage space with an application form in SUI.

All this can be done online and the storage quota is available after your application has been approved by your organisation's IDA contact person. Detailed instructions can be found at http://openscience.fi/how-to-apply-for-ida-storage.

1.4 I wish to add/remove a project member, how can I do that?

The project PI can add and remove members in the Scientist's User Interface (SUI). The instructions to add and remove project members can be found at http://openscience.fi/adding-an-ida-user.

1.5 I'm changing my home organisation, can I still use IDA?

Access to IDA is available as long as the user has a Haka ID and/or a CSC account. More information can be found at CSC Terms of Use. The project PI can also add people from other organisations as project members, provided that they are entitled to use IDA as described above.

1.6 Can a foreign research associate of the research project be given access to IDA?

A foreign research associate working in the Finnish research system can be given access to IDA, if the project is working under a Finnish higher education institute or another entity entitled to use IDA. The PI of the IDA project can decide which users can access the project. No exceptions are made regarding e.g. user federations.

1.7 How can we change the IDA project PI?

The new PI fills in an IDA storage application. Detailed instructions can be found at http://openscience.fi/how-to-apply-for-ida-storage.

The new PI and the old PI together contact the CSC Service Desk (servicedesk (a) csc.fi) and ask the data to be moved from the old project to the new, and that the old project with its data is removed.

The new PI invites users to the new project. The instructions can be found at http://openscience.fi/adding-an-ida-user.

1.8 Can the home organisation or its faculty claim access to the data in IDA service if needed?

The owner the data defines its access rights and using IDA does not change the ownership of the data. It is recommended to make agreement on the rights to the data early on, e.g. what happens when someone leaves the project.

2 Questions about the data saved in IDA

2.1 What kind of data is IDA meant for?

In IDA it is possible to store raw research data and new datasets, as well as published research data. IDA is not intended for a heavy-duty computing disk service, because IDA cannot guarantee the speed required for this kind of parallel access. It is either not ideal to connect IDA directly to an instrument that constantly pushes data onto the disk. What is relevant is how IDA is used, not what kind of research data is being stored. However, IDA is not meant for sensitive personal data.

2.2 What are the different IDA user interfaces suitable for?

The most suitable user interface for you depends on your needs and the computers you are using. The browser user interface (SUI) is easy to use and it doesn't require installing new software. It is well suited for storing smaller (less than 10 GB) files. If you wish to move multiple files at once with the browser user interface, you will need to zip them into a single file. Notice that the zip file will not be extracted in IDA.

The command line use is recommended for larger files (tens of gigabytes) and for transferring multiple files simultaneously with Linux operating system. The file transfer program Cyberduck offers similar file transfer features for Windows and Mac users, but the transfers are somewhat slower.

2.3 How fast are the file transfers to IDA? Can I increase the speed somehow?

IDA storage system runs multiple storage operations for each uploaded file. This is why a transfer to IDA is slower than just copying a file over the internet. During the transfer IDA e.g. decides which server to use for the storage, makes another copy of the file on the different computer, calculates a SHA checksum and generates metadata before the transfer is completed on the client' perspective.

For the reasons listed above single files are recommended to be compressed (zipped) into larger packages to facilitate the transfer. When doing big uploads (lots of files, deep directory structures, etc.) to IDA, we recommend using a special script: https://github.com/ida-csc/iput_wrapper. Other useful scripts: https://github.com/ida-csc

This picture gives an example of how file size affects the speed of the transfer.

2.4 How can I share a file stored in IDA outside the service?

You can share a file stored in IDA outside the service by editing the metadata of the file and its directory. You can set your file openly and publicly downloadable, which means that anyone who has its download link (url) can download the file on their own computer. When using IDA with Etsin it is also possible to share files based on applications, which you can approve or reject on a case-by-case basis. The instructions for sharing files can be found at http://openscience.fi/ida-browser-sharing.

2.5 Why is IDA not suitable for all research data?

IDA is not suitable for storing classified information, such as sensitive personal data (as in law http://www.finlex.fi/fi/laki/ajantasa/1999/19990523) or biometric identifiers, because network traffic with command line and Cyberduck/iRODS are not encrypted. Other user interfaces such as browser use through CSC's customer portal encrypt network traffic.

For this reason IDA is not at the increased security level according to government regulation https://www.vahtiohje.fi/web/guest/keskeiset-vaatimukset-ict-ympariston-tietoturvallisuuden-toteuttamiseksi but at the base level which is sufficient for otherwise restricted materials.

Please note that there are special requirements for handling personal data in research. Read more on personal data and its anonymisation from Finnish Social Science Data Archive's data management guidelines: http://www.fsd.uta.fi/aineistonhallinta/en/anonymisation-and-identifiers.html

2.6 Our IDA project quota is filling up. How can I apply for additional storage capacity?

Additional storage quota can be applied by the IDA project PI by sending a request to ida (a) csc.fi, from which we will forward the request to the responsible organisation. The additional storage application will soon be available also online.

2.7. How can I see how much of my project's storage space is used?

A report file that shows the project's used storage space is saved into each IDA project. The text file's name is in format "ida_space_report_<organisation>_<project>.txt", and it is situated in the root of the project folder. The <organisation> and <project> in the file name are generated based on the project in question. The file is automatically updated every morning 7 a.m. (UTC+2). We aim to develop the project view in Scientist's User Interface so that the used space would be visible there.

3 Questions about metadata

Show questions / hide questions

 

4 Question about connecting and logging in IDA

Show questions / hide questions

 

5 Questions about command line use

Show questions / hide questions