Operational Steps

Creating data-management plan

It is becoming an expectation that researchers create a data-management plan before data collection. You can find useful resources here: https://dmptool.org/, Cessda_DMP, PSA Template

Writing a data management plan is not only useful to help you manage your data by structuring it but also it is now often required as part of the grant proposal submission process. You can create an offline data management plan by yourself, but there are also free, open-source online DMP software available.

In case you do it by yourself, your plan has to contain the following sections:

  • Title of the project
  • Date and version of the current plan
  • Description of your project (what is your research question)
  • The origin of your data
  • Project collaborators (including leader of the project, and role descriptions)
  • Funding reference number
  • The organization, who is responsible for data producing
  • Project data contact
  • The owner of the data
  • Roles regarding data management (roles in the project, but also, who is responsible for keeping the DMP up to date)
  • Costs regarding the DMP

To get a feeling of how your DMP should look like, you may want to check this example, a generic plan, based on a fictional research.

If you decide to use an online DMP tool, https://dmptool.org/ is a good solution.

Creating time-management plan

Whereas one can build a time plan for shorter (hours) to longer (years) time period, most people struggle with organising and executing the task on a daily basis. Some guidelines can be read here: https://www.highspeedtraining.co.uk/hub/time-management-plan/

Open Science Framework

How to register?

The first time you arrive at the OSF front page, it provides you with a “Get started” button to start registration. After clicking on it, as an ELTE student, you should choose to register with ORCID option. Here you should choose the institutional registration option, and select ELTE – you can identify yourself with your CEASAR account. After validating yourself, you have to create an ORCID ID for yourself so the system can connect this with your CEASAR account.

Alternatively, if you do not want to connect your OSF account with your institution, you may also register with your primary email address.

Organize your projects

When you sign in to OSF, you can click on the “Create a New Project” button, which takes you to a form.

There you can enter the “Title” of your project, add an “Institutional affiliation” (optional), and choose a “Storage location”. It is recommended to use a European storage location for Europeans, because of the GDPR laws.

You can also add a “Description” of your project by clicking the “More options” menu.

Later, when you have opened your project, you can add a Licensing option for it by pressing the “Add license” [see more: Licensing preprints]

Finally, press the “Create” button to have your brand new project on OSF!

With the same procedure, you can create a “Component” inside a Project, within the “Components” box with the “Add Component” button. A Component is a sub-project below the Project, which helps you segment your work. It is not necessary to make a new project for every subpart of your work, instead, it is recommended to make components inside it.

Store your study in one place

Creating an OSF project is a good way to organize information about your study. It is also useful for storing your data, uploading your study design and your data analysis plan. OSF also lets you preregister your work [see “Preregistration” tab in this document].

OSF also offers a way for you to have multiple projects, and manage them under the “My Projects” menu. Here you can see all your projects, all your registrations and you can bookmark other studies, which can also be visited from here easily.

By clicking on a project title here, you can see a preview of it with its description and recent activities.

In the left menu, you can filter the listed projects by Contributors or Tags and you can also create Collections of projects with a desired name (e.g.: “Reaction Time Studies”). You can Rename or Delete existing collections later.

Maintain a wiki for your lab or projects

An OSF project inherently contains a Wiki tab on the upper bar. A Wiki is a useful way to share additional information and establish a transparent structure for your lab or your studies.

To edit your wiki, open the “Wiki” tab on the upper bar, then press the upper-right button within the “Wiki” box. This will take you to your Wiki’s “Home”, where you can push the “Edit” button to edit your content.

When the “Edit” panel is turned on, you will see a new box with a page, where you can write new text and format it.

You can add files to your Wiki, by uploading the file to the OSF Storage and embedding it in the text. The syntax for embedding images: @[osf](GUID)

GUID is an ID you get for your file when you update it to OSF – you can copy it from the URL.

It is possible for multiple contributors to edit the same Wiki page simultaneously as OSF lets parallel online, cloud-based working on these files.

Collaboration

OSF allows you to share your work with the people who are working on the same project as you do (you can give access to anybody you would like to). Once you make it a good practice in your team to continuously upload every file you created, everybody will have access to the same updated information. This way even when somebody leaves the project, the files they have been working on do not get lost.

Invite your team members

Open your project, then click the “Contributors tab in the navigation bar. At the top of the page, click on the “Add” button. This point you can decide the depth of access you would like to give – it can be read-only, you can grant reading and writing access or delegate administrator position. You can also decide whether the person is a “bibliographic” or a “non-bibliographic” contributor.

Request access to specific projects

On OSF you are able to see public projects (and to use their dataset), but additionally, you are also able to request contributorship. Just open the project page, click on the ellipsis button on the top right corner, and from the drop-down list choose the “request access”. After that the button will change to “access requested”, and it means, that the admin of the project was notified by an email of your request. When the admin grants you access, you will be notified by an email, too.

Control who sees what

On OSF you are able to divide your project into different components. This also allows you to control exactly who has access to what in your project. When you divide your projects into components, adding a new person to your project, OSF automatically asks you this by opening a “select components” menu in which you can select the components you would like your new contributor to see. It is important, that only the administrator of a component is able to give access to the given component.

Preregistering the study

What is Preregistration?

Preregistration is the practice of giving a detailed description of your study at some point in your research, preferably before data collection (see the details in the following sections). In general, these documents describe the exact methods and analysis plan. The goal of preregistration is preventing QRPs, increasing transparency and quality of research.

Where should I preregister?

One can preregister their study in multiple sites. Two will be mentioned and detailed here. One is Open Science Framework, OSF for short, the other is AsPrediced. The former requires you to be more specific than the latter.

OSF

This platform was created to make science more transparent and open. It grants its users multiple services from data storage through project-management tool until preregistration. You can create a project on the site which will belong to you and to those who you add. You can create preregistrations for any project. After you finalize the preregistration process your registration will receive a timestamp. You can choose to embargo it (i.e. keep it private) up to four years or make it public right away. If needed you can later withdraw your preregistration which will delete every data but the metadata.

AsPredicted

This platform is much simpler than the other, it is made for preregistration purposes exclusively, there are no accounts or projects. The submitting author has to answer 9 brief questions then the other authors have to approve it. Once everyone has approved, the preregistration receives a timestamp. It can be made public or be kept private forever.

How to preregister?

One can find a step-by-step guide for creating and submitting preregistrations on osf here.

When should I preregister?

The ideal time of preregistration relates to the idealistic research scenario.

The scientist makes observations in the world and generates a research question and a hypothesis. Then a study design and analysis plan are created. Preregistration should come after this phase and only adds a small burden – by having to upload the design and plan to an independent registry. After that, everything goes, as usual, data collection is conducted according to the design and analysis is done by the analysis plan. This ends up in testing the hypothesis.

Why should I preregister?

It is important for scientific advancement to distinguish between prediction and postdiction. This lets researchers to construct hypotheses from existing data and observations (prediction) and to test these new hypotheses to acquire new remarks (postdiction).

In Null-Hypothesis Significance Testing, P value is only diagnostic if it was based on predictions only. Also, P value is most diagnostic if it is corrected for the number of tests done during analysis.

Preregistration is the most useful way to let researchers separate predictions and postdictions clearly by reporting data testing methods before data analysis. This enhances replicability of a research study and also eliminates the possibility of biased test-selection.

By preregistering your research before data collection and analysis, it immediately becomes clear what your predictions are. This is the best way to reduce alternating between postdictions and predictions by post-hoc theorizing and hypothesizing.

There is evidence that preregistration correlates with increased replicability and reduced selection, reporting and publication biases.

I can’t follow my preregistration plan, help!

There could be some challenges during research, which could interfere with the preregistered plans on the independent registry:

  1. There is a change in the procedure
    • This is easily solved by changing the preregistration and reporting the changes and their causes. This makes it easier to asses their impacts and makes any deviations transparent.
  2. There is a violation in one of the assumptions in the preregistration.
    • It is possible to define stages of preregistering data-analysis. For example, one can preregister an evaluation phase, and after accomplishing it, one can preregister the next steps.
    • It is also possible to report a decision tree inside the preregistration.
    • Establishing a Standard Operating Procedure (SOP) could also solve this issue – this describes decision rules and there could be protocols for one or more preregistration inside.
  3. Data are preexisting:
    • This could undermine the distinction of predictions and postdictions and violate impartiality. However, reporting and preregistering the analysis plan all that was known and not known in advance could decently counter this violation.
  4. Competing predictions
    • It is entirely possible to preregister two or more competing predictions. This also makes the inference of favoring one over the other stronger.

For reference, see: Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606.

Back to summary