Release Code That Someone Else Can Inspect and Attempt to Run

Why this matters

A paper links to a repository, but readers cannot see how to recreate a figure or prepare the data. A reachable repository shows that material is available to inspect; it does not show which files produced the reported results, whether the necessary inputs are available, or whether the code has actually been run in the stated configuration. An analysis often includes interconnected commands, scripts, programs, and pre- and post-processing steps, all of which may matter to the final result. A useful research-code release gives another person a route through the work and enough detail to make a good-faith attempt to run it. That attempt may still fail because of unavailable data, changed computing systems, or other constraints; documentation and access are evidence of transparency, not proof of a successful rerun.

A repository needs a path through the work

Think of a repository as a guided route, not merely a folder of files. A reader should be able to answer five questions:

1. What are the inputs? Identify raw data, example data, metadata, or instructions for obtaining data that cannot be included.

2. How are inputs prepared? Provide data-preparation scripts—the files that clean, transform, combine, or otherwise turn inputs into analysis-ready data.

3. Where does the analysis start? Identify an analysis entry point: a clearly named script or command that starts the main workflow. A controller or driver script can contain the steps from start to finish, including parameters and data input/output instructions.

4. Which outputs should appear? Name the tables, figures, model summaries, and intermediate outputs that the workflow creates, and state where they should be found.

5. What environment is needed? State the software, packages, and system requirements needed to attempt the run.

A README is a short, human-readable overview placed at the top level of a project. It should state the project purpose, current contact information, and examples of how to run cleaning or analysis tasks.

A dependency manifest is a file that lists the external software packages or libraries required by the project, ideally with their versions. A dependency is something your code relies on but that is not part of the code itself. Making dependencies and requirements explicit can be done with a project-level requirements file or a “Getting Started” section in the README.

A release or archive is a preserved, identifiable snapshot of the code package associated with a study or paper. It is different from a changing working repository: version tracking makes it possible to retrieve a specific version of a whole project when responding to questions or providing supporting material.

A simple layout can make the route visible:

project/
├── README.md
├── LICENSE
├── CITATION
├── requirements.txt
├── data/
│   ├── raw/
│   └── example/
├── src/
│   ├── 01_prepare_data.py
│   ├── 02_run_analysis.py
│   ├── 03_make_figures.py
│   └── run_all.py
├── results/
│   ├── intermediate/
│   ├── tables/
│   └── figures/
├── docs/
│   ├── parameters.md
│   ├── manual-steps.md
│   └── CHANGELOG.md
└── tests/
    └── test_example_data.py

The exact folder names are less important than clarity and consistency. Separate raw data and metadata from generated files, and distinguish the code that performs the analysis from the script that runs the full workflow.

Record the details that change results

A script alone may not identify the computation that produced a result. The result can depend on the software version, package versions, input files, parameter values, preprocessing choices, and the exact state of the custom code. For each important step, record the program name and version, exact inputs, and exact parameter values.

A parameter is a value supplied to a script or program that changes what it does—for example, a threshold, model setting, file path, or number of iterations. Put important parameters in a documented configuration file, a dedicated parameters document, or the command shown in the README. Avoid asking readers to edit lines of code without explaining what those lines control.

A random seed is a starting value used by a random-number generator. When an analysis includes randomness, the same inputs and parameters may otherwise produce slightly different outputs on different runs. Record the seed and identify which steps use randomness.

Record manual steps too. Manual changes can include selecting records, copying values, changing a spreadsheet, choosing a graphical setting, or downloading a subset from a remote source. Where possible, replace manual manipulation with a script. When it cannot be avoided, state exactly what was done, to which files, and for what purpose.

Keep meaningful intermediate outputs when practical. An intermediate output is a file produced between raw input and final result, such as a cleaned dataset or a model-ready table. These files can help a reader inspect the path through the workflow, locate where differences arise, and rerun only part of a process.

Use version control or another systematic way to preserve changes to scripts and documentation. Version control is a system that stores snapshots of project files so that earlier states can be retrieved and changes can be traced. The particular version of a script used for a reported result may be necessary to produce that result again.

These practices make inspection and an attempted rerun more realistic. They do not guarantee that code will execute successfully, reproduce an output exactly, or validate a research claim. A repository can be online and reachable while still lacking an entry point, required inputs, environment details, or evidence that the released version was run.

Share what you safely can

A licence is a statement of what other people may and may not do with material. Include a clear licence file for the code and state separately which terms apply to data, documentation, and other materials when needed. Without a licence, default copyright rules apply and others may not reproduce, distribute, or create derivative works from the source code.

Never include secrets—such as passwords, private keys, or other security credentials—in a repository. Do not place legally restricted or sensitive data in a public code repository.

Sometimes authors cannot share all code. For example, code may embed restricted data access procedures, confidential material, credentials, or components that cannot safely be redistributed. Do not imply that code is openly available when it is not. Instead, write a direct non-sharing statement: identify what cannot be shared, give the relevant reason at an appropriate level of detail, state whether access is possible by another route, and release safe documentation or examples that help readers inspect the workflow.

A restriction statement is more useful when it distinguishes unavailable material from available supporting evidence. Even when full access is impossible, authors can share the main data and source code that are safe to release as supplementary material and be prepared to provide further methodological detail when appropriate.

Worked example

The following fictional project examines patterns in learners’ use of course activities. Its structure separates data preparation, analysis, figure production, documentation, and outputs.

learning-activity-patterns/
├── README.md
├── LICENSE
├── CITATION
├── requirements.txt
├── data/
│   ├── raw/
│   │   └── README.md
│   └── example/
│       └── activity_log_example.csv
├── src/
│   ├── 01_prepare_activity_log.py
│   ├── 02_fit_model.py
│   ├── 03_make_figure_1.py
│   └── run_all.py
├── config/
│   └── analysis_parameters.yml
├── results/
│   ├── intermediate/
│   │   └── prepared_activity_log.csv
│   ├── tables/
│   │   └── model_summary.csv
│   └── figures/
│       └── figure_1.png
├── docs/
│   ├── data-access.md
│   ├── manual-steps.md
│   ├── expected-outputs.md
│   └── CHANGELOG.md
└── tests/
    └── test_example_workflow.py

The project has an example dataset and a test because a small known input and expected output can help a user determine whether a program is working in their environment.

README excerpt that is too vague

# Learning activity patterns

Code for the paper.

Run the scripts in `src/` to reproduce the analysis.
Data are available on request.

This README identifies neither the starting command nor the order of work. It does not tell the reader which scripts prepare data, where outputs should appear, which dependencies are needed, or what “available on request” means.

Revised README excerpt

# Learning activity patterns

This repository contains the data-preparation, analysis, and figure-generation
scripts for the study.

## What is available

- `data/example/activity_log_example.csv` is safe example data for testing.
- The study data are not included because they contain restricted learner records.
  See `docs/data-access.md` for the access restriction and the variables required
  by the preparation script.
- `src/01_prepare_activity_log.py` prepares an activity log for analysis.
- `src/02_fit_model.py` runs the analysis.
- `src/03_make_figure_1.py` creates the reported figure.
- `src/run_all.py` runs the example workflow in this order.

## Before you start

Install the software and package versions listed in `requirements.txt`.
The analysis settings, including the random seed where used, are recorded in
`config/analysis_parameters.yml`.

## Try the example workflow

Run:

    python src/run_all.py --input data/example/activity_log_example.csv

Expected outputs are written to `results/`. Compare them with the file names
and descriptions in `docs/expected-outputs.md`.

## Version and licence

This archived release is the version associated with the study.
See `LICENSE` for reuse terms and `CITATION` for how to cite the package.

The revised version gives a reader a usable starting point without claiming that the restricted study data or a successful full rerun are available. It identifies the safe input, the workflow order, the environment information, the parameters, expected outputs, the release context, and the terms for reuse.