Build and submit an SDRF sample sheet

BS3 asks CZ Biohub submitters for an SDRF sample sheet. You build the sheet in MOSAIC, a Biohub web app, and upload it to the Mass Spec, Illumina or PacBio submission form. This page walks through every step.

Submissions for a CZ Biohub institute must use SDRF. BS3 turns the classic CSV sample sheet off for them.

1. Sign in to MOSAIC at https://mosaic.apps.czbiohub.org/ with your CZ Biohub Okta account.
2. Build the sheet in the Single Sheet Builder. Pick the Biohub template for your platform, describe your sample groups, fill in the facility columns for each sample and click Generate sheet.
3. Export it and fill in what is left. MOSAIC downloads sdrf.tsv and writes "Please fill this value" in every BL[...], Illumina[...] or PacBio[...] cell you must fill in but left blank. Replace each one with your real value: BS3 rejects a sheet that still has one.
4. Upload the .tsv file in the BS3 form, with Use SDRF selected under Sample Sheet Type.

What an SDRF sheet is

SDRF stands for Sample and Data Relationship Format. An SDRF sheet is a table saved as tab-separated text. Each row is one sample or measurement, and each column is one property of it. The proteomics community uses the format as SDRF-Proteomics. Biohub uses the same layout for mass spec and sequencing, so every platform describes samples the same way.

Columns come in families. The family is the word in front of the square brackets.

The public SDRF specification uses factor value[...] columns for the variables of an experiment. Biohub sheets use the fixed ones, such as factor value[organism], for the ID that goes with the name in the matching characteristics[...] column. Study variables you add in MOSAIC still export as factor value columns that hold the variable's value.

Ontology terms

Many columns take terms from public ontologies, curated vocabularies where every term has a label and a stable ID. Labels have synonyms and sometimes change. IDs don't, which is why a sheet carries both. MOSAIC looks terms up for you in the EBI Ontology Lookup Service and fills in the IDs, so you rarely type one.

Developmental stage uses the stage ontology for the organism, such as HsapDv for human, MmusDv for mouse and ZFA for zebrafish. MOSAIC picks the right one once you choose the organism.

When there is no value

Every cell needs something in it. Two phrases cover the gaps:

• "not available" means the value exists but you don't know it.
• "not applicable" means the property doesn't apply, like a passage number for a tissue sample.

Never leave a cell blank, and don't write none, NA, N/A, nan or null. BS3 treats all of those as empty and rejects the sheet. A few columns accept neither phrase, such as replicate numbers. The column reference marks them.

A small example

A handful of columns from a Mass Spec sheet. A real sheet has 30 or more.

Why BS3 asks for SDRF

BS3 passes your sheet to Deriva, Biohub's data catalog, where each row becomes a sample record attached to your data. The ontology IDs are what let anyone find, for example, every human liver sample across projects. A classic CSV sheet has to be converted first, and the conversion guesses at terms a scientist would pick better.

Read more in the EBI introduction to SDRF.

What MOSAIC is

MOSAIC, the Multi-Omics Sample Annotation & Creator, is the Biohub web app that builds SDRF sheets for mass spec, Illumina and PacBio. It knows which columns each platform needs, searches the ontologies as you type and fills in every ID.

• Open MOSAIC and click Login with Okta. Use your CZ Biohub Okta account.
• MOSAIC keeps one working sheet per account and saves your changes as you go. The status line reads "Saved to account". Reset sheet starts a new sheet. Discard draft drops the changes you haven't turned into a sheet with Generate sheet.
• Column Help in the left menu explains every BL, Illumina and PacBio column. Sign in first.

Check two settings once

Open Settings in MOSAIC and make sure both of these switches are on. New accounts start with both on.

• Enable Biohub Columns in output tsv adds the BL[...], Illumina[...] and PacBio[...] columns.
• Include Factor Value Ontology Term ID Columns adds the factor value[...] ID columns. MOSAIC says this "may cause SDRF validation issues". That warning is about public SDRF validators. BS3 needs these columns.

With either switch off, MOSAIC refuses to export Illumina and PacBio sheets, and BS3 rejects Mass Spec sheets for missing columns.

Build your sheet in MOSAIC

Click Enter samples on the MOSAIC home page, or Single Sheet Builder in the left menu. The builder has five steps, and each one opens when the previous one is done. The MOSAIC screenshots on this page show version 1.6.1.

1. Template

Pick your platform's template in the Experiment template list, then click Continue to experimental groups.

The template sets the sheet type, so only the Illumina and PacBio templates make sequencing sheets. It locks once you add a group. Delete all groups to change it.

2. Experimental groups

A group is a set of samples that share their biology and condition, such as untreated HeLa cells. Click Add your first experimental group to open the Add Sample Group dialog.

• Experiment/Group ID is a short name for the group, such as "control". It becomes part of every sample name.
• Organism. Type at least three letters and pick the term from the list. MOSAIC won't accept an organism typed as a custom value, because it has no ID.
• Tissue. Choose the Tissue Type first. It decides what else you must fill in. A cell line needs Cell Line and Passage Number, a cell culture needs Cell Type, a tissue or organoid needs Organism Part or Tissue, and an organelle needs Organelle.
• Replication. Biological Replicates is how many samples the group has. Technical Replicates repeat each one. Fraction Count appears on Mass Spec sheets only. MOSAIC makes one row for every combination.
• Conditions. Every group needs a Condition, such as "untreated".
• Everything else is optional. Fill in what you know. MOSAIC exports the rest as "not available", or as "not applicable" where a field does not fit the tissue type, such as Passage Number on anything but a cell line.

Click Add Group. To make a similar group, click the copy icon on its card. The copy gets a numbered Experiment/Group ID, such as control2, so rename it and change what differs. A card marked Needs repair has a problem MOSAIC can point to. Click Repair group to fix it.

Every ontology field searches as you type. Type at least three characters and pick a result. Some fields also list Suggested defaults before you type. The list starts with Standard options, Not applicable and Not available, for samples with no term. Use custom value only when no term fits, because a custom value has no ID.

3. Source names

Choose how MOSAIC names each sample. Automatic builds names from the group, Choose fields builds them from up to five fields you pick, and Serial numbers counts up. Check the Name examples, then click Continue. The button names the next step: Continue to Biohub (BL) columns on a Mass Spec sheet, Continue to Illumina columns or Continue to PacBio columns on a sequencing sheet.

4. Biohub columns

This step is named for your platform: Biohub (BL) columns on a Mass Spec sheet, Illumina columns or PacBio columns on a sequencing sheet. It lists every sample, with a cell for each facility column. The columns BS3 needs a real value in are marked Needed. Fill in real values wherever you can.

• Type each sample's value. To give every sample the same value, pick the column under Fill a column, enter the value and click Fill every sample. You can also paste a block copied from a spreadsheet into any cell.
• On an Illumina sheet MOSAIC checks each value as you type, against the BS3 upload rules that don't depend on your run, and refuses one that breaks them. For example, Illumina[Sample_ID] takes only letters, numbers, "-" and "_". Index lengths and unique names are still checked by BS3 at upload. On a PacBio sheet MOSAIC refuses only tabs, line breaks and double quotes.
• For an Illumina index column your run doesn't use, such as Illumina[Index2_ID] and Illumina[Index2] on a single-index run, click Not used on this run under its header. MOSAIC writes "not available" in it on every sample.
• A Needed cell you leave blank exports as "Please fill this value". The step counts those cells and reminds you that BS3 rejects the sheet until you fill them, here or in the downloaded file.

Click Continue to sheet target. Blank cells don't stop you, but the sheet target stays locked until you click it.

5. Sheet target

The sheet target applies to every group.

• Mass Spec sheets need a Technology Type, such as "proteomic profiling by mass spectrometry", then an Analysis Type and a Mass Spec Platform.
• Illumina and PacBio sheets need an EFO Assay, such as "RNA-Seq". MOSAIC writes it to the technology type columns.

Click Generate sheet. The finished table appears under Last generated sheet. Once a sheet exists the button reads Update sheet. Changes you make to groups reach the sheet, and its export, only after you click Update sheet.

Shortcuts for bigger experiments

• Power User Mode, a switch in the left menu, adds Bulk Sample Creation. It builds many groups from a CSV file.
• Save as Template in the group dialog stores a group's fields so you can reuse them.
• Build with AI drafts groups from a description. Check every field it fills in.
• Experimental Designs describe an experiment once and derive a sheet from it.

Export and finish the sheet

1. Under Last generated sheet, click Export SDRF.
2. MOSAIC warns you about the facility columns and counts the cells, if any, that will download as "Please fill this value". Tick "I have read and understand what to fill in for the ... columns" and click Continue Export.
3. Your browser downloads sdrf.tsv. MOSAIC may also show a pop-up listing Experiment/Group IDs it adjusted for BS3.

MOSAIC writes the facility values you entered in the Biohub columns step. A Needed cell you left blank gets "Please fill this value", and the other blank cells get "not available". If the export warning counted any "Please fill this value" cells, fill them in that step, click Continue to sheet target, then Update sheet, and export again. Or open sdrf.tsv in a spreadsheet and replace each one. The checklist for your platform below says which columns matter.

Editing the TSV without breaking it

• Google Sheets works well. Use File, Import, Upload, choose sdrf.tsv and keep the tab separator. When you finish, use File, Download, Tab-separated values.
• Excel works with care. Save with File, Save As, Text (Tab delimited), then rename the file so it ends in .tsv. Never choose Unicode Text. It saves UTF-16, which BS3 can't read.
• Spreadsheets reformat values that look like dates or numbers, such as the gene name MARCH1 or an ID with leading zeros. Check those cells before you save.
• Google Sheets turns true and false into TRUE and FALSE. BS3 accepts either.

Whatever you edit with:

• Keep every header exactly as MOSAIC wrote it. Header names are case-sensitive.
• Don't delete columns, even ones full of "not available".
• Don't leave blank lines, including one at the end of the file, or tabs after the last column.
• Leave bh[sdrf version] and sdrf_conversion_notes as they are.

Mass Spec checklist

• Template. One of the Biohub MS templates listed in step 1.
• Sheet target. The Technology Type that matches your analysis: proteomic, metabolomic, lipidomic or imaging profiling by mass spectrometry. Then the Analysis Type and the Mass Spec Platform.
• Facility columns. Fill in BL[preparation] and BL[sample label] on every row, and BL[multiplexing] for multiplexed samples. Fill in the other BL columns when you can. Leave out BL[injection volume], BL[method file], BL[plate id], BL[well position], BL[workflow], BL[searchgroup] and BL[experiment alias]. The Mass Spec team adds those. MOSAIC's BL column help describes each one.
• Rules BS3 checks on Mass Spec sheets.
  • characteristics[experiment_or_group_id]: Every row needs a real value. "not available" is not accepted. Letters and numbers, with single hyphens between them, for example "3-0382". No spaces or underscores. "not available" is not accepted.
  • characteristics[organism]: Every row needs a value. Letters, numbers and spaces only. Strain names with hyphens or periods fail.
  • Replicate and fraction numbers: Every row needs a value. A whole number, 1 or higher.
  • Only comment[cleavage agent details] (one per enzyme) and comment[modification parameters] (one per PTM) may appear more than once. Keep those headers as MOSAIC writes them, without a number added to make them unique.
• Upload. Use the Mass Spec submission form and its "Upload the sample sheet" field. Your project must already be approved, and the PI, institute, project ID and organization must match it. BS3 counts the samples from the rows of your sheet.

Illumina checklist

• Template. Biohub Illumina Template.
• Sheet target. The EFO Assay for your libraries, such as RNA-Seq.
• Facility columns. Fill in these on every row: Illumina[Study_Description], Illumina[BioSample_Description], Illumina[Sample_ID], Illumina[Sample_Name], Illumina[Sample_Project] and Illumina[Index]. The exception is a pool of one library with no index read. Leave Illumina[Index] empty for it. "not available" is also accepted there.
• Study_ID and Index_ID. Fill in Illumina[Study_ID] on every row, or write "not available" on every row, and do the same for Illumina[Index_ID]. BS3 rejects a column that mixes values with "not available". MOSAIC writes "not available" in each of these cells you leave blank, so in its Biohub columns step fill each column for every sample or for none.
• Index 2. Illumina[Index2_ID] and Illumina[Index2]: Fill in every row, or leave every row empty or "not available". Fill both for dual-index runs. Write "not available" in both for single-index runs. BS3 rejects a pool that mixes the two.
• Demultiplexing checks. The sequencing pipeline demultiplexes your run from these columns, so BS3 checks them with the classic sample sheet rules. The sheet needs one row per library in the pool. Index sequences use only A, C, G and T and have the index lengths you enter in the form, and every index pair is different. Sample_ID, Sample_Name and Sample_Project start with a letter and use only letters, numbers, "-" and "_". Sample_ID and Sample_Name are unique. Sample_Project is spelled with the same upper and lower case in every row. BioSample_Description has more than 5 characters. None of these cells contains commas or special characters. A "not available" counts as empty, so BS3 rejects it in every facility column except Illumina[Study_ID], Illumina[Index_ID] and the Index 2 columns. Illumina[Study_ID] and Illumina[Index_ID] take it only when every row uses it. Illumina[Index] accepts it, or an empty cell, only for the exception above. MOSAIC's Illumina column help explains every column.
• Upload. Use the Illumina submission form for San Francisco or New York and its "Upload the sample sheet" field. The number of libraries in the pool and the index lengths you enter in the form must match the sheet.

PacBio checklist

• Template. Biohub PacBio Template.
• Sheet target. The EFO Assay for your libraries.
• Facility columns. Fill in these on every row: PacBio[Study_ID], PacBio[Study_Description], PacBio[BioSample_Description], PacBio[Sample_Name], PacBio[Sample_Project] and PacBio[Index_ID]. The exception is a pool of one library with no index read. Leave PacBio[Index_ID] as "not available" for it.
• BS3 rejects "not available" in PacBio[Study_Description], PacBio[BioSample_Description], PacBio[Sample_Name] and PacBio[Sample_Project], and only checks that PacBio[Study_ID] and PacBio[Index_ID] are filled in. The Genomics team needs real values in all of them. MOSAIC's PacBio column help explains every column.
• Upload. Use the PacBio submission form and its "Upload barcode sample sheet" field. If the form shows an error, choose the file again before you resubmit, because the browser drops it. BS3 takes the submission's organism from characteristics[organism].

Upload it to BS3

1. Open the submission form for your platform. Each checklist above links to it.
2. Log in before you fill anything in, because logging in reloads the page. Use the yellow Log in with Okta link at the top of the form, which brings you back to it. CZ Biohub institutes only appear in the institute list after you log in.
3. Choose your institute. For a CZ Biohub institute BS3 turns off the classic option and keeps Use SDRF selected.
4. Pick your Deriva Study in the Metadata Hierarchy fields: Program, then Investigation, then Study. Submissions for a CZ Biohub institute need one.
5. Under Sample Sheet Type keep Use SDRF, then choose your .tsv file in the upload field.
6. Fill in the rest of the form and submit it.

BS3 checks the sheet when you submit. It runs the checks in stages and stops at the first stage that fails. First come the file type and encoding, then the header and the shape of the rows, then each column's rules, then empty cells anywhere in the sheet, and last the organism IDs and duplicate rows. The column-rule and empty-cell stages list every failing column at once. Fixing one problem can reveal the next, so a hand edited sheet can take a few rounds.

The Illumina form shows sheet errors at the top of the page. The Mass Spec and PacBio forms show them under the upload field.

Fixing upload errors

The messages below are quoted exactly. Text in angle brackets, such as <column>, stands for the column or rows the message names.

When MOSAIC refuses to export

Column reference

Every column BS3 checks, per platform. This list is generated from the same checks BS3 runs at upload, so it always matches them. A sheet may carry extra columns, as long as every cell in them is filled in.

Questions

Can I build an SDRF sheet without MOSAIC?

You can, but you would type every header and ontology ID yourself. MOSAIC writes the exact headers BS3 expects and looks up the IDs. The column reference lists everything a hand-made sheet needs.

Can one sheet hold several organisms?

Yes. Each organism name has to keep the same ID on every row.

Can I change a value after I export?

Yes. Edit the TSV as described in Export and finish the sheet, or change the group in MOSAIC, click Update sheet under Sheet target and export again. MOSAIC exports the last sheet you generated, so group edits you haven't applied with Update sheet are not in the file. A new export writes only what MOSAIC has, so a value you typed only into the TSV is lost. Enter facility values in the Biohub columns step where you can.

How do I start a second sheet?

MOSAIC keeps one working sheet per account. Export the current one, then click Reset sheet and pick a template.

My pool mixes single-index and dual-index libraries

BS3 can't take that in one SDRF sheet. Email the Genomics team before you submit.

Getting help

Mass Spec:

• Chan Zuckerberg Biohub San Francisco, mass-spec@sf.czbiohub.org
• Chan Zuckerberg Biohub Chicago, mass-spec@chicago.czbiohub.org

Illumina and PacBio:

• Chan Zuckerberg Biohub San Francisco, seq-team@sf.czbiohub.org
• Chan Zuckerberg Biohub New York, seq-team@ny.czbiohub.org

Problems with MOSAIC itself go through Report a bug in the MOSAIC header. Problems with BS3 go through Report a bug at the bottom of this page.

Glossary

SDRF

Sample and Data Relationship Format, a tab-separated table with one row per sample or measurement.

MOSAIC

The Biohub web app that builds SDRF sheets.

Ontology term

An entry in a curated vocabulary, with a label such as "liver" and an ID such as UBERON:0002107.

OLS

The EBI Ontology Lookup Service, which MOSAIC searches when you type in an ontology field.

Facility columns

The BL, Illumina and PacBio columns, which carry what the Mass Spec and Genomics teams need.

TSV

Tab-separated values, a text file where a tab separates the cells of each row.

Deriva

Biohub's data catalog. BS3 sends it your sheet with your data.