Last updated: 2015-05-21
Code version: e932f0a3dde0a58de5c41d4437a6e46184afcd2f
When performing QC by lane, we noticed that not every lane of single cell sequencing data had 96 fastq files. Furthermore, this was not a problem with the processing pipeline; the original fastq files were missing! We checked the data that had been independently downloaded to an external hard drive. It was missing some files as well, but they were different from those missing on the cluster. Thus we suspected there was some error in the download process. We confirmed this suspicion by checking the Genomics Core FTP site, which contained the proper 96 files in each lane.
Returning to the log files contained the wget output, I re-confirmed that the files did not contain the words error or fail (e.g. grep -i error raw/*log). Using the name of a file that I knew failed, I found that wget cryptically returns the message “No such file” or “No such directory” when a download fails, which doesn’t make a whole lot of sense because how did it know to search for that file or directory in its recursive search if it does not exist.
grep "No such" raw/*.lograw/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1.log:No such directory `Genomics_Data/NGS/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1/FastQ/Project_YG-SR100-1/Sample_19239_2_C04'.
raw/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1.log:unlink: No such file or directory
raw/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1.log:No such directory `Genomics_Data/NGS/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1/FastQ/Project_YG-SR100-1/Sample_19239_2_H10'.
raw/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1.log:unlink: No such file or directory
raw/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1.log:No such directory `Genomics_Data/NGS/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1/FastQ/Project_YG-SR100-1/Sample_19239_2_E09'.
raw/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1.log:unlink: No such file or directory
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:No such directory `Genomics_Data/NGS/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2/FastQ/Project_YG-SR100-2/Sample_19239_3_B01'.
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:unlink: No such file or directory
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:No such file `SampleSheet.csv'.
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:No such file `19098_2_A04_GTCGTTGA_L001_R1_001.fastq.gz'.
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:No such directory `Genomics_Data/NGS/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2/FastQ/Project_YG-SR100-2/Sample_19101_2_C11'.
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:unlink: No such file or directory
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:No such directory `Genomics_Data/NGS/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2/FastQ/Project_YG-SR100-2/Sample_19239_3_E10'.
raw/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2.log:unlink: No such file or directory
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such file `SampleSheet.csv'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such file `19239_2_F08_CACTTCCG_L007_R1_001.fastq.gz'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such directory `Genomics_Data/NGS/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3/FastQ/Project_YG-SR100-3/Sample_19239_2_G04'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:unlink: No such file or directory
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such file `19239_3_G01_CTAGGAAT_L004_R1_001.fastq.gz'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such directory `Genomics_Data/NGS/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3/FastQ/Project_YG-SR100-3/Sample_19239_3_G11'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:unlink: No such file or directory
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such file `SampleSheet.csv'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such file `19239_3_F02_GATTATGC_L004_R1_001.fastq.gz'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:No such directory `Genomics_Data/NGS/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3/FastQ/Project_YG-SR100-3/Sample_19239_3_F03'.
raw/150402_700819F_0305_AC723YACXX-YG-SR100-FC-3.log:unlink: No such file or directory
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:No such directory `Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19101_1_D08'.
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:unlink: No such file or directory
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:No such directory `Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19239_3_A03'.
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:unlink: No such file or directory
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:No such file `SampleSheet.csv'.
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:No such file `19101_2_D03_GGTTTGTC_L002_R1_001.fastq.gz'.
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:No such directory `Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19101_1_C03'.
raw/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4.log:unlink: No such file or directoryOverall this is scary. While it easy to identify files that are completely misssing, this could have also caused the more cryptic problem of files incompletely downloading. To eliminate this concern entirely, I confirmed the md5 checksums for all the files.
Furthermore, I identified the missing files, downloaded them, checked their md5 checksums, and renamed them.
All the following code was run from the external data directory on gluster.
Run run-md5sum.sh on all the files that were already downloaded. Because the pipeline takes so long to run, we want to identify only those files that failed to download properly.
submit-array.sh run-md5sum.sh 2g fastq/*fastq.gzFind the failed jobs.
ls md5sum | wc -l
cat ~/log/run-md5sum.sh/* | grep success | wc -l
grep -l failure ~/log/run-md5sum.sh/*Re-run failed jobs (6 of the 2,641 jobs failed).
for F in `grep failure ~/log/run-md5sum.sh/* | cut -f2`
do
run-md5sum.sh $F
doneCombine into one file.
cat md5sum/* > $ssc/data/md5_downloaded.txtImport md5 checksums of files downloaded to cluster.
md5_cluster <- read.table("../data/md5_downloaded.txt", stringsAsFactors = FALSE)
colnames(md5_cluster) <- c("md5", "path")
head(md5_cluster) md5
1 3949dc07f75b5d37789ce2ce0d215d67
2 02c28413d7e4905966fd94ba55e5a321
3 2cca289e8080964efc95b4c7328a0d5f
4 e6b6b504ed4d0cd793d36f55cf4fcf79
5 b16345bdc7ed23699d2f5e4a32ccf07a
6 7caeaf4d6b5ba147fbd42aa728dddd3c
path
1 fastq/19098.1.A01.ATTAGACG.L002.R1.C6WYKACXX.fastq.gz
2 fastq/19098.1.A01.ATTAGACG.L003.R1.C6WURACXX.fastq.gz
3 fastq/19098.1.A01.ATTAGACG.L006.R1.C723YACXX.fastq.gz
4 fastq/19098.1.A02.ACGAGTCT.L002.R1.C6WYKACXX.fastq.gz
5 fastq/19098.1.A02.ACGAGTCT.L003.R1.C6WURACXX.fastq.gz
6 fastq/19098.1.A02.ACGAGTCT.L006.R1.C723YACXX.fastq.gzParse filename.
md5_cluster$basename <- basename(md5_cluster$path)
md5_cluster$basename <- sub(".fastq.gz", "", md5_cluster$basename)
file_parts <- strsplit(md5_cluster$basename, "\\.")
md5_cluster$ind <- sapply(file_parts, "[", 1)
md5_cluster$batch <- sapply(file_parts, "[", 2)
md5_cluster$well <- sapply(file_parts, "[", 3)
md5_cluster$index <- sapply(file_parts, "[", 4)
md5_cluster$lane <- sapply(file_parts, "[", 5)
md5_cluster$flow_cell <- sapply(file_parts, "[", 7)
head(md5_cluster) md5
1 3949dc07f75b5d37789ce2ce0d215d67
2 02c28413d7e4905966fd94ba55e5a321
3 2cca289e8080964efc95b4c7328a0d5f
4 e6b6b504ed4d0cd793d36f55cf4fcf79
5 b16345bdc7ed23699d2f5e4a32ccf07a
6 7caeaf4d6b5ba147fbd42aa728dddd3c
path
1 fastq/19098.1.A01.ATTAGACG.L002.R1.C6WYKACXX.fastq.gz
2 fastq/19098.1.A01.ATTAGACG.L003.R1.C6WURACXX.fastq.gz
3 fastq/19098.1.A01.ATTAGACG.L006.R1.C723YACXX.fastq.gz
4 fastq/19098.1.A02.ACGAGTCT.L002.R1.C6WYKACXX.fastq.gz
5 fastq/19098.1.A02.ACGAGTCT.L003.R1.C6WURACXX.fastq.gz
6 fastq/19098.1.A02.ACGAGTCT.L006.R1.C723YACXX.fastq.gz
basename ind batch well index lane
1 19098.1.A01.ATTAGACG.L002.R1.C6WYKACXX 19098 1 A01 ATTAGACG L002
2 19098.1.A01.ATTAGACG.L003.R1.C6WURACXX 19098 1 A01 ATTAGACG L003
3 19098.1.A01.ATTAGACG.L006.R1.C723YACXX 19098 1 A01 ATTAGACG L006
4 19098.1.A02.ACGAGTCT.L002.R1.C6WYKACXX 19098 1 A02 ACGAGTCT L002
5 19098.1.A02.ACGAGTCT.L003.R1.C6WURACXX 19098 1 A02 ACGAGTCT L003
6 19098.1.A02.ACGAGTCT.L006.R1.C723YACXX 19098 1 A02 ACGAGTCT L006
flow_cell
1 C6WYKACXX
2 C6WURACXX
3 C723YACXX
4 C6WYKACXX
5 C6WURACXX
6 C723YACXXPieter sent us the md5 checksums of the files stored at the core facility.
(md5_core_files <- list.files("../data", "md5$", full.names = TRUE))[1] "../data/YG-SR-100-FC1-file.md5" "../data/YG-SR-100-FC2-file.md5"
[3] "../data/YG-SR-100-FC3-file.md5" "../data/YG-SR-100-FC4-file.md5"md5_core <- NULL
for (md5_core_f in md5_core_files) {
print(md5_core_f)
md5_core_tmp <- read.table(md5_core_f, stringsAsFactors = FALSE)
md5_core <- rbind(md5_core, md5_core_tmp)
}[1] "../data/YG-SR-100-FC1-file.md5"
[1] "../data/YG-SR-100-FC2-file.md5"
[1] "../data/YG-SR-100-FC3-file.md5"
[1] "../data/YG-SR-100-FC4-file.md5"colnames(md5_core) <- c("md5", "path")
head(md5_core) md5
1 06330063422aa97adf4ac293b103433d
2 59ae9f03da954fb5d619af90cbfdb437
3 fdc66c05a059142b7c0157ba38aab081
4 3b09236fef1c6e16bcec9db0a9679a79
5 14bd09a52995bb5b2340a7c076b7f9be
6 6b508a0b35e56ce201aa97f053541829
path
1 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19098_3_bulk/SampleSheet.csv
2 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19098_3_bulk/19098_3_bulk_CTACATCA_L001_R1_001.fastq.gz
3 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19098_3_bulk/19098_3_bulk_CGGAGTTA_L001_R1_001.fastq.gz
4 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19101_3_bulk/SampleSheet.csv
5 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19101_3_bulk/19101_3_bulk_ACACAACT_L001_R1_001.fastq.gz
6 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19101_3_bulk/19101_3_bulk_CGTCGCAT_L001_R1_001.fastq.gzRemove checksums of sample sheets (“SampleSheet.csv”).
md5_core <- md5_core[grep("fastq.gz", md5_core$path), ]Parse filenames.
path_parts <- strsplit(md5_core$path, "/")
# Remove .fastq.gz file extension
path_parts <- lapply(path_parts, function(x) sub(".fastq.gz", "", x))
file_parts <- sapply(path_parts, function(x) strsplit(x[9], "_"))
md5_core$ind <- sapply(file_parts, "[", 1)
md5_core$batch <- sapply(file_parts, "[", 2)
md5_core$well <- sapply(file_parts, "[", 3)
md5_core$index <- sapply(file_parts, "[", 4)
md5_core$lane <- sapply(file_parts, "[", 5)
md5_core$flow_cell <- sapply(path_parts, function(x) strsplit(x[5], "_")[[1]][4])
# Remove leading A or B
md5_core$flow_cell <- substr(md5_core$flow_cell, 2, 10)
head(md5_core) md5
2 59ae9f03da954fb5d619af90cbfdb437
3 fdc66c05a059142b7c0157ba38aab081
5 14bd09a52995bb5b2340a7c076b7f9be
6 6b508a0b35e56ce201aa97f053541829
8 43b6df964a05ad75d450d0c564ffedf8
9 27db904bde3c78039936c6077625095a
path
2 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19098_3_bulk/19098_3_bulk_CTACATCA_L001_R1_001.fastq.gz
3 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19098_3_bulk/19098_3_bulk_CGGAGTTA_L001_R1_001.fastq.gz
5 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19101_3_bulk/19101_3_bulk_ACACAACT_L001_R1_001.fastq.gz
6 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19101_3_bulk/19101_3_bulk_CGTCGCAT_L001_R1_001.fastq.gz
8 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19239_3_bulk/19239_3_bulk_TGCCCAGA_L001_R1_001.fastq.gz
9 /media/DATA1/NewSequencerRuns/150320_700819F_0303_AC6WYKACXX/Unaligned_YG-SR100-FC1/Project_YG-SR100-1/Sample_19239_3_bulk/19239_3_bulk_GTCAATCC_L001_R1_001.fastq.gz
ind batch well index lane flow_cell
2 19098 3 bulk CTACATCA L001 C6WYKACXX
3 19098 3 bulk CGGAGTTA L001 C6WYKACXX
5 19101 3 bulk ACACAACT L001 C6WYKACXX
6 19101 3 bulk CGTCGCAT L001 C6WYKACXX
8 19239 3 bulk TGCCCAGA L001 C6WYKACXX
9 19239 3 bulk GTCAATCC L001 C6WYKACXXMerge the data from the core and the cluster.
md5_merge <- merge(md5_core, md5_cluster,
by = c("ind", "batch", "well", "index", "lane", "flow_cell"),
all = TRUE, sort = TRUE, suffixes = c(".core", ".cluster"))
stopifnot(nrow(md5_merge) == nrow(md5_core))Check for incomplete downloads.
sum(md5_merge$md5.core != md5_merge$md5.cluster, na.rm = TRUE)[1] 0Every file that downloaded did so successfully.
Find the files that did not download at all.
missing_data <- md5_merge[is.na(md5_merge$md5.cluster), ]23 files failed to download.
missing_data[, c("ind", "batch", "well", "index", "lane", "flow_cell")] ind batch well index lane flow_cell
306 19098 2 A04 GTCGTTGA L001 C6WURACXX
806 19098 3 F09 GTACTCGG L003 C72JMACXX
976 19101 1 C03 TCATGCAC L004 C72JMACXX
1027 19101 1 D08 GGATACGT L004 C72JMACXX
1297 19101 2 C11 TGATTACC L007 C6WURACXX
1307 19101 2 D03 GGTTTGTC L002 C72JMACXX
2025 19239 1 G09 GCGTCTAA L001 C72JMACXX
2153 19239 2 C01 TTGTACAG L003 C6WYKACXX
2162 19239 2 C04 CTAATGAC L003 C6WYKACXX
2246 19239 2 E08 TCTAGTTG L003 C6WYKACXX
2249 19239 2 E09 TCCCATGG L003 C6WYKACXX
2284 19239 2 F08 CACTTCCG L007 C723YACXX
2308 19239 2 G04 CTTTACCT L007 C723YACXX
2351 19239 2 H07 TTAGCGGT L003 C6WYKACXX
2360 19239 2 H10 TACAACGT L003 C6WYKACXX
2377 19239 3 A03 TGACGTGC L006 C72JMACXX
2405 19239 3 B01 TCGACATC L002 C6WURACXX
2548 19239 3 E10 CGTCGCAT L002 C6WURACXX
2561 19239 3 F02 GATTATGC L004 C723YACXX
2564 19239 3 F03 ACCATAGA L004 C723YACXX
2566 19239 3 F04 TAAGTCCT L002 C6WURACXX
2594 19239 3 G01 CTAGGAAT L004 C723YACXX
2624 19239 3 G11 GGCAGACT L004 C723YACXXCreate file of URLs.
Pieter: 150327_700819F_0304_AC6WURACXX/Unaligned_YG-SR100-2/Project_YG-SR100-2/Sample_19098_2_A04/19098_2_A04_GTCGTTGA_L001_R1_001.fastq.gz FTP: Genomics_Data/NGS/150320_700819F_0303_AC6WYKACXX-YG-SR100-FC-1/FastQ/Project_YG-SR100-2/ cluster: 19098.1.A01.ATTAGACG.L002.R1.C6WYKACXX.fastq.gz
ftp_url <- "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS"
file_url_parts <- strsplit(missing_data$path.core, "/")
convert_filename <- function(parts) {
# Convert filename from Pieter's data to the format on the FTP server.
# From Pieter: /media/DATA1/NewSequencerRuns/150327_700819F_0304_AC6WURACXX/Unaligned_YG-SR100-2/Project_YG-SR100-2/Sample_19098_2_A04/19098_2_A04_GTCGTTGA_L001_R1_001.fastq.gz
# On FTP server: 150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2/FastQ/Project_YG-SR100-2/Sample_19098_2_A04/19098_2_A04_GTCGTTGA_L001_R1_001.fastq.gz
flow_cell <- paste0(parts[5], "-", substr(parts[7], 9, 22))
flow_cell <- sub("SR100", "SR100-FC", flow_cell)
result <- file.path(flow_cell, "FastQ", parts[7], parts[8], parts[9])
return(result)
}
file_url <- sapply(file_url_parts, convert_filename)
file_url <- file.path(ftp_url, file_url)
head(file_url)[1] "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2/FastQ/Project_YG-SR100-2/Sample_19098_2_A04/19098_2_A04_GTCGTTGA_L001_R1_001.fastq.gz"
[2] "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19098_3_F09/19098_3_F09_GTACTCGG_L003_R1_001.fastq.gz"
[3] "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19101_1_C03/19101_1_C03_TCATGCAC_L004_R1_001.fastq.gz"
[4] "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19101_1_D08/19101_1_D08_GGATACGT_L004_R1_001.fastq.gz"
[5] "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS/150327_700819F_0304_AC6WURACXX-YG-SR100-FC-2/FastQ/Project_YG-SR100-2/Sample_19101_2_C11/19101_2_C11_TGATTACC_L007_R1_001.fastq.gz"
[6] "ftp://fgfftp.uchicago.edu/Genomics_Data/NGS/150402_700819F_0306_BC72JMACXX-YG-SR100-FC-4/FastQ/Project_YG-SR100-4/Sample_19101_2_D03/19101_2_D03_GGTTTGTC_L002_R1_001.fastq.gz"write(file_url, "../data/missing-files.txt")Download the files.
wget --user=gilad --password='<password>' -i ../data/missing-files.txtfastq_files <- list.files(pattern = "001.fastq.gz")
for (fq in fastq_files) {
# Calculate md5 of downloaded file
fq_md5_cluster <- tools::md5sum(fq)
fq_parts <- unlist(strsplit(fq, "_"))
fq_data <- subset(missing_data, ind == fq_parts[1] &
batch == fq_parts[2] &
well == fq_parts[3] &
index == fq_parts[4] &
lane == fq_parts[5])
fq_md5_core <- fq_data[, "md5.core"]
# Confirm that file checksums match
stopifnot(nrow(fq_md5_core) == 1, # Luckily none of the samples are flow cell replicates
fq_md5_cluster == fq_md5_core)
# Rename file
new_name <- paste(fq_data$ind, fq_data$batch, fq_data$well, fq_data$index,
fq_data$lane, "R1", fq_data$flow_cell, "fastq.gz",
sep = ".")
file.rename(fq, new_name)
}Change the permission to read-only.
chmod uga=r *fastq.gzMove the files.
mv *.fastq.gz $ssd/fastqRemove missing files document.
unlink("../data/missing-files.txt")sessionInfo()R version 3.1.1 (2014-07-10)
Platform: x86_64-unknown-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] knitr_1.10.5
loaded via a namespace (and not attached):
[1] digest_0.6.4 evaluate_0.7 formatR_0.10 htmltools_0.2.6
[5] rmarkdown_0.5.1 stringr_0.6.2 tools_3.1.1 yaml_2.1.13