Dear all,
On the topic of FastQC: if you see FastQC failing on large fastq files, here's the fix.
The issue is that FastQC crashes as it runs out of memory, but then exits with a zero exit code, and without logging an error.
The reason why it runs out of memory is that its startup script sets the JVM max memory parameter (-Xmx) to 250MB. That number stems straight from the dark ages, when 2GB was a lot already. :-)
Two fixes: 1. Edit the fastqc startup script, look for '-Xmx250m' and change this to e.g. '-Xmx8g'. Downside: you must remember to redo this after reinstallation or upgrade. 2. Run fastqc with '-t 32' (or some other large number). In theory this makes it multithread on 32 threads, but in practice it still runs on a single CPU[1]. The upside (despite the lack of speed benefits) is that the startup script now allocates 32 x 250MB memory.
Cheers Marco
[1] Presumably it uses Java (software) threads, rather than system processes / threads.