https://divingintogeneticsandgenomics.com/tags/fastq/ Recent content in FASTQ on Chatomics Hugo -- gohugo.io en-us © 2018 Ming 'Tommy' Tang Tue, 06 Oct 2015 15:21:00 -0700 https://divingintogeneticsandgenomics.com/archive/convert-bam-to-fastq-and-remap-it-with-bwa-mem/ Tue, 06 Oct 2015 15:21:00 -0700 https://divingintogeneticsandgenomics.com/archive/convert-bam-to-fastq-and-remap-it-with-bwa-mem/ Why we want to revert bam files back to fastq file? Sometimes, you get a processed bam file from other resources, but you want to map it with another aligner. For example, the old BWA aln does not keep the information for soft-cliped reads which are useful for structural variant detection, the new BWA mem is more suitable for this. So, I want to dump bam to fastq first and then realign it with BWA MEM. https://divingintogeneticsandgenomics.com/archive/rename-a-bunch-of-files-with-bash-by-regular-expression/ Thu, 14 Aug 2014 07:44:00 -0700 https://divingintogeneticsandgenomics.com/archive/rename-a-bunch-of-files-with-bash-by-regular-expression/ I am at the MSU NGS 2014 course. My colleague wanted to follow the khmer protocol with his own data, but one of the steps has to use a certain file name convention. In the protocol it requires fastq files listed as: *001_R1.fastq.gz 001 is the replicate number, it can be 002 or 003 or any number of replicates you have. ( for RNA-seq, sequence as many as biological samples as possible ! https://divingintogeneticsandgenomics.com/archive/quality-control-of-your-fastq-file-my-first-post-after-i-got-my-phd/ Thu, 26 Jun 2014 07:06:00 -0700 https://divingintogeneticsandgenomics.com/archive/quality-control-of-your-fastq-file-my-first-post-after-i-got-my-phd/ I passed my defense on Tuesday, and now I am a PhD! what a long process, it took me six years. I've been enjoying the PhD experience though. OK, let me go back to this post. things to remember: 1) You must quality control your fastq files first by fastqc. Trim the adapters and low quality bases by trimmomatic. 2) The fastq files dumped from the SRA GEO repository are usually the raw sequence, you need to quality control them. https://divingintogeneticsandgenomics.com/archive/qseq-to-fastq-conversion/ Tue, 18 Mar 2014 19:08:00 -0700 https://divingintogeneticsandgenomics.com/archive/qseq-to-fastq-conversion/ I was looking at some old ChIP-seq data with raw file in qseq format. I want to convert them to fastq file for mapping with bowtie. A quick google I found: http://www.biostars.org/p/6682/ the qseqtofastq C++ program http://www.dna.bio.keio.ac.jp/~krisp/qseq2fastq/ has to be compiled by scons, and I encountered some compiling problem. I gave it up and used this java program http://sourceforge.net/projects/snpeff/files/qseq2fastq.jar/download Usage: zcat myfile.qseq.tgz | java -jar qseqtofastq.jar -phred64 > myfile.fastq it took me around 1 hour to finish the conversion of a 600MB tgz file on the computing cluster (single cpu 4GB ram). https://divingintogeneticsandgenomics.com/archive/use-sed-to-print-out-every-n-lines-split-pair-end-fastq-file/ Tue, 25 Jun 2013 13:00:00 -0700 https://divingintogeneticsandgenomics.com/archive/use-sed-to-print-out-every-n-lines-split-pair-end-fastq-file/ tommy@tommy-ThinkPad-T420:~$ cat test.txt 1 2 3 4 5 6 7 8 9 10 11 12 #print out the first two lines of every 4 lines. -n flag suppress all of the other lines and only print the line you specified. -e option tells sed to accept multiple p (print) command. tommy@tommy-ThinkPad-T420:~$ sed -ne '1~4p;2~4p' test.txt 1 2 5 6 9 10 This trick would be useful if you have a pair-end FASTq file and want to split it into two files. https://divingintogeneticsandgenomics.com/archive/fasta-q-file-manipulation-tool-readfq/ Tue, 11 Jun 2013 08:24:00 -0700 https://divingintogeneticsandgenomics.com/archive/fasta-q-file-manipulation-tool-readfq/ it is developed by Heng Li in princeton. https://github.com/lh3/readfq it is super fast and I will have a try. Readfq is a collection of routines for parsing the FASTA/FASTQ format. It seamlessly parses both FASTA and multi-line FASTQ with a simple interface. Readfq is first implemented in a single C header file and then ported to Lua, Perl and Python as a single function less than 50 lines. For users of scripting languages, I encourage to copy-and-paste the function instead of using readfq as a library. https://divingintogeneticsandgenomics.com/archive/awk-command-to-count-total-unique-and-the-most-abundant-read-in-a-fastq-file/ Thu, 16 May 2013 08:11:00 -0700 https://divingintogeneticsandgenomics.com/archive/awk-command-to-count-total-unique-and-the-most-abundant-read-in-a-fastq-file/ see post here: http://gettinggeneticsdone.blogspot.com/2012/04/awk-command-to-count-total-unique-and.html Awk Command to Count Total, Unique, and the Most Abundant Read in a FASTQ file I was reading through a paper on comparative ChIP-Seq when I found this awk gem that lets you get some very basic stats very quickly on next generation sequencing reads. To use, simply cat the fastq file (or gunzip -c) and pipe that to this awk command: cat myfile.fq | awk '((NR-2)%4==0){read=$1;total++;count[read]++}END{for(read in count){if(! https://divingintogeneticsandgenomics.com/archive/cool-use-of-unix-paste-with-ngs-sequences-fastq-to-fasta/ Thu, 04 Apr 2013 19:29:00 -0700 https://divingintogeneticsandgenomics.com/archive/cool-use-of-unix-paste-with-ngs-sequences-fastq-to-fasta/ http://thegenomefactory.blogspot.com.au/search?updated-max=2012-09-08T12:11:00%2B10:00&max-results=7 Cool use of Unix paste with NGS sequences While browsing SeqAnswers.com today I came across a post where Uwe Appelt provided a couple of lines of Unix shell wizadry to solve some problem. What attracted my attention was the following: paste - - - - < in.fq | filter | tr "\t" "\n" > out.fq Now, I've done a reasonable amount of shell one-liners in my life, but I'd never seen this before.