Archive for the ‘Uncategorized’ Category.

« Previous Entries
Next Entries »

Simulate dataset with samtools, align with Bowtie, BWA, MAQ and pileup

September 30, 2011, 11:19 pm

This is an old blog post, migrated from my blogspot account. How much of this information is still valid I don’t know.

Place your reference in to phi.fa (phix reference). Bowtie in bowtie subdirectory. Samtools in samtools subdirectory. Bwa in bwa subdirectory. Maq in the maq subdirectory. cd in to each directory and type make to build them. The phix reference I used had additional characters in the reference tag line which confused samtools when used with bwa. The sed in the following script removes them. The script also aligns each end of the simulated pair independently.

#!/bin/bash

#Generate simulated dataset
./samtools/misc/wgsim ./phi.fa ./sim_phi1.fq ./sim_phi2.fq > sim_phi.info

# Bowtie Create binary reference for bowtie
./bowtie/bowtie-build ./phi.fa phi

# Bowtie Align ends independently
./bowtie/bowtie -t phi ./sim_phi1.fq ./sim_phi1.fq.bowtie.align
./bowtie/bowtie -t phi ./sim_phi2.fq ./sim_phi2.fq.bowtie.align

# Bowtie Convert to SAM
./samtools/misc/bowtie2sam.pl ./sim_phi1.fq.bowtie.align > ./sim_phi1.fq.bowtie.align.sam
./samtools/misc/bowtie2sam.pl ./sim_phi2.fq.bowtie.align > ./sim_phi2.fq.bowtie.align.sam

# Clean up SAM reference tags (which in this case were screwy)
sed 's/ Coliphage phiX174, complete genome//' ./sim_phi1.fq.bowtie.align.sam > ./sim_phi1.fq.bowtie.align.sam.clean
sed 's/ Coliphage phiX174, complete genome//' ./sim_phi2.fq.bowtie.align.sam > ./sim_phi2.fq.bowtie.align.sam.clean

# Create sam phi index
./samtools/samtools faidx phi.fa

# Bowtie Convert to BAM
./samtools/samtools import ./phi.fa.fai ./sim_phi1.fq.bowtie.align.sam.clean ./sim_phi1.fq.bowtie.align.bam
./samtools/samtools import ./phi.fa.fai ./sim_phi2.fq.bowtie.align.sam.clean ./sim_phi2.fq.bowtie.align.bam

# Bowtie Sort alignments
./samtools/samtools sort ./sim_phi1.fq.bowtie.align.bam ./sim_phi1.fq.bowtie.align.bam.sorted
./samtools/samtools sort ./sim_phi2.fq.bowtie.align.bam ./sim_phi2.fq.bowtie.align.bam.sorted

# Bowtie Index alignments
./samtools/samtools index ./sim_phi1.fq.bowtie.align.bam.sorted.bam
./samtools/samtools index ./sim_phi2.fq.bowtie.align.bam.sorted.bam

# Bowtie Pileup
./samtools/samtools pileup -cf ./phi.fa ./sim_phi1.fq.bowtie.align.bam.sorted.bam > ./sim_phi1.fq.bowtie.align.pileup
./samtools/samtools pileup -cf ./phi.fa ./sim_phi2.fq.bowtie.align.bam.sorted.bam > ./sim_phi2.fq.bowtie.align.pileup

# Bowtie Filter to SNP locations
./samtools/misc/samtools.pl varFilter -p -D 200000 ./sim_phi1.fq.bowtie.align.pileup &> ./sim_phi1.fq.bowtie.align.pileup.snps
./samtools/misc/samtools.pl varFilter -p -D 200000 ./sim_phi2.fq.bowtie.align.pileup &> ./sim_phi2.fq.bowtie.align.pileup.snps

# BWA Build index
./bwa/bwa index phi.fa

# BWA
./bwa/bwa aln phi.fa sim_phi1.fq > sim_phi1.fq.bwa.align.sai
./bwa/bwa aln phi.fa sim_phi2.fq > sim_phi2.fq.bwa.align.sai

# BWA Convert to SAM
./bwa/bwa samse phi.fa sim_phi1.fq.bwa.align.sai ./sim_phi1.fq > sim_phi1.fq.bwa.align.sam
./bwa/bwa samse phi.fa sim_phi2.fq.bwa.align.sai ./sim_phi2.fq > sim_phi2.fq.bwa.align.sam

# BWA Convert to BAM
./samtools/samtools import ./phi.fa.fai ./sim_phi1.fq.bwa.align.sam ./sim_phi1.fq.bwa.align.bam
./samtools/samtools import ./phi.fa.fai ./sim_phi2.fq.bwa.align.sam ./sim_phi2.fq.bwa.align.bam

# BWA Sort alignments
./samtools/samtools sort ./sim_phi1.fq.bwa.align.bam ./sim_phi1.fq.bwa.align.bam.sorted
./samtools/samtools sort ./sim_phi2.fq.bwa.align.bam ./sim_phi2.fq.bwa.align.bam.sorted

# BWA Index alignments
./samtools/samtools index ./sim_phi1.fq.bwa.align.bam.sorted.bam
./samtools/samtools index ./sim_phi2.fq.bwa.align.bam.sorted.bam

# BWA Pileup
./samtools/samtools pileup -cf ./phi.fa ./sim_phi1.fq.bwa.align.bam.sorted.bam > ./sim_phi1.fq.bwa.align.pileup
./samtools/samtools pileup -cf ./phi.fa ./sim_phi2.fq.bwa.align.bam.sorted.bam > ./sim_phi2.fq.bwa.align.pileup

# BWA Filter to SNP locations
./samtools/misc/samtools.pl varFilter -p -D 200000 ./sim_phi1.fq.bwa.align.pileup &> ./sim_phi1.fq.bwa.align.pileup.snps
./samtools/misc/samtools.pl varFilter -p -D 200000 ./sim_phi2.fq.bwa.align.pileup &> ./sim_phi2.fq.bwa.align.pileup.snps

# MAQ Convert reference to binary fasta format
./maq/maq fasta2bfa ./phi.fa ./phi.bfa

# MAQ Convert reads to binary fastq format
./maq/maq fastq2bfq ./sim_phi1.fq ./sim_phi1.bfq
./maq/maq fastq2bfq ./sim_phi2.fq ./sim_phi2.bfq

# MAQ Align read to the reference
./maq/maq match ./sim_phi1.maq.map ./phi.bfa sim_phi1.bfq
./maq/maq match ./sim_phi2.maq.map ./phi.bfa sim_phi2.bfq

# MAQ Mapcheck
./maq/maq mapcheck phi.bfa sim_phi1.maq.map &> sim_phi1.mapcheck
./maq/maq mapcheck phi.bfa sim_phi2.maq.map &> sim_phi2.mapcheck

# MAQ Call SNPs
./maq/maq assemble ./sim_phi1.maq.cns ./phi.bfa ./sim_phi1.maq.map &> ./sim_phi1.maq.cns.log
./maq/maq assemble ./sim_phi2.maq.cns ./phi.bfa ./sim_phi2.maq.map &> ./sim_phi2.maq.cns.log

# MAQ Build Consensus
./maq/maq cns2fq ./sim_phi1.maq.cns > ./sim_phi1.maq.cns.fq
./maq/maq cns2fq ./sim_phi2.maq.cns > ./sim_phi2.maq.cns.fq

# MAQ Extract SNPs
./maq/maq cns2snp ./sim_phi1.maq.cns > ./sim_phi1.maq.snps
./maq/maq cns2snp ./sim_phi2.maq.cns > ./sim_phi2.maq.snps

# MAQ Alignments to SAM
./samtools/misc/maq2sam-long ./sim_phi1.maq.map > ./sim_phi1.maq.sam
./samtools/misc/maq2sam-long ./sim_phi2.maq.map > ./sim_phi2.maq.sam

# MAQ SAM to BAM
./samtools/samtools import ./phi.fa.fai ./sim_phi1.maq.sam ./sim_phi1.maq.bam
./samtools/samtools import ./phi.fa.fai ./sim_phi2.maq.sam ./sim_phi2.maq.bam

# MAQ Sort alignments
./samtools/samtools sort ./sim_phi1.maq.bam ./sim_phi1.maq.bam.sorted
./samtools/samtools sort ./sim_phi2.maq.bam ./sim_phi2.maq.bam.sorted

# MAQ Index alignments
./samtools/samtools index ./sim_phi1.maq.bam.sorted.bam
./samtools/samtools index ./sim_phi2.maq.bam.sorted.bam

# MAQ Run MAQ alignments against samtools for SNP calling
./samtools/samtools pileup -cf ./phi.fa ./sim_phi1.maq.bam.sorted.bam > ./sim_phi1.maq.pileup
./samtools/samtools pileup -cf ./phi.fa ./sim_phi2.maq.bam.sorted.bam > ./sim_phi2.maq.pileup

# MAQ Filter to SNP locations
./samtools/misc/samtools.pl varFilter -p -D 200000 ./sim_phi1.maq.pileup &> ./sim_phi1.maq.pileup.snps
./samtools/misc/samtools.pl varFilter -p -D 200000 ./sim_phi2.maq.pileup &> ./sim_phi2.maq.pileup.snps
Category: Uncategorized  |  Comment

Nokia n770 internally mounted SD card

September 30, 2011, 11:16 pm

This is an old post migrated from my blogspot account for posterity…

This post describes the internal mounting of an SD card on the Nokia n770. Unless using an SD card gives you a significant speed boost (I’ve yet to test this) I’m not sure it gives you any real advantage over using an RSMMC. However, nowhere around here would sell me an 2Gb RSMMC and I fancied some hacking. I have to say the SD does feel very nippy (In my case a 2Gb Corsair 60x card). The disadvantage of this mod is that you’ll lose the MMC socket on the n770. It might be possible to mount a full size SD card socket internally somewhere which I might try in the future. OK anyway, lets get on with it…

Disassemble the n770

First you need to remove the 5 screws which keep the n770 together, they are in the locations below (nothing under the sticker!):

You should then be able to pull the n770 apart. There is a small piece of plastic covering the ports at the bottom of the unit. This will unclip. The rest of the cover should then clip away. There are no other screws internally.

The LCD will kind of flop around, so be careful you don’t damage to cable. The pcb is attached to a plastic mount, the display PCB and mount should come out of the case as a single unit. Attached to the plastic mount is the Mic, remove the connector that err connects this to the PCB and move it to one side. Detach the PCB from the mount. Once this is done you can turn the PCB over and detach the LCD. Unplug the LCD socket, the LCD socket is hooked to the board, to detach it press down and pull. OK you n770 should be all in bits now…

Great isn’t it!

Internal SD Card mod

Now comes the actual mod which is very simple. To be honest I didn’t take any intermediate pictures here, so you’ll just have to look at the final result and follow the (simple) instructions. Firstly, it makes sense to short the MMC detect switch (this is the thing that tells the n770 if the door is open or closed. You don’t have to do this, you could just keep the door closed all the time, but hey while we’re here… 🙂

The above picture shows the location of the switch solder a length of wire between both sides (wirewap wire works best for surface mount components). As a side note, it would be nice if just by shorting this and leaving the MMC door open you could insert an SD card. Unfortunately the RSMMC socket isn’t thick (high) enough for an MMC card, a nice mod would be simply to replace the socket with one able to take thicker cards.

Next remove the cover of the RSMMC socket. It’s soldered to the board round the edges, you’ll need to desolder these which maybe difficult as being metal it will dissipate the heat, you might find it easier to hack it off (carefully!) with a craft knife… You’ll then expose the MMC connect which you can see already in the previous image.

Now connect up the SD card. I removed the SD PCB from is casing before mounting it in the n770, it makes it a little thinner (there’s not much space in there) and it makes soldering to the connector easier. You may not have this option if your SD cards IC is molded in to the casing. Anyway, solder the SD card pin for pin to the connector. Start on the left side and work your way to the right. You’ll need to skip the first pin and last couple (these are not present on MMC cards) make sure your connecting the correct pins by comparing the SD card connections to an MMC card.

That’s it! You may get and error saying the SD Card is corrupt, I needed to reformat the card (from the File manager) in order to fix this.

Mount the SD Card somewhere internally. I simply mounted it over the SD Socket, there seems to be just enough space.

Now, stick the n770 back together and your done!

That’s the basic mod, I may try mounting a full SD connector. Or perhaps swapping the RSMMC for a MicroSD socket. If I remember correctly more than once device can be connected to the MMC bus, so there may be an option to add multiple SD Micro SD cards internally?

Apologies for the poor quality of this howto (it’s 2am I’m knackered). If there is any interested I’ll tidy it up later.

Disclaimer: Don’t do any of this it’s stupid

Category: Uncategorized  |  Comment

mplayer recursive play all files in directory

September 30, 2011, 11:03 pm 
#!/bin/bash

find . -name "*.mp3" > playlist
mplayer -playlist playlist
Category: Uncategorized  |  1 Comment

Battery powered soldering iron review

September 30, 2011, 11:00 pm

I purchased a battery powered soldering iron last time I was in Japan. I wanted something small that would fit in my electronics kit bag and that could be taken on a plane (ruling out gas powered irons, which are generally more effective).

I bought a Engineer Inc “Kotenosuke SKB-01”. It takes 3 AA batteries. I finally tried it out for an extended soldering session last night.

To switch the soldering iron on you need to both move the switch to the on position and keep the button depressed. As such the soldering iron is only actually on when your holding it, as so as you put it down it switches itself off.

The iron takes about 10 seconds to heat up, which is pretty good, but to be honest even that 10 seconds can get annoying if you’re constantly picking it up and putting it down (as it turns itself off). The batteries last a good few hours, enough for an evening soldering I would guess.

One advantage of the “press to activate” button is that you can remove heat before you remove the iron. That means if you’ve got something particularly springy to solder you can apply a little pressure with the iron while soldering and allow to contact to set before removing the iron.

The top itself is quite strong, and fine enough for most microelectronics work. It’s interchangeable but I’ve not investigated purchasing replacement tips yet.

All in all, I’d say it’s a useful tool but probably not a replacement for a mains powered iron of course, in particular having to constantly wait for the iron to heat up, lack of temperature control etc. is annoying.

Below is some soldering I did with the iron, I used the narrowest gauge solder I could find. It’s not so pretty but it did the job:

Category: Uncategorized  |  Comment
« Previous Entries
Next Entries »