%\VignetteDepends{AdvancedR2011Data}
%\VignetteIndexEntry{Integration}
%\VignettePackage{AdvancedR2011}

\documentclass{article}

\usepackage{BioconductorVignette}
\SweaveOpts{keep.source=TRUE,width=9,height=8,eps=FALSE,include=FALSE}

\title{Integration \\ Self-Study Exercises}
\author{Bioconductor Team \\ Fred Hutchinson Cancer Research Center}
\date{17-18 February, 2011}


\begin{document}
\maketitle
<<setup, echo=FALSE>>=
options(width = 40)
@ 


\section{Introduction} 

These exercises pull together the various aspects of
data manipulation and class construction and methods
development covered in this the Advanced \R{} course.

\begin{Ext}
In this section you will compute linkage disequilibrium on the 
snp data, filter out snps based on a particular
result value and annotate the remaining snps.

One of the challenges of this exercise is to construct
your function such that linkage disequilibrium is computed for all
snps in the dataset (except the last `width` number of snps). 
Note that of the snps input to the \Rcode{.cld} function, linkage 
disequilibrium results are only computed for (\# snps - width) 
snps. This is because the number of linkage disequilibrium comparisons 
made are equal to the \Rcode{width} argument. Thus if 100 snps are 
input with a width of 5, the output will be a matrix of the first 95 
snps (one row per snp) by 5 columns. The 5 columns represent the 
comparisons with the five snps to the right of the snp. 

At your disposal you have the \Rcode{GWASdata} class that can
hold the snp data in the \Rcode{AdvancedR2011Data}
package. Also useful may be the method for composite 
linkage disequilibrium which accepts the 
\Rcode{GWASdata} as input. See `?cld` and the
code defining the generic and method in \Rcode{cld.R}.

<<LDsetup, echo=TRUE>>=
library(AdvancedR2011)
library(StudentGWAS)
dataPath <- system.file("extdata", "snpData.nc",
                        package="AdvancedR2011Data")
metadataPath <- system.file("extdata", "metadata.sqlite",
                            package="AdvancedR2011Data")

## The GWASdata class
data <- GWASdata(dataPath, metadataPath)

## The cld method for GWASdata
res <- cld(data, 1, 20) 
@
\end{Ext}

\begin{Question} 
Write a function that computes composite linkage disequilibrium on 
the snpData.nc file. Iterate through the data in chunks. 
Filter out snps that have a linkage disequilibrium result greater 
than 0.1 with any of its neighbors within the specified width.
A subset of the filtered snps will be annotated in the next exercise. 
\end{Question}

\begin{Solution}
<<LDallsnps, results=hide, echo=FALSE>>=

## Helper function to create index
createidx <- function(first, last, chunk, width)
{
    part <- ceiling((last-first)/(chunk-width))
    first <- c(first, (chunk-width+1)*(seq_len(part-1)))
    last <- c(chunk, chunk + first[-c(1,part)], last)

    if (any(abs(first - last) < width))
        stop("Adjust the chunk size or width to ensure the
        number of snps are greater than the width in all sections.")
    list(first = first, last = last)
}

## Function to compute LD and filter snps
filterLD <- function(first, last, data, width)
{
    res <- cld(data, first, last, width) 
    idx <- which(abs(res) > 0.1, arr.ind=TRUE)
    rownames(res)[-unique(idx[,1])]
}

## mapply to process all data 
width <- 5
idx <- createidx(1, 100000, 10000, width)
lst <- mapply(filterLD, first=idx$first, last=idx$last, 
              MoreArgs=c(data, width), 
              USE.NAMES=FALSE)

snps <- unlist(lst)
@
\end{Solution}


\begin{Ext} 
At this point a natural question to wonder about is what all these
SNPs actually are?  Previous to this point, you created a metadata
database that mapped the SNPs onto their respective entrez gene IDs
(where applicable).  In case you ever need to know, the mapping of
these SNPs and onto their Entrez Gene IDs was already done for you by
using the dbSNP anntoation package along with the GenomicFeatures
package.  To get other relevant information, you have several options
to explore which I will describe briefly here.  1) You could make use
of the annotation packages such as org.Hs.eg.db.  We didn't cover this
in this course since it is an introductory topic, but that also means
that it is pretty easy to do.  Basically you can just load up a
package, and make use of the provided mapping objects.  So here is one quick
example of how you can do this sort of approach:

<<orgExample,echo=TRUE>>=
## load a lib
library(org.Hs.eg.db)
## list the mappings:
head(ls("package:org.Hs.eg.db")) ## etc.
## The mappings act like subsettable R objects so you can subset:
org.Hs.egPATH[c("10","100")]
## or you can convert that subsetted mapping to a data.frame:
toTable(org.Hs.egPATH[c("10","100")])
## and you can merge that with other data using merge() etc.
merge(toTable(org.Hs.egPATH[c("10","100")]), 
      toTable(org.Hs.egSYMBOL[c("10","100")]),
      by.x ="gene_id", by.y="gene_id",
      all.x=TRUE, all.y=TRUE)
@ 

OR you could also do the more advanced approach that we taught you
about before in the final exercise and use a cross database SQL join
to get your data.  This is more difficult than simply using an
annotation package, but it also does not require that you have a
standard org package in order to use it.  So any SQLite database that
contains the data you need should work with this approach.


\end{Ext}

\begin{Question}
Using the getSnps() method that you developed for the StudentGWAS
package, retrieve the annotations for all the SNPs that you searched
and then subset that using the result from before.  After this 1st
step, you should have a data.frame containing the SNPs of interest
mapped to the entrez gene IDs.  And now you can choose how advanced
you want to go and do one (or all) of the strategies described above
to additionally map these SNPs to gene symbols.  To speed things up
for those of you who are interested in the 2nd option, be aware that
the gene symbols are stored in the gene\_info table.  Also, it is a
good idea while working on this to initially work with just a subset
(maybe the 1st 10 or so) SNPs until you get everything working.  Then
annotate the whole set.
\end{Question}
  



\begin{Solution}
<<annotSolutions, results=hide,echo=FALSE>>=
library(org.Hs.eg.db)
library(StudentGWAS)
## both approaches require that we recover the entrez gene IDs
dbPath <- system.file("extdata","metadata.sqlite",package="AdvancedR2011Data")
snpAnn <- getSnps(dbPath)


## TEMP work with just the 1st 10 snps
snps <- snpAnn[["snp_id"]][1:10]
## end TEMP


## subset the data
snpAnn <- snpAnn[snpAnn[["snp_id"]] %in% snps,]

## 1st lets do the easy way:
## Before we start,
snpGeneIds <- snpAnn[["gene_id"]] ## get the IDs
snpGeneIds <- snpGeneIds[!is.na(snpGeneIds)] ## drop NAs
snpGeneIds <- unique(snpGeneIds) ## make them unique

symbAnn <- merge(snpAnn, toTable(org.Hs.egSYMBOL[snpGeneIds]), 
                 by.x ="gene_id", by.y="gene_id", all.x=TRUE)



## then lets do it the SQL way:

drv <- SQLite()
con <- dbConnect(drv, dbname=dbPath)
orgPath <- system.file("extdata", "org.Hs.eg.sqlite",
                       package="org.Hs.eg.db")
## Attach the foreign db from the dbPath
dbGetQuery(con, sprintf("ATTACH '%s' AS db", orgPath))

## now we might want a bit more information about that 2nd DB:
con2 <- dbConnect(drv, dbname=orgPath)
dbListTables(con2)
dbListFields(con2, "gene_info") 

## format the snpIDs
snpIDs <- paste(snps, collapse = "','")

## a simple join will work here (restricted by just the IDs
sql <- paste("SELECT * FROM db.gene_info AS gi, db.genes as g, snps as s
                   WHERE gi._id=g._id AND g.gene_id=s.gene_id AND s.snp_id 
              IN ('", snpIDs,"')",sep="")
allAnn2 <- dbGetQuery(con, sql)


## And just to be tidy lets put away our DB connections.
dbDisconnect(con)
dbDisconnect(con2)
@ 
\end{Solution}
  


\end{document}
