##############################################################################
##############################################################################
###
### Running command:
###
###   /Library/Frameworks/R.framework/Resources/bin/R CMD check --install=check:goSorensen.install-out.txt --library=/Library/Frameworks/R.framework/Resources/library --no-vignettes --timings goSorensen_1.15.0.tar.gz
###
##############################################################################
##############################################################################


* using log directory ‘/Users/biocbuild/bbs-3.24-bioc/meat/goSorensen.Rcheck’
* using R version 4.6.0 Patched (2026-05-01 r89994)
* using platform: aarch64-apple-darwin23
* R was compiled by
    Apple clang version 17.0.0 (clang-1700.3.19.1)
    GNU Fortran (GCC) 14.2.0
* running under: macOS Tahoe 26.3.1
* using session charset: UTF-8
* current time: 2026-05-21 23:50:43 UTC
* using option ‘--no-vignettes’
* checking for file ‘goSorensen/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘goSorensen’ version ‘1.15.0’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘goSorensen’ can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking code files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking contents of ‘data’ directory ... OK
* checking data for non-ASCII characters ... OK
* checking data for ASCII and uncompressed saves ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘test_gosorensen_funcs.R’
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes ... OK
* checking running R code from vignettes ... SKIPPED
* checking re-building of vignette outputs ... SKIPPED
* checking PDF version of manual ... OK
* DONE

Status: OK

##############################################################################
##############################################################################
###
### Running command:
###
###   /Library/Frameworks/R.framework/Resources/bin/R CMD INSTALL goSorensen
###
##############################################################################
##############################################################################


* installing to library ‘/Library/Frameworks/R.framework/Versions/4.6/Resources/library’
* installing *source* package ‘goSorensen’ ...
** this is package ‘goSorensen’ version ‘1.15.0’
** using staged installation
** R
** data
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (goSorensen)

R version 4.6.0 Patched (2026-05-01 r89994) -- "Because it was There"
Copyright (C) 2026 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin23

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

> library(goSorensen)

Attaching package: 'goSorensen'

The following object is masked from 'package:utils':

    upgrade

> 
> # A contingency table of GO terms mutual enrichment
> # between gene lists "atlas" and "sanger":
> data("cont_atlas.sanger_BP4")
> cont_atlas.sanger_BP4
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE   127   159
            FALSE   19  3018
> ?cont_atlas.sanger_BP4
cont_atlas.sanger_BP4        package:goSorensen        R Documentation

_E_x_a_m_p_l_e _o_f _t_h_e _o_u_t_p_u_t _p_r_o_d_u_c_e_d _b_y _t_h_e _f_u_n_c_t_i_o_n '_b_u_i_l_d_E_n_r_i_c_h_T_a_b_l_e'. _I_t
_c_o_n_t_a_i_n_s _t_h_e _e_n_r_i_c_h_m_e_n_t _c_o_n_t_i_n_g_e_n_c_y _t_a_b_l_e _f_o_r _t_w_o _l_i_s_t_s _a_t _l_e_v_e_l _4 _o_f
_o_n_t_o_l_o_g_y _B_P.

_D_e_s_c_r_i_p_t_i_o_n:

     A contingency 2x2 table with the number of joint enriched GO terms
     (TRUE-TRUE); the number of GO terms enriched only in one list but
     not in the other one (FALSE-TRUE and TRUE-FALSE); and the number
     of GO terms not enriched in either of the two lists.

_U_s_a_g_e:

     data(cont_atlas.sanger_BP4)
     
_F_o_r_m_a_t:

     An object of class "table"

_D_e_t_a_i_l_s:

     Consider this object only as an illustrative example, which is
     valid exclusively for the lists atlas and sanger from the data
     'allOncoGeneLists' contained in this package. Note that gene
     lists, GO terms, and Bioconductor may change over time. The
     current version of these results were generated with Bioconductor
     version 3.20.


> class(cont_atlas.sanger_BP4)
[1] "table"
> 
> # Sorensen-Dice dissimilarity on this contingency table:
> ?dSorensen
dSorensen              package:goSorensen              R Documentation

_C_o_m_p_u_t_a_t_i_o_n _o_f _t_h_e _S_o_r_e_n_s_e_n-_D_i_c_e _d_i_s_s_i_m_i_l_a_r_i_t_y

_D_e_s_c_r_i_p_t_i_o_n:

     Computation of the Sorensen-Dice dissimilarity

_U_s_a_g_e:

     dSorensen(x, ...)
     
     ## S3 method for class 'table'
     dSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'matrix'
     dSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'numeric'
     dSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'character'
     dSorensen(x, y, check.table = TRUE, ...)
     
     ## S3 method for class 'list'
     dSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'tableList'
     dSorensen(x, check.table = TRUE, ...)
     
_A_r_g_u_m_e_n_t_s:

       x: either an object of class "table", "matrix" or "numeric"
          representing a 2x2 contingency table, or a "character" vector
          (a set of gene identifiers) or "list" or "tableList" object.
          See the details section for more information.

     ...: extra parameters for function 'buildEnrichTable'.

check.table: Boolean. If TRUE (default), argument 'x' is checked to
          adequately represent a 2x2 contingency table, by means of
          function 'nice2x2Table'.

       y: an object of class "character" representing a vector of valid
          gene identifiers (e.g., ENTREZ).

_D_e_t_a_i_l_s:

     Given a 2x2 arrangement of frequencies (either implemented as a
     "table", a "matrix" or a "numeric" object):

       n_{11}   n_{10} 
       n_{01}  n_{00}, 
      
     this function computes the Sorensen-Dice dissimilarity

                   { n_10 + n_01}/{2 n_11 + n_10 + n_01}.               
     
     The subindex '11' corresponds to those GO terms enriched in both
     lists, '01' to terms enriched in the second list but not in the
     first one, 10' to terms enriched in the first list but not
     enriched in the second one and '00' corresponds to those GO terms
     non enriched in both gene lists, i.e., to the double negatives, a
     value which is ignored in the computations.

     In the "numeric" interface, if 'length(x) >= 3', the values are
     interpreted as(n_11, n_01, n_10, n_00), always in this order and
     discarding extra values if necessary. The result is correct,
     regardless the frequencies being absolute or relative.

     If 'x' is an object of class "character", then 'x' (and 'y') must
     represent two "character" vectors of valid gene identifiers (e.g.,
     ENTREZ). Then the dissimilarity between lists 'x' and 'y' is
     computed, after internally summarizing them as a 2x2 contingency
     table of joint enrichment. This last operation is performed by
     function 'buildEnrichTable' and "valid gene identifiers (e.g.,
     ENTREZ)" stands for the coherency of these gene identifiers with
     the arguments 'geneUniverse' and 'orgPackg' of 'buildEnrichTable',
     passed by the ellipsis argument '...' in 'dSorensen'.

     If 'x' is an object of class "list", the argument must be a list
     of "character" vectors, each one representing a gene list
     (character identifiers). Then, all pairwise dissimilarities
     between these gene lists are computed.

     If 'x' is an object of class "tableList", the Sorensen-Dice
     dissimilarity is computed over each one of these tables. Given k
     gene lists (i.e. "character" vectors of gene identifiers) l1, l2,
     ..., lk, an object of class "tableList" (typically constructed by
     a call to function 'buildEnrichTable') is a list of lists of
     contingency tables t(i,j) generated from each pair of gene lists i
     and j, with the following structure:

     $l2

     $l2$l1$t(2,1)

     $l3

     $l3$l1$t(3,1), $l3$l2$t(3,2)

     ...

     $lk

     $lk$l1$t(k,1), $lk$l2$t(k,2), ..., $lk$l(k-1)t(k,k-1)

_V_a_l_u_e:

     In the "table", "matrix", "numeric" and "character" interfaces,
     the value of the Sorensen-Dice dissimilarity. In the "list" and
     "tableList" interfaces, the symmetric matrix of all pairwise
     Sorensen-Dice dissimilarities.

_M_e_t_h_o_d_s (_b_y _c_l_a_s_s):

        • 'dSorensen(table)': S3 method for class "table"

        • 'dSorensen(matrix)': S3 method for class "matrix"

        • 'dSorensen(numeric)': S3 method for class "numeric"

        • 'dSorensen(character)': S3 method for class "character"

        • 'dSorensen(list)': S3 method for class "list"

        • 'dSorensen(tableList)': S3 method for class "tableList"

_S_e_e _A_l_s_o:

     'buildEnrichTable' for constructing contingency tables of mutual
     enrichment, 'nice2x2Table' for checking contingency tables
     validity, 'seSorensen' for computing the standard error of the
     dissimilarity, 'duppSorensen' for the upper limit of a one-sided
     confidence interval of the dissimilarity, 'equivTestSorensen' for
     an equivalence test.

_E_x_a_m_p_l_e_s:

     # Sorensen-Dice dissimilarity from scratch, directly from two gene lists:
     
     # Manually define a 2 x 2 enrichment contingency table
     contTable <- as.table(matrix(c(127, 19, 159, 3018),
       nrow = 2,
       dimnames = list(
         "Enriched in List 1" = c(TRUE, FALSE),
         "Enriched in List 2" = c(TRUE, FALSE)
       )
     ))
     contTable
     
     # Calculation of the dissimilarity value using the joint enrichment
     # contingency matrix
     dSorensen(contTable)
     

> dSorensen(cont_atlas.sanger_BP4)
[1] 0.412037
> 
> # Standard error of this Sorensen-Dice dissimilarity estimate:
> ?seSorensen
seSorensen             package:goSorensen              R Documentation

_S_t_a_n_d_a_r_d _e_r_r_o_r _o_f _t_h_e _s_a_m_p_l_e _S_o_r_e_n_s_e_n-_D_i_c_e _d_i_s_s_i_m_i_l_a_r_i_t_y, _a_s_y_m_p_t_o_t_i_c
_a_p_p_r_o_a_c_h

_D_e_s_c_r_i_p_t_i_o_n:

     Standard error of the sample Sorensen-Dice dissimilarity,
     asymptotic approach

_U_s_a_g_e:

     seSorensen(x, ...)
     
     ## S3 method for class 'table'
     seSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'matrix'
     seSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'numeric'
     seSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'character'
     seSorensen(x, y, check.table = TRUE, ...)
     
     ## S3 method for class 'list'
     seSorensen(x, check.table = TRUE, ...)
     
     ## S3 method for class 'tableList'
     seSorensen(x, check.table = TRUE, ...)
     
_A_r_g_u_m_e_n_t_s:

       x: either an object of class "table", "matrix" or "numeric"
          representing a 2x2 contingency table, or a "character" (a set
          of gene identifiers) or "list" or "tableList" object. See the
          details section for more information.

     ...: extra parameters for function 'buildEnrichTable'.

check.table: Boolean. If TRUE (default), argument 'x' is checked to
          adequately represent a 2x2 contingency table. This checking
          is performed by means of function 'nice2x2Table'.

       y: an object of class "character" representing a vector of gene
          identifiers (e.g., ENTREZ).

_D_e_t_a_i_l_s:

     This function computes the standard error estimate of the sample
     Sorensen-Dice dissimilarity, given a 2x2 arrangement of
     frequencies (either implemented as a "table", a "matrix" or a
     "numeric" object):

       n_{11}   n_{10} 
       n_{01}  n_{00}, 
      
     The subindex '11' corresponds to those GO terms enriched in both
     lists, '01' to terms enriched in the second list but not in the
     first one, 10' to terms enriched in the first list but not
     enriched in the second one and '00' corresponds to those GO terms
     non enriched in both gene lists, i.e., to the double negatives, a
     value which is ignored in the computations.

     In the "numeric" interface, if 'length(x) >= 3', the values are
     interpreted as(n_11, n_01, n_10), always in this order.

     If 'x' is an object of class "character", then 'x' (and 'y') must
     represent two "character" vectors of valid gene identifiers (e.g.,
     ENTREZ). Then the standard error for the dissimilarity between
     lists 'x' and 'y' is computed, after internally summarizing them
     as a 2x2 contingency table of joint enrichment. This last
     operation is performed by function 'buildEnrichTable' and "valid
     gene identifiers (e.g., ENTREZ)" stands for the coherency of these
     gene identifiers with the arguments 'geneUniverse' and 'orgPackg'
     of 'buildEnrichTable', passed by the ellipsis argument '...' in
     'seSorensen'.

     In the "list" interface, the argument must be a list of
     "character" vectors, each one representing a gene list (character
     identifiers). Then, all pairwise standard errors of the
     dissimilarity between these gene lists are computed.

     If 'x' is an object of class "tableList", the standard error of
     the S orensen-Dice dissimilarity estimate is computed over each
     one of these tables. Given k gene lists (i.e. "character" vectors
     of gene identifiers) l1, l2, ..., lk, an object of class
     "tableList" (typically constructed by a call to function
     'buildEnrichTable') is a list of lists of contingency tables
     t(i,j) generated from each pair of gene lists i and j, with the
     following structure:

     $l2

     $l2$l1$t(2,1)

     $l3

     $l3$l1$t(3,1), $l3$l2$t(3,2)

     ...

     $lk

     $lk$l1$t(k,1), $lk$l2$t(k,2), ..., $lk$l(k-1)t(k,k-1)

_V_a_l_u_e:

     In the "table", "matrix", "numeric" and "character" interfaces,
     the value of the standard error of the Sorensen-Dice dissimilarity
     estimate. In the "list" and "tableList" interfaces, the symmetric
     matrix of all standard error dissimilarity estimates.

_M_e_t_h_o_d_s (_b_y _c_l_a_s_s):

        • 'seSorensen(table)': S3 method for class "table"

        • 'seSorensen(matrix)': S3 method for class "matrix"

        • 'seSorensen(numeric)': S3 method for class "numeric"

        • 'seSorensen(character)': S3 method for class "character"

        • 'seSorensen(list)': S3 method for class "list"

        • 'seSorensen(tableList)': S3 method for class "tableList"

_S_e_e _A_l_s_o:

     'buildEnrichTable' for constructing contingency tables of mutual
     enrichment, 'nice2x2Table' for checking the validity of enrichment
     contingency tables,'dSorensen' for computing the Sorensen-Dice
     dissimilarity, 'duppSorensen' for the upper limit of a one-sided
     confidence interval of the dissimilarity, 'equivTestSorensen' for
     an equivalence test.

_E_x_a_m_p_l_e_s:

     # Manually define a 2 x 2 enrichment contingency table
     contTable <- as.table(matrix(c(127, 19, 159, 3018),
       nrow = 2,
       dimnames = list(
         "Enriched in List 1" = c(TRUE, FALSE),
         "Enriched in List 2" = c(TRUE, FALSE)
       )
     ))
     contTable
     
     # Calculation of the standard error using the joint enrichment contingency
     # matrix
     seSorensen(contTable)
     

> seSorensen(cont_atlas.sanger_BP4)
[1] 0.02818626
> 
> # Upper 95% confidence limit for the Sorensen-Dice dissimilarity:
> ?duppSorensen
duppSorensen            package:goSorensen             R Documentation

_U_p_p_e_r _l_i_m_i_t _o_f _a _o_n_e-_s_i_d_e_d _c_o_n_f_i_d_e_n_c_e _i_n_t_e_r_v_a_l (_0, _d_U_p_p] _f_o_r _t_h_e _S
_o_r_e_n_s_e_n-_D_i_c_e _d_i_s_s_i_m_i_l_a_r_i_t_y

_D_e_s_c_r_i_p_t_i_o_n:

     Upper limit of a one-sided confidence interval (0, dUpp] for the S
     orensen-Dice dissimilarity

_U_s_a_g_e:

     duppSorensen(x, ...)
     
     ## S3 method for class 'table'
     duppSorensen(
       x,
       dis = dSorensen.table(x, check.table = FALSE),
       se = seSorensen.table(x, check.table = FALSE),
       conf.level = 0.95,
       z.conf.level = qnorm(1 - conf.level),
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
     ## S3 method for class 'matrix'
     duppSorensen(
       x,
       dis = dSorensen.matrix(x, check.table = FALSE),
       se = seSorensen.matrix(x, check.table = FALSE),
       conf.level = 0.95,
       z.conf.level = qnorm(1 - conf.level),
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
     ## S3 method for class 'numeric'
     duppSorensen(
       x,
       dis = dSorensen.numeric(x, check.table = FALSE),
       se = seSorensen.numeric(x, check.table = FALSE),
       conf.level = 0.95,
       z.conf.level = qnorm(1 - conf.level),
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
     ## S3 method for class 'character'
     duppSorensen(
       x,
       y,
       conf.level = 0.95,
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
     ## S3 method for class 'list'
     duppSorensen(
       x,
       conf.level = 0.95,
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
     ## S3 method for class 'tableList'
     duppSorensen(
       x,
       conf.level = 0.95,
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
_A_r_g_u_m_e_n_t_s:

       x: either an object of class "table", "matrix" or "numeric"
          representing a 2x2 contingency table, or a "character" (a set
          of gene identifiers) or "list" or "tableList" object. See the
          details section for more information.

     ...: additional arguments for function 'buildEnrichTable'.

     dis: Sorensen-Dice dissimilarity value. Only required to speed
          computations if this value is known in advance.

      se: standard error estimate of the sample dissimilarity. Only
          required to speed computations if this value is known in
          advance.

conf.level: confidence level of the one-sided confidence interval, a
          numeric value between 0 and 1.

z.conf.level: standard normal (or bootstrap, see arguments below)
          distribution quantile at the '1 - conf.level' value. Only
          required to speed computations if this value is known in
          advance. Then, the argument 'conf.level' is ignored.

    boot: boolean. If TRUE, 'z.conf.level' is computed by means of a
          bootstrap approach instead of the asymptotic normal approach.
          Defaults to FALSE.

   nboot: numeric, number of initially planned bootstrap replicates.
          Ignored if 'boot == FALSE'. Defaults to 10000.

check.table: Boolean. If TRUE (default), argument 'x' is checked to
          adequately represent a 2x2 contingency table. This checking
          is performed by means of function 'nice2x2Table'.

       y: an object of class "character" representing a vector of gene
          identifiers (e.g., ENTREZ).

_D_e_t_a_i_l_s:

     This function computes the upper limit of a one-sided confidence
     interval for the Sorensen-Dice dissimilarity, given a 2x2
     arrangement of frequencies (either implemented as a "table", a
     "matrix" or a "numeric" object):

       n_{11}   n_{10} 
       n_{01}  n_{00}, 
      
     The subindex '11' corresponds to those GO terms enriched in both
     lists, '01' to terms enriched in the second list but not in the
     first one, 10' to terms enriched in the first list but not
     enriched in the second one and '00' corresponds to those GO terms
     non enriched in both gene lists, i.e., to the double negatives, a
     value which is ignored in the computations, except if 'boot ==
     TRUE'.

     In the "numeric" interface, if 'length(x) >= 4', the values are
     interpreted as(n_11, n_01, n_10, n_00), always in this order and
     discarding extra values if necessary.

     Arguments 'dis', 'se' and 'z.conf.level' are not required. If
     known in advance (e.g., as a consequence of previous computations
     with the same data), providing its value may speed the
     computations.

     By default, 'z.conf.level' corresponds to the 1 - conf.level
     quantile of a standard normal N(0,1) distribution, as the
     studentized statistic (^d - d) / ^se) is asymptotically N(0,1). In
     the studentized statistic, d stands for the "true" Sorensen-Dice
     dissimilarity, ^d to its sample estimate and ^se for the estimate
     of its standard error. In fact, the normal is its limiting
     distribution but, for finite samples, the true sampling
     distribution may present departures from normality (mainly with
     some inflation in the left tail). The bootstrap method provides a
     better approximation to the true sampling distribution. In the
     bootstrap approach, 'nboot' new bootstrap contingency tables are
     generated from a multinomial distribution with parameters 'size ='
     n11 + n01 + n10 + n00 and probabilities %. Sometimes, some of
     these generated tables may present so low frequencies of
     enrichment that make them unable for Sorensen-Dice computations.
     As a consequence, the number of effective bootstrap samples may be
     lower than the number of initially planned bootstrap samples
     'nboot'. Computing in advance the value of argument 'z.conf.level'
     may be a way to cope with these departures from normality, by
     means of a more adequate quantile function. Alternatively, if
     'boot == TRUE', a bootstrap quantile is internally computed.

     If 'x' is an object of class "character", then 'x' (and 'y') must
     represent two "character" vectors of valid gene identifiers (e.g.,
     ENTREZ). Then the confidence interval for the dissimilarity
     between lists 'x' and 'y' is computed, after internally
     summarizing them as a 2x2 contingency table of joint enrichment.
     This last operation is performed by function 'buildEnrichTable'
     and "valid gene identifiers (e.g., ENTREZ)" stands for the
     coherency of these gene identifiers with the arguments
     'geneUniverse' and 'orgPackg' of 'buildEnrichTable', passed by the
     ellipsis argument '...' in 'dUppSorensen'.

     In the "list" interface, the argument must be a list of
     "character" vectors, each one representing a gene list (character
     identifiers). Then, all pairwise upper limits of the dissimilarity
     between these gene lists are computed.

     In the "tableList" interface, the upper limits are computed over
     each one of these tables. Given gene lists (i.e. "character"
     vectors of gene identifiers) l1, l2, ..., lk, an object of class
     "tableList" (typically constructed by a call to function
     'buildEnrichTable') is a list of lists of contingency tables
     t(i,j) generated from each pair of gene lists i and j, with the
     following structure:

     $l2

     $l2$l1$t(2,1)

     $l3

     $l3$l1$t(3,1), $l3$l2$t(3,2)

     ...

     $lk

     $lk$l1$t(k,1), $lk$l2$t(k,2), ..., $lk$l(k-1)t(k,k-1)

_V_a_l_u_e:

     In the "table", "matrix", "numeric" and "character" interfaces,
     the value of the Upper limit of the confidence interval for the
     Sorensen-Dice dissimilarity. When 'boot == TRUE', this result also
     haves a an extra attribute: "eff.nboot" which corresponds to the
     number of effective bootstrap replicats, see the details section.
     In the "list" and "tableList" interfaces, the result is the
     symmetric matrix of all pairwise upper limits.

_M_e_t_h_o_d_s (_b_y _c_l_a_s_s):

        • 'duppSorensen(table)': S3 method for class "table"

        • 'duppSorensen(matrix)': S3 method for class "matrix"

        • 'duppSorensen(numeric)': S3 method for class "numeric"

        • 'duppSorensen(character)': S3 method for class "character"

        • 'duppSorensen(list)': S3 method for class "list"

        • 'duppSorensen(tableList)': S3 method for class "tableList"

_S_e_e _A_l_s_o:

     'buildEnrichTable' for constructing contingency tables of mutual
     enrichment, 'nice2x2Table' for checking contingency tables
     validity, 'dSorensen' for computing the Sorensen-Dice
     dissimilarity, 'seSorensen' for computing the standard error of
     the dissimilarity, 'equivTestSorensen' for an equivalence test.

_E_x_a_m_p_l_e_s:

     # Computing the Upper confidence limit:
     
     # Manually define a 2 x 2 enrichment contingency table
     contTable <- as.table(matrix(c(127, 19, 159, 3018),
       nrow = 2,
       dimnames = list(
         "Enriched in List 1" = c(TRUE, FALSE),
         "Enriched in List 2" = c(TRUE, FALSE)
       )
     ))
     contTable
     
     # Calculation of the Upper Confidence Limit Using the Joint Enrichment
     # Contingency Matrix
     duppSorensen(contTable)
     

> duppSorensen(cont_atlas.sanger_BP4)
[1] 0.4583993
> # This confidence limit is based on an assimptotic normal N(0,1)
> # approximation to the distribution of (dSampl - d) / se, where
> # dSampl stands for the sample dissimilarity, d for the true dissimilarity
> # and se for the sample dissimilarity standard error estimate.
> 
> # Upper confidence limit but using a Student's t instead of a N(0,1)
> # (just as an example, not recommended -no theoretical justification)
> df <- sum(cont_atlas.sanger_BP4[1:3]) - 2
> duppSorensen(cont_atlas.sanger_BP4, z.conf.level = qt(1 - 0.95, df))
[1] 0.4585415
> 
> # Upper confidence limit but using a bootstrap approximation
> # to the sampling distribution, instead of a N(0,1)
> set.seed(123)
> duppSorensen(cont_atlas.sanger_BP4, boot = TRUE)
[1] 0.4589157
attr(,"eff.nboot")
[1] 10000
> 
> # Some computations on diverse data structures:
> badConti <- as.table(matrix(c(501, 27, 36, 12, 43, 15, 0, 0, 0),
+                             nrow = 3, ncol = 3,
+                             dimnames = list(c("a1","a2","a3"),
+                                             c("b1", "b2","b3"))))
> tryCatch(nice2x2Table(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(badConti): Not a 2x2 table>
> 
> incompleteConti <- badConti[1,1:min(2,ncol(badConti)), drop = FALSE]
> incompleteConti
    b1  b2
a1 501  12
> tryCatch(nice2x2Table(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(incompleteConti): Not a 2x2 table>
> 
> contiAsVector <- c(32, 21, 81, 1439)
> nice2x2Table(contiAsVector)
[1] TRUE
> contiAsVector.mat <- matrix(contiAsVector, nrow = 2)
> contiAsVector.mat
     [,1] [,2]
[1,]   32   81
[2,]   21 1439
> contiAsVectorLen3 <- c(32, 21, 81)
> nice2x2Table(contiAsVectorLen3)
[1] TRUE
> 
> tryCatch(dSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> 
> # Apparently, the next order works fine, but returns a wrong value!
> dSorensen(badConti, check.table = FALSE)
[1] 0.05915493
> 
> tryCatch(dSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> dSorensen(contiAsVector)
[1] 0.6144578
> dSorensen(contiAsVector.mat)
[1] 0.6144578
> dSorensen(contiAsVectorLen3)
[1] 0.6144578
> dSorensen(contiAsVectorLen3, check.table = FALSE)
[1] 0.6144578
> dSorensen(c(0,0,0,45))
[1] NaN
> 
> tryCatch(seSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> tryCatch(seSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> seSorensen(contiAsVector)
[1] 0.04818012
> seSorensen(contiAsVector.mat)
[1] 0.04818012
> seSorensen(contiAsVectorLen3)
[1] 0.04818012
> seSorensen(contiAsVectorLen3, check.table = FALSE)
[1] 0.04818012
> tryCatch(seSorensen(contiAsVectorLen3, check.table = "not"), error = function(e) {return(e)})
<simpleError in seSorensen.numeric(contiAsVectorLen3, check.table = "not"): Argument 'check.table' must be logical>
> seSorensen(c(0,0,0,45))
[1] NaN
> 
> tryCatch(duppSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> tryCatch(duppSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> duppSorensen(contiAsVector)
[1] 0.6937071
> duppSorensen(contiAsVector.mat)
[1] 0.6937071
> set.seed(123)
> duppSorensen(contiAsVector, boot = TRUE)
[1] 0.6922658
attr(,"eff.nboot")
[1] 10000
> set.seed(123)
> duppSorensen(contiAsVector.mat, boot = TRUE)
[1] 0.6922658
attr(,"eff.nboot")
[1] 10000
> duppSorensen(contiAsVectorLen3)
[1] 0.6937071
> # Bootstrapping requires full contingency tables (4 values)
> set.seed(123)
> tryCatch(duppSorensen(contiAsVectorLen3, boot = TRUE), error = function(e) {return(e)})
<simpleError in duppSorensen.numeric(contiAsVectorLen3, boot = TRUE): Bootstraping requires a numeric vector of 4 frequencies>
> duppSorensen(c(0,0,0,45))
[1] NaN
> 
> # Equivalence test, H0: d >= d0 vs  H1: d < d0 (d0 = 0.4444)
> ?equivTestSorensen
equivTestSorensen          package:goSorensen          R Documentation

_E_q_u_i_v_a_l_e_n_c_e _t_e_s_t _b_a_s_e_d _o_n _t_h_e _S_o_r_e_n_s_e_n-_D_i_c_e _d_i_s_s_i_m_i_l_a_r_i_t_y

_D_e_s_c_r_i_p_t_i_o_n:

     Equivalence test based on the Sorensen-Dice dissimilarity,
     computed either by an asymptotic normal approach or by a bootstrap
     approach.

_U_s_a_g_e:

     equivTestSorensen(x, ...)
     
     ## S3 method for class 'character'
     equivTestSorensen(
       x,
       y,
       d0 = 1/(1 + 1.25),
       conf.level = 0.95,
       boot = FALSE,
       nboot = 10000,
       check.table = TRUE,
       ...
     )
     
_A_r_g_u_m_e_n_t_s:

       x: either an object of class "table", "matrix", "numeric",
          "character", "list" or "tableList". See the details section
          for more information.

     ...: extra parameters for function 'buildEnrichTable'.

       y: an object of class "character" representing a list of gene
          identifiers (e.g., ENTREZ).

      d0: equivalence threshold for the Sorensen-Dice dissimilarity, d.
          The null hypothesis states that d >= d0, i.e., inequivalence
          between the compared gene lists and the alternative that d <
          d0, i.e., equivalence or dissimilarity irrelevance (up to a
          level d0).

conf.level: confidence level of the one-sided confidence interval, a
          value between 0 and 1.

    boot: boolean. If TRUE, the confidence interval and the test
          p-value are computed by means of a bootstrap approach instead
          of the asymptotic normal approach. Defaults to FALSE.

   nboot: numeric, number of initially planned bootstrap replicates.
          Ignored if 'boot == FALSE'. Defaults to 10000.

check.table: Boolean. If TRUE (default), argument 'x' is checked to
          adequately represent a 2x2 contingency table (or an aggregate
          of them) or gene lists producing a correct table. This
          checking is performed by means of function 'nice2x2Table'.

_D_e_t_a_i_l_s:

     This function computes either the normal asymptotic or the
     bootstrap equivalence test based on the Sorensen-Dice
     dissimilarity, given a 2x2 arrangement of frequencies (either
     implemented as a "table", a "matrix" or a "numeric" object):

       n_{11}   n_{10} 
       n_{01}  n_{00}, 
      
     The subindex '11' corresponds to those GO terms enriched in both
     lists, '01' to terms enriched in the second list but not in the
     first one, '10' to terms enriched in the first list but not
     enriched in the second one and '00' corresponds to those GO terms
     non enriched in both gene lists, i.e., to the double negatives, a
     value which is ignored in the computations.

     In the "numeric" interface, if 'length(x) >= 4', the values are
     interpreted as(n_11, n_01, n_10, n_00), always in this order and
     discarding extra values if necessary.

     If 'x' is an object of class "character", then 'x' (and 'y') must
     represent two "character" vectors of valid gene identifiers (e.g.,
     ENTREZ). Then the equivalence test is performed between 'x' and
     'y', after internally summarizing them as a 2x2 contingency table
     of joint enrichment. This last operation is performed by function
     'buildEnrichTable' and "valid gene identifiers (e.g., ENTREZ)"
     stands for the coherency of these gene identifiers with the
     arguments 'geneUniverse' and 'orgPackg' of 'buildEnrichTable',
     passed by the ellipsis argument '...' in 'equivTestSorensen'.

     If 'x' is an object of class "list", each of its elements must be
     a "character" vector of gene identifiers (e.g., ENTREZ). Then all
     pairwise equivalence tests are performed between these gene lists.

     Class "tableList" corresponds to objects representing all mutual
     enrichment contingency tables generated in a pairwise fashion:
     Given gene lists l1, l2, ..., lk, an object of class "tableList"
     (typically constructed by a call to function 'buildEnrichTable')
     is a list of lists of contingency tables tij generated from each
     pair of gene lists i and j, with the following structure:

     $l2

     $l2$l1$t21

     $l3

     $l3$l1$t31, $l3$l2$t32

     ...

     $lk$l1$tk1, $lk$l2$tk2, ..., $lk$l(k-1)tk(k-1)

     If 'x' is an object of class "tableList", the test is performed
     over each one of these tables.

     The test is based on the fact that the studentized statistic (^d -
     d) / ^se is approximately distributed as a standard normal. ^d
     stands for the sample Sorensen-Dice dissimilarity, d for its true
     (unknown) value and ^se for the estimate of its standard error.
     This result is asymptotically correct, but the true distribution
     of the studentized statistic is not exactly normal for finite
     samples, with a heavier left tail than expected under the Gaussian
     model, which may produce some type I error inflation. The
     bootstrap method provides a better approximation to this
     distribution. In the bootstrap approach, 'nboot' new bootstrap
     contingency tables are generated from a multinomial distribution
     with parameters 'size =' (n11 + n01 + n10 + n00) and probabilities
     %. Sometimes, some of these generated tables may present so low
     frequencies of enrichment that make them unable for Sorensen-Dice
     computations. As a consequence, the number of effective bootstrap
     samples may be lower than the number of initially planned ones,
     'nboot', but our simulation studies concluded that this makes the
     test more conservative, less prone to reject a truly false null
     hypothesis of inequivalence, but in any case protects from
     inflating the type I error.

     In a bootstrap test result, use 'getNboot' to access the number of
     initially planned bootstrap replicates and 'getEffNboot' to access
     the number of finally effective bootstrap replicates.

_V_a_l_u_e:

     See method-specific documentation.

     For all interfaces (except for the "list" and "tableList"
     interfaces), the result is a list of class "equivSDhtest" which
     inherits from "htest", with the following components:

     statistic
          The value of the studentized statistic (dSorensen(x) - d0) /
          seSorensen(x).

     p.value
          The p-value of the test.

     conf.int
          The one-sided confidence interval (0, dUpp].

     estimate
          The Sorensen dissimilarity estimate, dSorensen(x).

     null.value
          The value of 'd0'.

     stderr
          The standard error of the Sorensen dissimilarity estimate,
          seSorensen(x), used as denominator in the studentized
          statistic.

     alternative
          A character string describing the alternative hypothesis.

     method
          A character string describing the test.

     data.name
          A character string giving the names of the data.

     enrichTab
          The 2x2 contingency table of joint enrichment on which the
          test was based.

     For the "list" and "tableList" interfaces, the result is an object
     of class "equivSDhtestList", a list of all pairwise comparisons,
     each one being an object of class "equivSDhtest".

_M_e_t_h_o_d_s (_b_y _c_l_a_s_s):

        • 'equivTestSorensen(character)': S3 default method.

_S_e_e _A_l_s_o:

     'nice2x2Table' for checking and reformatting data, 'dSorensen' for
     computing the Sorensen-Dice dissimilarity, 'seSorensen' for
     computing the standard error of the dissimilarity, 'duppSorensen'
     for the upper limit of a one-sided confidence interval of the
     dissimilarity. 'getTable', 'getPvalue', 'getUpper', 'getSE',
     'getNboot' and 'getEffNboot' for accessing specific fields in the
     result of these testing functions. 'update' for updating the
     result of these testing functions with alternative equivalence
     limits, confidence levels or to convert a normal result in a
     bootstrap result or the reverse.

_E_x_a_m_p_l_e_s:

     ## The following example is highly time-consuming and is therefore not run
     ## automatically during R CMD check.
     
     ## Not run:
     
     ## i) Obtaining ENTREZ identifiers for the gene universe of humans:
     library(org.Hs.eg.db)
     humanEntrezIDs <- keys(org.Hs.eg.db, keytype = "ENTREZID")
     
     ## ii) Gene lists to be explored for analysis:
     data(allOncoGeneLists)
     
     # iii) Calculation of Calculation of the equivalence test of all joint
     # enrichment contingency tables obtained from the BP ontology at the GO 4
     # level.
     eqTest_all_BP4 <- equivTestSorensen(allOncoGeneLists,
       geneUniverse = humanEntrezIDs,
       orgPackg = "org.Hs.eg.db",
       onto = "BP",
       GOLevel = 4,
       d0 = 0.4444,
       conf.level = 0.95
     )
     eqTest_all_BP4
     ## End(Not run)
     
     
     # Since running this example may take several minutes, the result has been
     # pre-computed and is accessible as the following:
     data(eqTest_all_BP4)
     eqTest_all_BP4
     # This shortcut applies only to this example; for your own gene-list data,
     # the computation must be performed explicitly.
     
     # For a complete overview of this function's use, see the section 4 of the
     # vignette "Introduction to goSorensen". You can do this by consulting the
     # general package documentation or by directly running the following code in
     # the R console:
     # vignette("goSorensen_Introduction", package = "goSorensen")
     

> equiv.atlas.sanger <- equivTestSorensen(cont_atlas.sanger_BP4)
> equiv.atlas.sanger

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  cont_atlas.sanger_BP4
(d - d0) / se = -1.1498, p-value = 0.1251
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4583993
sample estimates:
Sorensen dissimilarity 
              0.412037 
attr(,"se")
standard error 
    0.02818626 

> getTable(equiv.atlas.sanger)
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE   127   159
            FALSE   19  3018
> getPvalue(equiv.atlas.sanger)
  p-value 
0.1251215 
> 
> tryCatch(equivTestSorensen(badConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> tryCatch(equivTestSorensen(incompleteConti), error = function(e) {return(e)})
<simpleError in nice2x2Table.table(x): Not a 2x2 table>
> equivTestSorensen(contiAsVector)

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  contiAsVector
(d - d0) / se = 3.5287, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6937071
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> equivTestSorensen(contiAsVector.mat)

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  contiAsVector.mat
(d - d0) / se = 3.5287, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6937071
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> set.seed(123)
> equivTestSorensen(contiAsVector.mat, boot = TRUE)

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  contiAsVector.mat
(d - d0) / se = 3.5287, p-value = 0.9996
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6922658
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> equivTestSorensen(contiAsVectorLen3)

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  contiAsVectorLen3
(d - d0) / se = 3.5287, p-value = 0.9998
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.6937071
sample estimates:
Sorensen dissimilarity 
             0.6144578 
attr(,"se")
standard error 
    0.04818012 

> 
> tryCatch(equivTestSorensen(contiAsVectorLen3, boot = TRUE), error = function(e) {return(e)})
<simpleError in equivTestSorensen.numeric(contiAsVectorLen3, boot = TRUE): Bootstraping requires a numeric vector of 4 frequencies>
> 
> equivTestSorensen(c(0,0,0,45))

	No test performed due non finite (d - d0) / se statistic

data:  c(0, 0, 0, 45)
(d - d0) / se = NaN, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                   NaN 
attr(,"se")
standard error 
           NaN 

> 
> # Sorensen-Dice computations from scratch, directly from gene lists
> data(allOncoGeneLists)
> ?allOncoGeneLists
allOncoGeneLists          package:goSorensen           R Documentation

_7 _g_e_n_e _l_i_s_t_s _p_o_s_s_i_b_l_y _r_e_l_a_t_e_d _w_i_t_h _c_a_n_c_e_r

_D_e_s_c_r_i_p_t_i_o_n:

     An object of class "list" of length 7. Each one of its elements is
     a "character" vector of gene identifiers (e.g., ENTREZ). Only gene
     lists of length almost 100 were taken from their source web. Take
     these lists just as an illustrative example, they are not
     automatically updated.

_U_s_a_g_e:

     data(allOncoGeneLists)
     
_F_o_r_m_a_t:

     An object of class "list" of length 7. Each one of its elements is
     a "character" vector of ENTREZ gene identifiers .

_S_o_u_r_c_e:

     <http://www.bushmanlab.org/links/genelists>


> 
> library(org.Hs.eg.db)
Loading required package: AnnotationDbi
Loading required package: stats4
Loading required package: BiocGenerics
Loading required package: generics

Attaching package: 'generics'

The following objects are masked from 'package:base':

    as.difftime, as.factor, as.ordered, intersect, is.element, setdiff,
    setequal, union


Attaching package: 'BiocGenerics'

The following objects are masked from 'package:stats':

    IQR, mad, sd, var, xtabs

The following objects are masked from 'package:base':

    Filter, Find, Map, Position, Reduce, anyDuplicated, aperm, append,
    as.data.frame, basename, cbind, colnames, dirname, do.call,
    duplicated, eval, evalq, get, grep, grepl, is.unsorted, lapply,
    mapply, match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,
    rank, rbind, rownames, sapply, saveRDS, table, tapply, unique,
    unsplit, which.max, which.min

Loading required package: Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.

Loading required package: IRanges
Loading required package: S4Vectors

Attaching package: 'S4Vectors'

The following object is masked from 'package:utils':

    findMatches

The following objects are masked from 'package:base':

    I, expand.grid, unname


> humanEntrezIDs <- keys(org.Hs.eg.db, keytype = "ENTREZID")
> # First, the mutual GO node enrichment tables are built, then computations
> # proceed from these contingency tables.
> # Building the contingency tables is a slow process (many enrichment tests)
> normTest <- equivTestSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
+                               listNames = c("atlas", "sanger"),
+                               onto = "BP", GOLevel = 5,
+                               geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")

> normTest

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -1.9354, p-value = 0.02647
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4392392
sample estimates:
Sorensen dissimilarity 
             0.4097744 
attr(,"se")
standard error 
    0.01791328 

> 
> # To perform a bootstrap test from scratch would be even slower:
> # set.seed(123)
> # bootTest <- equivTestSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #                               listNames = c("atlas", "sanger"),
> #                               boot = TRUE,
> #                               onto = "BP", GOLevel = 5,
> #                               geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # bootTest
> 
> # It is much faster to upgrade 'normTest' to be a bootstrap test:
> set.seed(123)
> bootTest <- upgrade(normTest, boot = TRUE)
> bootTest

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -1.9354, p-value = 0.0269
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4393869
sample estimates:
Sorensen dissimilarity 
             0.4097744 
attr(,"se")
standard error 
    0.01791328 

> # To know the number of planned bootstrap replicates:
> getNboot(bootTest)
[1] 10000
> # To know the number of valid bootstrap replicates:
> getEffNboot(bootTest)
[1] 10000
> 
> # There are similar methods for dSorensen, seSorensen, duppSorensen, etc. to
> # compute directly from a pair of gene lists.
> # They are quite slow for the same reason as before (many enrichment tests).
> # dSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #           listNames = c("atlas", "sanger"),
> #           onto = "BP", GOLevel = 5,
> #           geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # seSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #            listNames = c("atlas", "sanger"),
> #            onto = "BP", GOLevel = 5,
> #            geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> #
> # duppSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #              listNames = c("atlas", "sanger"),
> #              onto = "BP", GOLevel = 5,
> #              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> #
> # set.seed(123)
> # duppSorensen(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
> #              boot = TRUE,
> #              listNames = c("atlas", "sanger"),
> #              onto = "BP", GOLevel = 5,
> #              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # etc.
> 
> # To build the contingency table first and then compute from it, may be a more flexible
> # and saving time strategy, in general:
> ?buildEnrichTable
buildEnrichTable          package:goSorensen           R Documentation

_C_r_e_a_t_e _e_n_r_i_c_h_m_e_n_t _c_o_n_t_i_n_g_e_n_c_y _t_a_b_l_e_s _f_r_o_m _g_e_n_e _l_i_s_t_s

_D_e_s_c_r_i_p_t_i_o_n:

     Generic function to build 2x2 enrichment contingency tables from
     gene lists, or all pairwise contingency tables for a "list" of
     gene lists.

_U_s_a_g_e:

     buildEnrichTable(x, ...)
     
     ## Default S3 method:
     buildEnrichTable(
       x,
       y,
       listNames = c("gene.list1", "gene.list2"),
       check.table = TRUE,
       geneUniverse,
       orgPackg,
       onto,
       GOLevel = NULL,
       storeEnrichedIn = TRUE,
       pAdjustMeth = "BH",
       pvalCutoff = 0.01,
       qvalCutoff = 0.05,
       keyType = "ENTREZID",
       parallel = FALSE,
       nOfCores = 1,
       ...
     )
     
     ## S3 method for class 'list'
     buildEnrichTable(
       x,
       check.table = TRUE,
       geneUniverse,
       orgPackg,
       onto,
       GOLevel = NULL,
       storeEnrichedIn = TRUE,
       pAdjustMeth = "BH",
       pvalCutoff = 0.01,
       qvalCutoff = 0.05,
       keyType = "ENTREZID",
       parallel = FALSE,
       nOfCores = min(parallel::detectCores() - 1, length(x) - 1),
       ...
     )
     
_A_r_g_u_m_e_n_t_s:

       x: A list of gene lists (each element must be a character vector
          of gene identifiers).

     ...: Additional parameters for internal use (not used for the
          moment)

       y: An object of class "character" (or coerzable to "character")
          representing a vector of gene identifiers (e.g., ENTREZ).

listNames: a character(2) with the gene lists names originating the
          cross-tabulated enrichment frequencies. Only in the
          "character" or default interface.

check.table: Logical The resulting table must be checked. Defaults to
          TRUE.

geneUniverse: character vector containing the universe of genes from
          where gene lists have been extracted. This vector must be
          obtained from the annotation package declared in 'orgPackg'.
          For more details, refer to vignette goSorensen_Introduction.

orgPackg: A string with the name of the genomic annotation package
          corresponding to a specific species to be analysed, which
          must be previously installed and activated. For more details,
          refer to vignette goSorensen_Introduction.

    onto: string describing the ontology. Either "BP", "MF" or "CC".

 GOLevel: Integer specifying the GO level to analyze. If NULL, the
          analysis is performed without restricting GO terms to a
          specific level.

storeEnrichedIn: logical, the matrix of enriched (GO terms) x (gene
          lists) TRUE/FALSE values, must be stored in the result? See
          the details section

pAdjustMeth: string describing the adjust method, either "BH", "BY" or
          "Bonf", defaults to 'BH'.

pvalCutoff: adjusted pvalue cutoff on enrichment tests to report

qvalCutoff: qvalue cutoff on enrichment tests to report as significant.
          Tests must pass i) pvalueCutoff on unadjusted pvalues, ii)
          pvalueCutoff on adjusted pvalues and iii) qvalueCutoff on
          qvalues to be reported

 keyType: keyType Character string specifying the type of gene
          identifier used in the input, such as '"ENTREZID"' or
          '"SYMBOL"'.

parallel: Logical. Defaults to FALSE but put it at TRUE for parallel
          computation.

nOfCores: Number of cores for parallel computations. Only in "list"
          interface.

_D_e_t_a_i_l_s:

     Specific methods are implemented for different input classes.

     If the argument 'storeEnrichedIn' is TRUE (the default value), the
     result of 'buildEnrichTable()' includes an additional attribute
     'enriched' with a matrix of TRUE/FALSE values. Each row indicates
     whether a given GO term is enriched or not in each one of the gene
     lists (columns). To save space, only GO terms enriched in at least
     one of the gene lists are included in this matrix.

     Also, to avoid redundancies and save space, the result of
     'buildEnrichTable.list()' (an object of class "tableList", which
     is itself an aggregate of 2x2 contingency tables of class "table")
     has the attribute 'enriched', but its table members do not have
     this attribute.

     The default value of argument 'parallel' is FALSE, and you may
     consider the trade-off between the time spent initializing
     parallelization and the possible time gain from parallel
     execution. Although it is difficult to establish a general
     guideline, parallelization is usually worthwhile only when
     analyzing many gene lists, on the order of 30 or more, although
     this depends on the computer and the application.

_V_a_l_u_e:

     See method-specific documentation.

     in the "character" interface, an object of class "table". It
     represents a 2x2 contingency table, the cross-tabulation of the
     enriched GO terms in two gene lists: "Number of enriched GO terms
     in list 1 (TRUE, FALSE)" x "Number of enriched Go terms in list 2
     (TRUE, FALSE)". In the "list" interface, the result is an object
     of class "tableList" with all pairwise tables. Class "tableList"
     corresponds to objects representing all mutual enrichment
     contingency tables generated in a pairwise fashion: Given gene
     lists (i.e. "character" vectors of gene identifiers) l1, l2, ...,
     lk, an object of class "tableList" is a list of lists of
     contingency tables t(i,j) generated from each pair of gene lists i
     and j, with the following structure:

     $l2

     $l2$l1$t(2,1)

     $l3

     $l3$l1$t(3,1), $l3$l2$t(3,2)

     ...

     $lk

     $lk$l1$t(k,1), $lk$l2$t(k,2), ..., $lk$l(k-1)t(K,k-1)

     An object of class "tableList" containing all pairwise enrichment
     contingency tables.

_M_e_t_h_o_d_s (_b_y _c_l_a_s_s):

        • 'buildEnrichTable(default)': Creates a 2x2 enrichment
          contingency table from two gene lists

        • 'buildEnrichTable(list)': Builds all pairwise enrichment
          contingency tables from a list of gene lists

_E_x_a_m_p_l_e_s:

     ## The following example is highly time-consuming and is therefore not run
     ## automatically during R CMD check.
     
     ## Not run:
     
     ## i) Obtaining ENTREZ identifiers for the gene universe of humans:
     library(org.Hs.eg.db)
     humanEntrezIDs <- keys(org.Hs.eg.db, keytype = "ENTREZID")
     
     ## ii) Gene lists to be explored for analysis:
     data(allOncoGeneLists)
     
     # iii) Calculation of the joint enrichment matrix directly from gene lists at
     # the GO 4 level and the BP ontology.:
     cont_all_BP4 <- buildEnrichTable(allOncoGeneLists,
       geneUniverse = humanEntrezIDs,
       orgPackg = "org.Hs.eg.db",
       onto = "BP",
       GOLevel = 4
     )
     cont_all_BP4
     ## End(Not run)
     
     
     # Since running this example may take several minutes, the result has been
     # pre-computed and is accessible as the following:
     data(cont_all_BP4)
     cont_all_BP4
     # This shortcut applies only to this example; for your own gene-list data,
     # the computation must be performed explicitly.
     
     # For a complete overview of this function's use, see the section 3 of the
     # vignette "Introduction to goSorensen". You can do this by consulting the
     # general package documentation or by directly running the following code in
     # the R console:
     # vignette("goSorensen_Introduction", package = "goSorensen")
     

> tab <- buildEnrichTable(allOncoGeneLists[["atlas"]], allOncoGeneLists[["sanger"]],
+                         listNames = c("atlas", "sanger"),
+                         onto = "BP", GOLevel = 5,
+                         geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> 
> tab
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE   314   410
            FALSE   26  8015
> 
> # (Here, an obvious faster possibility would be to recover the enrichment contingency
> # table from the previous normal test result:)
> tab <- getTable(normTest)
> tab
                 Enriched in sanger
Enriched in atlas TRUE FALSE
            TRUE   314   410
            FALSE   26  8015
> 
> tst <- equivTestSorensen(tab)
> tst

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -1.9354, p-value = 0.02647
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4392392
sample estimates:
Sorensen dissimilarity 
             0.4097744 
attr(,"se")
standard error 
    0.01791328 

> set.seed(123)
> bootTst <- equivTestSorensen(tab, boot = TRUE)
> bootTst

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -1.9354, p-value = 0.0269
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4393869
sample estimates:
Sorensen dissimilarity 
             0.4097744 
attr(,"se")
standard error 
    0.01791328 

> 
> dSorensen(tab)
[1] 0.4097744
> seSorensen(tab)
[1] 0.01791328
> # or:
> getDissimilarity(tst)
Sorensen dissimilarity 
             0.4097744 
attr(,"se")
standard error 
    0.01791328 
> 
> duppSorensen(tab)
[1] 0.4392392
> getUpper(tst)
   dUpper 
0.4392392 
> 
> set.seed(123)
> duppSorensen(tab, boot = TRUE)
[1] 0.4393869
attr(,"eff.nboot")
[1] 10000
> getUpper(bootTst)
   dUpper 
0.4393869 
> 
> # To perform from scratch all pairwise tests (or other Sorensen-Dice computations)
> # is even much slower. For example, all pairwise...
> # Dissimilarities:
> # # allPairDiss <- dSorensen(allOncoGeneLists,
> # #                          onto = "BP", GOLevel = 5,
> # #                          geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # # allPairDiss
> #
> # # Still time consuming but potentially faster: compute in parallel (more precisely,
> # # build all enrichment tables in parallel):
> # allPairDiss <- dSorensen(allOncoGeneLists,
> #                          onto = "BP", GOLevel = 4,
> #                          geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db",
> #                          parallel = TRUE)
> # allPairDiss
> # # Not always parallelization results in speed-up, take into account the trade-off between
> # # parallelization initialization and possible gain in speed. For a few gene lists (like
> # # in this example, 7 lists, a negative speed-up will be the most common scenario)
> 
> # Standard errors:
> # seSorensen(allOncoGeneLists,
> #            onto = "BP", GOLevel = 5,
> #            geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> #
> # Upper confidence interval limits:
> # duppSorensen(allOncoGeneLists,
> #              onto = "BP", GOLevel = 5,
> #              geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # All pairwise asymptotic normal tests:
> # allTests <- equivTestSorensen(allOncoGeneLists,
> #                               onto = "BP", GOLevel = 5,
> #                               geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # getPvalue(allTests, simplify = FALSE)
> # getPvalue(allTests)
> # p.adjust(getPvalue(allTests), method = "holm")
> # To perform all pairwise bootstrap tests from scratch is (slightly)
> # even more time consuming:
> # set.seed(123)
> # allBootTests <- equivTestSorensen(allOncoGeneLists,
> #                                   boot = TRUE,
> #                                   onto = "BP", GOLevel = 5,
> #                                   geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # Not all bootstrap replicates may conduct to finite statistics:
> # getNboot(allBootTests)
> 
> # Given the normal tests (object 'allTests'), it is much faster to upgrade
> # it to have the bootstrap tests:
> # set.seed(123)
> # allBootTests <- upgrade(allTests, boot = TRUE)
> # getPvalue(allBootTests, simplify = FALSE)
> 
> # Again, the faster and more flexible possibility may be:
> # 1) First, build all pairwise enrichment contingency tables (slow first step):
> # allTabsBP.4 <- buildEnrichTable(allOncoGeneLists,
> #                                 onto = "BP", GOLevel = 5,
> #                                 geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db")
> # allTabsBP.4
> 
> # Better, directly use the dataset available at this package, goSorensen:
> data("cont_all_BP4")
> cont_all_BP4
$cangenes
$cangenes$atlas
                    Enriched in atlas
Enriched in cangenes TRUE FALSE
               TRUE     0     0
               FALSE  286  3037
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4


$cis
$cis$atlas
               Enriched in atlas
Enriched in cis TRUE FALSE
          TRUE    37     2
          FALSE  249  3035

$cis$cangenes
               Enriched in cangenes
Enriched in cis TRUE FALSE
          TRUE     0    39
          FALSE    0  3284
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4


$miscellaneous
$miscellaneous$atlas
                         Enriched in atlas
Enriched in miscellaneous TRUE FALSE
                    TRUE    98     3
                    FALSE  188  3034

$miscellaneous$cangenes
                         Enriched in cangenes
Enriched in miscellaneous TRUE FALSE
                    TRUE     0   101
                    FALSE    0  3222
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$miscellaneous$cis
                         Enriched in cis
Enriched in miscellaneous TRUE FALSE
                    TRUE    22    79
                    FALSE   17  3205


$sanger
$sanger$atlas
                  Enriched in atlas
Enriched in sanger TRUE FALSE
             TRUE   127    19
             FALSE  159  3018

$sanger$cangenes
                  Enriched in cangenes
Enriched in sanger TRUE FALSE
             TRUE     0   146
             FALSE    0  3177
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$sanger$cis
                  Enriched in cis
Enriched in sanger TRUE FALSE
             TRUE    28   118
             FALSE   11  3166

$sanger$miscellaneous
                  Enriched in miscellaneous
Enriched in sanger TRUE FALSE
             TRUE    69    77
             FALSE   32  3145


$Vogelstein
$Vogelstein$atlas
                      Enriched in atlas
Enriched in Vogelstein TRUE FALSE
                 TRUE   134    20
                 FALSE  152  3017

$Vogelstein$cangenes
                      Enriched in cangenes
Enriched in Vogelstein TRUE FALSE
                 TRUE     0   154
                 FALSE    0  3169
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$Vogelstein$cis
                      Enriched in cis
Enriched in Vogelstein TRUE FALSE
                 TRUE    27   127
                 FALSE   12  3157

$Vogelstein$miscellaneous
                      Enriched in miscellaneous
Enriched in Vogelstein TRUE FALSE
                 TRUE    74    80
                 FALSE   27  3142

$Vogelstein$sanger
                      Enriched in sanger
Enriched in Vogelstein TRUE FALSE
                 TRUE   132    22
                 FALSE   14  3155


$waldman
$waldman$atlas
                   Enriched in atlas
Enriched in waldman TRUE FALSE
              TRUE   169    45
              FALSE  117  2992

$waldman$cangenes
                   Enriched in cangenes
Enriched in waldman TRUE FALSE
              TRUE     0   214
              FALSE    0  3109
attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4

$waldman$cis
                   Enriched in cis
Enriched in waldman TRUE FALSE
              TRUE    33   181
              FALSE    6  3103

$waldman$miscellaneous
                   Enriched in miscellaneous
Enriched in waldman TRUE FALSE
              TRUE    94   120
              FALSE    7  3102

$waldman$sanger
                   Enriched in sanger
Enriched in waldman TRUE FALSE
              TRUE   105   109
              FALSE   41  3068

$waldman$Vogelstein
                   Enriched in Vogelstein
Enriched in waldman TRUE FALSE
              TRUE   116    98
              FALSE   38  3071


attr(,"onto")
[1] "BP"
attr(,"GOLevel")
[1] 4
attr(,"class")
[1] "tableList" "list"     
attr(,"enriched")
           atlas cangenes   cis miscellaneous sanger Vogelstein waldman
GO:0001649  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0030278  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0030282  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0045778  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0060688  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0061138  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0002263  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE    TRUE
GO:0030168  TRUE    FALSE FALSE          TRUE   TRUE      FALSE    TRUE
GO:0042118 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0050866  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0050867  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0072537  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0002260 FALSE    FALSE  TRUE         FALSE   TRUE       TRUE   FALSE
GO:0001818  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0001819  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE   FALSE
GO:0002367  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0002534  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0010573  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032602  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032612  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032613  TRUE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:0032615  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032623  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032633 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0032635  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0071604  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0071706  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0002562 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002566 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0016445 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002443  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002697  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE    TRUE
GO:0002698  TRUE    FALSE  TRUE         FALSE  FALSE      FALSE    TRUE
GO:0002699  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0043299  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002218  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0034101  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE    TRUE
GO:0002377  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002700  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002701 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0002702  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0002461 FALSE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:0002200 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0048534  TRUE    FALSE  TRUE         FALSE  FALSE      FALSE    TRUE
GO:0002685  TRUE    FALSE  TRUE          TRUE  FALSE      FALSE    TRUE
GO:1903706  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0002695  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0050777  TRUE    FALSE  TRUE         FALSE  FALSE      FALSE    TRUE
GO:0050858  TRUE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:1903707  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE    TRUE
GO:0002687  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0002696  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:1903708  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0007548  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0045137  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048608  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0060008 FALSE    FALSE FALSE         FALSE  FALSE       TRUE   FALSE
GO:0003012  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0022600  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0001666  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0002931  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0006970  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0006979  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0009408  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0009611  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0034405  TRUE    FALSE FALSE         FALSE  FALSE       TRUE   FALSE
GO:0035902 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0035966  TRUE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:0042594  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0055093  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0002437 FALSE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0042092  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0031023 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0032886  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0045786  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0045787  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0051321 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0007162  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0031589  TRUE    FALSE  TRUE          TRUE  FALSE      FALSE    TRUE
GO:0045785  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0030010  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032878 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0061245  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0061339  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0009755  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0009756  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0038034  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0008366  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0007389  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0007566  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0046660  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0046661  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0048736  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0003002  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0009798  TRUE    FALSE FALSE          TRUE   TRUE       TRUE   FALSE
GO:0009880  TRUE    FALSE FALSE          TRUE  FALSE       TRUE    TRUE
GO:0061450 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0007611 FALSE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0032922  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0042752  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0006413  TRUE    FALSE FALSE          TRUE  FALSE      FALSE   FALSE
GO:0032259 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0032963  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0072593  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:2001057  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0010463  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0014009  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0033002  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048144  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0050673  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0051450 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0061351  TRUE    FALSE FALSE         FALSE   TRUE      FALSE    TRUE
GO:0070661  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0072089  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0072111  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0009612  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0009649 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0032102  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0042330  TRUE    FALSE  TRUE          TRUE  FALSE      FALSE    TRUE
GO:0071496  TRUE    FALSE  TRUE         FALSE  FALSE       TRUE    TRUE
GO:0002347  TRUE    FALSE  TRUE         FALSE   TRUE      FALSE   FALSE
GO:0002833  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0009595  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0071216  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE   FALSE
GO:1990840  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0009266  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0009314  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0051602  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0070482  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0071214  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0001763  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0003151  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0003179  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0003206  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0010171 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0031069 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0035107  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048532  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0048598  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0048729  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0060323 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0060325 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0060411  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0060560  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0061383  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0072028  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0001558  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0030307  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0030308  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048588  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0046718  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0035019 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:1902459  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:2000036  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0071695  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0007051  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0007059  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0007062 FALSE    FALSE FALSE         FALSE  FALSE       TRUE   FALSE
GO:0007098  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0008608  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0010948  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0044786 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0045023  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0051304  TRUE    FALSE FALSE          TRUE  FALSE      FALSE   FALSE
GO:0090068  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0090399  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:1903046 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0022405  TRUE    FALSE FALSE          TRUE  FALSE       TRUE    TRUE
GO:0046883  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0099177  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0046887  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032970  TRUE    FALSE  TRUE          TRUE  FALSE      FALSE    TRUE
GO:0031295  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0007584  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0031669  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:1905952  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0031503 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0046620  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0046622  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0060419  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0051851  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0045926  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0045927  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048638  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0040013  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0050920  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:2000146  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0050921  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0001101 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0006935  TRUE    FALSE  TRUE          TRUE  FALSE      FALSE    TRUE
GO:0009410  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0009636 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0010038  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0035094  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0046677  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE   FALSE
GO:0046683  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:1902074  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0022404  TRUE    FALSE FALSE          TRUE  FALSE       TRUE    TRUE
GO:0042633 FALSE    FALSE FALSE          TRUE  FALSE       TRUE    TRUE
GO:0044849 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0014745  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0043502  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0043697  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0051701  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0007565  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0043368  TRUE    FALSE  TRUE         FALSE   TRUE      FALSE   FALSE
GO:0002274  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0002366  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE    TRUE
GO:0048640  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048639  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:1903829  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:1904181 FALSE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:1904951  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:1905954  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0051051  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:1900047  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0032388  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0044089  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0051781 FALSE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0090314 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:1905898  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:2000773  TRUE    FALSE FALSE          TRUE  FALSE      FALSE   FALSE
GO:0060759  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0090287  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:1900076  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0045739  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:1903846  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0031348  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0045738  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:1903035  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0035264 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0035265  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0042246  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0055017  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0042698 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0001704  TRUE    FALSE FALSE         FALSE  FALSE       TRUE   FALSE
GO:0003272  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0030220  TRUE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:0035148  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0048645  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0034103  TRUE    FALSE FALSE         FALSE  FALSE       TRUE    TRUE
GO:0034104  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0046849  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0001541  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0001942  TRUE    FALSE FALSE          TRUE  FALSE       TRUE    TRUE
GO:0002088 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0003170  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0003205  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0003279  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0021510  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0021537  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0021543  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0021675  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0021772 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0021987  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0021988 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0030900  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0030901  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0031018  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0031099  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0032835  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0036302  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0048286 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0060021  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0060324 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0060711 FALSE    FALSE FALSE         FALSE  FALSE       TRUE   FALSE
GO:0072006  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:1904888 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0001708  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0001709  TRUE    FALSE FALSE         FALSE  FALSE       TRUE   FALSE
GO:0045165  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0001659  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0001894  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0048872  TRUE    FALSE  TRUE         FALSE   TRUE       TRUE    TRUE
GO:0060249  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0097009  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0033500  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0040008  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:1900046  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:2000241 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0010453  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0048634 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0090183 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:1901861 FALSE    FALSE  TRUE         FALSE  FALSE      FALSE    TRUE
GO:0031641 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0051302  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:1900117  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:2000772  TRUE    FALSE FALSE          TRUE  FALSE       TRUE   FALSE
GO:0007596  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0050819  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0002523  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0030595  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0071674  TRUE    FALSE  TRUE          TRUE  FALSE      FALSE    TRUE
GO:0097529  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0032370  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0045807  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0051222  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0051048 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0051961  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:1901343  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0045684  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0045830 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0051798  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0051962  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:1900006 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:1904018  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:1905332  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0051656  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0051651  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0010632  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0120161  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0010718  TRUE    FALSE FALSE          TRUE  FALSE       TRUE    TRUE
GO:0120162  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:2000833  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0010633  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0014741  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0120163 FALSE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0055057 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0050000  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:1990849  TRUE    FALSE FALSE         FALSE   TRUE      FALSE    TRUE
GO:0009615  TRUE    FALSE FALSE          TRUE  FALSE      FALSE   FALSE
GO:0009620  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0104004  TRUE    FALSE FALSE          TRUE   TRUE       TRUE    TRUE
GO:0034219 FALSE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:0008360  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0031647  TRUE    FALSE FALSE          TRUE   TRUE       TRUE   FALSE
GO:0043114  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0050803  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0050878  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0090066  TRUE    FALSE FALSE          TRUE  FALSE      FALSE    TRUE
GO:0090559  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0099072  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:0099149 FALSE    FALSE FALSE         FALSE   TRUE      FALSE   FALSE
GO:0044092  TRUE    FALSE FALSE          TRUE   TRUE      FALSE    TRUE
GO:0044093  TRUE    FALSE  TRUE          TRUE   TRUE       TRUE    TRUE
GO:0051098  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0090132  TRUE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0099645  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:1900119  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0019827  TRUE    FALSE FALSE         FALSE   TRUE       TRUE   FALSE
GO:0030193  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0030195  TRUE    FALSE FALSE         FALSE  FALSE      FALSE   FALSE
GO:0044771 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
GO:0061982 FALSE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:1900182  TRUE    FALSE FALSE         FALSE   TRUE       TRUE    TRUE
GO:1901993 FALSE    FALSE FALSE         FALSE  FALSE      FALSE    TRUE
attr(,"enriched")attr(,"nTerms")
[1] 3323
> class(cont_all_BP4)
[1] "tableList" "list"     
> # 2) Then perform all required computatios from these enrichment contingency tables...
> # All pairwise tests:
> allTests <- equivTestSorensen(cont_all_BP4)
> allTests
$cangenes
$cangenes$atlas

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$cis
$cis$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 10.569, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.8233323
sample estimates:
Sorensen dissimilarity 
             0.7723077 
attr(,"se")
standard error 
    0.03102077 


$cis$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$miscellaneous
$miscellaneous$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 1.578, p-value = 0.9427
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.5447161
sample estimates:
Sorensen dissimilarity 
             0.4935401 
attr(,"se")
standard error 
    0.03111282 


$miscellaneous$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$miscellaneous$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 4.7162, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.769861
sample estimates:
Sorensen dissimilarity 
             0.6857143 
attr(,"se")
standard error 
    0.05115758 



$sanger
$sanger$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -1.1498, p-value = 0.1251
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4583993
sample estimates:
Sorensen dissimilarity 
              0.412037 
attr(,"se")
standard error 
    0.02818626 


$sanger$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$sanger$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 5.7276, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7699121
sample estimates:
Sorensen dissimilarity 
             0.6972973 
attr(,"se")
standard error 
    0.04414665 


$sanger$miscellaneous

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -0.082785, p-value = 0.467
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.503861
sample estimates:
Sorensen dissimilarity 
             0.4412955 
attr(,"se")
standard error 
    0.03803709 



$Vogelstein
$Vogelstein$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -1.9482, p-value = 0.0257
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4361093
sample estimates:
Sorensen dissimilarity 
             0.3909091 
attr(,"se")
standard error 
    0.02747975 


$Vogelstein$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$Vogelstein$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 6.4873, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7901267
sample estimates:
Sorensen dissimilarity 
             0.7202073 
attr(,"se")
standard error 
    0.04250801 


$Vogelstein$miscellaneous

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -0.67266, p-value = 0.2506
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4803412
sample estimates:
Sorensen dissimilarity 
             0.4196078 
attr(,"se")
standard error 
    0.03692323 


$Vogelstein$sanger

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -16.292, p-value < 2.2e-16
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.152757
sample estimates:
Sorensen dissimilarity 
                  0.12 
attr(,"se")
standard error 
    0.01991484 



$waldman
$waldman$atlas

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -4.9937, p-value = 2.961e-07
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3636725
sample estimates:
Sorensen dissimilarity 
                 0.324 
attr(,"se")
standard error 
    0.02411914 


$waldman$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$waldman$cis

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = 8.0759, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7991502
sample estimates:
Sorensen dissimilarity 
             0.7391304 
attr(,"se")
standard error 
    0.03648945 


$waldman$miscellaneous

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -1.2577, p-value = 0.1043
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4571484
sample estimates:
Sorensen dissimilarity 
             0.4031746 
attr(,"se")
standard error 
    0.03281373 


$waldman$sanger

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -0.89641, p-value = 0.185
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4676368
sample estimates:
Sorensen dissimilarity 
             0.4166667 
attr(,"se")
standard error 
    0.03098765 


$waldman$Vogelstein

	Normal asymptotic test for 2x2 contingency tables based on the
	Sorensen-Dice dissimilarity

data:  tab
(d - d0) / se = -2.5378, p-value = 0.005577
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4180967
sample estimates:
Sorensen dissimilarity 
             0.3695652 
attr(,"se")
standard error 
    0.02950506 



attr(,"class")
[1] "equivSDhtestList" "list"            
> class(allTests)
[1] "equivSDhtestList" "list"            
> set.seed(123)
> allBootTests <- equivTestSorensen(cont_all_BP4, boot = TRUE)
> allBootTests
$cangenes
$cangenes$atlas

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$cis
$cis$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 10.569, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.8198006
sample estimates:
Sorensen dissimilarity 
             0.7723077 
attr(,"se")
standard error 
    0.03102077 


$cis$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 



$miscellaneous
$miscellaneous$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 1.578, p-value = 0.9418
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.5458334
sample estimates:
Sorensen dissimilarity 
             0.4935401 
attr(,"se")
standard error 
    0.03111282 


$miscellaneous$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$miscellaneous$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 4.7162, p-value = 0.9999
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7673133
sample estimates:
Sorensen dissimilarity 
             0.6857143 
attr(,"se")
standard error 
    0.05115758 



$sanger
$sanger$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -1.1498, p-value = 0.1304
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4590251
sample estimates:
Sorensen dissimilarity 
              0.412037 
attr(,"se")
standard error 
    0.02818626 


$sanger$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$sanger$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 5.7276, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7681412
sample estimates:
Sorensen dissimilarity 
             0.6972973 
attr(,"se")
standard error 
    0.04414665 


$sanger$miscellaneous

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -0.082785, p-value = 0.4683
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.5051565
sample estimates:
Sorensen dissimilarity 
             0.4412955 
attr(,"se")
standard error 
    0.03803709 



$Vogelstein
$Vogelstein$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -1.9482, p-value = 0.0333
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4386614
sample estimates:
Sorensen dissimilarity 
             0.3909091 
attr(,"se")
standard error 
    0.02747975 


$Vogelstein$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$Vogelstein$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 6.4873, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7882428
sample estimates:
Sorensen dissimilarity 
             0.7202073 
attr(,"se")
standard error 
    0.04250801 


$Vogelstein$miscellaneous

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -0.67266, p-value = 0.2531
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4809647
sample estimates:
Sorensen dissimilarity 
             0.4196078 
attr(,"se")
standard error 
    0.03692323 


$Vogelstein$sanger

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -16.292, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.1580463
sample estimates:
Sorensen dissimilarity 
                  0.12 
attr(,"se")
standard error 
    0.01991484 



$waldman
$waldman$atlas

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -4.9937, p-value = 9.999e-05
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.3664417
sample estimates:
Sorensen dissimilarity 
                 0.324 
attr(,"se")
standard error 
    0.02411914 


$waldman$cangenes

	No test performed due not finite (d - d0) / se statistic

data:  tab
(d - d0) / se = Inf, p-value = NA
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
   0 NaN
sample estimates:
Sorensen dissimilarity 
                     1 
attr(,"se")
standard error 
             0 


$waldman$cis

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = 8.0759, p-value = 1
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.7974428
sample estimates:
Sorensen dissimilarity 
             0.7391304 
attr(,"se")
standard error 
    0.03648945 


$waldman$miscellaneous

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -1.2577, p-value = 0.1079
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.0000000 0.4585292
sample estimates:
Sorensen dissimilarity 
             0.4031746 
attr(,"se")
standard error 
    0.03281373 


$waldman$sanger

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -0.89641, p-value = 0.179
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.468985
sample estimates:
Sorensen dissimilarity 
             0.4166667 
attr(,"se")
standard error 
    0.03098765 


$waldman$Vogelstein

	Bootstrap test for 2x2 contingency tables based on the Sorensen- Dice
	dissimilarity (10000 bootstrap replicates)

data:  tab
(d - d0) / se = -2.5378, p-value = 0.007799
alternative hypothesis: true equivalence limit d0 is less than 0.4444444
95 percent confidence interval:
 0.000000 0.419616
sample estimates:
Sorensen dissimilarity 
             0.3695652 
attr(,"se")
standard error 
    0.02950506 



attr(,"class")
[1] "equivSDhtestList" "list"            
> class(allBootTests)
[1] "equivSDhtestList" "list"            
> getPvalue(allBootTests, simplify = FALSE)
                   atlas cangenes    cis miscellaneous     sanger Vogelstein
atlas         0.00000000      NaN 1.0000     0.9418058 0.13038696 0.03329667
cangenes             NaN        0    NaN           NaN        NaN        NaN
cis           1.00000000      NaN 0.0000     0.9999000 1.00000000 1.00000000
miscellaneous 0.94180582      NaN 0.9999     0.0000000 0.46825317 0.25307469
sanger        0.13038696      NaN 1.0000     0.4682532 0.00000000 0.00009999
Vogelstein    0.03329667      NaN 1.0000     0.2530747 0.00009999 0.00000000
waldman       0.00009999      NaN 1.0000     0.1078892 0.17898210 0.00779922
                 waldman
atlas         0.00009999
cangenes             NaN
cis           1.00000000
miscellaneous 0.10788921
sanger        0.17898210
Vogelstein    0.00779922
waldman       0.00000000
> getEffNboot(allBootTests)
          cangenes.atlas                cis.atlas             cis.cangenes 
                     NaN                    10000                      NaN 
     miscellaneous.atlas   miscellaneous.cangenes        miscellaneous.cis 
                   10000                      NaN                    10000 
            sanger.atlas          sanger.cangenes               sanger.cis 
                   10000                      NaN                    10000 
    sanger.miscellaneous         Vogelstein.atlas      Vogelstein.cangenes 
                   10000                    10000                      NaN 
          Vogelstein.cis Vogelstein.miscellaneous        Vogelstein.sanger 
                   10000                    10000                    10000 
           waldman.atlas         waldman.cangenes              waldman.cis 
                   10000                      NaN                    10000 
   waldman.miscellaneous           waldman.sanger       waldman.Vogelstein 
                   10000                    10000                    10000 
> 
> # To adjust for testing multiplicity:
> p.adjust(getPvalue(allBootTests), method = "holm")
          cangenes.atlas.p-value                cis.atlas.p-value 
                             NaN                       1.00000000 
            cis.cangenes.p-value      miscellaneous.atlas.p-value 
                             NaN                       1.00000000 
  miscellaneous.cangenes.p-value        miscellaneous.cis.p-value 
                             NaN                       1.00000000 
            sanger.atlas.p-value          sanger.cangenes.p-value 
                      1.00000000                              NaN 
              sanger.cis.p-value     sanger.miscellaneous.p-value 
                      1.00000000                       1.00000000 
        Vogelstein.atlas.p-value      Vogelstein.cangenes.p-value 
                      0.39956004                              NaN 
          Vogelstein.cis.p-value Vogelstein.miscellaneous.p-value 
                      1.00000000                       1.00000000 
       Vogelstein.sanger.p-value            waldman.atlas.p-value 
                      0.00149985                       0.00149985 
        waldman.cangenes.p-value              waldman.cis.p-value 
                             NaN                       1.00000000 
   waldman.miscellaneous.p-value           waldman.sanger.p-value 
                      1.00000000                       1.00000000 
      waldman.Vogelstein.p-value 
                      0.10138986 
> 
> # If only partial statistics are desired:
> dSorensen(cont_all_BP4)
                  atlas cangenes       cis miscellaneous    sanger Vogelstein
atlas         0.0000000        1 0.7723077     0.4935401 0.4120370  0.3909091
cangenes      1.0000000        0 1.0000000     1.0000000 1.0000000  1.0000000
cis           0.7723077        1 0.0000000     0.6857143 0.6972973  0.7202073
miscellaneous 0.4935401        1 0.6857143     0.0000000 0.4412955  0.4196078
sanger        0.4120370        1 0.6972973     0.4412955 0.0000000  0.1200000
Vogelstein    0.3909091        1 0.7202073     0.4196078 0.1200000  0.0000000
waldman       0.3240000        1 0.7391304     0.4031746 0.4166667  0.3695652
                waldman
atlas         0.3240000
cangenes      1.0000000
cis           0.7391304
miscellaneous 0.4031746
sanger        0.4166667
Vogelstein    0.3695652
waldman       0.0000000
> duppSorensen(cont_all_BP4)
                  atlas cangenes       cis miscellaneous    sanger Vogelstein
atlas         0.0000000      NaN 0.8233323     0.5447161 0.4583993  0.4361093
cangenes            NaN        0       NaN           NaN       NaN        NaN
cis           0.8233323      NaN 0.0000000     0.7698610 0.7699121  0.7901267
miscellaneous 0.5447161      NaN 0.7698610     0.0000000 0.5038610  0.4803412
sanger        0.4583993      NaN 0.7699121     0.5038610 0.0000000  0.1527570
Vogelstein    0.4361093      NaN 0.7901267     0.4803412 0.1527570  0.0000000
waldman       0.3636725      NaN 0.7991502     0.4571484 0.4676368  0.4180967
                waldman
atlas         0.3636725
cangenes            NaN
cis           0.7991502
miscellaneous 0.4571484
sanger        0.4676368
Vogelstein    0.4180967
waldman       0.0000000
> seSorensen(cont_all_BP4)
                   atlas cangenes        cis miscellaneous     sanger
atlas         0.00000000        0 0.03102077    0.03111282 0.02818626
cangenes      0.00000000        0 0.00000000    0.00000000 0.00000000
cis           0.03102077        0 0.00000000    0.05115758 0.04414665
miscellaneous 0.03111282        0 0.05115758    0.00000000 0.03803709
sanger        0.02818626        0 0.04414665    0.03803709 0.00000000
Vogelstein    0.02747975        0 0.04250801    0.03692323 0.01991484
waldman       0.02411914        0 0.03648945    0.03281373 0.03098765
              Vogelstein    waldman
atlas         0.02747975 0.02411914
cangenes      0.00000000 0.00000000
cis           0.04250801 0.03648945
miscellaneous 0.03692323 0.03281373
sanger        0.01991484 0.03098765
Vogelstein    0.00000000 0.02950506
waldman       0.02950506 0.00000000
> 
> 
> # Tipically, in a real study it would be interesting to scan tests
> # along some ontologies and levels inside these ontologies:
> # (which obviously will be a quite slow process)
> # gc()
> # set.seed(123)
> # allBootTests_BP_MF_lev4to8 <- allEquivTestSorensen(allOncoGeneLists,
> #                                                    boot = TRUE,
> #                                                    geneUniverse = humanEntrezIDs, orgPackg = "org.Hs.eg.db",
> #                                                    ontos = c("BP", "MF"), GOLevels = 4:8)
> # getPvalue(allBootTests_BP_MF_lev4to8)
> # getEffNboot(allBootTests_BP_MF_lev4to8)
> 
> proc.time()
   user  system elapsed 
 64.448   0.778  65.525