GO-Elite: A New Method for Identifying Minimal Redundant Annotations from Gene Expression Data

Nathan G. Salomonis1,2, Stan Gaj3,4, Alexander C. Zambon1,5, Karen Vranizan1,6, Chris Evelo3 and  Bruce R. Conklin1,2

1Gladstone Institute of Cardiovascular Disease, University of California, San Francisco, CA 94141
2Pharmaceutical Sciences and Pharmacogenomics Graduate Program, University of California, San Francisco, CA 94141
3Department of Bioinformatics, BiGCaT Bioinformatics, Maastricht University, Maastricht, The Netherlands
4Nutrigenomics Consortium, Top Institute Food and Nutrition, Wageningen, The Netherlands 
5Department of Pharmacology, University of California at San Diego, La Jolla, CA 92093
6Functional Genomics Laboratory, University of California, Berkeley, CA 94720

Corresponding author:
Bruce R. Conklin, 
Gladstone Institute of Cardiovascular Disease
1650 Owens St
San Francisco, CA 94158, USA
Phone: (415) 734-2712
Fax: (415) 415-355-0960
E-mail: bconklin@gladstone.ucsf.edu
Web address: http://www.ConklinLab.org
Abstract
Background: Microarray analysis has become a powerful means to elucidate the genetic and expression networks that regulate cell function.  Tools which annotate user data based on Gene Ontology (GO) annotations and pathways can provide a high level view of the biological associations linked to such data.  These methods often generate results in the form of gene associations to nested GO terms, where all child gene associations are included with each set of parent associations. While providing more rich annotations at every level within the GO hierarchy, this approach is disadvantages in its ability to report a minimal set of significant GO associations, with often many parent and children terms highlighted. 
Implementation: To improve on this technique, we have written a stand-alone program, named GO-Elite (http://www.genmapp.org/go_elite), with an efficient algorithm to prune existing GO results and perform new MAPPFinder analyses in order to provide a minimal set of non-redundant GO terms for describing gene expression criterion. This approach considers the relative position and empirically derived over-representation score of significant GO terms within the GO hierarchy, to choose the highest scoring terms at each trunk and branch in hierarchy to identify a minimal set of required terms from the original GO categories. GO-Elite is able to reduce the number of MAPPFinder GO results by as much as 90%, while retaining as much as a 100% of the original associated genes and biological trends. When combined with gene-redundancy pruning, GO-Elite can further reduce GO and pathway results. In addition redundancy filtering, GO-Elite provides multiple levels of gene annotation along with pathway level gene data averaging. To facilitate community contribution, this software has built in tools for supporting new gene and species relationships, simple user interface and underlying data structure.

Introduction
In recent years, the use of pathway/ontology over-representation analysis (ORA) has become the gold standard method for obtaining biological insights into data from genome scale experiments. This method has broad ranging applications, from understanding the basis of phenotypic differences in genetic cell and animal models from whole genome mRNA expression data, to identifying DNA polymorphisms which co-occur within biologically pathways 1.  With an ever-increasing amount of data produced from microarray and high-throughput sequencing technologies, the need for efficient and informative pathway-level analyses are in great demand.
We originally described a freely available tool called MAPPFinder 2 for linking user genomic datasets to curated biological pathways and hierarchically organized GO terms 3 for ORA. Unlike many other tools, MAPPFinder, which is component of the GenMAPP application4, is able to use user-defined statistical criterion to perform ORA on curated and custom pathways and nested GO terms, display the ORA results in a hierarchically ordered tree and the further visualize gene level changes for any biological term back in GenMAPP. More recent versions of MAPPFinder perform a permutation analysis of z-scores (normal approximation to the hypergeometric distribution) to determine an overall and multi-hypothesis corrected likelihood that over-representation of each biological category is not due to chance. This has been a highly popular application among computational biologists and typical bench biologists, given the software’s user friendly nature and ease of use.
Both GO and pathway level perspectives can provide distinct but useful insights into user data. The GO has a hierarchical structured nature and thus allowing for distinct annotations and ORA at each level of the hierarchy. This structure allows for the user to assess changes for each parent, child and sibling GO-term. The genomic coverage of GO terms is typically far greater than that observed for pathways (~70% versus ~ 25%) (PMID: 17588266) and can include electronically inferred content without having to infer and test biological interactions. Nonetheless, pathway information can provide detailed interactions, additional cellular context and annotation information as well as highlighting critical rate limiting steps.
In addition to the approach used by MAPPFinder, several other methods have been developed that exploit additional relationships either from the user data or from pathway interactions to additionally weight their ORA statistics.  One approach, Gene Set Enrichment Analysis (GSEA) 5 uses a prior ranking of the gene data (e.g. by expression clustering) to identify more highly related sets of genes (based on their distance by ranking) which are specifically enriched in certain pathways, GO categories and genomic loci. An advantage of this approach is that stringent filtering of the data may not be necessary since ranking can be based on the pattern and robustness of the change, however, such a ranking requires that user be able to appropriately rank their genes and that such a ranking is biologically significant for ORA. Another intriguing strategy is used by the tool Pathway-Express: 6, a component of Onto-Tools 7) which in addition to weighting gene changes based on magnitude of the change but also considers the position of the protein within the context of known interaction networks to determine the overall impact of that change and others on the entire network. 
A principle challenge for all of these approaches is the identification of a minimal set of non-redundant terms to describe user genome level results. Such methods are necessary for efficiently summarizing ORA results in a publication ready manner.  The most popular GO ORA tools, including MAPPFinder, nest child gene associations up to parents often producing many related GO terms (sharing overlapping gene content) as being over-represented. Methods such as GO-Slim, provide a slimmed down version of GO-hierarchy to address this, but can result in lost associations to highly specific terms. Although some algorithms provide methods to cluster (DAVID) (PMID: 12734009) and identify the most statistically enriched GO terms among a set of ORA results (TopGO) (PMID: 16606683), these tools are relatively few with little control over how to best identify an optimal set of descriptive terms based on different enrichment options and obtain gene associations for these terms. Redundancy is also an issue for pathway/network ORA, where often highly related pathways are present within a single pathway archive.
To address these challenges we have developed a new approach called GO-Elite, for eliminating or highlighting redundant GO terms or pathways from user genomic datasets. This method can be applied to either raw user data or to existing ORA data from other sources to reduce these results to a manageable, optimal set of descriptive terms. Along with GO-Elite terms, this approach includes maximally informative gene level annotation and data summarization results, which allow users to easily view associated data at glance.  In addition to GO terms, GO-Elite includes pathway level ORA for supplied and custom relationships along with the ability to assess gene redundancy between pathways. This software was written with the common bench lab biologist in mind and thus requires no prior expertise with any bioinformatics applications. With a modular structure and simple built-in tools for database file creation, GO-Elite will allow any user to analyze data for any given number of custom species or gene relationship systems.

Methods
Over-representation analysis.  There are three critical steps in the GO-Elite analysis: 1) building ORA files, 2) establishing criterion for filtering and 3) GO-Elite pruning and gene/data summarization.  Input ORA files for the GO-Elite analysis can be created in one of three ways; 1) built directly using the mappfinder option in GO-Elite from raw gene level data, 2) using pre-built MAPPFinder 2.0 (a component of the GenMAPP 2.0 application (http://www.GenMAPP.org, Dahlquist et. al 2002) results or 3) output from other GO analysis programs re-formatted into the MAPPFinder layout. When building ORA from scratch in GO-Elite, users must begin with at least two input gene lists; a set of regulated genes (numerator) and a denominator list with all gene IDs initially examined, stored in an existing species-specific directory. Multiple gene ID systems are supported in GO-Elite, including Affymetrix, Ensembl and EntrezGene, however, additional systems can be easily added by modifying or adding database text files (see building new relationship). By default, the user has the option of using either Ensembl or EntrezGene as their primary database gene system to link to GO or supplied GenMAPP pathways, where GO relationships are supplied by the respective resources. For non-primary gene systems, such Affymetrix, relationships to Ensembl and EntrezGene are stored in database file folders along with the program, which can be augmented by users using built-in tools. 
GO-Elite uses the latest version of the OBO files locally present (supplied with program or updated by users from files posted at GeneOntology.org) along with gene to GO relationship files to build a nested tree of gene to GO relationships, similar to MAPPFinder.  Unique nested gene to GO relationships are assessed for each GO-term and a permuted non-adjusted p-value from the mappfinder z-score and a Benjamini-Hochberg false discovery rate p-value are exported using the function as non-GO-Elite MAPPFinder output files. Z-scores are calculated using a normal approximation to the hypergeometric distribution to calculate a as described by Doniger et al2., where a particular gene or array ID is counted only once per GO-term independent of the number of times present in the regulated gene list or redundant gene associations present to IDs in the GO-term. To calculate a p-value for each GO term, the mappfinder module (referred to in lower case) of GO-Elite randomly selects the same number of input genes or array IDs in the user’s input gene list from the denominator list, 2000 times (or user defined) to determine the likelihood of obtaining a z score greater than or equal to the empirically derived z score.  This p-value is adjusted for multiple hypothesis testing, using the Benjamini-Hochberg correction method and saved as a second set of p-values in the output file (ref). Run-time for ORA in GO-Elite is typically one to dozens of minutes per gene list, depending on (a) the number of genes in the regulated list and (b) number permutations selected.
As an alternative to running ORA directly in GO-Elite, such results can also be supplied by from either GenMAPP or other methods formatted with the MAPPFinder layout. The MAPPFinder analysis method in GenMAPP is similar to that employed by GO-Elite’s mappfinder module, with exception to the multiple hypothesis correction method used is based on the Westfall-Young method (ref). GO-Elite can use the raw-text files automatically produced by MAPPFinder along with corresponding input gene lists to build a minimal non-redundant set of GO terms or pathways matching the original user’s GenMAPP criterion along with gene/data summary information.
Filtering statistics. The over-represenation z-score, number of genes changed and non-adjusted permutation p-value generated by either the mappfinder module or GenMAPP are the default statistics used for pruning GO terms and pathways in GO-Elite.  When ORA data from other GO and pathway analysis programs are used as input for GO-Elite filtering, analogous statistics are recommended.  Upon import of ORA data, only those GO terms and pathways that meet the user defined minimum filters (by default, permuted p-value < 0.05, z-score > 2 and number of genes change > 2) are processed for redundancy. Once imported, GO-Elite will compare related GO terms based one of three possible options: 1) z-score, 2) number genes changed or 3) combination.  The z-score option ranks GO terms only based on the MAPPFinder calculated z-score, ranked from higher to lowest. The gene number option allows the user to rank based on the number of genes changed in the GO hierarchy, again from highest to lowest.  The combination option is a weighted metric based of both number of genes changed and z-score, generated by multiplying the z-score times the log base 2 of the number of genes changed for a given GO term. These scores are used for selecting which GO terms will be reported by GO-Elite.
GO-Elite Algorithm. GO-Elite can process different types of ORA files, corresponding to either GO results (file suffix “-GO.txt”) or pathway results (file suffix “-local.txt”).  For GO-level results, after GO terms are initially filtered based on user defined statistics, all possible parent-child relationships are built and stored for these GO terms, where each parent is the key in the database (Python dictionary object) and all of its children are the value.  This full database is stored for later queries, while the full parent-child paths (agglomerated path relationships) for all entries are generated by iterating this process. The program then searches these relationships in a hierarchical nature to identify the most significant scoring GO term that either has a higher score (see Algorithm Statistics) than all of it’s children (along that branch of the tree) or sibling terms (children of a single parent, each representing distinct branches) where at least one of the sibling terms on a branch scores greater than the parent. For these sibling terms, if one sibling branch scores higher than the parent and another branch does not, the highest scoring term from the latter sibling branch is also selected for the GO-Elite output, but the parent term is not.  A visual representation of this pruning strategy is shown for a theoretical set of parent-child relationships with corresponding z-scores (figure 1).
This process allows the user to view the highest scoring term(s) for a particular branch of GO terms and eliminates redundancy of GO terms within the same global category (e.g. biological process, molecular function and cellular component), without needing to consider associated gene content. Since some terms and branches are replicated within the GO hierarchy (redundant), already eliminated or selected GO terms are removed from the results from these other branches.  Finally, GO-Elite reports the list of pruned GO terms along with a summary of gene symbols associated with input regulated genes, user data and other reported GO-Elite terms that have gene content that is redundant with that term (if gene lists supplied by user).  This data is stored to a results file for each input gene list, a combined file with all lists run in the batch analysis and detailed gene association files with gene ID, symbol, description, array IDs and raw data provided in the user input files.
Building New Relationship Files. GO-Elite was built with a modular design, that allows for addition of new gene ID systems (primary and secondary) or support for new species, with minimal work any no expertise.  Gene ID system and species support are controlled by two configuration files in the main database directory (Databases) of GO-Elite (species.txt and source_data.txt). New species support requires addition of species code (two letter designation species genus code) and species name to the species configuration file (via text or spreadsheet editor), creation of new species directories in the input and Databases directories (see existing structure) and creation of relationship text files. The latter can be done in an automated fashion using either Affymetrix microarray annotation files for the species (relationships extracted directly by GO-Elite), the cross-species GO-EntrezGene relationship file provided by NCBI (ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2go.gz) and/or direct downloads for Ensembl relationships from BioMart.  The resulting tables, along with GO hierarchy files supplied at http://www.geneontology.org, are all that is required to analyze data for a new species. To add additional gene ID systems, the user must add a new gene ID system name, system code (optionally included in the gene input file to ensure proper gene ID system selection) and an indicator if the system is a possible primary ID (which links directly to GO). For new primary ID systems, the user must supply a gene annotation table, gene-GO relationship table and any array ID or unique ID to system tables in the corresponding folders. GO-Elite will automatically build intermediate tables from this information (e.g. nested gene-GO relationships). For more information, please see the online or packaged GO-Elite Help file.

Compatibility and Installation. GO-elite is a Python (Python 2.3.4) program which is provided as a stand-alone Windows executable application and as cross-platform source-code, compressed as a ZIP file. Program files and additional documentation can be found at http://www.genmapp.org/go_elite/ and at http://source_forge.com. No additional files are required, beyond those typically packaged with the operating system (Python for Mac OS X and Linux, but not required for PCs when using the stand-alone exe).

Results and Discussion
GO-Elite Increases Gene Ontology ORA Specificity for Example Queries. To estimate the specificity to which GO-Elite is able to summarize biological associations for a given set of genes, without including redundant or non-specific terms we compared this method to different ORA strategies for test data.  As a test case, we presume that given a list of genes matching a single GO category, ORA should ideally report only the most descriptive GO term, the input GO term.  Although we are only adding genes belonging to one GO category, with typical ORA, we would expect an increasing number of additional non-specific and child GO terms to be reported with an increasing number of genes associated with the input parent category.  Therefore, we tested the ability of GO-Elite to report a minimal, non-redundant set of GO terms, for input gene lists containing all genes for a GO-category with a large (apoptosis), mid-range (response to unfolded protein) and small number (stem cell division) of gene associations.  
For each of three input GO category gene sets, GO-Elite was compared to typical ORA, non-nested ORA and GO-Slim association, all derived using the mappfinder method of GO-Elite using different ontology tables or nesting options (table 1a). Analysis by conventional ORA accurately assigned the highest z-score for each query to the same input GO category supplied (e.g. apoptosis was the highest scoring term for the apoptosis input list, with 100% of genes ‘regulated’). Nonetheless, even for the smallest input gene set, stem cell division, 57 additional categories were reported using ORA default filters. For non-nested and GO-Slim ORA only 6 and 5 additional GO categories were associated with the stem cell division test list, respectively, however, with both methods, the category stem cell division was not included (although the child term, somatic stem cell division was selected by non-nested ORA) (table 1b – comparing results for the 4 approaches and redundant sets), reflecting the lack of specificity for reporting nested results for specific terms. GO-Elite analysis for this test case retained stem cell division as the top scoring GO term in addition to 13 additional non-specific terms, eliminating 44 related-redundant terms.  For the two additional test cases, GO-Elite reduced the outputs by a similar magnitude. While the overall number of categories reported by GO-Slim was also relatively small, this method produced associations to less specific or unrelated terms with much lower overall z-scores.  In the case of response to unfolded protein, GO-Elite did not report the test GO category but rather it’s parent term response to protein stimulus. Since both the parent and the child terms contained identical possible gene content, GO-Elite selected the parent term, since additional specificity could not be found in the child.  Furthermore, when gene content redundancy is considered, only a single GO-Elite term remained for each test case (the positive control), since all other categories contained a subset of genes from this ‘regulated’ input GO category. Thus for at least the three test cases examined, GO-Elite was able to select a representative set of high-scoring non-redundant GO-terms that reduced the number of results produced by typical approaches. 
GO-Elite Eliminates up to 90% of Redundant GO terms without Decreasing Associated Gene Content from Experimental Data. To observe how GO-Elite performs with real data, we examined published microarray datasets for two biological processes highlighted by large phenotypic transitions: 1) differentiation of human embryonic stem cells (hESCs) to cardiomyocytes and 2) mouse uterine gestation 8. The first of these datasets, hESC differentiation, possess over 1,500 upregulated genes (fold >2 and ttest p<0.05). For the second dataset, we looked at two criterion: A) up or down-regulation across 7 time-points (fold>2 versus non-pregnant and ftest<0.05) corresponding to ~2,500 genes or B) a smaller subset of these genes only containing ~150 genes that were specifically expressed in the myometrium just prior to the onset of labor (HOPACH clustering) 8. For each of these datasets, the highlighted pathways and GO terms can be correlated to physiologically relevant functions, in this case both being regulation of contraction and structural myocyte changes. 
To determine if sensitivity is affected by GO-Elite pruning, we compared excluded GO terms to the retained GO-Elite terms and corresponding gene content to see what information may be lost upon filtering (tables 2a and b). For this analysis, we compared the three built-in filtering strategies which can be employed by GO-Elite: 1) ordering by z-score, 2) number of genes changed or 3) z-score weighted by genes changed (aka combined). Traditional ORA for each of the datasets and default filtering resulted in 520, 845 and 56 GO terms being reported for hESC regulated, uterine gestation and term regulated criterion, respectively. GO-Elite pruning, ranking by z-score alone, resulted in a 61- 88% reduction in the number of GO categories reported, where the percentage of GO terms excluded is inversely proportional to the number originally filtered ORA categories. This is effect is due to the improved ability of GO-Elite to filter terms if there are more related sets of terms. 
When comparing total gene content, before and after pruning, we observe a loss of 11-14% of genes associated with input GO-categories as compared to GO-Elite terms. Examining these lost genes revealed that they typically align to more general GO-categories with relatively high gene content. This data therefore suggests that GO-Elite pruning dramatically reduces the amount of redundant content without compromising biological sensitivity.
While a 14% loss in associated gene content is largely acceptable, given the large decrease in redundant GO-terms, we wished to see what the cost versus benefit of alternate GO ranking strategies were. For both gene number and combined methods we were able to observe an increase in the associated gene content, relative to using the z-score method alone. In fact, when ranking based on gene number, no loss in gene content was observed, while further decreasing the number of reported GO categories to as much as 13% the number of original ORA terms (hESC differentiation)(table 2a). While this observation suggests that gene number is the preferred method for GO-Elite analysis, this method can result in agglomeration of highly descriptive child terms, into large, less descriptive parents, which may prevent highlighting important biology.
Combination filtering, on the other hand while also resulting in a large preservation of gene content and decrease in the number of reported terms, did not largely comprise the biological description of these datasets, compared to z-score alone or the published reports. This is likely because, gene number ranking will always favor the GO-term that is highest up on the tree and thus contains more genes, while the z-score ranking can commonly favor more specific child terms with sibling terms, that will be retained by GO-Elite, since they are on distinct branches.  Combination however, will tend to favor parents that can occur before a bifurcation as opposed to child terms, since the scores are close but lower in the parent since they have a large denominator.  In the case of uterine term pattern genes, 15 GO-terms were shared between the combined and z-score methods, whereas five GO terms were unique to combination and eight for z-score.  Comparison of these distinct results reveals that the GO-terms B cell differentiation, epidermis development, cellular structure morphogenesis, and morphogenesis of an epithelium, found using the z-score method are represented by their common upstream parent term, anatomical structure development which is a general development category. Therefore, we conclude that the combination z-score approach is suitable for retaining associated gene content from ORA analyses, but that produces a loss of some biologically descriptive content.
Gene Content Redundancy Additionally Increases Specificity for Both GO and Pathways. We have shown that pruning of the hierarchy is a useful means of selecting a set of fairly non-redundant GO terms.  However, this method does not take into account terms which can not be directly related to each other within the GO-hierarchy but contain overlapping gene content. In addition to the primary GO-Elite analysis strategy, considering gene-redundancy, or which GO-Elite categories have genes entirely described in another category, can be useful in further pruning down GO terms and even non-GO pathways with redundant gene content. 
	In GO-Elite, two additional columns of information are included with the summary results, terms redundant with a given category and vise versa.  As an example, the terms gas transport (biological process) and oxygen transport activity (molecular function) are two terms reported from the uterine gestation GO-Elite analysis, which have overlapping gene content, where oxygen transport activity transport genes (n=4) are a subset of gas transport (n=5). As a result, in the first redundancy column, gas transport will be reported as having a super-set of genes for oxygen transport activity.  In the second column, GO-Elite reports the reciprocal relationship, being that oxygen transport activity is redundant with gas transport.  These annotations allow the user to filter for relationships which are maximally descriptive for their dataset, eliminating either categories containing redundant gene content with another or the inverse, which can highlight or more specialized, less generic terms. Considering the three experimental datasets analyzed here, after removing terms annotated as being redundant with other GO terms, we observe as much as an additional 38% decrease in redundant terms (table 2a). Since these terms have entirely redundant gene content, there is no decrease in the total associated gene content.  As a result however, more descriptive GO-categories with fewer genes tend to be ‘absorbed’ into larger, less descriptive categories (e.g. creatine kinase activity is redundant with the term cytoplasm).  Therefore, users can use this annotation present in the result summary file as a guide to decide which general categories (encompassing the redundant terms and genes) or more specific categories do not introduce additional novel biological information, prior to removal.
While the GO-Elite method can only be applied to data from hierarchically organized ontology data, the gene redundancy annotations can be applied non-ontology data, such as GenMAPP pathway associations, also computed by mappfinder ORA. This approach can be useful for when a set of genes are regulate that correspond to core biological processes (e.g. apoptosis, cell cycle, integrin-mediated signaling), that are often described among several cell-type or cell response varying versions of those pathways (see http:/www.wikipathways.org). In addition to providing pre-packaged GenMAPP-gene association data, GO-Elite has a simple format for adding or replacing the existing non-GO pathway content for ORA and gene-redundancy analysis. Thus this method is a general multi-purpose approach for assessing GO-term or pathway gene-content redundancy.
Extended Annotation and Quantization of GO-Elite Level Terms. A common limitation of pathway/ontology analysis tools is efficient gene annotation and interpretation of user data after ORA. For example, MAPPFinder allows users to export lists of associated user input IDs associated with a particular biological term or view color criterion for associated genes in the context of a GO list or GenMAPP pathway.  However, in order to access the set of associated genes (as opposed to an array ID) along with annotations and associated data (e.g. gene expression statistics and annotations), complex queries are required against the user ORA results and the input data. A second challenge is how to summarize these gene annotations and associated quantitative data in an efficient format for direct publication and/or downstream analyses (e.g. clustering of mean pathway expression values at different time-points).
Given that both annotation and quantization present issues for typical ORA, where hundreds to thousands of results are reported, GO-Elite provides a more efficient infrastructure for such analyses. After automatic GO-Elite pruning, three methods of gene summarization are provided with the GO-Elite output: 1) associated gene symbol column in the GO and pathway summary files, 2) specific gene associations, annotations and data for each GO term and pathway (gene-association file) and 3) gene ranking for over-representation among GO terms and pathways (gene-ranking file).  Gene symbols provided in the annotation file provide a simple means to summarize associated data with little real-estate costs. Detailed gene-associations allow the user to not only see which user input IDs link to primary gene IDs for each biological category, but access detailed gene annotations (symbol and description) in addition to any user data provided in the input ID file. Since these are GO-Elite terms reported, there is typically minimal redundant genes that are linked to multiple biological categories. Nonetheless, to better assess if certain genes are over-represented among certain GO-Elite categories, the user can view the gene-ranking files which show which genes tend to be most represented among GO-Elite terms, providing a low-level means of eliminating GO-terms and pathways with redundant content. This method not only highlights promiscuous genes which tend to be associated with several of GO-Elite biological categories, but provides a gene centric view of associated GO-Elite terms.
In addition to summarizing gene annotations, GO-Elite is capable of summarizing numerical data for pruned GO terms and pathways. This method is similar to a previously described approach called GO-Quant9, which links prior MAPPFinder ORA biological terms to user gene IDs in a GenMAPP expression file to calculate mean expression values for each biological term, for any given number of time-points/conditions. The analogous option implemented in GO-Elite, allows any column of numerical data present with the input gene IDs (e.g. array IDs) to be averaged first at the level of primary gene IDs (e.g. Ensembl, EntrezGene) and next at the level of GO-Elite terms for inclusion in the GO-Elite summary results file. An example of this process is illustrated with log2 fold changes for all differentially regulated genes for the different time-points of uterine gestation compared to baseline, non-pregnant mouse uterus (figure 2).  Here, GenMAPP pathway MAPPs generated by GO-Elite’s basic ORA algorithm are clustered for all uterine time-point mean pathway fold changes relative to baseline non-pregnant uterine mRNA profiles. This method of analysis allows for a global view of gene-expression changes directly linked to biological processes.  While we have used the mean of fold changes for summarization, this could just easily be applied to any other values linked to the input gene ID data.

Conclusions:
A critical challenge in the analysis of large scale genomic datasets has been proper description and summarization of gene-associated changes in the content of known biology.  Given the rate at which this data is produced and commonly utilized in publications, researchers require tools capable of highlighting the most relevant biological associations.  In addition to GO categories, this includes over-representation of pathways, efficient summarization of gene content and associated gene data and customizable tools that can be easily updated and optimized for new species/gene analyses. 
Here, we describe a new stand-alone software package called GO-Elite for the analysis of user genomic data to quickly identify a minimal set of non-redundant GO terms, pathways and associated gene data.  While the method itself is not entirely novel10, it is the only available application we are aware of that is currently capable of performing such integrated analyses.  GO-Elite was specifically written with the needs of the genomic community mind, providing flexibility in the type of input gene IDs, species and types applications required. In addition to pruning of existing Ontology and pathway data, GO-Elite is able to perform batch ORA analyses of user gene data, with superior performance compared our previous application, MAPPFinder.
Application of this method to both mouse and human expression data was successful in dramatically reducing the scale of data produced by conventional ORA, with a reduction of up to 90% of the original GO categories (when including gene-content redundancy pruning), while preserving both descriptive categories and associated gene content.  While this method of ORA is relatively basic as compared to more recent methods, such as GSEA 7 or pathway topology analysis (Pathway-Express6, it can be made to be compatible with nearly any ORA data, given the proper input (see associated documentation).  It should be noted however that this method does attempt to select the most statistically significant term relative to other related terms, but rather provides different options for users to rank terms using different ORA statistics.
In addition to these methods, GO-Elite includes multiple levels of gene annotation GO-Elite terms and support for pathway level gene data summarization. We anticipate this approach will be an important addition to the tool-kit used by biologists for large-scale genome-level over-representation analyses in the years to come. 

Acknowledgments:

We would like to the thank Gary Howard and Benjamin Phillips or their editorial help with manuscript along with the GO-Elite user community for the helpful feedback and suggestions. This work is supported by the National Heart, Lung, and Blood Institute grant HL66621. 

Figure and Table Legends:

Figure 1: Gene GO-Elite Node Selection Strategy. A theoretical GO tree with GO paths (black text) is displayed along with associated z-scores (blue text). Here, a single GO term can be represented by multiple path IDs, since the GO is represented as a directed acyclic graph. Red boxes indicate the "reported significant term" selected by GO-Elite.  If the GO paths A.B.2 and A.C.1 associate with the same GO ID, only A.B would be reported since A.B.2 is redundant with it’s a parent A.B. As a result 16 distinct GO paths would be reduced to five GO IDs. The order of GO-Elite operations is presented.

Figure 2: Pathway Level Analysis of Numerical Gene Data. The mean pathway fold change (log2) at multiple time-points of mouse uterine gestations are shown for Affymetrix probe-sets linked to GO-Elite selected GenMAPP MAPPs. These fold changes are relative to non-pregnant (NP) mouse uterus, for days post fertilization (14.5, 16.5, 17.5 and 18.5 days) in addition to postpartum (PP) time-points (6 and 24hrs). This data was clustered using the program HOPACH (ref), where red indicates up-regulation and green, down-regulated mean expression changes for all regulated gene linked probe-sets, for a given pathway.

Table 1: Comparison of GO-Elite to Alternative ORA Methods Using Simulated Data. A summary of ORA and GO-Elite analyses using different strategies to determine which methods produce the least and most specific set of GO associations given all genes associated with 3 different GO categories (Response to unfolded protein, stem cell division and apoptosis). ORA methods include, GO-Elite’s mappfinder algorithm selecting non-nested GO terms (genes specifically associated with each term), nested (nesting child associated terms to parents), GO-Slim, GO-Elite filtering and GO-Elite filtering including gene content redundancy filtering (GCR).  (A) The number of GO-terms that were produced (after common default filtering) for each input GO-category list. (B) The top ranked GO-term (based on z-score) for each method along with the GO-term’s z-score value are shown, where “same” indicates that the top ranked GO-term is the same as the GO-category input gene list.

Table 2: GO-Elite Filtering For Muscle Differentiation and Remodeling Paradigms. Different GO-Elite filtering options are compared with typical mappfinder ORA. The three pruning methods shown are z-score, gene-count and combination (gene-count weighted z-scores), obtained from the input ORA files. (A) The number of GO-terms changed using the different stategies are shown, with the number of additional GO-terms that are redundant based on gene content in parentheses (e.g. for hESC cardiac differentiation, 146 highlighted GO-terms with the GO-Elite z-score method, 47 have gene content redundant with other GO-Elite terms). (B) Number of genes associated with each set results produced by the different GO-Elite filtering methods.

Table 3: Comparison of Uterine Term Gestation Elite and Non-Elite GO terms. (A) 22 GO-Elite terms for a cluster of genes, maximally up-regulated just prior to the onset of labor from a mouse uterine microarray analysis. (B) Those 34 GO-terms that were filtered out of the original set of 56 ORA highlighted terms are shown. 

Tables

Table 1:
A.
GO-terms changed	non-nested	GO-Slim	nested	GO-Elite	GCR GO-Elite	
Response to unfolded protein	39	16	160	38	1	
stem cell division	6	5	58	14	1	
apoptosis	571	48	1639	254	1	
GCR: Gene Content Redundancy filtering					

B.
Top scoring association	non-nested	GO-Slim	nested	GO-Elite		
response to unfolded protein	same	protein complex	same	response to protein stimulus	top-term	
	120	15	134	134	z-score	
stem cell division	same	multicellular organismal development	same	same	top-term	
	110	5	134	134	z-score	
apoptosis	same	protein binding	same	same	top-term	
	91	14	134	134	z-score	

Table 2:
A.
		GO-Elite Pruning Options	
GO-terms changed	mappfinder	z-score	gene count	combination	
hESC cardiac differentiation	520	146 (47)	110 (42)	109 (37)	
uterine gestation time-course	845	103 (50)	20 (11)	69 (32)	
Uterine term pregnancy	56	22 (6)	16 (7)	19 (6)	

B.
		GO-Elite Pruning Options	
Associated Genes	mappfinder	z-score	gene count	combination	
hESC cardiac differentiation	872	753	872	847	
uterine gestation time-course	2553	2362	2553	2438	
uterine term pregnancy	111	98	111	106	

Table 3:
A.
GO Name	GO Type	Number Changed	Z Score	PermuteP	AdjustedP	redundant with terms	inverse redundant	
Intercellular junction	C	8	6.35	0	0	 	intercalated disc	
B cell differentiation	P	4	5.38	0.0005	0.352	 	 	
intermediate filament	C	6	5.03	0.0005	0.352	 	 	
Intercalated disc	C	4	4.98	0	0	intercellular junction	 	
Basolateral plasma membrane	C	5	4.54	0.0015	0.5025	 	 	
structural constituent of cytoskeleton	F	6	4.28	0.0055	1	 	 	
cytokine and chemokine mediated signaling pathway	P	4	4.16	0.0015	0.5025	cell development, death	 	
positive regulation of growth	P	3	3.98	0.0125	1	 	 	
extracellular region	C	43	3.89	0	0	 	endopeptidase inhibitor activity, serine-type endopeptidase activity	
Erythrocyte homeostasis	P	3	3.20	0.016	1	cell development	 	
endopeptidase inhibitor activity	F	5	3.17	0.011	1	extracellular region	 	
apical plasma membrane	C	3	3.13	0.016	1	 	 	
cellular structure morphogenesis	P	12	3.00	0.0055	1	 	 	
regulation of cell adhesion	P	3	2.91	0.0175	1	 	 	
amine biosynthetic process	P	3	2.85	0.018	1	 	 	
morphogenesis of an epithelium	P	5	2.71	0.011	1	 	 	
intrinsic to membrane	C	45	2.70	0.016	1	 	 	
carbohydrate biosynthetic process	P	3	2.45	0.041	1	 	 	
cell development	P	21	2.26	0.0185	1	 	cytokine and chemokine mediated signaling pathway, death, erythrocyte homeostasis	
serine-type endopeptidase activity	F	5	2.26	0.029	1	extracellular region	 	
Epidermis development	P	3	2.17	0.0445	1	 	 	
Death	P	13	2.14	0.0265	1	cell development	cytokine and chemokine mediated signaling pathway	

B.
GO Name	GO Type	Z Score		GO Name	GO Type	Z Score	
apical junction complex	C	6.25		biological adhesion	P	2.57	
Apicolateral plasma membrane	C	6.11		cell adhesion	P	2.57	
intermediate filament cytoskeleton	C	4.97		structural molecule activity	F	2.54	
tight junction	C	4.70		integral to membrane	C	2.53	
plasma membrane	C	3.75		lymphocyte differentiation	P	2.27	
cell junction	C	3.60		enzyme regulator activity	F	2.26	
Protease inhibitor activity	F	3.13		hemopoietic or lymphoid organ development	P	2.24	
apical part of cell	C	3.11		developmental process	P	2.23	
serine-type endopeptidase inhibitor activity	F	3.11		growth	P	2.22	
Membrane	C	3.07		regulation of growth	P	2.18	
cell morphogenesis	P	3.00		nitrogen compound biosynthetic process	P	2.17	
B cell activation	P	2.92		cell death	P	2.14	
enzyme inhibitor activity	F	2.88		serine hydrolase activity	F	2.11	
anatomical structure morphogenesis	P	2.83		serine-type peptidase activity	F	2.11	
plasma membrane part	C	2.80		endopeptidase activity	F	2.08	
anatomical structure development	P	2.71		immune system development	P	2.05	
membrane part	C	2.60		apoptosis	P	1.97	

References:

1.	Wang, K., Li, M., and Bucan, M.,Pathway-Based Approaches for Analysis of Genomewide Association Studies. American journal of human genetics 81, (2007).
2.	Doniger, S.W., Salomonis, N., Dahlquist, K.D., Vranizan, K., Lawlor, S.C., and Conklin, B.R.,MAPPFinder: using Gene Ontology and GenMAPP to create a global gene-expression profile from microarray data. Genome biology 4, R7 (2003).
3.	Ashburner, M., Ball, C.A., Blake, J.A., Botstein, D., Butler, H., Cherry, J.M., Davis, A.P., Dolinski, K., Dwight, S.S., Eppig, J.T., Harris, M.A., Hill, D.P., Issel-Tarver, L., Kasarskis, A., Lewis, S., Matese, J.C., Richardson, J.E., Ringwald, M., Rubin, G.M., and Sherlock, G.,Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nature genetics 25, 25 (2000).
4.	Salomonis, N., Hanspers, K., Zambon, A.C., Vranizan, K., Lawlor, S.C., Dahlquist, K.D., Doniger, S.W., Stuart, J., Conklin, B.R., and Pico, A.R.,GenMAPP 2: new features and resources for pathway analysis. BMC bioinformatics 8, 217 (2007).
5.	Subramanian, A., Tamayo, P., Mootha, V.K., Mukherjee, S., Ebert, B.L., Gillette, M.A., Paulovich, A., Pomeroy, S.L., Golub, T.R., Lander, E.S., and Mesirov, J.P.,Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proceedings of the National Academy of Sciences of the United States of America 102, 15545 (2005).
6.	Draghici, S., Khatri, P., Tarca, A.L., Amin, K., Done, A., Voichita, C., Georgescu, C., and Romero, R.,A systems biology approach for pathway level analysis. Genome research 17, 1537 (2007).
7.	Khatri, P., Voichita, C., Kattan, K., Ansari, N., Khatri, A., Georgescu, C., Tarca, A.L., and Draghici, S.,Onto-Tools: new additions and improvements in 2006. Nucleic acids research 35, W206 (2007).
8.	Salomonis, N., Cotte, N., Zambon, A.C., Pollard, K.S., Vranizan, K., Doniger, S.W., Dolganov, G., and Conklin, B.R.,Identifying genetic networks underlying myometrial transition to labor. Genome biology 6, R12 (2005).
9.	Yu, X., Griffith, W.C., Hanspers, K., Dillman, J.F., 3rd, Ong, H., Vredevoogd, M.A., and Faustman, E.M.,A system-based approach to interpret dose- and time-dependent microarray data: quantitative integration of gene ontology analysis for risk assessment. Toxicol Sci 92, 560 (2006).
10.	Barriot, R., Sherman, D.J., and Dutour, I.,How to decide which are the most pertinent overly-represented features during gene set enrichment analysis. BMC bioinformatics 8, 332 (2007).