The effect of prior assumptions over the weights in BayesPI with application to study proteinDNA interactions from ChIPbased highthroughput data.  
Jump to Full Text  
MedLine Citation:

PMID: 20684785 Owner: NLM Status: MEDLINE 
Abstract/OtherAbstract:

BACKGROUND: To further understand the implementation of hyperparameters reestimation technique in Bayesian hierarchical model, we added two more prior assumptions over the weight in BayesPI, namely Laplace prior and Cauchy prior, by using the evidence approximation method. In addition, we divided hyperparameter (regularization constants alpha of the model) into multiple distinct classes based on either the structure of the neural networks or the property of the weights. RESULTS: The newly implemented BayesPI was tested on both synthetic and real ChIPbased highthroughput datasets to identify the corresponding protein binding energy matrices. The results obtained were encouraging: 1) there was a minor effect on the quality of predictions when prior assumptions over the weights were altered (e.g. the prior probability distributions to the weights and the number of classes to the hyperparameters) in BayesPI; 2) however, there was a significant impact on the computational speed when tuning the weight prior in the model: for example, BayesPI with a Laplace weight prior achieved the best performance with regard to both the computational speed and the prediction accuracy. CONCLUSIONS: From this study, we learned that it is absolutely necessary to try different prior assumptions over the weights in Bayesian hierarchical model to design an efficient learning algorithm, though the quality of the final results may not be associated with such changes. In future, the evidence approximation method can be an alternative to Monte Carlo methods for computational implementation of Bayesian hierarchical model. 
Authors:

Junbai Wang 
Related Documents
:

17298235  Neighborhood propertybased pattern selection for support vector machines. 23996115  Secondgeneration minimal physiologicallybased pharmacokinetic model for monoclonal an... 19923045  Regularized negative correlation learning for neural network ensembles. 19029555  Efficient recognition of highly similar 3d objects in range images. 25250445  Two genera foersteria szépligeti, 1896 and polydegmon foerster, 1862 (hymenoptera, bra... 20551505  Deformable adult human phantoms for radiation protection dosimetry: anthropometric data... 
Publication Detail:

Type: Journal Article; Research Support, NonU.S. Gov't Date: 20100804 
Journal Detail:

Title: BMC bioinformatics Volume: 11 ISSN: 14712105 ISO Abbreviation: BMC Bioinformatics Publication Date: 2010 
Date Detail:

Created Date: 20100816 Completed Date: 20101013 Revised Date:  
Medline Journal Info:

Nlm Unique ID: 100965194 Medline TA: BMC Bioinformatics Country: England 
Other Details:

Languages: eng Pagination: 412 Citation Subset: IM 
Affiliation:

Department of Pathology, The Norwegian Radium Hospital, Oslo University Hospital, Montebello 0310 Oslo, Norway. junbai.wang@rrresearch.no 
Export Citation:

APA/MLA Format Download EndNote Download BibTex 
MeSH Terms  
Descriptor/Qualifier:

Algorithms Bayes Theorem* Chromatin Immunoprecipitation* DNA / metabolism* DNABinding Proteins / metabolism* Monte Carlo Method Oligonucleotide Array Sequence Analysis* Sequence Analysis, DNA 
Chemical  
Reg. No./Substance:

0/DNABinding Proteins; 9007492/DNA 
Comments/Corrections 
Full Text  
Journal Information Journal ID (nlmta): BMC Bioinformatics ISSN: 14712105 Publisher: BioMed Central 
Article Information Copyright ©2010 Wang; licensee BioMed Central Ltd. openaccess: Received Day: 5 Month: 12 Year: 2009 Accepted Day: 4 Month: 8 Year: 2010 collection publication date: Year: 2010 Electronic publication date: Day: 4 Month: 8 Year: 2010 Volume: 11First Page: 412 Last Page: 412 Publisher Id: 1471210511412 PubMed Id: 20684785 DOI: 10.1186/1471210511412 
The effect of prior assumptions over the weights in BayesPI with application to study proteinDNA interactions from ChIPbased highthroughput data  
Junbai Wang1  Email: junbai.wang@rrresearch.no 
1Department of Pathology, The Norwegian Radium Hospital, Oslo University Hospital, Montebello 0310 Oslo, Norway 
In our previous study, we developed a Bayesian neural network type of model  BayesPI  to study proteinDNA interactions, using ChIPbased highthroughput data [^{1}]. In BayesPI, the model error function (data error) is interpreted as defining a likelihood function, and the model regularizer (a penalty term to the error function) corresponds to a prior probability distribution over the weights, and such a framework is considered as a Bayesian hierarchical model. In addition to the common model parameters, BayesPI includes unknown hyperparameters (e.g. weight decay rate α and model noise level β) that need to be learned from the data. There are three possible implementations to control the model hyperparameters when using Bayesian neural networks to infer the model parameters: 1) using Markov chain Monte Carlo methods to simulate the probability distribution  MCMC [^{2}]; 2) integrating out the model hyperparameters analytically before the application of Gaussian approximation of posterior distribution, and subsequently maximizing the true posterior over the model parameters  Maximum A Posterior Probability (MAP) [^{3}]; and 3) integrating out the model parameters first, and then maximizing the resulting evidence over the hyperparameters  the Evidence Approximation [^{4}]. Descriptions of the first two implementations can be found in the earlier papers [^{2},^{3}], and in this study, we will focus only on the last approach (the evidence approximation) implemented in BayesPI.
Three motivations inspired us to pursue an investigation on the effect of prior assumptions over the weights (the evidence approximation) in Bayesian neural networks to study proteinDNA interactions from ChIPbased highthroughput data: 1) With regard to others' concern, before BayesPI paper was published, we received some criticisms about the treatment of hyperparameters in Bayesian neural networks. For example, do alternative definitions of hyperparameters according to the model parameters (e.g. divide the hyperparameters α into several classes based on either the structure of neural networks or the property of the model parameters) strongly influence the model inference? 2) With regard to our own interest, how significant will a different assignment of prior distribution (e.g. Gaussian prior, Laplace prior or Cauchy prior) to weights affect the outcome of Bayesian neural networks (e.g. prediction accuracy and computational time cost)? 3) With respect to a general survey of the application of Bayesian inferences in ChIPbased experiments, we searched PubMed using the keywords "Bayesian, chip" or "Bayesian, ChIPchip," and then downloaded the search results that had been recorded before May 28, 2010. From this search, we obtained 33 papers that contained the abovementioned keywords. Subsequently, we carried out a literature study of these 33 papers. To our surprise, only 14 of the 33 papers had applied Bayesian methods on issues related to motif discovery (e.g. DNA binding site identification) by using ChIPbased highthroughput data, and the remaining 19 papers had applied Bayesian methods in data integration, clustering and network reconstructions, etc. A detailed examination of the 14 papers relevant to proteinDNA interaction study reveal that BayesPI applied used evidence approximation to solve the posterior distribution in Bayesian inference, while the remaining 12 papers utilized the sampling methods (e.g. MCMC and Gibbs sampling) to simulate the posterior distribution of the Bayesian models (one paper cannot be determined because of lack of method description; detailed information of the 33 papers is available in [Additional file 1: Supplemental Data]). Though the present implementation (the evidence approximation) in BayesPI for handling hyperparameters has been rarely applied earlier, there are clear advantages of using it to solve the data mining problems [^{5}]. Thus, by being motivated by the last finding along with the earlier two inspirations, we decided to carry out a followup study on the effect of prior assumption over the weights in BayesPI. Our study may pave the way for the future development of evidence approximation in Bayesian inferences as well as for the further application of the Bayesian methods in bioinformatics research.
To evaluate the performance of BayesPI under (15) different prior assumptions over the weights, we first tried each of them on the same set of simulated ChIPchip experiments (16 synthetic ChIPchip datasets), where the synthetic DNA sequences and ChIPchip log ratios were generated using MATLAB Bay Net toolbox and MATLAB buildin random number generator, respectively [^{1}]. The accuracy of the predictions was accessed from motif similarity scores by comparing the predicted motif energy matrix with the corresponding SGD consensus sequences [^{6}]. In Figure 1, we have illustrated the outcomes of the abovementioned simulations in 15 different prior assumptions, where both the CPU hours required for the calculation and distribution of the motif similarity scores among all the tests are shown. The results are very interesting because no significant changes of the prediction quality could be observed across the tests after changing either the prior probability assumption or the number of subclasses for α hyperparameters, except for the tests with Gaussian approximation (e.g. comparing the distribution of motif similarity scores using Wilcoxon ranksum test: Gaussian vs. Cauchy, p < 0.03; Laplace vs. Cauchy, p < 0.04). However, the CPU hours used for various tests differed significantly. Particularly, the selection of prior probability assumption over the weights in Bayesian neural networks had a much stronger impact on the cost of CPU hours than that by tuning the number of subclasses of hyperparameters. For examples, by using a Laplace assumption over the weights in BayesPI, the CPU hours used for the calculations were shortened by almost two to five times when compared with the assumptions of the weights by the other two probability distributions (e.g. comparing the distribution of used CPU hours by Wilcoxon ranksum test: Gaussian vs. Laplace, p < 1.4e9; Cauchy vs. Laplace, p < 5.8e8). It is worth noting that the assignment of Laplace prior probability to weights utilizes the least CPU hours for the calculation, but provides the best prediction accuracy. Thus, we can expect Laplace approximation over the weights to provide the most efficient computation for BayesPI if real ChIPchip datasets are used.
After testing the effect of prior assumptions over the weights in BayesPI using the synthetic ChIPchip datasets, we tried it on the real proteinDNA interaction datasets from ChIPchip experiments. We collected ChIPchip datasets for nine yeast TFs in rich medium condition [^{7}], among which four (SWI4, INO4, ACE2, and XBP1) had the same consensus sequences as the TFs in the synthetic datasets. In the earlier tests, we neither found a significant variation in the prediction accuracy nor observed a strong perturbation of the computational time cost (Figure 1) through tuning the number of subclasses of α hyperparameters: hence, we decided to select only four subclasses for the hyperparameters in the rest of the studies. The results of these tests both with and without the inclusion of nucleosome information are presented in Figure 2 that shows that there is little difference in the prediction accuracies among the tests regarding the selection of prior probability assumptions and the inclusion of the nucleosome information. A comparison between the motif similarity scores provided by the three prior weight assumptions in BayesPI and those obtained by MatrixREDUCE is presented in Table 1. The results indicate that all poor predictions are caused by stressinduced transcription factors (e.g. ROX1, MSN2, and XBP1). Though BayesPI may provide some reasonable answers to TFs that are nonfunctional under certain growth conditions (e.g. Table 1, SKN7 and MSN2 in the YPD condition), its computational speed is much slower than that by the popular program MatrixREDUCE [^{8}]. Nevertheless, the CPU hours used by BayesPI among the three prior weight assumptions differ significantly (e.g. comparing the distribution of used CPU hours without considering the nucleosome information by Wilcoxon ranksum test: Gaussian vs. Laplace, p < 4.2e5; Gaussian vs. Cauchy, p < 4.2e5; Cauchy vs. Laplace, p < 2.9e4). Furthermore, we found that the cost of the CPU hours for the computation was slightly reduced with regard to the nucleosome information. Taken together, it can be concluded that the computational efficiency of the Laplace prior assumption over the weights in the Bayesian neural networks clearly surpasses that of the other two weight priors, and that the Laplace prior may be suitable for the further improvement of BayesPI algorithm.
After the successful application of the earlier tests on ChIPchip datasets, we tried the new BayesPI program on three human ChIPSeq datasets [^{9}] by applying three different prior assumptions (e.g. Gaussian, Lasso, and Cauchy) over the weights with predefined four groups of regularization constants α. Here, the inputs to BayesPI were preprocessed raw ChIPSeq measurements, which are a set of putative protein binding sites (e.g. there are 5814, 26815, and 73957 putative TF binding sites for NRFS, CTCF, and STAT1, respectively.), and the corresponding tag densities obtained from SISSRs method [^{9}]. The results of these tests are shown in Figure 3, which demonstrates that the Laplace prior requires much less CPU hours when compared with that required by the other two assumptions. For instance, to complete the same calculation, Laplace approximation needs only 50 percent to 25 percent of the CPU hours that is used by either Cauchy or Gaussian approximation. However, interestingly, the accuracy of the predictions does not differ significantly among various prior assumptions similar to the previous tests that employed the ChIPchip datasets: tuning of the prior assumption over the weights does not seem to affect the quality of the predictions, but is rather observed to bring strong impact on the CPU requirement. Thus, a careful design of the weight priors in a Bayesian model may significantly reduce the computational cost for the calculation.
Nowadays, chromatin immunoprecipitation followed by massively paralleled sequencing (ChIPSeq) is being used widely in various molecular biological researches such as investigating genomewide proteinDNA interactions [^{7}] and histone modification studies [^{10}]. It is possible that the ChIPSeq experiment may replace ChIPchip technology completely [^{11}] in future. That is because the ChIPSeq experiment produces higher quality and higher resolution data than the ChIPchip, which also avoids several pitfalls that accompany with the ChIPchip technology: for example, array probespecific behavior and dye bias [^{12}]. In this work, we studied the effect of prior assumptions over the weight in BayesPI to predict the protein binding energy matrices from ChIPbased highthroughput datasets. The results on both synthetic and real experimental datasets were consistent: in general, the prior assumptions over the weights and the classification of regularization constants (e.g. hyperparameters α) into several classes did not strongly affect the final outcome of BayesPI (e.g. Figures 1, 2, and 3) if sufficient training datasets were provided; particularly, a change in the number of classes over the regularization constants had a much weaker impact on the requirement of computational resource than a change in the weight prior in BayesPI; nevertheless, the selection of prior approximation over the weights had the most significant influence on the CPU hours that were used for calculation (e.g. by using a Laplace prior, the computational time was reduced by more than 50 percent when compared with that utilized by the old BayesPI [^{1}], a Gaussian prior.) Thus, the present study reveals the importance of defining a right weight prior to a Bayesian hierarchical model, which may dramatically speed up the calculation when the program is applied to a large dataset.
In addition to the abovementioned findings that the computation efficiency of BayesPI is highly associated with prior assumptions over the weights, we also provided a detailed illustration of the hyperparameter reestimation technique by using the evidence approximation method. We presume that the evidence method may become a popular approximate method for computational implementation of Bayesian hierarchical model (a deterministic algorithm), as well as become an alternative to Monte Carlo methods that are currently being widely used in bioinformatics research fields [^{13}]. Particularly, the evidence method can overcome some of the inherent limitations of the sampling approaches, such as nonreproducible results, long burning period, and unknown stopping time.
The present study has clarified several doubts in the early implementation of BayesPI: 1) prediction accuracy of BayesPI is robust against dividing the hyperparameters (e.g. regularization constants α) into multiple distinct groups; 2) there is a minor effect on the quality of predictions by selecting alternative prior assumptions over the weights in BayesPI; 3) however, there is a strong impact on the computational requirement for calculation when a proper weight prior is chosen. Overall, we have derived the new reestimation formulas for both Laplace prior and Cauchy prior over the weights in the Bayesian neural networks, and the new implements have been tested successfully in both synthetic and real ChIPbased highthroughput datasets.
In this study, we have only focused on the effect of prior assumption over the parameters in Bayesian neural networks. The descriptions of the biophysical background behind BayesPI and the implementation of the Bayesian predicative model to estimate the protein binding parameters by combining ChIPbased datasets with DNA sequence information will not be repeated, because they are available in the previous paper [^{1}]. First, we regularized the objective function
(1)
M(w)=β•ED(Dw,Λ,η)+α•Ew(wΛ,Γ) 
which can be used by the Bayesian neural networks [^{4}] to determine the parameters (e.g. w, α, β). In the abovementioned equation, E_{D}, E_{w}, D, and ⟨Λ, η, Γ⟩ are the model error function (data error), the model regularizer (a penalty term to the error function), the input data, and the hypothesis model space (e.g. Λ is the protein binding probability and Γ is the regularization function), respectively; α and β are the two unknown hyperparameters (e.g. weight decay rate and model noise level) that must be determined from the input data; and w indicates the model parameters (e.g. weights in the Bayesian neural networks), which represents the inferred the protein binding energy matrix and the chemical potentials from ChIPbased highthroughput data [^{1}].
Here, a Gaussian model error function E_{D }is assumed throughout as
(2)
ED=12∑i=1g(ti−Yi)2 
where t_{i }is the measured ChIPbased data to gene i, and Y_{i }is the predicted TF occupancy data for that gene, according to a predefined TF binding probability (e.g. either inclusion or exclusion of nucleosome binding information in the proteinbinding probability [^{1}]). For the model regularizer E_{w}, three types of weight prior assumptions were selected [^{3}]: 1) Gaussian prior assumption over the weights,
(3)
Ew=12∑q=1Qwq2 
in which Q is the number of parameters in the model, such as w; 2) Laplace prior distribution as the model regularization function,
(4)
Ew=∑q=1Qwq 
3) Cauchy prior assumption over the model parameters w in BayesPI
(5)
Ew=1α∑q=1Qlog(1+α2wq2) 
Based on the abovementioned three weight priors, we applied the evidence approximation method [^{4}] to determine the corresponding reestimation formulas for both α and β, which can be used by Bayesian neural networks to fit the model (e.g. to learn the model parameters w from the data).
Based on Bayes' theorem, a posterior distribution of the model parameters, can be given as
(6)
P(wD,α,β,Λ,η,Γ)=exp(−M(w))ZM(α,β) 
where M is a probability framework of the objective function described in equation (1) and Z_{M }is a normalization factor [^{4}]. By employing a Gaussian approximation of the posterior probability, we have
(7)
P(wD,α,β,Λ,η,Γ)≈exp(−MMP)exp(−12(w−wMP)TA(w−wMP))ZM' 
in which ZM'=∫dwexp(−MMP(w)) and A is the Hessian of M (e.g. A = α∇∇E_{w }+ β∇∇E_{D}) evaluated at w_{MP}. Here, we first assume that the most probable model parameters w_{MP }are known (integrating out the model parameters), and then infer the hyperparameters through Bayes' rule
(8)
P(α,βD,Λ,η,Γ)=P(Dα,β,Λ,η,Γ)P(α,βΛ,η,Γ)P(DΛ,Γ,η) 
where we also assume equal priors P(α, βΛ, η, Γ) to the alternative models and a constant term to the P(DΛ, Γ, η). Thus
(9)
P(α,βD,Λ,η,Γ)≈P(Dα,β,Λ,η,Γ) 
where P(Dα, β, Λ, η, Γ) is the evidence for the overall model,
(10)
P(Dα,β,Λ,η,Γ)=ZM(α,β)Zw(α)ZD(β) 
including both the model architecture and the regularizing parameters [^{4}], where Z_{w}(α) and Z_{D}(β) are the normalization factors given by Z_{w}(α) = ∫ dwexp(αE_{w}) and Z_{D}(β) = ∫ dDexp(βE_{D}), respectively. By maximizing the log evidence of equation (10), we can determine the reestimation formulas for hyperparameters α and β according to the weight assumptions E_{w }in BayesPI.
The log evidence for hyperparameters is
(11)
log(P(Dα,β,Λ,η,Γ))=logZM−logZw−logZD 
where a Gaussian prior, equation (3), is used for E_{w }and
(12)
ZM≈exp(−MMP)(2π)k/2•detA−1/2 
(13)
Zw≈(2πα)k/2 
(14)
ZD≈(2πβ)N/2 
After replacing Z_{M}, Z_{w}, Z_{D }by equations (12), (13), and (14), respectively, equation (11) becomes
(15)
log(P(Dα,β,Λ,η,Γ))≈−MMP−12logdetA+k2logα−N2log2π+N2logβ 
To determine the conditions that are satisfied at the maximum log evidence, we differentiated equation (15) with respect to α and β, and then set the derivative to zero from which we can obtain the reestimation formulas for both α and β as follows
(16)
α=k−αTrace(A−1I)2Ew 
(17)
β=N−k+αTrace(A−1)2ED 
Let
(18)
γ=k−αTrace(A−1) 
The equations (16) and (17) can be rewritten as
(19)
α=γ2Ew 
(20)
β=N−γ2ED 
where γ are eigenvectors of A. For example, equation (18) can be transformed to
(21)
γ=∑qλqλq+α 
Where λ_{q }are the eigenvalues of the β∇∇E_{D }and the negative λ_{q }are omitted from the sum. Thus, for a Gaussian weight prior, we used equation (21) to update the hyperparameters α and β through equations (19) and (20).
By using equation (4) as a prior assumption over the weights, the Hessian of M becomes
(22)
A=β∇∇ED 
The log evidence for the hyperparameters is
(23)
log(P(Dα,β,A,η,R))≈−MMP+k2log2π−12logdetA−logZw−logZD 
where
(24)
Zw≈(2α)k 
(25)
ZD≈(2πβ)N/2 
After inserting equations (24) and (25) into the log evidence, we get
(26)
log(P(Dα,β,A,η,R))≈−MMP−12logdetA+k2log2π−klog2+klogα−N2log2π+N2logβ 
To maximize the log evidence over the hyperparameters, we differentiated equation (26) with respect to α and β, and the derivative was set to zero, then we obtained the following reestimation formulas
(27)
α=kEw 
(28)
β=N−k2ED 
for the hyperparameters, when assuming a Laplace prior over the weights.
Here, we used equation (5) as the prior assumption over the weights, and the log evidence for the model can be given as
(29)
log(P(Dα,β,A,η,R))≈−MMP+k2log2π−12logdetA−logZw−logZD 
where
(30)
Zw≈(πα)k 
(31)
ZD≈(2πβ)N/2 
After inserting equations (30) and (31) into equation (29), the log evidence becomes
(32)
L(α,β)=−MMP−12logdetA+k2log2π−klogπ+klogα−N2log2π+N2logβ 
where we assume that α is known for E_{w}, ∇E_{w}, and ∇∇E_{w}. To determine the conditions suitable for the maximum log evidence, equation (32) was differentiated with respect to hyperparameters, and the derivative was set to zero. Then the reestimation formulas for α and β are
(33)
α=k+γ2Ew 
(34)
β=N−γ2ED 
where
(35)
γ=∑qλqλq+α∇∇Ew 
where λ_{q }are the eigenvalues of data error β∇∇E_{D}. Thus, for a Cauchy prior, equation (35) can be used to compute hyperparameters α and β through equations (33) and (34). Detailed derivations of hyperparameters update functions for above three priors are available in [Additional file 1: Supplemental Methods].
In equation (35), there is a second derivative ∇∇E_{w}, which can be estimated from an efficient Rpropagation algorithm of Pearlmutter [^{14}]. The algorithm applies a differential operator R() on the Backpropagation neural networks. For example, let us assume that equation (5) is used by the model regularizer E_{w}. Then
(36)
Ew=1αa2 
where α_{2 }is the node of the output layer
(37)
a2=∑qlog(1+Hq2) 
in which H_{q }is the node of the hidden layer
(38)
Hq=αa1q 
and α_{1}^{q }is the node of the input layer
(39)
a1q=wq 
After completing the abovementioned forward computation of the neural networks, a backward pass can be subsequently obtained as
(40)
∂E∂a2=1α∂E∂Hq=1α2Hq1+Hq2∂E∂a1q=2Hq1+Hq2∂E∂wq=2Hq1+Hq2 
and Rforward computation can be carried out as follows
(41)
R(a1q)=VwqR(Hq)=αR(a1q)R(a2)=∑q2Hq1+Hq2R(Hq) 
Furthermore, the Rbackward computation can be carried out as follows
(42)
R(∂E∂wq)=−2(Hq2−1)(Hq2+1)2R(Hq) 
By following the abovementioned Rbackpropagation procedures, R(∂E∂wq) can be estimated, which is equivalent to computing the second derivative ∇∇E_{w }[^{14}]. Detailed description of application of Rpropagation algorithm is available in [Additional file 1: Supplemental Methods]. The source code of BayesPI2 is public available http://folk.uio.no/junbaiw/bayesPI2.
For simplicity, we assumed that there is only one class of weights in BayesPI [^{1}]. For example, the weights are modeled as coming from a single Gaussian prior (e.g. equation (3)). However, in a real study, weights may fall into multiple distinct groups [^{4}]. Therefore, it is desirable to divide the weights into several classes c, with independent regularization constants α_{c}. In the new version of BayesPI, there are five types of assignment of weight decay rate α to each of the three weight priors (e.g. Gaussian, Laplace, and Cauchy). The term αE_{w }in equation (1) is replaced by ∑cαcEwc, in which c is the number of classes to the regularization constants α: 1) if c equals 1, then all the weights have the same regularization constant α; 2) if c equals 2, then we can divide the weights into two groups, namely the weights in the hidden layer and the weights in the output layer; 3) if c equals 3, then it suggests that there are two distinct weight classes in the hidden layer (e.g. weights from the motif energy matrix and weight from the chemical potential), but only a single weight class in the output layer [^{1}]); 4) if c equals 4, then it suggests that there are two independent weight classes in both the hidden layer and output layer; 5) if c is greater than 5, then it suggests that each binding position of the motif energy matrix has its own regularization constant α_{c }as well as the chemical potential, and that the two weights in the output layer have their own regularization constants, respectively (e.g. if TF motif length equals 8, then the regularization constant α has 11 classes).
To access the quality of the predicted motif binding sites, we used a published method (motif similarity score [^{15}]) to estimate the similarity between the predicted motif energy matrices and the corresponding consensus sequences from the SGD database [^{16}]. Detailed description of these calculations can be found in the previous publication [^{1}]. Synthetic ChIPchip datasets and real ChIPchip experiments for nine yeast transcription factors were adopted from the earlier works [^{1},^{7}]. ChIPSeq datasets for three human TFs (STAT1, NRSF, and CTCF) were obtained from Jothi et al. [^{9}]. More information about the preprocessing of both ChIPchip and ChIPSeq datasets are available in [^{1}].
The author declares that they have no competing interests.
JW conceived and designed the study, implemented program, performed data analysis and drafted manuscript.
Supplementary information to the paper. Here we provide detailed description of derivation of hyperparameters update functions for three different priors (e.g. Gaussian, Cauchy, and Laplace), the implementation of Rpropagation algorithm, and the full information of 33 papers that were obtained from PubMed on May 28^{th}, 2010 by searching the keywords (e.g. Chip, Bayesian).
Junbai Wang is supported by the Norwegian Cancer Society, the cluster facilities of the University of Oslo and the NOTUR project.
References
Wang J,BayesPI  a new model to study proteinDNA interactions: a case study of conditionspecific protein binding parameters for Yeast transcription factorsBMC bioinformaticsYear: 20091034510.1186/147121051034519857274  
Neal RM,Bayesian Learning for Neural NetworksPhD thesisYear: 1994University of Toronto  
Williams PM,Bayesian Regularization and Pruning Using a Laplace PriorNeural ComputationYear: 19957111714310.1162/neco.1995.7.1.117  
Mackay D,Bayesian Methods for Adaptive ModelsPhD thesisYear: 1991California Institute of Technology  
Mackay DJC,Comparison of Approximate Methods for Handling HyperparametersNeural ComputationYear: 19991151035106810.1162/089976699300016331  
Chen CY,Tsai HK,Hsu CM,May Chen MJ,Hung HG,Huang GT,Li WH,Discovering gapped binding sites of yeast transcription factorsProc Natl Acad Sci USAYear: 200810572527253210.1073/pnas.071218810518272477  
Harbison CT,Gordon DB,Lee TI,Rinaldi NJ,Macisaac KD,Danford TW,Hannett NM,Tagne JB,Reynolds DB,Yoo J,Transcriptional regulatory code of a eukaryotic genomeNatureYear: 200443170049910410.1038/nature0280015343339  
Foat BC,Morozov AV,Bussemaker HJ,Statistical mechanical modeling of genomewide transcription factor occupancy data by MatrixREDUCEBioinformaticsYear: 20062214e14114910.1093/bioinformatics/btl22316873464  
Jothi R,Cuddapah S,Barski A,Cui K,Zhao K,Genomewide identification of in vivo proteinDNA binding sites from ChIPSeq dataNucleic Acids ResYear: 200836165221523110.1093/nar/gkn48818684996  
Schones DE,Zhao K,Genomewide approaches to studying chromatin modificationsNature reviewsYear: 20089317919110.1038/nrg227018250624  
Buck MJ,Lieb JD,ChIPchip: considerations for the design, analysis, and application of genomewide chromatin immunoprecipitation experimentsGenomicsYear: 200483334936010.1016/j.ygeno.2003.11.00414986705  
Qi Y,Rolfe A,MacIsaac KD,Gerber GK,Pokholok D,Zeitlinger J,Danford T,Dowell RD,Fraenkel E,Jaakkola TS,Highresolution computational models of genome binding eventsNature biotechnologyYear: 200624896397010.1038/nbt123316900145  
Wilkinson DJ,Bayesian methods in bioinformatics and computational systems biologyBriefings in bioinformaticsYear: 20078210911610.1093/bib/bbm00717430978  
Pearlmutter BA,Fast exact multiplication by the HessianNeural ComputationYear: 19946110.1162/neco.1994.6.1.147  
Tsai HK,Huang GT,Chou MY,Lu HH,Li WH,Method for identifying transcription factor binding sites in yeastBioinformaticsYear: 200622141675168110.1093/bioinformatics/btl16016644789  
Cherry JM,Adler C,Ball C,Chervitz SA,Dwight SS,Hester ET,Jia Y,Juvik G,Roe T,Schroeder M,SGD: Saccharomyces Genome DatabaseNucleic Acids ResYear: 1998261737910.1093/nar/26.1.739399804  
Gorner W,Durchschlag E,MartinezPastor MT,Estruch F,Ammerer G,Hamilton B,Ruis H,Schuller C,Nuclear localization of the C2H2 zinc finger protein Msn2p is regulated by stress and protein kinase A activityGenes DevYear: 199812458659710.1101/gad.12.4.5869472026  
Lee J,Godon C,Lagniel G,Spector D,Garin J,Labarre J,Toledano MB,Yap1 and Skn7 control two specialized oxidative stress response regulons in yeastJ Biol ChemYear: 199927423160401604610.1074/jbc.274.23.1604010347154  
Raitt DC,Johnson AL,Erkine AM,Makino K,Morgan B,Gross DS,Johnston LH,The Skn7 response regulator of Saccharomyces cerevisiae interacts with Hsf1 in vivo and is required for the induction of heat shock genes by oxidative stressMol Biol CellYear: 20001172335234710888672  
Deckert J,Perini R,Balasubramanian B,Zitomer RS,Multiple elements and autorepression regulate Rox1, a repressor of hypoxic genes in Saccharomyces cerevisiaeGeneticsYear: 19951393114911587768429  
Mai B,Breeden L,Xbp1, a stressinduced transcriptional repressor of the Saccharomyces cerevisiae Swi4/Mbp1 familyMol Cell BiolYear: 19971711649165019343412 
Figures
Tables
Comparing motif similarity scores of nine yeast TFs from four different calculations.
TF Name (consensus sequence length)  Activated in stress conditions  BayesPI  Gaussian prior  BayesPI  Laplace prior  BayesPI  Cauchy prior  MatrixREDUCE 

ACE2 (6)  No  0.89  0.95  0.96  0.90 


MSN2 (6)  Yes[^{17}]  0.76  0.93  0.79  NA 


SWI4 (7)  No  0.96  0.94  0.94  0.95 


YAP1 (7)  Yes[^{18}]  0.93  0.92  0.92  0.93 


INO4 (8)  No  0.90  0.92  0.94  0.97 


SKN7 (9)  Yes[^{19}]  0.86  0.87  0.86  0.82 


FHL1 (10)  No  0.95  0.95  0.93  0.88 


ROX1 (12)  Yes[^{20}]  0.72  0.72  0.78  0.75 


XBP1 (12)  Yes[^{21}]  0.76  0.77  0.76  NA 
For the nine yeast TFs, the ChIPchip datasets were obtained from [^{7}]; the regularization constants α in BayesPI were divided into four classes; MatrixREDUCE program was downloaded from the publication [^{8}] and its default parameters were used in the present study. Here, the motif similarity scores greater than 0.85 represents a good match between the prediction and the SGD consensus sequences [^{1}]. Poor predictions are marked by bold text. NA indicates that no results are available owing to the program reason. All the programs were applied on the same datasets and were run under a PC cluster (a dualcore CPU SUN X6220 blade node with 16 GB of RAM).
Article Categories:

Previous Document: Splenic artery embolization in a woman with bleeding gastric varices and splenic vein thrombosis: a ...
Next Document: Evaluation of a system of structured, proactive care for chronic depression in primary care: a rand...