Bayesian Predictive Checking And Parameter Estimation

Bayesian Predictive Checking

• Given that the problem has geometric prior distribution with mean 100. The prior distribution is:

P(N)=(1/100)(99/100)^N-1

Given that the far number 203 is being observed here. Hence, it can be expected to be at least 203 cars. Any of the cars has equal probability to be the number of the last car. The likelihood function can be evaluated here as:

P(y/N) = (1/N), for each N>202,

• , else where.

The posterior distribution here will be:

P(N/y) α P( N)*P(y/N)

=(1/N)(0.01)(0.99)^N-1

α (1/N)(0.99)^N

• A probability distribution adds up to 1 and the posterior probability has to sum up to 1. Therefore, the normalizing constants here are:

P(X) = ∑(1/N)(0.99)^N      N=203(1) ∞

Therefore, the posterior mean is:

E[p(N/X)] = ∑_N=203¹⁰⁰⁰⁰ N = ∑_N=203^{10000 = 279.0885}

And, the standard deviation is:

sd[p(N/X)] = √ = 79.96458

N = 203:10000

p.x. = sum((1/N)*(99/100)*(N-1))

e.n. = sum(((99/100)*(N-1))/p.x)

e.n.

s.d.N = sqrt(sum((N-e.n.)^2*((1/N)*(99/100)*(N-1)/p.x.))

s.d.

• Let the problem has constant prior distribution. The prior distribution is:

P(N)=k.

Given that the far number 203 is being observed here. Hence, it can be expected to be at least 203 cars. Any of the cars has equal probability to be the number of the last car. The likelihood function can be evaluated here as:

P(y/N) = (1/N), for each N>202,

• , else where.

The posterior distribution here will be:

P(N/y) α P( N)*P(y/N)

=(1/N)*k

A probability distribution adds up to 1 and the posterior probability has to sum up to 1. Therefore, the normalizing constants here are:

P(X) = ∑(1/N)*k     N=203(1) ∞

Therefore, the posterior mean is:

E[p(N/X)] = ∑_N=203¹⁰⁰⁰⁰ N 

And, the standard deviation is:

sd[p(N/X)] = √

N = 203:10000

p.x. = k*sum(1/N)

e.n. = k*sum(1/p.x)

e.n.

s.d.N = k*sqrt(sum((N-e.n.)^2*((1/N)/p.x.))

s.d.Ns

Given is the that on survey on the number of bicycles and other vehicles recorded in an neighbourhood area of university of California, Berkeley on the division of two routes, one is for bike route and the other is non-bike route in an residential area. Given is the observed dataset:

A probability model is to be applied for this dataset for i=1(1)2. It can be easily said that the observed number of bicycles in the said area is binomial, that is,

y1,y2,…yi ~ ind Bin(ni,θi),

where, θ is the parameter of the model that resembles the proportion of  bike traffic at the jth location. The prior distribution for the parameters can be said to be beta distribution, that is ,

θi ~ iid Beta(α,β),

(b)

Therefore, the prior distribution here is :

P(θi) = Beta(α,β) α θiⁿⁱ(1-θ_i)^ni-θi

(c)

The likelihood function is:

P(yi /θi) α θi^(α-1)( 1-θ_i)^(β-1)

P(θi/yi) α P(θi)*P(yi/θi) α (θiⁿⁱ(1-θ_i)^ni-θi)*( θi^(α-1)( 1-θ_i)^(β-1)) = θi^(ni+α-1)(1- θi)^ni-θi+β-1

The required simulation result is:

α=0.152, β=0.220.

From popn 1:

P(yi /θ1) α θ1^(α-1)( 1-θ₁)^(β-1)

P(θi/yi) α P(θi)*P(yi/θi) α (θiⁿⁱ(1-θ_i)^ni-θi)*( θi^(α-1)( 1-θ_i)^(β-1)) = θi^(ni+α-1)(1- θi)^ni-θi+β-1 

=β(ni+α, ni-θi+β)

R codes are:

x=seq(0,1,length=100)

l=dbeta(x,10.152,9.9827)

l

plot(l)

For popn 2:

P(θ2) α θ2^(α-1)( 1-θ₂)^(β-1) ~ β(α, β)

P(θ2/yi) α β(n2+α, n2-θ2+β)

Simulated graph:

R codes are:

x2=seq(0,1,length=100)

k=dbeta(x1,8.152,8.1226)

k

 (d)

It can be said from the above plot that with a simulation of values with mean of first dataset minus mean of second dataset, their would not have been much difference in the plot of it. Therefore, it would not create much difference.   

The given data set is :

Residencial (with bike route): 16/58, 9/90, 10/48, 13/57, 19/103, 20/57, 18/86, 17/112, 35/273,55/64.

Residencial (without bike route): 12/113, 1/18, 2/14, 4/44, 9/208, 7/67, 9/29, 8/154.

The dataset are the  counts of bikes and other vehicles used, observed within a period of an hour in 10 blocks of a city. The dataset are being observed  in some residential area.

• Given is the that on survey on the number of bicycles and other vehicles recorded in an neighbourhood area of university of California, Berkeley on the division of two routes, one is for bike route and the other is non-bike route in an residential area. Given is the observed dataset:

A probability model is to be applied for this dataset for i=1(1)2. It can be easily said that the observed number of bicycles in the said area is binomial, that is,

y1,y2,…yi ~ ind Bin(ni,θi),

where, θ is the parameter of the model that resembles the proportion of  bike traffic at the jth location. The prior distribution for the parameters can be said to be beta distribution, that is ,

θi ~ iid Beta(α,β),  with hyper pior density as: p(α,β) =(α+β)^-2

Therefore, the prior distribution here is :

P(θi) = Beta(α,β) = ∏ θiⁿⁱ(1-θ_i)^ni-θi

And,the hyper prior distribution is:

P(α,β)=(α+β)^-(5/2)

The likelihood function is:

P(yi /θi) = ∏Cⁿⁱ_yi θi^(α-1)( 1-θ_i)^(β-1)

The posterior density is:

P(θi,α,β/yi) α P(θi)*P(yi/θi)*P(α,β) ⁼   ∏Cⁿⁱ_yi θi^(α-1)( 1-θ_i)^(β-1) *  ∏ θiⁿⁱ(1-θ_i)^ni-θi * (α+β)^-(5/2)

Therefore,

P(θi,α,β/yi) ~ (α+β)^{-(5/2) *}  * ∏θi^(α+yi-1)( 1-θ_i)^(β+ni-yi-1), that can also be written as:

P(θi,α,β/yi) ~ β(ni+α, ni-θi+β),

• The marginal distribution of α and β are:

P(α ,β /y) = 

The marginal density of the parameter is:

P(θ/α,β,y) = ∏ θi^(α+yi-1)( 1-θ_i)^(β+ni-yi-1)

P(α,β/y) α  (α+β)^-(5/2) *  ∏   

The comparison between the posterior distribution of  the parameter theta and thr original proportions can be made here:

It can be seen from the graph that the original values and the theta values are close to each other because the points and the line is close. It can be said that the distribution is a good fit.

(d)

For popn 1:

The expected value of the beta distribution with parameters   will be given by

Therefore,  expected value is 0.4086.

R codes for calculation :

p = 0.152/(0.152+0.220)

c1 = c(16/58, 9/90, 10/48, 13/57, 19/103,20/57, 18/86, 17/112, 35/273, 55/64)

quantile (p,c1)

(e)

The question demands for a prediction from posterior distribution. This can be made here by generating samples or generating new  from the beta(, that is

 ~ beta(

With the use of these sampled values, another sampling can be made from binomial distribution with a sample size of 100, that s n = 100.

y/ ~ Bin(100,

R cdes for simulation and confidence interval estimation are:

theta_new  = rbeta(S,alpha,beta)

Y_new = rbinom(S,100,theta_new)

Quantile(y_new,c(0.152,0.220))

(f)

It can be seen that the plot between   and y/n fits each other well. Therefore, it can be concluded  here that   has reasonable values.

References

Benavoli, A., Corani, G., Demšar, J. and Zaffalon, M., 2017. Time for a change: a tutorial for comparing multiple classifiers through Bayesian analysis. The Journal of Machine Learning Research, 18(1), pp.2653-2688.

Moore, B.R., Höhna, S., May, M.R., Rannala, B. and Huelsenbeck, J.P., 2016. Critically evaluating the theory and performance of Bayesian analysis of macroevolutionary mixtures. Proceedings of the National Academy of Sciences, 113(34), pp.9569-9574.