1suppose Yc=y coordinate of centre of mass
Yc = sum(MiYi) / sum(Mi)
Yc = ∫{Yi*dM} / ∫{dM}
divide the cone into horizontal disks of mass
dM = p*dV.
p (density) is constant
Xc = ∫{Xi*dV} /∫ {dV}
Now in this case, dV = pi*r^2 dy, where r is the radius of the cone at an arbitrary height y, and dy is the thin vertical slice of your disk. The radius is dependent on height y, and because you're integrating over dy you need to express r in terms of y. At y=0, r=R, and y=h r=0. The progression is linear, meaning a point-slope representation of r as a function of y should be simple; I get r=R-yR/h between y=0 and y=h.
Re-expressing, dV = pi*R^2(1-y/h)^2dy
Yc = ∫{y*dV} / Int{dV}
Numerator:
∫{y*dV} =∫[0,h] {pi*R^2(1-y/h)^2 y} dy
= pi*R^2*∫[0,h] {y-2y^2/h+y^3/h^2}dy
= pi*R^2*{y^2/2 - 2y^3/(3h) +y^4/(4h^2)} [0,h]
= pi*R^2*{h^2/2-2h^2/3+h^2/4}
= (1/12)*pi*R^2*h^2
Denominator next:
∫{dV} = ∫[0,h]{pi*R^2(1-y/h)^2} dy
= pi*R^2{-h/3(1-y/h)^3}[0,h]
= (1/3)pi*R^2h
So Yc = {(1/12)pi*R^2*h^2} / {(1/3)pi*R^2*h}
Yc = h/4
I think that gives you everything you need.