Let $X$ be a smooth projective curve over a perfect field $k$ of positive characteristic $p$. The differential (also called stratified) fundamental group of $X$ (at a base point $x\in X(k)$ is defined as the Tannakian dual of the category of stratified bundles on $X$. It is the pro-algebraic analogue of the etale fundamental group of $X$. In this talk we discuss the group cohomology of the stratified fundamental group of $X$ by comparing it with the stratified cohomology introduced by Ogus. We obtain some vanishing and non-vanishing results and make some suggestions on the structure of the fundamental group itself.