Yep...that's a baby frog coming out of a tiny mama frogs' mouth.
This species of frog, called the Gastric Brooding Frog, incubates its' young inside it's tummy...then the frogs come hopping out of the mouth when they develop past the tadpole stage. Scientists were most intrigued by how this species manages to "Turn off" production of hydrochloric acid (the digestive juices) when brooding the froglets.
The Gastric Brooding Frog was found in Australia. Sadly, not long after their discovery they dissapeared and are now believed to be extinct.
Also- Yes! That is a thumb you see behind the frog, to show how tiny even the mama is.