Pagan
The pagan religion has a great number of special rules that apply only to it. This article is a general overview of these rules.
Technological Penalties
Pagan countries get a bad tech multiplier because of the pagan state religion. So, pagans have much slower tech development even once they surmount their isolation and whiteman penalties, if they do.
Most pagan countries are in the exotic techgroup, which is the worst one. It has practically no neighbor bonus possible, and the highest multiplier for tech costs. However, this is not a feature of paganism per se, rather it is a connection set up by the scenario. The west African pagan nations are in the China techgroup, which is much better than exotic. It is possible to get a pagan nation in the European tech group during a game, although this is pretty unlikely without the player causing it happen.
War and Peace
Pagan countries can be force-annexed as a whole, not just when single provinces. Annexing them is badboy-free for European countries. Pagan countries can also be DoWed by Europeans with no badboy cost. And finally, the colonial AIs treat pagans differently: they love to DoW them, and they do not invite their allies into wars with them.
Religion and Conversion
Pagan countries may have their state religion changed via force-conversion by countries of any other religion.
On the upside, pagan countries have a special 20% bonus when sending missionaries to attempt religious conversion. Also, pagan countries get 6 religious tolerance sliders from the start of the game, so that they can fully tolerate Christians, Muslims, and pagans, which is a good fraction of the world.
Pagan provinces are special in that converting them via missionary work also converts their culture.
Miscellaneous
Pagan countries are not eligible to receive province defections of non-pagan provinces.
CoTs owned by pagan countries seem to be much less "magnetic" in drawing generated trade from provinces. Also, provinces owned by pagans are practically never chosen for the placement of new CoTs.
Pagan provinces do not seem to get revolt risk from nationalism, although they do have a sort of hidden nationalism for the purpose of determining cultural change in colonial cities.
When you capture a pagan capital, you do not get their maps.