PHP is one Docbook example I've been looking at from all angles because it does seem to work. I'd be curious to hear how it got there (not just what the steps are but incentives, encouragement, mandates, motivation, and also ways to keep people focused on producing in Docbook -- because there are other methods and I have observed discussions in another project where the pull is to produce in the wiki).
Part of the solution appears to be:
* Thoroughly documenting how to produce documentation
* Organizing the documentation so it is very thin-sliced (you can successfully produce a very small section of the documentation)
* Using -- and if necessary, creating -- tools that fit in the authors' workflows
* Incentives (such as acknowledging authors, and even the subtle use of second person -- "viewing your changes")
* Easing the way (and ensuring stylistic consistency) by providing clear templates (
http://doc.php.net/php/dochowto/chapter-skeletons.php )
Hidden behind this are the discussions, people, etc. who moved this project into a documentation mindset. Part of the decision to Docbook had to be the need for translation, but even beyond that was a decision that documentation is essential to the project. Such incentives exist (it produces better code, it encourages wider participation). My guess is there was one or several people who were key to making documentation part of this project's culture.