Markup is Text

At its base XML, SGML, and the rest are purely text. The content is text, the markup is text. You can treat it as binary (and in SGML, even perhaps make it look binary), but the basic notion is simply text plus labels.
Benefits That textual nature has exposed content to manipulation in ways that were barely considered possible with binary formats. Text is reasonably amenable to human intervention, and most languages have libraries of some sort for working directly with text.
Costs Verbosity is the most obvious, but there are also lingering questions about text processing being the poor stepchild of computing. (Perl is always suspect!)