How I use Pandoc to create programming eBooks
Posted on January 21, 2021
In the past four years, I wrote three eBooks about programming. Itās like an addiction.
I would be terrible at talking about writing techniques, so I wonāt. But the content is not everything when you write an eBook. You have to care about creating beautiful files to release: PDF, ePub, Kindleā¦
Here is the story of how I built my eBooks, especially the last one. After some trial and errors, in the end, I used the same recipe as the two previous ones, and it involves a fantastic tool: Pandoc.
Note: since I wrote this post, I converted all by book content to an interactive course, solving some of the questions I asked myself in the conclusion. Check out the course at useEffect.dev.
Give me a beautiful PDFā¦
I wanted my last eBook to be a practical guide, not an academic manual. So I imagined a very custom design, with a dark theme and many colors (it is not designed to be printed).
Since it was about coding, I also wanted to include some source code examples with syntax highlighting.
Bust most of all, I wanted an efficient workflow.
⦠with an efficient workflow (please)
I intended to release a first version of the book ASAP, then iterate thanks to readersā feedback and new ideas I had. For this reason, I wanted to focus on the content and on the appearance independently.
I didnāt want to spend time on the presentation each time I updated the content. Iām a web developer; it wouldnāt occur to me to edit my CSS each time I update the HTML content. Same here.
My first try with mainstream tools
I thought: why not just use MS Word or its Apple version, Pages? Well, I tried. It didnāt go well.
You can customize pretty much everything you want. Every color, font family, font size⦠But it gets trickier when you want to include source code.
Since I wanted to have syntax highlighting, the less bad solution I found was to copy and paste from VS Code. At first, it looked (kind of) okay, but as soon as I wanted to modify it, it went terribly: the formatting was lost, spaces were inserted where I didnāt want themā¦
Definitely, these tools are not made to write content with embedded source code.
I also tried Illustrator and its open-source alternative, Scribus.
It was worse.
I admire people able to use these applications. They seem to require a lot of training and practice. Unfortunately, I intended to release my book in the next two years, so I didnāt have time to learn them. š
Iām used to writing content using Markdown. Or in Slack. Or in Notion (for this blog post). Today, we have such perfect tools for writing. I didnāt want to struggle each time I update some examples.
So, in the end, I relied on the tool I used for my previous books: Pandoc.
The never-disappointing Pandoc
Pandoc is a command-line tool whose main feature is converting some text (usually Markdown) to another format: HTML, ePub, PDF, etc.
I used it for my other books, and itās perfect for āacademicā work (thesis, research articles). But I wasnāt sure that creating something a bit more original, with shiny colors and such, was possible.
I spent around 5 hours to write the content, and lost the same amount of time looking for a publishing software or Saas to generate beautiful PDF for developer-oriented content, with source code. The winner ended being the good old Pandoc š
ā SĆ©bastien Castiel (@scastiel) December 14, 2020
To write the content, I could now write using Markdown, with the basic formatting options I needed: titles, italic, source code⦠And the source code was highlighted!
By the way, syntax highlighting is themeable, and I really wanted to use Dracula, which I already used for my VSCode. So I created a custom theme for it. š
The only thing I find problematic with Pandoc is its documentation. It contains a lot of information, but itās on one page.
One page!
So you scroll, then scroll, and scroll again. And when you want to find something, you do Cmd/Ctrl+F, type ācolorā, oh cool, there is something. Then you realize youāre in the section concerning the generation of some obscure and unknown document formatā¦
Anyway, you can usually find what youāre looking for. Iād just prefer if it used something more modern, such as Gitbook or Docusaurus, to maintain the documentation and make it easier to navigate.
Pandoc is very good at generating HTML or an ePub file. You can give it some CSS to customize anything you want. It would be a dream if you were able to do the same to generate a customized PDF.
Unfortunately, it is slightly more complicatedā¦
The tricky LaTeX customization
To generate PDFs, Pandoc uses pdflatex, which relies on LaTeX. The LaTeX language is used to write documents with a powerful but unusual syntax. These documents are then converted to PDF, Postscript, etc.
So, a little like Pandoc. But with a weird syntax instead of beautiful Markdown.
Long story short, if you want to customize the PDF rendered by Pandoc with pdflatex, you donāt have a choice: you need to provide some LaTeX code, a bit like CSS.
And I didnāt want to spend several months learning LaTeXāitās truly fantastic, you can do anything you want with it. So I did what every good developer hates doing: search on Google and copy-paste pieces of code.
Here is, for example, how I managed to have the book title in the page footer: (sorry if you are a LaTeX guru, it probably will hurt your eyes)
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\lfoot{
\begin{tikzpicture}[overlay,baseline={(0,0)}]
\node[rectangle, outer sep=0pt, text=white, anchor=west ]
(pageno)
{A React Developerās Guide to Hooks, Ā© Sebastien Castiel};
\draw[fill=comment, draw=comment]
([xshift=-\oddsidemargin-1in]pageno.north west)
rectangle
([yshift=1.5pt]pageno.south east);
\end{tikzpicture}
}
Well, to be fair, it would have been much more straightforward if I just wanted the book title. But I wanted it in an indigo rectangle starting from the far left of the page.
Still, most of the time, I was able to get what I wanted. It just took more time than I wanted to spend on it.
You can find here the complete LaTeX header file I used for my book. I cleaned it a little and added some comments.
Although I didnāt interact with them, Iām sure the LaTeX community is fantastic because you can find many resources on the Internet: answers on forums, tutorials, manuals, etc.
Polishing the workflow with a Makefile
I think the last time I used the GNU make utility was like ten years ago. I had this idea that it was only used by C or C++ projects.
If you never heard of GNU make, itās a utility program to run specific commands to generate files. Itās mostly used to compile and build programs, but you can use it for any command.
My first intuition was to write a shell script to generate my book in several formats. Quickly, I realized a Makefile would make more sense:
- I had content files and resource files (images, the LaTeX header, for example)
- I needed to generate intermediate files, such as the cover as a PDF from a PNG
- Some formats (ePub) were used to create other formats (Kindle)
Using a Makefile, I regenerated only the needed parts, based on which files were updated since the latest build. I could run make pdf
, make epub
, make mobi
, or just make
to generate all formats.
Here is the portion of my Makefile used to generate the ePub and mobi (Kindle) files:
BOOK_TITLE = A\ React\ Developerās\ Guide\ to\ Hooks\ -\ Sebastien\ Castiel
epub: dist/${BOOK_TITLE}.epub
@echo 'ā
ePub'
kindle: dist/${BOOK_TITLE}.mobi
@echo āā
Kindleā
dist/${BOOK_TITLE}.epub: dist content/*.md images/* resources/dracula.theme resources/metadata.xml resources/epub.css
@pandoc content/foreword.md content/ch*.md content/resources.md \
--output=dist/${BOOK_TITLE}.epub \
--highlight-style resources/dracula.theme \
--standalone \
--epub-metadata=resources/metadata.xml \
-c resources/epub.css \
--epub-cover-image=cover.png \
--toc --toc-depth=1 \
--metadata title="A React Developerās Guide to Hooks"
dist/${BOOK_TITLE}.mobi: dist/${BOOK_TITLE}.epub
@kindlegen dist/${BOOK_TITLE}.epub -o ${BOOK_TITLE}.mobi >/dev/null 2>&1
Okay, maybe slightly overkill for my needs, but I felt so powerful! šŖš»
You can find here the complete Makefile for my book.
What about the cover or the examples?
I embedded the examplesā source code in a small React application to run them locally. The user can start it by running yarn start
and navigate easily through them.
Now, I feel a bit silly not to have thought of using Storybook, which could have been a perfect solution.
I created the cover using Apple Pages, sufficient for what I wanted to do. I should ~~probably~~ definitely hire someone to create a more professional one.
I often design graphics using Inkscape (for example, the banner for the Gumroad page), even if at the end I manipulate and generate bitmap images. I know itās not its first purpose, but I find it easier to use than Gimp or commercial alternatives.
If you know a better choice, please please please tell me!
What I have to do better
From the beginning, I really wanted a custom presentation with a dark theme.
Now, I realize this wish took me a lot of extra time. And I would have preferred spending that time on the content. Plus, I donāt know if the readers actually like this theme or notā¦
Also, it may have been better to focus first on the web support. Docusaurus could do an excellent job: readers could navigate the sections, search for keywords, etc.
The PDF or ePub version would be secondary content: available, but maybe with a generic appearance. So what?
In the end, if the user finds the content useful, shouldnāt it be all that matters?
I will probably consider this change soon. Plus, Iād like to create a course (without video) from the bookās contentā¦
Regarding the workflow with Pandoc, Iām quite satisfied with it. Writing using Markdown is lovely to focus on the content without caring how it will render. And since Markdown is text, itās easy to keep track of the history using Git. š
It takes some time to set it up, especially to get the rendering you have in mind. But once itās done, the workflow is perfect: write in Markdown, focusing on the content, run your build command, and done!
ā SĆ©bastien Castiel (@scastiel) December 14, 2020
As I wanted, I spent time working on the appearance, time focusing on the content, but both independently. I canāt imagine what it would have been if I used MS Word.
A nightmare, for sure. š
What about you?
Did you publish eBooks? Which tools did you use? What was your workflow?
I canāt wait to hear about it!
Mention me on Twitter to share your thoughts!
Check my latest articles
- š A better learning path for React with server components (May 26, 2023)What if we took advantage of React Server Components not only to improve how we use React, but also how we help people learn it from the beginning?
- š Display a view counter on your blog with React Server Components (April 24, 2023)A short tutorial with a cool use case for React Server Components, Streaming and Suspense with Next.js: adding a view counter on a blog, calling the Plausible analytics API.
- š Using Zod & TypeScript for more than user input validation (March 8, 2023)If you have ever created an API or a form accepting user input, you know what data validation is, and how tedious it can be. Fortunately, libraries can help us, such as Yup or Zod. But recently, I realized that these libraries allow patterns that go much farther than input validation. In this post, Iāll show you why I now use them in most of my TypeScript projects.