Automation is great, but it won't work for my books

The most common feedback from publishers when chatting about automation is the belief that their publications are beyond what automation can do.

As a book designer and typesetter with over 20 years’ experience in producing almost every kind of book possible, I get it.

The hours dedicated to ensuring each line is hyphenated correctly; taking care of widows, orphans, grunts, and rivers; deciding where to place images (with the perfect crop); appropriately placing artwork in relation to its reference in the content; adjusting leading and layouts to ensure the content fits a publication correctly; and so it goes, page after page.

As a book designer, I was so much more than a machine, and it brought in the bread and butter⁠—and some jam every now and then.

How could any software replace what I had taken decades to accomplish?

I knew there were systems that created PDFs with the push of a button. While enticing, they were really expensive, and would require reskilling with a new set of software tools.

Meanwhile, the ‘total solution’ systems didn’t appear to have a middle ground—it was an all or nothing commitment.

The thought of losing production time, letting down clients and colleagues with inferior work, or missing deadlines while getting to grips with a new solution was frightening. And, quite frankly, I was terrified of losing control of the process.

So, I let any thoughts of automation languish, until it suddenly became a necessity.

Dipping a toe in the water

I took my first step with ‘semi-automation’ on a book that was fairly complex. The manuscript was unstyled (everything had the paragraph style ‘Normal’ with bolds and italics and manual character styling, argghh), and the deadline for typesetting did not compensate for the untagged manuscript.

Bless the editor, though—at least most of the manual styling was fairly consistent!

I ran in the manuscript and paused. There had to be some way to fix this.

With some analysis, I noticed that certain text occurred regularly. Some paragraphs started with “Exercise 1” in bold and 12pt, and I realised that this could serve as a unique identifier for that type of paragraph.

I decided a search and replace was in order. I could have used Microsoft Word, but Adobe InDesign was my everyday tool and I was familiar with its features.

So, I did some research in online forums. I fiddled (and failed), and fiddled, and eventually I had a reasonable grasp on InDesign’s powerful GREP search and replace.

There was an initial investment in time to learn the GREP codes, which was more than a little stressful as the deadline was looming. At the time I wasn’t sure whether the steep learning curve was worth the effort.

I remember thinking, “If I had just done it the old manual way, I would have at least created pages and have something to show for my effort.”

But I kept at it, as online articles and forum discussions provided evidence that I was on the right track.

Sure enough, once I had a grasp on the GREP codes, and with some trial and error, it took just a few keystrokes to bring the manuscript into shape, and for the pages to start looking like the spec design.

I was pretty amazed with my new bionic self—part human and part machine! I had taken control of the all ‘Normal’ manuscript and turned it into a book with some basic search and replace, in a fraction of the time that it would have taken if I’d plodded along as usual.

Now I was thinking, “Why didn’t I try this before?”

By replacing the drudge of monotonous work with a few keystrokes, I could now focus more on the creative process of laying out the pages, and the proofs went back a little earlier than the deadline.

Developing my new superpowers

With the books that followed, I looked at every job through two lenses: first, using the spec design from the client to understand the layout and aesthetics, and secondly, analysing the content’s structure (paragraph and character styles, and editor’s instructions) to see how many design features I could insert using my newfound GREP skills.

I used this approach on almost every project to varying degrees, and while some projects were more successful than others, over time the gains were exponential.

Looking back on this learning experience, a few observations stand out.

My automation skills were transferable

Once I started using very basic semi-automation on one book, I could then transfer those skills to other types of books. The transfer was not always obvious, but every book levelled up my skills.

While InDesign was my main go-to software, there were occasions when working in Word was more effective. I found I could also transfer the knowledge across software, as well as content types.

Semi-automating with InDesign offered numerous benefits

Working with InDesign kept me relevant with my publishing clients. Even as I became more productive, clients and colleagues could still access files to update any last-minute changes before print, or pull the files from the archive to make reprint changes manually.

(I’ve since learned that if I had leveraged JavaScript to further automate my workflow, the InDesign documents would still be accessible to clients and colleagues for manual updates. The scripts only massage the content within the file, and do not create a new format other than what you can already export out of InDesign.)

I retained the spec design for each publication or series of publications—supplied in the form of a marked-up InDesign file—as the source of truth. Since nearly all my automation was achieved using InDesign, it was convenient to reference the design elements in the spec and use them with my semi-automated workflow.

GREP is great, but it’s not the answer for everything

Quickly I realised that GREP search and replace was not a silver bullet. Some things were beyond what was possible with GREP, or would require more powerful automation tools such as JavaScript.

And that’s fine. Gains made on semi-automating many books more than made up for the manual process still required for other publications.

The lowdown: Almost any type of publication can be (and has been) automated

As a bionic book designer of many years now, I’ve learned that the diversity of books automatically created in InDesign is simply extraordinary.

Adobe provides access to InDesign’s underbelly for scripting, and a scout around the internet quickly reveals that there are some very smart people who have leveraged this for almost every kind of publication and most types of content.

With InDesign’s scripting model, it is highly probable that if it has been made in InDesign, it can be automated—even if only partially.

While Microsoft Word continues to be the pervasive authoring tool in publishing, it lacks one critical feature—a convenient method for marking up structure. Automation is difficult without this, and getting people to do additional work, or even to work differently within the same software tool, can be challenging.

Fortunately, nowadays there are sophisticated Word plug-ins and tools that help authors and editors insert structural markup effortlessly.

Publishers often experience a gear-shift once source content has been given some structure—they come to realise that their source content is dynamic, rather than static. Going from source to multiple outputs (PDF, EPUB, HTML, and so on) through automation suddenly becomes not only possible, but realistic.

I’m more convinced than ever that Word and InDesign continue to offer a lot for publishing, even after decades as the industry standards. Both are mature and scriptable, and there is a large pool of publishing talent skilled in this software to draw from.

Complementing powerful software and talented people with a workflow that facilitates automation allows publishers of all sizes to be more creative and enhance their productivity—without massive disruption.

Some level of automation probably will work for your books. Why not make 2020 your year to give it a go?

An abridged version of this article first appeared on the BookMachine Production blog.