Simple Scanning Station

Mostly out of Corona-induced boredom I decided I wanted a document scanning pipeline to get any relevant paperwork into a document management system like doscpell, with as little effort (both cost-wise and from a UX perspective) as possible. So I built a simple and hands-off scanning station with my raspberry home automation system, running on 64bit Ubuntu and with mostly standard packages / components. The scanning station works with a Fujitsu S1100 portable scanner, and implements a very simple workflow:

  1. turn on scanner (by opening the front and top covers)
  2. feed in the document to be scanned; this will start the scan and result in a pdf version of the document file appearing in the output folder (to be ingested by docspell)
  3. the single button of the S1100 is used to implement a multi-page document mode:
    • push the button to start multi-page mode
    • feed in document pages
    • push button again to end multi-page mode, resulting in a single pdf file with all scanned pages placed in output folder
  4. close the scanner covers - done

This article gives a quick rundown of the configuration steps necessary to (re)create this setup. It is intended to work with a document management system like docspell, which consumes the generated scans and makes them available in a document database.

saned

The fundamental building block for our scanner pipeline is of course sane. Basic setup for the Fujitsu S1100 is quite simple and the device works out of the box - after adding the .nal driver files from either the Windows driver CABs or the internet to /usr/share/sane/epjitsu/

This should be enough to fire up the scanner:

  • start saned: sudo systemctl start saned.socket
  • try sane-find-scanner -q, make sure the device is listed
  • then try scanimage -L, which should result in the scanner buzzing a bit and a message that sane has identified and set up the device

If this results in permission errors, make sure the user you’re running this with is a mamber of the scanner group (Ubuntu default setup), or prefix a sudo to these commands.

scanservjs

At this point it is possible to go ahead with a UI frontend for sane-scanning like scanservjs. I like to put things into containers as much as possible, for ease of deployment (especially, easy of re-creating deployments). scanservjs is available as a docker image of course - below I include a simple docker-compose file that pulls up scanservjs to work with saned.

There is one thing needed to make this play seamlessly from inside a docker container: while it is possible to map (usb) device inodes into the container, this only works as long as the /dev/bus/usb/... entry for our scanner doesn’t change. Which it will as soon as we close/open the device cover flaps in between scanning sessions.

To get around that, we can tell the natively running saned to expose device access via a network port. To do this, add the container’s IP address range to the /etc/saned.conf file in the Access List section. To find out what IP address the scanserv container is running with, use this command:

docker container inspect scanservjs | grep IPAddress

In my case I’m adding the following address block to the saned.conf file:

## Access List
172.21.0.1/24

Inside scanservjs’ docker container the sane service now needs to use the net driver, and connect to the saned instance running on the host system. A rudimentary docker-compose file to make all of this happen looks like this:

version: "3"
services:
  scanservjs:
    container_name: scanservjs
    image: masterokhan/pi-scanservjs:latest
    restart: unless-stopped
    extra_hosts:
      - "host.docker.internal:host-gateway"
    environment:
      - SANED_NET_HOSTS=<host-address>
    volumes:
      - '/opt/scanserv/output:/app/data/output'
      - '/opt/scanserv/config:/app/config'
    restart: unless-stopped

scanbd and scanbm

scanservjs is a smooth frontend for interactive scanning, and if that’s what is needed you’re done at this point. However I want to build an automated scanning pipeline with as little user interaction needed as possible. So we are not looking for an interactive UI, but rather want to streamline the entire process.

To do this we need a service that continuously polls the scanner for events like “button pressed” or “paper detected”, and immediately initiates the scanning process. As usual people have been there before, and have created the scanbd service to do just that. A quick apt install scanbd gets the necessary things in place, however our setup for scanbd is slightly more involved. I am not going to rehash the scanbd setup here, there are multiple articles out there describing the steps.

What is important:

  • really copy all files from /etc/sane.d/ to the /etc/scanbd/ configuration; I initially forgot to include sane.conf, which results in scanbm not working as expected
  • … scanbm is a management-proxy in front of scanbd, which monitors client interactions on the saned port, and stops scanbd polling to allow other clients access to the scanner

More interesting is setting up the script actions for scanbd to execute on various scanner events. To enable the workflow described above, my configuration reacts to the paperload and scan events, which represent paper being fed and the button being pressed, respectively.

I wanted to use the powersave event as well, to clean up the multipage environment (see below), thinking that it would occur when the scanner covers are being closed. Turns out the device enters powersave after a few seconds of inactivity, so I had to dismiss that idea.

If you want to know what events your scanner supports, scanimage -A can give you an overview. Also have a look at the files in /etc/scanbd/scanner.d, maybe there is something in there that works with your device.

The scanbd.conf file, plus the scripts in scanbd/scripts, contain the logic to implement our hands-off scanning station workflow. In principle there are two parts to make this work:

  • on paperload the scripts/scan.script is started, which tells the scanner to
    • grab a tiff from the inserted document, into a tmp directory
    • convert that into a pdf file
    • move the pdf to the final output folder and clean up, unless we are in multipage mode (see below)
  • on scan button press we initiate multi-page mode, which
    • creates a multipage-flag file on first press
    • if that flag is set during page scan, created documents are not moved to the final output folder but stay in the tmp folder, until
    • the scan button is pressed a second time,
    • which collates all the collected single-page documents into one multipage pdf,
    • moves that into the output folder
    • and cleans everything up including the multipage-flag file

Look at the scripts below for details on how things work. The main thing to be mindful of is setting up the scripts and folders with the necessary permissions so that everything can work as desired. I am doing the following:

  • scanbd.conf and scripts/* are managed in a gitlab repository, and checked out locally into /opt/scanserv/scanbd
  • /etc/scanbd/scanbd.conf and /etc/scanbd/scripts/ are links to that local repo
  • the output and tmp folders are also located in /opt/scanserv/
  • saned and scanbd deamons run as user saned, group scanner by default, so we allow the required access to these folders:
sudo chmgrp -R scanner /opt/scanserv/output /opt/scanserv/temp
sudo chmod -R g+w /opt/scanserv/output /opt/scanserv/tmp

Testing all of this can be done by running the scripts individually - just keep in mind that you’re likely going to do that with a different user than what saned will be using. The scripts contain syslog output that can he observed using tail -f /var/log/syslog. Once everything appears to work, start the scanbd and scanbm deamons using

sudo systemctl start scanbd.service
sudo systemctl start scanbm.socket

Once all of this is running, instruct the document management system to pick up files from the output folder defined in scanbd/scripts/vars.scripts - this is also where other relevant directories and properties like scanning mode and resolution are defined.

Waymarks

  • What is missing from the setup is some user feedback when multipage mode is active - I’ve no good ideas how to do that, as there is not way to get the scanner to make noises or change the LED settings.
  • Otherwise this is a nifty setup, works well even with a cheap simplex scanner.
  • Setting up docspell on a 4B Raspi requires tuning down language processing settings, and docspell in general hasbn’t been the most robust experience for me. Might write a dedicated post about that setup.

Appendix

scanbd/scripts/vars.script

#!/bin/bash

BASEDIR=/opt/scanserv
TMPDIR="$BASEDIR"/scanbd/tmp
OUTPUTDIR="$BASEDIR"/output
MULTIPAGE_MODE="$BASEDIR"/scanbd/multimode.flag

MODE=Gray
RESOLUTION=300

scanbd/scripts/scan.script

#!/bin/bash
source /etc/scanbd/scripts/vars.script

# We need tiff2pdf installed
if ! command -v tiff2pdf &> /dev/null
then
  logger -t "scanbd: $0" "ERROR scanning on $SCANBD_DEVICE: missing tiff2pdf executable!"
  exit
fi


logger -t "scanbd: $0" "Begin of $SCANBD_ACTION for device $SCANBD_DEVICE"

if [[ ! -d "$TMPDIR" ]]; then
  mkdir -p "$TMPDIR"
fi

filename=$(echo "Scan_" | awk '{ print $1 strftime("%Y%m%d_%H%M%S") }')
scanimage -o "$TMPDIR/$filename".tiff --format=tiff --mode="$MODE" --resolution="$RESOLUTION" -d epjitsu
tiff2pdf -p A4 -F -d -o "$TMPDIR/$filename".pdf "$TMPDIR/$filename".tiff


# If we're not in MULTIPAGE_MODE move file to OUTPUT and clean up
if [[ ! -f "$MULTIPAGE_MODE" ]]; then
  mv "$TMPDIR/$filename".pdf "$OUTPUTDIR"
  rm -fr "$TMPDIR"/*

else
  rm -f "$TMPDIR/$filename".tiff
  logger -t "scanbd: $0" "MULTIPAGE scan on device $SCANBD_DEVICE, leaving scan in place in $TMPDIR"

fi

logger -t "scanbd: $0" "End of $SCANBD_ACTION for device $SCANBD_DEVICE"

scanbd/scripts/multipage.script

#!/bin/bash
source /etc/scanbd/scripts/vars.script


# We need tiff2pdf installed
# (gs is also used below, but should be available by default in most Linux environments)
if ! command -v tiff2pdf &> /dev/null
then
  logger -t "scanbd: $0" "ERROR entering MULTIPAGE mode for $SCANBD_DEVICE: missing tiff2pdf executable!"
  exit
fi


# are we already in multipage mode?
if [[ -f "$MULTIPAGE_MODE" ]]; then
  if [[ ! -d "$TMPDIR" ]]; then
    logger -t "scanbd: $0" "ERROR finalizing multipage session for device $SCANBD_DEVICE: TMPDIR not available ($TMPDIR)"
    /etc/scanbd/scripts/multipage_cleanup.script
    exit
  fi

  # wrap up multipage session - collate scanned documents, move to downstream processing, clean up
  logger -t "scanbd: $0" "Finalize multipage session for device $SCANBD_DEVICE"

  COUNT=$(find "$TMPDIR" -maxdepth 1 -name '*.pdf' | wc -l)
  if [[ "$COUNT" -gt 0 ]]; then
    logger -t "scanbd: $0" "Concatenating $COUNT files for multipage session on device $SCANBD_DEVICE"

    # collate all pdfs into one file and move that to output folder
    filename=$(echo "MultiScan_" | awk '{ print $1 strftime("%Y%m%d_%H%M%S") }')
    gs -dBATCH -dNOPAUSE -q -sDEVICE=pdfwrite -dAutoRotatePages=/None -sOutputFile="$TMPDIR"/"$filename".pdf "$TMPDIR"/*.pdf
    mv "$TMPDIR/$filename".pdf "$OUTPUTDIR"

  else
      logger -t "scanbd: $0" "No scanned files found for session for device $SCANBD_DEVICE"

  fi

  /etc/scanbd/scripts/multipage_cleanup.script
  logger -t "scanbd: $0" "Multipage session for device $SCANBD_DEVICE wrapped up"

else
  # prepare multipage session
  logger -t "scanbd: $0" "Enter multipage session for device $SCANBD_DEVICE"

  # create tmp dir for our multi-page scans, set MULTIPAGE flag to true
  mkdir -p "$TMPDIR"
  touch "$MULTIPAGE_MODE"

  logger -t "scanbd: $0" "Multipage session for device $SCANBD_DEVICE set up"
fi

scanbd/scripts/multipage_cleanup.script

#!/bin/bash
source /etc/scanbd/scripts/vars.script

logger -t "scanbd: $0" "Cleaning up multipage-mode residuals"

rm -fr "$TMPDIR"/*
rm -f "$MULTIPAGE_MODE"

excerpt from scanbd/scanbd.conf`

action scan {
    filter = "^scan.*"
        numerical-trigger {
            from-value = 1
            to-value   = 0
        }
    desc   = "Trigger multipage mode" 
    script = "multipage.script"
}

action paperload {
    filter = "^page-loaded.*"
        numerical-trigger {
            from-value = 0
            to-value   = 1
        }
    desc   = "Scan to file"
    script = "scan.script"
}