Calibre Web: home in-browser online-library with Nextcloud storage and Moon+ Reader on Android

By | 01/07/2020
 

I’m using Calibre for my e-books library management for many years, but until today always used its desktop version.

A couple of days ago I found the Calibre Web project – the same library, working with the same database, but accessible via a browser.

The idea is to move away from storing books in Google Books and using its e-book reader on mobile and via a browser but still to be able to access my library from anywhere and to have have a book’s reading position synchronization ability, as it is in the Google Books mobile and browser readers.

So, now I have my library itself is stored in a Nextcloud (see the Nextcloud: running in Docker Compose on Debian with Let’s Encrypt SSL post), and I’ll use the Moon+ Reader application on my Android devices as it can synchronize read position via WebDAV.

Also, you can use Google Drive as a storage backend for Calibre Web, read the docs.

In this post we will:

  • run a Calibre Web instance in Docker – a web interface for a library
  • will mount a directory from the Nextcloud with Calibre’s library
  • and will use WebDAV from our phone to connect to the Nextcloud using Moon+ Reader

NGINX, Let’s Encrypt

NGINX + Let’s Encrypt set up already described in the Bitwarden: an organization’s password manager self-hosted version installation on an AWS EC2 post.

Calibre Web start

Check the library’s directory under Nextcloud which will be used by our Calibre Web:

[simterm]

$ ll /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/ | head
total 1212
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:53 A. A. Orlov
drwxr-xr-x  7 www-data www-data   4096 Dec 13 07:53 A. Avramienko
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:55 A. Bielash
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:53 A. B_iuli
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:55 Adol_f Gitlier
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:55 A. Fomienko, G. Nosovskii
drwxr-xr-x  4 www-data www-data   4096 Dec 13 07:55 A. Kamienistyi
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:55 A. Karr
drwxr-xr-x  3 www-data www-data   4096 Dec 13 07:55 Al_bier Kamiu
...

[/simterm]

Run a Docker container with Calibre, mount the library’s directory inside as /books:

[simterm]

$ docker run -ti --name books -p 8083:8083 -v /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/:/books technosoft2000/calibre-web

[/simterm]

Check if metadata.db is present:

[simterm]

$ docker exec -ti books ls -l /books/metadata.db
-rw-r--r-- 1 xfs xfs 752640 Dec 13 07:51 /books/metadata.db

[/simterm]

Open your URL in a browser, and set a path to the /books directory which has to contain the metadata.db file used by Calibre to store information about all books in the library:

Log in with the default admin:admin123:

And you’ll see your library now:

The Categories menu here are just Tags from the Calibre:

And as in the common Calibre – you can view, add, edit and read books from the browser:

A browser’s reader:

“Failed to create path /books/ (Permission denied)” error

If you’ll try to upload a file now you can face the following error:

Failed to create path /books/Mark Menson/Tonkoe iskusstvo pofigizma_ Paradoksal’nyi sposob zhit’ schastlivo (Permission denied).

For me, it was an expected error as the directory /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/ is used now by both Nextcloud and Calibre Web Docker containers.

Check the user in the Calibre container:

[simterm]

$ docker exec -ti books ps | grep cali
68 calibre   0:08 /usr/bin/python /calibre-web/app/cps.py

[/simterm]

And his UID:

[simterm]

$ docker exec -ti books id calibre
uid=15000(calibre) gid=15000(calibre) groups=15000(calibre),15000(calibre)

[/simterm]

While the local directory on the host is owned by the:

[simterm]

$ ls -l /data/nextcloud/app/data/setevoy/files/Books/
total 4
drwxr-xr-x 117 www-data www-data 4096 Dec 16 09:32 CALIBRE

[/simterm]

And the www-data user’s UID:

[simterm]

$ id www-data
uid=33(www-data) gid=33(www-data) groups=33(www-data)

[/simterm]

I.e. Calibre uses the calibre user with UID 15000, but NGINX on the host and the Nextcloud container both are using www-data user with UID 33.

To solve this issue – on the host create a new user calibre with UID 15000:

[simterm]

$ sudo useradd -u 15000 calibre
$ id calibre
uid=15000(calibre) gid=15000(calibre) groups=15000(calibre)

[/simterm]

Change the directory ownership:

[simterm]

$ sudo chown -R calibre:calibre /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/

[/simterm]

Restart Calibre:

[simterm]

$ docker restart calibre-web
calibre-web

[/simterm]

Check upload now – must work now:

Nextcloud PUT 403

But now Nexcloud is broken – on a desktop client I got 403 errors during sync, for example:

172.29.0.2 – setevoy [16/Dec/2019:13:22:24 +0000] “PUT /remote.php/dav/files/setevoy/Books/CALIBRE/Pol’%20Diubua/MySQL%20(68)/cover.jpg HTTP/1.1” 403 759 “-” “Mozilla/5.0 (Linux) mirall/2.6.1git (Nextcloud)”

To solve this issue – add newly added the calibre user created above to the www-data group:

[simterm]

$ usermod -a -G www-data calibre

[/simterm]

Check his groups now:

[simterm]

$ id calibre
uid=15000(calibre) gid=15000(calibre) groups=15000(calibre),33(www-data)

[/simterm]

Recursively change ownership – give full permissions to the www-data group members, e,g, for the www-data and the  calibre users:

[simterm]

$ find /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/ -type d -exec chmod g+rwx {} +

[/simterm]

And change the owner:

[simterm]

$ chown -R calibre:www-data /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/

[/simterm]

This is done, sync is working now as well.

Docker Compose

To have data persistence first let’s find where are data stored in the container, e.g. the application’s database with all its settings.

You can read the documentation (the recommended way!), or connect to the container:

[simterm]

$ docker exec -ti calibre-web bash

[/simterm]

Find all *.db files:

[simterm]

bash-4.4# find / -name "*.db"
/calibre-web/app/app.db
/calibre-web/app/gdrive.db
...

[/simterm]

On the host create directories for the Docker Compose file and for the application’s data:

[simterm]

$ mkdir /opt/calibre-web/
$ mkdir -p /data/calibre/app
$ chown -R calibre:calibre /data/calibre/

[/simterm]

In the /opt/calibre-web/ directory create a calibre-compose.yml file:

version: '3'

services:

  calibre-web:
    user: "root:root"
    image: technosoft2000/calibre-web
    container_name: calibre-web
    ports:
      - 8083:8083
    volumes:
      - /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/:/books
      -  /data/calibre/app:/calibre-web/app/ 
      - /var/run/docker.sock:/tmp/docker.sock:ro
    restart: unless-stopped

Start the service:

[simterm]

$ docker-compose -f calibre-compose.yml up -d

[/simterm]

A unit-file creation is described in the Linux: systemd сервис для Docker Compose, in this case, I’d added a /etc/systemd/system/calibre.service file:

[Unit]
Description=Calibre library service
Requires=docker.service
After=docker.service

[Service]
Restart=always
WorkingDirectory=/opt/calibre-web
ExecStart=/usr/local/bin/docker-compose -f calibre-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f calibre-compose.yml down

[Install]
WantedBy=multi-user.target

Calibre Web themes

Out from the box Calibre Web has two themes – the light one, default:

And the dark one:

Pitfalls – merging the metadata.db with Nextcloud

This is issue relates to this current setup with the Nextcloud as storage, as it’s running on my own server and sometimes can be offline which can cause data un-sync on desktops.

The problem is, that the main Calibre’s file – the metadata.db – will be updated on the host’s filesystem when you’ll edit a book from the browser using Calibre Web, but this change will NOT trigger local Nextcloud instance to push this change to its clients when it’ll become back online as it’s running just like a server – it’s not client part.

Then, if your Calibre Desktop  is configured to use the same Nextcloud and has a desktop client like nextcloud-desktop, if you’ll run a local Calibre Desktop and will make any changes to its library – this will affect your metadata.db file and this will trigger your Nextcloud client to push these changes to the Nextcloud server, which will overwrite changes on the server.

So, to avoid it, need to manually synchronize your desktop’s Caliber’s directory content with the Nextcloud server before using it.

To do so, just issue the following command:

[simterm]

$ nextcloudcmd –user setevoy -p PASSWORD /home/setevoy/Nextcloud/Books/CALIBRE/ https://cloud.example.org.ua/Books/CALIBRE

[/simterm]

This will grab all the changes from the https://cloud.example.org.ua/Books/CALIBRE (i.e. /data/nextcloud/app/data/setevoy/files/Books/CALIBRE/ on the Nextcloud’s host) to the local /home/setevoy/Nextcloud/Books/CALIBRE/ directory (which is used by the desktop Calibre application).

Moon+ Reader

The Moon+ Reader application is from my recent findings.

Can synchronize reading positions in books when using G DriveDropbox/WebDAV, sync comments, notes, bookmarks, etc.

Install it on your phone, add a WebDAV catalog:

And you have access to your Calibre’s library now:

Synchronization settings

To make sync works in the Moon+ – open a book, then three dots in the top right corner > Miscellaneous, scroll bottom and enable the Sync reading positions:

Done.