Serra MtG Collection Tracker
Dockerized in WSL2 Debian
Prerequisite: Docker setup done. Based on:
- https://nickjanetakis.com/blog/install-docker-in-wsl-2-without-docker-desktop
- https://nickjanetakis.com/blog/setting-up-docker-for-windows-and-wsl-to-work-flawlessly
- https://nickjanetakis.com/blog/docker-tip-60-what-really-happens-when-you-run-docker-compose-up
- https://nickjanetakis.com/blog/tag/docker-tips-tricks-and-tutorials
- https://docs.docker.com/engine/install/debian/#install-using-the-convenience-script
- https://docs.docker.com/engine/install/linux-postinstall/
- https://docs.docker.com/compose/install/linux/
- https://www.geeksforgeeks.org/running-commands-inside-docker-container/
- https://github.com/docker/compose
- https://www.reddit.com/r/OpenMediaVault/comments/v6zvn9/how_do_i_backup_and_restore_exactly_where_i_left/
Initial Serra setup
Pull latest git trunk
$ mkdir ~/docker
$ cd docker/
~/docker$ git clone https://github.com/noqqe/serra.git
Pull latest serra binary
~/docker$ cd serra/ && wget https://github.com/noqqe/serra/releases/download/1.5.1/serra_1.5.1_Linux_x86_64.tar.gz
~/docker/serra$ tar xvf serra_1.5.1_Linux_x86_64.tar.gz
Start container
~/docker/serra$ docker compose up -d
[+] Running 10/10 ⠿ mongo Pulled 29.2s ⠿ eaead16dc43b Pull complete 5.1s ⠿ 8a00eb9f68a0 Pull complete 5.1s ⠿ f520612e53a0 Pull complete 5.5s ⠿ 14a357eb129f Pull complete 5.5s ⠿ c44e86010cfc Pull complete 5.6s ⠿ 962d899e9427 Pull complete 5.6s ⠿ cfecd4270f9a Pull complete 5.7s ⠿ 9dd246a7f5dd Pull complete 27.1s ⠿ b962974cdbb0 Pull complete 27.2s [+] Running 4/4 ⠿ Network serra_default Created 0.0s ⠿ Volume "serra_mongodb" Created 0.0s ⠿ Volume "serra_mongodbbackup" Created 0.0s ⠿ Container serra-mongo-1 Started
Check container
~/docker/serra$ docker stats
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS dc75fbcb0c7d serra-mongo-1 0.99% 247.3MiB / 5.791GiB 4.17% 27.7MB / 210MB 0B / 0B 36
List container(s), same as ’docker container ls’
~$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dc75fbcb0c7d mongo "docker-entrypoint.s…" 2 months ago Up 4 days 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp serra-mongo-1
Enable bash completion
After this, I need to either open a new terminal instance or source /etc/bash_completion.d/serra.
# ./serra completion bash > /etc/bash_completion.d/serra
Before adding cards to database, connect to it:
$ export MONGODB_URI='mongodb://root:root@localhost:27017'
In order to do backups and restorations for database, install mongo-tools.
~/docker/serra$ sudo aptitude install mongo-tools
This allows mongodump to do a database dump in the container,
because of docker exec:
$ docker exec -it serra-mongo-1 mongodump -u root -p root --authenticationDatabase admin -d serra -o /backup/
Check that database dump resulted in a OK copy in container filesystem
Open bash instance in the container in order to examine /backup/ content:
$ docker exec -it serra-mongo-1 bash
root@dc75fbcb0c7d:/# ls -alR /backup/
/backup/: total 12 drwxr-xr-x 3 1000 1000 4096 Dec 9 19:02 . drwxr-xr-x 1 root root 4096 Dec 7 14:03 .. -rw-r--r-- 1 1000 1000 0 Dec 6 09:58 .placeholder drwxr-xr-x 2 root root 4096 Dec 9 19:02 serra /backup/serra: total 524 drwxr-xr-x 2 root root 4096 Dec 9 19:02 . drwxr-xr-x 3 1000 1000 4096 Dec 9 19:02 .. -rw-r--r-- 1 root root 8408 Dec 9 19:02 cards.bson -rw-r--r-- 1 root root 172 Dec 9 19:02 cards.metadata.json -rw-r--r-- 1 root root 497872 Dec 9 19:02 sets.bson -rw-r--r-- 1 root root 171 Dec 9 19:02 sets.metadata.json -rw-r--r-- 1 root root 65 Dec 9 19:02 total.bson -rw-r--r-- 1 root root 172 Dec 9 19:02 total.metadata.json root@dc75fbcb0c7d:/# exit
Backup seems to be also hosted outside container at /serra/_backup/ for easier access from host OS.
Do a collection export to json as a backup
$ docker exec -it serra-mongo-1 mongoexport -u root -p root --authenticationDatabase admin -d serra -c cards > /backup/cards.json
Do restore of the backup
$ docker exec -it serra-mongo-1 mongorestore --drop -u root -p root --authenticationDatabase admin --port=27017 /backup/
Mass add card collection with the fish shell wrapper script
Install fish shell as here: https://clubmate.fi/how-to-install-and-use-fish-shell
Update docker container
~/docker/serra$ git pull origin main
remote: Enumerating objects: 87, done. remote: Counting objects: 100% (87/87), done. remote: Compressing objects: 100% (28/28), done. remote: Total 87 (delta 54), reused 79 (delta 46), pack-reused 0 Unpacking objects: 100% (87/87), 15.90 KiB | 397.00 KiB/s, done. From https://github.com/noqqe/serra * branch main -> FETCH_HEAD 5913f30..bd7536b main -> origin/main Updating 5913f30..bd7536b Fast-forward go.mod | 1 + go.sum | 1 + readme.md | 1 + src/serra/add.go | 6 +++--- src/serra/card.go | 12 ++++++++---- src/serra/env.go | 28 ++++++++++++++++++++++++++++ src/serra/gains.go | 17 +++++++++++------ src/serra/helpers.go | 21 ++++++++++++++------- src/serra/missing.go | 2 +- src/serra/remove.go | 2 +- src/serra/root.go | 1 - src/serra/scryfall.go | 65 +++++++++++++++++++++++++++++++++++++---------------------------- src/serra/set.go | 23 ++++++++++++++--------- src/serra/stats.go | 8 ++++++-- src/serra/storage.go | 59 ++++++++++++++++++++++++++++++++++------------------------- src/serra/update.go | 75 +++++++++++++++++++++++++++++++++++++++++++++++---------------------------- src/serra/utils.go | 4 ---- 17 files changed, 207 insertions(+), 119 deletions(-) create mode 100644 src/serra/env.go
OR, try next time just to do: docker compose pull
: None of the above are needed as I can just wget the latest Serra
binary, which is enough to upgrade Serra.
To upgrade to latest Serra:
$ wget https://github.com/noqqe/serra/releases/download/3.20.0/serra_Linux_x86_64.tar.gz
$ tar xvzf serra_Linux_x86_64.tar.gz
Then run Serra:
$ ./serra web
And access it in your browser:
$ xdg-open 127.0.0.1:8080
To upgrade from 1.5.1 to any later version
See steps at https://github.com/noqqe/serra/issues/16
Consider upgrading so that you scratch the current container and rebuild it anew.
To upgrade from 2.x.x to any later version
$ wget https://github.com/noqqe/serra/releases/download/3.13.0/serra_Linux_x86_64.tar.gz
$ tar xvf serra_Linux_x86_64.tar.gz
Running MongoDB queries
Queries are located at ~/docker/serra/mongodb_queries.md
First launch bash in serra container:
$ docker exec -it serra-mongo-1 bash
root@3c069ea8c73c:/# cd bin/ # Launch mongo shell to interact with the MongoDB server root@3c069ea8c73c:/bin# mongosh "mongodb://localhost:27017" --username root --authenticationDatabase admin Enter password: **** Current Mongosh Log ID: 644959e57c7951071eb16b2e Connecting to: mongodb://<credentials>@localhost:27017/?directConnection=true&serverSelectionTimeoutMS=2000&authSource=admin&appName=mongosh+1.6.2 Using MongoDB: 6.0.4 Using Mongosh: 1.6.2 For mongosh info see: https://docs.mongodb.com/mongodb-shell/ ------ The server generated these startup warnings when booting 2023-04-26T05:47:43.282+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem 2023-04-26T05:47:44.444+00:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never' 2023-04-26T05:47:44.444+00:00: vm.max_map_count is too low ------ ------ Enable MongoDB's free cloud-based monitoring service, which will then receive and display metrics about your deployment (disk utilization, CPU, operation statistics, etc). The monitoring data will be available on a MongoDB website with a unique URL accessible to you and anyone you share the URL with. MongoDB may use this information to make product improvements and to suggest MongoDB products and deployment options to you. To enable free monitoring, run the following command: db.enableFreeMonitoring() To permanently disable this reminder, run the following command: db.disableFreeMonitoring() ------ test> show dbs admin 100.00 KiB config 48.00 KiB local 96.00 KiB serra 19.23 MiB test> use serra switched to db serra serra> show collections cards sets total serra>
Find cards that increased prices
db.cards.find({$expr: {$gt: [{$arrayElemAt: ["$serra_prices", -2]}, {$arrayElemAt: ["$serra_prices", -1]}]}}, {name:1})
NEXT Pipe cardlist to a file from mongosh
Utilize jq found from docker bin Something like jc dig example.com | jq -r ’.[].answer[].data’ (https://github.com/kellyjonbrazil/jc)
With that, utilize db.Cards.find().forEach(printjson) so that I get the output in json, then figure out how to strip everything else than the quantity and card name (https://stackoverflow.com/questions/22565231/printing-mongo-query-output-to-a-file-while-in-the-mongo-shell)
None of the above needs to be done, if I get csv export to work. As CSV allows me to run this ruby script and strip out everything. CSV also allows to import to other services, such as https://hottemax.github.io/MtgCsvHelper/.