Compare commits

..

159 Commits

Author SHA1 Message Date
scito
66c68fd9ef fix Python 3.7 compatibility 2023-01-02 22:41:30 +01:00
scito
6cfbc10e69 build wheel 2023-01-02 22:18:54 +01:00
scito
064fe81b2e docker remove python3-opencv, but add required libs; add debug mode 2023-01-02 22:11:56 +01:00
scito
cd5160f123 fix Alpine tests 2023-01-02 20:29:37 +01:00
scito
e1c8568ba2 fix protoc generation 2023-01-02 19:28:11 +01:00
scito
ef0fbc3586 update README 2023-01-02 19:09:28 +01:00
scito
b86c4f9a61 add ignore duplicate entries option and add quiet test; fixed -k stdout 2023-01-02 17:30:46 +01:00
scito
722009b172 build: add generate result files option 2023-01-02 14:16:30 +01:00
scito
1c106150b0 sort docker imports 2023-01-02 13:53:29 +01:00
scito
77e23b4ae4 refactor cv2 window and logging
- increase font and cut if too long
- refactor logging
- extract key handling
- refactor big methods
2023-01-02 13:52:04 +01:00
scito
fe3e371897 build: print restult at the end 2023-01-02 12:50:41 +01:00
scito
7c6d341270 rename github urls; update descriptions 2023-01-01 23:21:29 +01:00
scito
160a558825 some renaming and README updating 2023-01-01 22:47:12 +01:00
scito
8d8b993f12 add CV2 screenshot 2023-01-01 20:15:38 +01:00
scito
e177f860e1 abort instead of assertion 2023-01-01 19:55:28 +01:00
scito
8545dab7a5 extract draw_box and print_text functions 2023-01-01 19:22:46 +01:00
scito
16047a5b15 fix test compatibility with Alpine Linux 2023-01-01 19:19:41 +01:00
scito
604c461549 rename to build.sh 2023-01-01 19:19:21 +01:00
scito
f5acd1dee9 test docker locally after build, easier to reproduce 2023-01-01 17:56:06 +01:00
scito
1086e28056 fix pylint and mypy problems 2023-01-01 17:45:50 +01:00
scito
2c0cfd83ee docker: pip install -U 2023-01-01 17:38:08 +01:00
scito
a3bda6ff8e make PYTHON workaround uniform 2023-01-01 17:25:24 +01:00
scito
67c4f737c4 do catch AssertionError in camera capture 2023-01-01 15:21:18 +01:00
scito
fff74fc638 handle errors while reading QR from camera; log errors 2023-01-01 15:12:24 +01:00
scito
19c8e9df23 update docs 2023-01-01 10:03:44 +01:00
scito
13fcdcd022 fix problem of outdated colorama 2023-01-01 01:22:06 +01:00
scito
91b9b3671c ci: add missing colorama 2023-01-01 01:02:39 +01:00
scito
be6b9c8a7c ci: mypy only for latest Python on ubuntu 2023-01-01 00:57:18 +01:00
scito
3d61f1d316 ci: use pip install -e . for smoke tests 2023-01-01 00:56:19 +01:00
scito
a8559db6e0 ci: mypy only for latest Python on ubuntu 2023-01-01 00:52:28 +01:00
scito
9f725b227f ci: use requirements.txt for smoke tests 2023-01-01 00:48:52 +01:00
scito
869c404489 fix fileinput.input encoding only since Python 3.10 2023-01-01 00:41:11 +01:00
scito
003e122808 ignore types for stdout.reconfigure 2023-01-01 00:22:09 +01:00
scito
b3fc854078 colored warn and error messages
- add log_warn() and log_error()
- adapt tests
2023-01-01 00:14:56 +01:00
scito
fc1619d9c7 disable verbose unittest for Windows 2022-12-31 21:34:28 +01:00
scito
5be6e9c322 fix finput encoding problem on Windows 2022-12-31 21:27:40 +01:00
scito
739ae4c012 fix pytest.skipif 2022-12-31 21:16:19 +01:00
scito
1af6fe3161 fix camera type and enhance readme with pyzbar problem 2022-12-31 21:02:05 +01:00
scito
e311386a15 skip verbose tests for windows as there are encoding problems 2022-12-31 20:42:40 +01:00
scito
496564a605 avoid AttributeError in tests
StringIO in tests do not have all attributes, ignore it
2022-12-31 20:19:21 +01:00
scito
6406fcaef7 set encoding to utf-8 for stdout, needed for Windows 2022-12-31 19:52:09 +01:00
scito
7bb92f00b2 avoid mypy problems for duplicate Final
src/extract_otp_secrets.py:63: error: Incompatible import of "Final"
(imported name has type "typing_extensions._SpecialForm", local name has type "typing._SpecialForm")
[assignment]
2022-12-31 19:42:53 +01:00
scito
965f721caf ci_docker: run smoke tests before building 2022-12-31 19:20:01 +01:00
scito
61cca0c476 import Final from __future__ for Python 3.7 2022-12-31 19:11:37 +01:00
scito
ebd4d61f5f try workaround for Python 3.7: must use () for assignments 2022-12-31 19:06:57 +01:00
scito
e058010be3 fix docker invalid tags from comments 2022-12-31 18:59:01 +01:00
scito
463a9851be try workaround for Python 3.7: avoid Final in tuple assignments 2022-12-31 18:55:16 +01:00
scito
dcbb128e7c try workaround for Python 3.7: do not use Types in Final 2022-12-31 18:51:26 +01:00
scito
1b572fc9ab fix pipenv problem 2022-12-31 18:40:26 +01:00
scito
c3e9883216 try workaround for Python 3.7 NORMAL_COLOR = 255, 0, 255 problem 2022-12-31 18:21:13 +01:00
scito
3f9f7d2b8a better formatting of help 2022-12-31 18:00:49 +01:00
scito
0212e54f42 ci: runs on windows, exclude linting for Python 3.7 2022-12-31 17:52:19 +01:00
scito
3558eba93b use detect_and_decode since qrreader is fixed 2022-12-31 17:30:24 +01:00
scito
5225af0621 update README with --qr option 2022-12-31 16:42:12 +01:00
scito
1f04dd71e2 allow to choose qr reader for images 2022-12-31 15:54:21 +01:00
scito
2dea161cdc add argument for initial setting of qr_mode 2022-12-31 12:44:16 +01:00
scito
f731530f57 improve handling of wrong urls
- adapt tests
- improve messages for files
- show red box camera
2022-12-31 11:32:07 +01:00
scito
4c0bb8dc61 vscode debug: fix path to script: add missing src/ 2022-12-31 11:26:11 +01:00
scito
ad9c4a22db ci_docker: upload to only one repo 2022-12-31 11:25:33 +01:00
scito
2cdf2480a0 make bulding docker optional in script 2022-12-31 11:25:02 +01:00
scito
5aa1a35b8f add licence label to docker images 2022-12-31 11:23:44 +01:00
scito
3f3903cc81 add version to PYTHON todos 2022-12-30 21:22:03 +01:00
scito
97e4f084cb mv docker extract_otp_secrets_no_qr_reader -> extract_otp_secrets_only_txt 2022-12-30 20:43:51 +01:00
scito
549c128fb7 renaming extract_otp_secret_keys -> extract_otp_secrets and test file names 2022-12-30 20:37:38 +01:00
scito
10ff533a42 downgrade opencv for macos in requirements.txt and Pipfile 2022-12-30 19:26:45 +01:00
scito
7eb6f036ab ci: tryout downgrade opencv for macos 2022-12-30 18:51:13 +01:00
scito
652ecf57f0 ci: workaround macOS pytest segfauls -> exclude macOS (2) 2022-12-30 18:33:26 +01:00
scito
9592e6ebfe ci: workaround macOS pytest segfauls -> exclude macOS 2022-12-30 18:29:23 +01:00
scito
d6c285e59d ci: segfault: try python -m pytest 2022-12-30 18:23:31 +01:00
scito
5eed47364e ci: revert try out changes 2022-12-30 18:18:15 +01:00
scito
26e4632f90 ci: another try, install module 2022-12-30 18:14:16 +01:00
scito
c84ca46861 ci: try to avoid random test failures 2022-12-30 18:09:25 +01:00
scito
63f5ab37c4 docu pyproject.toml 2022-12-30 18:02:22 +01:00
scito
f97d7143c5 ci: fix mypy and pytest-cov after clean 2022-12-30 17:49:46 +01:00
scito
0566683203 reenable pytest --import-mode=importlib 2022-12-30 17:31:34 +01:00
scito
ee404576d5 set package_dir=src and clean option 2022-12-30 17:14:49 +01:00
scito
60d7362eee add protobuf_generated_python to py-modules 2022-12-30 15:47:02 +01:00
scito
1beba7587f enable setuptools-git-versioning 2022-12-30 15:44:58 +01:00
scito
144c9e6320 fixes after change to src-layout 2022-12-30 15:31:41 +01:00
scito
3e4476e317 change to src-layout 2022-12-30 12:37:05 +01:00
scito
7f5d4b37ee initial pyproject.toml 2022-12-30 11:14:15 +01:00
scito
82e43172c3 update setup.py 2022-12-30 09:23:31 +01:00
scito
149a548610 upgrade opencv 4.7.0 & run program at the end 2022-12-30 08:14:13 +01:00
scito
d8de89de36 improve README, add docker ci badge 2022-12-30 02:30:16 +01:00
scito
3c164fea28 coverage from yellowgreen to brightgreen 2022-12-30 02:01:10 +01:00
scito
23d8cfa151 ci: Pytest coverage comment only for 3.x and ubuntu-latest 2022-12-30 01:58:54 +01:00
scito
f5ee59368e python 3.7 compatibility: use TypedDict from typing_extensions 2022-12-30 01:44:11 +01:00
scito
b2a877061c add typing_extensions for compatibility 2022-12-30 01:37:13 +01:00
scito
c525c06480 quit on window close click 2022-12-30 01:22:12 +01:00
scito
fb43c6793c type hinting fixes 2022-12-30 01:22:05 +01:00
scito
58fc1b85ac type compatibility for Python < 3.11 2022-12-30 01:07:39 +01:00
scito
04d864c093 add code coverage bade & ci: fix pytest-cov 2022-12-30 00:58:52 +01:00
scito
51094a1a18 use PathLike type instead of str | Path 2022-12-29 23:17:31 +01:00
scito
a5768ba1e6 Workaround for PYTHON < 3.10: use Union[int, None] instead of int | None 2022-12-29 22:34:07 +01:00
scito
faafb61241 fix type hint compatibility with Python < 3.11 2022-12-29 22:12:07 +01:00
scito
d5a088135e ci: if matrix.python-version 2022-12-29 22:04:52 +01:00
scito
45a9693586 ci: fix if ${{ matrix.python-version }} 2022-12-29 21:58:44 +01:00
scito
66b41d86e6 ci: use ${{ }} for mypy python check 2022-12-29 21:52:54 +01:00
scito
89564448c6 ci: fix mypy if condition 2022-12-29 21:41:11 +01:00
scito
9ab33bd02b sort imports 2022-12-29 21:34:14 +01:00
scito
f4ab540283 rename protobuf module as pb 2022-12-29 21:32:19 +01:00
scito
201e6510f8 add type hints (Python 3.11) 2022-12-29 21:29:20 +01:00
scito
f933cd0d32 initial mypy type checking 2022-12-29 16:30:18 +01:00
scito
f4389ca8a3 fix linting 2022-12-29 15:52:17 +01:00
scito
b89a338246 enable mypy type checking 2022-12-29 14:24:12 +01:00
scito
631bacc409 use tmp_path fixture instead of clean_up() in pytests 2022-12-29 13:44:19 +01:00
scito
833afa7c13 use 3.11-slim-bullseye, add missing package 2022-12-29 11:50:19 +01:00
scito
4209a5db3d move abort to end, use SystemExit instead; improve colors 2022-12-29 11:18:57 +01:00
scito
d9a4c7ca9f fix docker image digest 2022-12-29 04:36:31 +01:00
scito
829fe65b1e ci: pip install pytest-mock 2022-12-29 04:24:00 +01:00
scito
c90526dcf2 fix undefined name 'abort' and 'qreader' 2022-12-29 04:15:36 +01:00
scito
47e84e4462 run docker capture version 2022-12-29 03:19:09 +01:00
scito
b4931856ba apk add --no-cache nano zlib jpeg && re-enable tests 2022-12-29 02:34:29 +01:00
scito
f532dc668d indent_size = 2 for yml 2022-12-29 01:59:54 +01:00
scito
1dee86668a rename ARG run_tests to RUN_TESTS 2022-12-29 01:48:00 +01:00
scito
aa0de699fe disable tests for alpine 2022-12-29 01:43:22 +01:00
scito
7e684ff19e --build-arg run_tests=false 2022-12-29 01:35:43 +01:00
scito
b159b9e70d apk add musl-dev and apk del .build-deps 2022-12-29 01:13:48 +01:00
scito
951878d027 apk add gcc 2022-12-29 01:02:41 +01:00
scito
2a44bbfa27 use $(apk --print-arch) 2022-12-29 00:53:47 +01:00
scito
540ae7438d use $(uname -m) 2022-12-29 00:42:13 +01:00
scito
c346c085b6 apk add python3-dev py3-setuptools 2022-12-29 00:33:52 +01:00
scito
7cb3b2ac21 always apk add on alpine 2022-12-29 00:29:26 +01:00
scito
0eb5014eb0 ci_docker: add docker/setup-buildx-action@v2 2022-12-29 00:25:29 +01:00
scito
d4f5eb243e change to "$TARGETARCH" == "arm64" 2022-12-29 00:02:14 +01:00
scito
b05decc10f apk add zlib-dev jpeg-dev only for alpine linux/arm64 2022-12-28 23:58:09 +01:00
scito
21ebccbba5 apk add zlib-dev 2022-12-28 23:44:30 +01:00
scito
912825034f add zlib for alpine/arm64 docker image 2022-12-28 23:37:54 +01:00
scito
95e7d73173 mock no_args tests 2022-12-28 23:37:21 +01:00
scito
9f0872c2d0 extract from camera
- add help description
- use f-strings
- handle plural correctly
- rename methods, use otp_url instead of line
- remove importlib.util
- move cv2 imports to top
- remove unnecessary global delcarations
- group image tests
2022-12-28 22:43:40 +01:00
scito
7964c687f6 make running tests optional in docker build 2022-12-28 09:02:02 +01:00
scito
1d0b568b1e setup QEMU 2022-12-27 11:36:53 +01:00
scito
aaa7bd3da1 build also on arm64 2022-12-27 10:34:32 +01:00
scito
5ab5f84ff3 add docker label for source 2022-12-27 01:11:36 +01:00
scito
a4c4badd54 fix docker hub username 2022-12-27 00:57:49 +01:00
scito
f272c35a1f enable docker push in ci_docker 2022-12-27 00:52:14 +01:00
scito
e4e5271c0f github actions docker build no_qr_reader 2022-12-27 00:43:30 +01:00
scito
158564e79a fix to_bytes() mandatory byte_order 2022-12-27 00:18:37 +01:00
scito
672d18a5ca build docker images, run tests in docker build
- qreader_available flag
- echo commands in upgrade_deps.sh
2022-12-26 23:57:38 +01:00
scito
0490e227e1 docker image with qreader, 2nd image without qreader
- organize imports
- add qreader pytest.mark
- relaxed mode for pytest
- run tests in docker
- more tests
2022-12-26 18:31:09 +01:00
scito
2bcaa35251 organize imports 2022-12-25 11:00:15 +01:00
scito
b0b4c29e7b improve README 2022-12-24 15:50:44 +01:00
scito
e754befb52 refactor; update setup.py
more verbose logging
better error messages
2022-12-24 15:30:17 +01:00
scito
06b8efff62 add zypper and dnf for libzbar0 2022-12-24 05:31:17 +01:00
scito
5d0feacdba update README 2022-12-24 05:12:52 +01:00
scito
343520acb8 support multiple infiles 2022-12-24 04:48:12 +01:00
scito
c2d7c905ff handle wrong stdin streams 2022-12-24 04:19:43 +01:00
scito
bc329e24d5 renable libzbar 2022-12-24 03:29:14 +01:00
scito
4612ab6e7f catch zbar import error 2022-12-24 03:25:10 +01:00
scito
05db190de3 test qreader import 2022-12-24 03:12:06 +01:00
scito
0ad3c2d8ed change name to nightly tests 2022-12-24 03:06:03 +01:00
scito
31bb2909da fix ci jobs 2022-12-24 03:05:07 +01:00
scito
c1a55fb874 test pypy only in nightly builds 2022-12-24 03:02:08 +01:00
scito
82da427d1a refactor code: extract method 2022-12-24 02:56:40 +01:00
scito
af0d7ffd5d dynamic import of QReader
since this module has a dependency to zbar lib
2022-12-24 02:46:36 +01:00
scito
9a308b148f fix macOS ci 2022-12-24 02:37:16 +01:00
scito
cd07851e30 install zbar lib 2022-12-24 02:29:43 +01:00
scito
f4934192ae WIP 2022-12-24 01:59:35 +01:00
qwertyca
483fcc0163 Add the ability so provide an image file as the infile. If the file contains a QR code generated by Google Authenticator's "Transfer Accounts" function, it will be decoded directly in a single step. This is meant to help users who need to access their secrets from Google Authenticator but don't have a QR code decoder and don't want to use an online one due to security concerns. 2022-12-23 10:17:04 +01:00
51 changed files with 1200 additions and 3468 deletions

3
.envrc
View File

@@ -1,3 +0,0 @@
source_url "https://raw.githubusercontent.com/cachix/devenv/95f329d49a8a5289d31e0982652f7058a189bfca/direnvrc" "sha256-d+8cBpDfDBj41inrADaJt+bDWhOktwslgoP5YiGJ1v0="
use devenv

View File

@@ -8,4 +8,4 @@ updates:
- package-ecosystem: "pip" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "daily"
interval: "weekly"

View File

@@ -2,15 +2,10 @@ name: tests
# https://docs.github.com/de/actions/using-workflows/workflow-syntax-for-github-actions
# https://docs.github.com/en/actions/using-workflows
# https://docs.github.com/en/actions/learn-github-actions/contexts
# https://docs.github.com/en/actions/learn-github-actions/expressions
on:
push:
paths-ignore:
- 'docs/**'
- '**.md'
# pull_request:
pull_request:
schedule:
# Run daily on default branch
- cron: '37 3 * * *'
@@ -20,19 +15,18 @@ jobs:
strategy:
matrix:
python-version: ["3.13", "3.12", "3.11", "3.10", "3.9", "3.8"]
python-version: ["3.x", "3.11", "3.10", "3.9", "3.8", "3.7"]
platform: [ubuntu-latest, macos-latest, windows-latest]
# exclude:
runs-on: ${{ matrix.platform }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
check-latest: ${{ github.event_name == 'schedule' }}
- name: Install zbar shared lib for QReader (Linux)
if: runner.os == 'Linux'
run: |
@@ -52,6 +46,7 @@ jobs:
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=200 --statistics
if: matrix.python-version != '3.7'
- name: Type checking with mypy
run: |
mypy --install-types --non-interactive src/*.py tests/*.py
@@ -59,7 +54,8 @@ jobs:
if: matrix.python-version == '3.x' && matrix.platform == 'ubuntu-latest'
- name: Test with pytest
run: pytest
if: (matrix.python-version != '3.x' || matrix.platform != 'ubuntu-latest') && (matrix.python-version != '3.10' && matrix.platform != 'macos-latest')
if: (matrix.python-version != '3.x' || matrix.platform != 'ubuntu-latest')
# && matrix.platform != 'macos-latest'
- name: Test with pytest (with code coverage)
run: pytest --cov=extract_otp_secrets_test --junitxml=pytest.xml --cov-report=term-missing | tee pytest-coverage.txt
if: matrix.python-version == '3.x' && matrix.platform == 'ubuntu-latest'
@@ -69,7 +65,4 @@ jobs:
with:
pytest-coverage-path: ./pytest-coverage.txt
junitxml-path: ./pytest.xml
if: |
false && matrix.python-version == '3.x' && matrix.platform == 'ubuntu-latest'
&& !contains(github.ref, 'refs/tags/')
if: matrix.python-version == '3.x' && matrix.platform == 'ubuntu-latest'

View File

@@ -2,8 +2,6 @@ name: docker
# https://docs.github.com/de/actions/using-workflows/workflow-syntax-for-github-actions
# https://docs.github.com/en/actions/using-workflows
# https://docs.github.com/en/actions/learn-github-actions/contexts
# https://docs.github.com/en/actions/learn-github-actions/expressions
# How to setup: https://event-driven.io/en/how_to_buid_and_push_docker_image_with_github_actions/
# How to run: https://aschmelyun.com/blog/using-docker-run-inside-of-github-actions/
@@ -11,28 +9,22 @@ name: docker
on:
# run it on push to the default repository branch
push:
paths-ignore:
- 'docs/**'
- '**.md'
tags-ignore:
- '**'
# branches is needed if tags-ignore is used
branches:
- '**'
branches: [master]
schedule:
# Run weekly on default branch
- cron: '47 3 * * 6'
jobs:
build-and-push-docker-debian-image:
name: Build Docker Bookworm image and push to repositories
# define job to build and publish docker image
build-and-push-docker-image:
name: Build Docker image and push to repositories
# run only when code is compiling and tests are passing
runs-on: ubuntu-latest
# steps to perform in job
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v3
# avoid building if there are testing errors
- name: Run smoke test
@@ -44,216 +36,60 @@ jobs:
pytest
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
uses: docker/setup-qemu-action@v2
# setup Docker build action
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
# Workaround for failing builds: https://github.com/docker/build-push-action/issues/761#issuecomment-1383822381
# TODO remove workaround when fixed
with:
driver-opts: |
image=moby/buildkit:v0.10.6
uses: docker/setup-buildx-action@v2
- name: Login to DockerHub
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Github Packages
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GHCR_IO_TOKEN }}
- name: "Build image and push to Docker Hub and GitHub Container Registry"
id: docker_build_qr_reader_latest
uses: docker/build-push-action@v5
with:
platforms: linux/amd64,linux/arm64
# relative path to the place where source code with Dockerfile is located
# TODO file:, move to docker/
context: .
file: docker/Dockerfile
# builder: ${{ steps.buildx.outputs.name }}
# Note: tags has to be all lower-case
build-args: |
BASE_IMAGE=python:3.13-slim-bookworm
pull: true
tags: |
scit0/extract_otp_secrets:latest
scit0/extract_otp_secrets:bookworm
ghcr.io/scito/extract_otp_secrets:latest
ghcr.io/scito/extract_otp_secrets:bookworm
# build on feature branches, push only on master branch
push: ${{ github.ref == 'refs/heads/master' && github.secret_source == 'Actions'}}
- name: Image digest
# TODO upload digests to assets
run: |
echo "extract_otp_secrets digests: ${{ steps.docker_build_qr_reader_latest.outputs.digest }}"
echo "${{ steps.docker_build_qr_reader_latest.outputs.digest }}" > digests.txt
- name: Save docker digests as artifacts
if: github.ref == 'refs/heads/master'
uses: actions/upload-artifact@v4
with:
name: debian_digests
path: digests.txt
build-and-push-docker-alpine-image:
name: Build Docker Alpine image and push to repositories
# run only when code is compiling and tests are passing
runs-on: ubuntu-latest
# steps to perform in job
steps:
- name: Checkout code
uses: actions/checkout@v4
# avoid building if there are testing errors
- name: Run smoke test
run: |
sudo apt-get install -y libzbar0
python -m pip install --upgrade pip
pip install -U -r requirements-dev.txt
pip install -U .
pytest
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# setup Docker build action
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Github Packages
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GHCR_IO_TOKEN }}
- name: "only_txt: Build image and push to Docker Hub and GitHub Container Registry"
- name: "no_qr_reader: Build image and push to Docker Hub and GitHub Container Registry"
id: docker_build_only_txt
uses: docker/build-push-action@v5
uses: docker/build-push-action@v2
with:
# relative path to the place where source code with Dockerfile is located
platforms: linux/amd64,linux/arm64
context: .
file: docker/Dockerfile_only_txt
file: Dockerfile_only_txt
# builder: ${{ steps.buildx.outputs.name }}
# Note: tags has to be all lower-case
pull: true
tags: |
scit0/extract_otp_secrets:only-txt
scit0/extract_otp_secrets:alpine
ghcr.io/scito/extract_otp_secrets:only-txt
ghcr.io/scito/extract_otp_secrets:alpine
scit0/extract_otp_secrets:latest-only-txt
ghcr.io/scito/extract_otp_secrets:latest-only-txt
# build on feature branches, push only on master branch
push: ${{ github.ref == 'refs/heads/master' && github.secret_source == 'Actions'}}
push: ${{ github.ref == 'refs/heads/master' }}
build-args: |
RUN_TESTS=true
- name: Image digest
# TODO upload digests to assets
run: |
echo "extract_otp_secrets:only-txt digests: ${{ steps.docker_build_only_txt.outputs.digest }}"
echo "${{ steps.docker_build_qr_reader_latest.outputs.digest }}" > digests.txt
- name: Save docker digests as artifacts
if: github.ref == 'refs/heads/master'
uses: actions/upload-artifact@v4
with:
name: alpine_digests
path: digests.txt
build-and-push-docker-bullseye-image:
name: Build Docker Bullseye image (for PyInstsaller) and push to repositories
# run only when code is compiling and tests are passing
runs-on: ubuntu-latest
# steps to perform in job
steps:
- name: Checkout code
uses: actions/checkout@v4
# avoid building if there are testing errors
- name: Run smoke test
run: |
sudo apt-get install -y libzbar0
python -m pip install --upgrade pip
pip install -U -r requirements-dev.txt
pip install -U .
pytest
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# setup Docker build action
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
# Workaround for failing builds: https://github.com/docker/build-push-action/issues/761#issuecomment-1383822381
# TODO remove workaround when fixed
with:
driver-opts: |
image=moby/buildkit:v0.10.6
- name: Login to DockerHub
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Github Packages
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GHCR_IO_TOKEN }}
- name: "Build image from Bullseye and push to GitHub Container Registry"
id: docker_build_bullseye
if: github.ref == 'refs/heads/master'
uses: docker/build-push-action@v5
- name: "qr_reader: Build image and push to Docker Hub and GitHub Container Registry"
id: docker_build_qr_reader
uses: docker/build-push-action@v2
with:
platforms: linux/amd64,linux/arm64
# relative path to the place where source code with Dockerfile is located
context: .
file: docker/Dockerfile
# builder: ${{ steps.buildx.outputs.name }}
build-args: |
BASE_IMAGE=python:3.13-slim-bullseye
# Note: tags has to be all lower-case
pull: true
tags: |
scit0/extract_otp_secrets:bullseye
push: ${{ github.secret_source == 'Actions' }}
scit0/extract_otp_secrets:latest
ghcr.io/scito/extract_otp_secrets:latest
# build on feature branches, push only on master branch
push: ${{ github.ref == 'refs/heads/master' }}
- name: Image digest
# TODO upload digests to assets
run: |
echo "extract_otp_secrets digests: ${{ steps.docker_build_qr_reader_latest.outputs.digest }}"
echo "${{ steps.docker_build_qr_reader_latest.outputs.digest }}" > digests.txt
- name: Save docker digests as artifacts
if: github.ref == 'refs/heads/master'
uses: actions/upload-artifact@v4
with:
name: bullseye_digests
path: digests.txt
echo "extract_otp_secrets: ${{ steps.docker_build_qr_reader.outputs.digest }}"
echo "extract_otp_secrets_only_txt: ${{ steps.docker_build_only_txt.outputs.digest }}"

View File

@@ -1,412 +0,0 @@
name: release
# https://data-dive.com/multi-os-deployment-in-cloud-using-pyinstaller-and-github-actions
# https://github.com/actions/upload-artifact
# https://github.com/actions/download-artifact
# https://github.com/docker/metadata-action
# https://github.com/marketplace/actions/generate-release-hashes
# https://github.com/oleksis/pyinstaller-manylinux
# https://github.com/pypa/manylinux
# https://github.com/batonogov/docker-pyinstaller
# https://docs.github.com/de/actions/using-workflows/workflow-syntax-for-github-actions
# https://docs.github.com/en/actions/using-workflows
# https://docs.github.com/en/actions/learn-github-actions/contexts
# https://docs.github.com/en/actions/learn-github-actions/expressions
# https://docs.github.com/en/rest/releases/releases
# https://peps.python.org/pep-0440/
# https://semver.org/
# macOS:
# https://pyinstaller.org/en/stable/usage.html#building-macos-app-bundles
# https://github.com/pyinstaller/pyinstaller/wiki/Recipe-OSX-Code-Signing
# Build matrix:
# - Linux x86_64 glibc 2.35: ubuntu-latest
# - Linux x86_64 glibc 2.31: extract_otp_secrets:bullseye
# - Linux x86_64 glibc 2.36: extract_otp_secrets:bookworm
# - Windows x86_64: windows-latest
# - MacOS x86_64: macos-11
# - Linux x86_64 glibc 2.28: extract_otp_secrets:buster
# - Linux aarch64 glibc 2.28: extract_otp_secrets:buster
# - MacOS universal2: macos-11
# - Windows arm64: [buildx + https://github.com/batonogov/docker-pyinstaller]
on:
push:
tags:
- 'v*' # Push events to matching v*, i.e. v1.0, v20.15.10
pull_request:
schedule:
# Run weekly on default branch
- cron: '47 4 * * 6'
jobs:
create-release:
name: Create Release
runs-on: ubuntu-latest
outputs:
date: ${{ steps.meta.outputs.date }}
version: ${{ steps.meta.outputs.version }}
inline_version: ${{ steps.meta.outputs.inline_version }}
tag_name: ${{ steps.meta.outputs.tag_name }}
tag_message: ${{ steps.meta.outputs.tag_message }}
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set meta data
id: meta
# Writing to env with >> $GITHUB_ENV is an alternative
run: |
echo "date=$(TZ=Europe/Zurich date +'%d.%m.%Y')" >> $GITHUB_OUTPUT
echo "version=${TAG_NAME/v/}" >> $GITHUB_OUTPUT
echo "inline_version=${TAG_NAME/v/_}" >> $GITHUB_OUTPUT
echo "tag_name=${{ github.ref_name }}" >> $GITHUB_OUTPUT
echo "tag_message=$(git tag -l --format='%(contents:subject)' ${{ github.ref_name }})" >> $GITHUB_OUTPUT
env:
TAG_NAME: ${{ github.ref_name }}
PYTHONHASHSEED: 31
- name: Create Release
id: create_release
if: startsWith(github.ref, 'refs/tags/v')
# https://docs.github.com/en/rest/releases/releases?apiVersion=2022-11-28#create-a-release
run: |
echo "date: ${{ steps.meta.outputs.date }}"
echo "version: ${{ steps.meta.outputs.version }}"
echo "inline_version: ${{ steps.meta.outputs.inline_version }}"
echo "tag_name: ${{ steps.meta.outputs.tag_name }}"
echo "tag_message: ${{ steps.meta.outputs.tag_message }}"
response=$(curl \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}"\
-H "X-GitHub-Api-Version: 2022-11-28" \
https://api.github.com/repos/scito/extract_otp_secrets/releases \
--silent \
--show-error \
-d '{"tag_name":"${{ github.ref }}","target_commitish":"master","name":"${{ steps.meta.outputs.version }} - ${{ steps.meta.outputs.date }}","body":"${{ steps.meta.outputs.tag_message }}\n\n## Executables\n\nDownload the executable for your platform and execute it, see [README.md](https://github.com/scito/extract_otp_secrets#readme)\n\n | Executable | Description |\n | --- | --- |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_linux_x86_64 | Linux x86_64/amd64 (glibc >= 2.31) |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_linux_arm64 | Linux arm64 (glibc >= 2.31) |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_win_x86_64.exe | Windows x86_64/amd64/x64 |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_win_arm64.exe | N/A |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_macos_x86_64.dmg | N/A, see [README.md](https://github.com/scito/extract_otp_secrets#readme) |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_macos_x86_64.pkg | N/A, see [README.md](https://github.com/scito/extract_otp_secrets#readme) |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_macos_x86_64 | MacOS x86_64/amd64 (bare executable, see [README.md](https://github.com/scito/extract_otp_secrets#readme); optional libzbar must be installed manually, see [README.md](https://github.com/scito/extract_otp_secrets#readme)) |\n | extract_otp_secrets${{ steps.meta.outputs.inline_version }}_macos_arm64 | N/A |\n","draft":true,"prerelease":false,"generate_release_notes":true}')
echo upload_url=$(jq '.upload_url' <<< "$response") >> $GITHUB_OUTPUT
echo $(jq -r '.upload_url' <<< "$response") > release_url.txt
echo $(jq -r '.id' <<< "$response") > release_id.txt
- name: Save Release URL File for publish
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/upload-artifact@v4
with:
name: release_url
path: release_url.txt
- name: Save asset upload id for publish
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/upload-artifact@v4
with:
name: release_id
path: release_id.txt
build-linux-executable-in-docker:
name: Build ${{ matrix.PLATFORM }} release in docker container
# run only when code is compiling and tests are passing
runs-on: ubuntu-latest
needs: create-release
strategy:
matrix:
include:
- PLATFORM: linux/amd64
EXE: extract_otp_secrets_linux_x86_64
ASSET_NAME: extract_otp_secrets${{ needs.create-release.outputs.inline_version }}_linux_x86_64
- PLATFORM: linux/arm64
EXE: extract_otp_secrets_linux_arm64
ASSET_NAME: extract_otp_secrets${{ needs.create-release.outputs.inline_version }}_linux_arm64
# steps to perform in job
steps:
- name: Checkout code
uses: actions/checkout@v4
# avoid building if there are testing errors
- name: Run smoke test
run: |
sudo apt-get install -y libzbar0
python -m pip install --upgrade pip
pip install -U -r requirements-dev.txt
pip install -U .
pytest
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
# setup Docker build action
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v3
# Workaround for failing builds: https://github.com/docker/build-push-action/issues/761#issuecomment-1383822381
# TODO remove workaround when fixed
with:
driver-opts: |
image=moby/buildkit:v0.10.6
- name: Login to DockerHub
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Github Packages
uses: docker/login-action@v3
if: github.secret_source == 'Actions'
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GHCR_IO_TOKEN }}
- name: Image digest
# TODO upload digests to assets
run: |
echo "extract_otp_secrets: ${{ steps.docker_build_bullseye.outputs.digest }}"
# TODO use local docker image https://stackoverflow.com/a/61155718/1663871
# https://github.com/multiarch/qemu-user-static
# https://hub.docker.com/r/multiarch/qemu-user-static/
- name: Run Pyinstaller in container for ${{ matrix.EXE }}
run: |
docker run --pull always --rm --privileged multiarch/qemu-user-static --reset -p yes
docker run --platform ${{ matrix.PLATFORM }} --pull always --entrypoint /bin/bash --rm -v "$(pwd)":/files -w /files scit0/extract_otp_secrets:bullseye -c 'apt-get update && apt-get -y install binutils && pip install -U -r /files/requirements.txt && pip install pyinstaller && PYTHONHASHSEED=31 && pyinstaller -y --add-data /usr/local/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile --name ${{ matrix.EXE }} --distpath /files/dist/ /files/src/extract_otp_secrets.py'
- name: Smoke tests linux/amd64
if: matrix.PLATFORM == 'linux/amd64'
run: |
dist/${{ matrix.EXE }} -V
dist/${{ matrix.EXE }} -h
dist/${{ matrix.EXE }} --debug
dist/${{ matrix.EXE }} example_export.png
dist/${{ matrix.EXE }} - < example_export.txt
dist/${{ matrix.EXE }} --qr ZBAR example_export.png
dist/${{ matrix.EXE }} --qr QREADER example_export.png
dist/${{ matrix.EXE }} --qr QREADER_DEEP example_export.png
dist/${{ matrix.EXE }} --qr CV2 example_export.png
dist/${{ matrix.EXE }} --qr CV2_WECHAT example_export.png
- name: Smoke tests linux/arm64
if: matrix.PLATFORM == 'linux/arm64'
run: |
docker run --platform ${{ matrix.PLATFORM }} --pull always --entrypoint /bin/bash --rm -v "$(pwd)":/files -w /files scit0/extract_otp_secrets -c 'dist/${{ matrix.EXE }} -V && dist/${{ matrix.EXE }} -h && dist/${{ matrix.EXE }} example_export.png && dist/${{ matrix.EXE }} - < example_export.txt && dist/${{ matrix.EXE }} --qr ZBAR example_export.png && dist/${{ matrix.EXE }} --qr QREADER example_export.png && dist/${{ matrix.EXE }} --qr QREADER_DEEP example_export.png && dist/${{ matrix.EXE }} --qr CV2 example_export.png && dist/${{ matrix.EXE }} --qr CV2_WECHAT example_export.png'
- name: Load Release URL File from release job
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/download-artifact@v4
with:
name: release_url
- name: Display structure of files
run: ls -R
- name: Upload EXE to artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.EXE }}
path: dist/${{ matrix.EXE }}
- name: Upload Release Asset
id: upload-release-asset
if: startsWith(github.ref, 'refs/tags/v')
run: |
response=$(curl \
-X POST \
-H "Accept: application/vnd.github+json" \
-H "Content-Type: application/x-executable" \
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}"\
-H "X-GitHub-Api-Version: 2022-11-28" \
--silent \
--show-error \
--data-binary @dist/${{ matrix.EXE }} \
$(cat release_url.txt)=${{ matrix.ASSET_NAME }})
build-native-executables:
name: Build native packages
needs: create-release
runs-on: ${{ matrix.os }}
strategy:
matrix:
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#choosing-github-hosted-runners
include:
- os: windows-latest
TARGET: windows
# TODO add --icon
# TODO add --manifest
# TODO find more elegant solution for pyzbar\libiconv.dll and pyzbar\libzbar-64.dll
# Files of Visual C++ 2013 Redistributable Package: https://support.microsoft.com/en-us/topic/update-for-visual-c-2013-redistributable-package-d8ccd6a5-4e26-c290-517b-8da6cfdf4f10
EXE: extract_otp_secrets.exe
ASSET_NAME: extract_otp_secrets${{ needs.create-release.outputs.inline_version }}_win_x86_64.exe
ASSET_MIME: application/vnd.microsoft.portable-executable
UPLOAD: true
CMD_BUILD: |
pyinstaller -y --add-data "$($Env:pythonLocation)\__yolo_v3_qr_detector:__yolo_v3_qr_detector" --add-binary "$($Env:pythonLocation)\Lib\site-packages\pyzbar\libiconv.dll:pyzbar" --add-binary "$($Env:pythonLocation)\Lib\site-packages\pyzbar\libzbar-64.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\msvcr120.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\msvcp120.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\vcamp120.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\vcomp120.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\vccorlib120.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120u.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120chs.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120cht.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120deu.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120enu.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120esn.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120fra.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120ita.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120jpn.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120kor.dll:pyzbar" --add-binary "$($Env:WinDir)\system32\mfc120rus.dll:pyzbar" --onefile --version-file build\win_file_version_info.txt --name extract_otp_secrets.exe src\extract_otp_secrets.py
- os: ubuntu-latest
TARGET: linux
EXE: extract_otp_secrets_ubuntu
ASSET_NAME: extract_otp_secrets${{ needs.create-release.outputs.inline_version }}_linux_x86_64_ubuntu_latest
ASSET_MIME: application/x-executable
UPLOAD: false
CMD_BUILD: |
pyinstaller -y --add-data $pythonLocation/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile --name extract_otp_secrets_ubuntu src/extract_otp_secrets.py
# TODO temp disable macos releases to due
# FileNotFoundError: Icon input file /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/PyInstaller/bootloader/images/icon-windowed.icns not found
# - os: macos-12
# TARGET: macos
# # https://pyinstaller.org/en/stable/spec-files.html#spec-file-options-for-a-macos-bundle
# EXE: extract_otp_secrets
# ASSET_NAME: extract_otp_secrets${{ needs.create-release.outputs.inline_version }}_macos_x86_64
# DMG: extract_otp_secrets.dmg
# ASSET_NAME_DMG: extract_otp_secrets${{ needs.create-release.outputs.inline_version }}_macos_x86_64.dmg
# ASSET_MIME: application/octet-stream
# UPLOAD: true
# CMD_BUILD: |
# VERSION_STR=$(setuptools-git-versioning) COPYRIGHT_YEARS='2020-2023' envsubst < installer/extract_otp_secrets_macos_template.spec > extract_otp_secrets_macos.spec
# pyinstaller -y extract_otp_secrets_macos.spec
# installer/build_dmg.sh
steps:
- name: Output path
if: runner.os == 'Windows'
run: echo "$($Env:Path)"
- name: List Windir
if: runner.os == 'Windows'
run: ls "$($Env:WinDir)\system32"
- uses: actions/checkout@v4
- name: Set macos macos_python_path
# TODO use variable for Python version
run: echo "macos_python_path=/Library/Frameworks/Python.framework/Versions/3.13" >> $GITHUB_ENV
- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: 3.13
check-latest: true
- name: Install zbar shared lib for QReader (Linux)
if: runner.os == 'Linux'
run: |
sudo apt-get install -y libzbar0
- name: Install zbar shared lib for QReader (macOS)
if: runner.os == 'macOS'
run: |
brew install zbar create-dmg
- name: Install dependencies
# TODO fix --use-pep517
run: |
python -m pip install --upgrade pip
pip install -U -r requirements-dev.txt
pip install -U .
- name: Create Windows win_file_version_info.txt
shell: bash
run: |
mkdir -p build/
VERSION_STR=$(setuptools-git-versioning) VERSION_MAJOR=$(cut -d '.' -f 1 <<< "$(setuptools-git-versioning)") VERSION_MINOR=$(cut -d '.' -f 2 <<< "$(setuptools-git-versioning)") VERSION_PATCH=$(echo $(cut -d '.' -f 3 <<< "$(setuptools-git-versioning)") | sed -E -n "s/^([0-9]+).*/\1/p") VERSION_BUILD=$(echo $(cut -d '.' -f 3 <<< "$(setuptools-git-versioning)") | sed -E -n -e"s/^[0-9]+.+/99/p")$(($(git rev-list --count $(git tag | sort -V -r | sed '1!d')..HEAD))) COPYRIGHT_YEARS='2020-2023' envsubst < installer/win_file_version_info_template.txt > build/win_file_version_info.txt
- name: Build with pyinstaller for ${{ matrix.TARGET }}
env:
# Reproducible build: https://pyinstaller.org/en/stable/advanced-topics.html#creating-a-reproducible-build
PYTHONHASHSEED: 31
run: ${{ matrix.CMD_BUILD }}
- name: Smoke tests for generated exe (general)
run: |
dist/${{ matrix.EXE }} -V
dist/${{ matrix.EXE }} -h
dist/${{ matrix.EXE }} --debug
dist/${{ matrix.EXE }} example_export.png
dist/${{ matrix.EXE }} --qr ZBAR example_export.png
dist/${{ matrix.EXE }} --qr QREADER example_export.png
dist/${{ matrix.EXE }} --qr QREADER_DEEP example_export.png
dist/${{ matrix.EXE }} --qr CV2 example_export.png
dist/${{ matrix.EXE }} --qr CV2_WECHAT example_export.png
- name: Smoke tests for generated exe (stdin)
if: runner.os != 'Windows'
run: |
dist/${{ matrix.EXE }} - < example_export.txt
- name: Load Release URL File from release job
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/download-artifact@v4
with:
name: release_url
- name: Load Release Id File from release job
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/download-artifact@v4
with:
name: release_id
- name: Display structure of files
run: ls -R
- name: Set meta data
id: meta
if: startsWith(github.ref, 'refs/tags/v')
shell: bash
run: |
echo "release_id=$(cat release_id.txt)" >> $GITHUB_OUTPUT
echo "upload_url=https://uploads.github.com/repos/scito/extract_otp_secrets/releases/$(cat release_id.txt)/assets?name=" >> $GITHUB_OUTPUT
- name: Upload EXE to artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.EXE }}
path: dist/${{ matrix.EXE }}
- name: Upload DMG to artifacts
uses: actions/upload-artifact@v4
if: runner.os == 'macOS'
with:
name: ${{ matrix.DMG }}
path: dist/${{ matrix.DMG }}
- name: Upload Release Asset
id: upload-release-asset
if: matrix.UPLOAD && startsWith(github.ref, 'refs/tags/v')
run: |
curl -X POST -H "Accept: application/vnd.github+json" -H "Content-Type: ${{ matrix.ASSET_MIME }}" -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" -H "X-GitHub-Api-Version: 2022-11-28" --show-error --data-binary @dist/${{ matrix.EXE }} ${{ steps.meta.outputs.upload_url }}=${{ matrix.ASSET_NAME }}
- name: Upload Release Asset DMG (macOS)
id: upload-release-asset-dmg
if: false && matrix.UPLOAD && startsWith(github.ref, 'refs/tags/v') && runner.os == 'macOS'
run: |
curl -X POST -H "Accept: application/vnd.github+json" -H "Content-Type: ${{ matrix.ASSET_MIME }}" -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" -H "X-GitHub-Api-Version: 2022-11-28" --show-error --data-binary @dist/${{ matrix.DMG }} ${{ steps.meta.outputs.upload_url }}=${{ matrix.ASSET_NAME_DMG }}
upload-hashes:
name: Upload hashes
if: startsWith(github.ref, 'refs/tags/v')
needs:
- build-linux-executable-in-docker
- build-native-executables
runs-on: ubuntu-latest
steps:
- name: Load Release Id File from release job
uses: actions/download-artifact@v4
with:
name: release_id
- name: Set meta data
id: meta
run: |
echo "release_id=$(cat release_id.txt)" >> $GITHUB_OUTPUT
echo "upload_url=https://uploads.github.com/repos/scito/extract_otp_secrets/releases/$(cat release_id.txt)/assets?name=" >> $GITHUB_OUTPUT
- name: Calculate and upload hashes from assets
run: |
GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }}
for asset_url in $(curl \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_TOKEN"\
-H "X-GitHub-Api-Version: 2022-11-28" \
--silent \
--show-error \
https://api.github.com/repos/scito/extract_otp_secrets/releases/${{ steps.meta.outputs.release_id }}/assets |
jq -r '.[].url'); do
echo "Download $asset_url"
name=$(curl \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer $GITHUB_TOKEN"\
-H "X-GitHub-Api-Version: 2022-11-28" \
--output-dir assets \
-L \
$asset_url |
jq -r '.name')
curl \
-H "Accept: application/octet-stream" \
-H "Authorization: Bearer $GITHUB_TOKEN"\
-H "X-GitHub-Api-Version: 2022-11-28" \
--create-dirs \
--output-dir assets \
-L \
-o $name \
$asset_url
done
(cd assets/ && sha256sum * > ../sha256_hashes.txt)
curl -X POST -H "Accept: application/vnd.github+json" -H "Content-Type: text/plain" -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" -H "X-GitHub-Api-Version: 2022-11-28" --show-error --data-binary @sha256_hashes.txt ${{ steps.meta.outputs.upload_url }}=sha256_hashes.txt
(cd assets/ && sha512sum * > ../sha512_hashes.txt)
curl -X POST -H "Accept: application/vnd.github+json" -H "Content-Type: text/plain" -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" -H "X-GitHub-Api-Version: 2022-11-28" --show-error --data-binary @sha512_hashes.txt ${{ steps.meta.outputs.upload_url }}=sha512_hashes.txt

View File

@@ -42,26 +42,26 @@ jobs:
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v3
uses: github/codeql-action/autobuild@v2
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
# If the Autobuild fails above, remove it and uncomment the following three lines.
# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
# - run: |
@@ -69,6 +69,6 @@ jobs:
# ./location_of_script_within_repo/buildscript.sh
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"

25
.gitignore vendored
View File

@@ -19,28 +19,3 @@ dist/
*.xml
pytest-coverage.txt
tests/reports/
dist_*/
file_version_info_python.txt
file_version_info_explorer.txt
file_version_info.txt
assets/*
extract_otp_secrets_linux_arm64.spec
extract_otp_secrets_linux_x86_64_bullseye.spec
extract_otp_secrets_linux_x86_64_bookworm.spec
extract_otp_secrets_linux_x86_64.spec
extract_otp_secrets.spec
test.txt
extract_otp_secrets.build/
extract_otp_secrets.dist/
extract_otp_secrets.onefile-build/
extract_otp_secrets.bin
# Devenv
.devenv*
devenv.local.nix
# direnv
.direnv
# pre-commit
.pre-commit-config.yaml

View File

@@ -2,7 +2,7 @@
"python.testing.pytestArgs": [
"."
],
"python.testing.unittestEnabled": false,
"python.testing.unittestEnabled": true,
"python.testing.pytestEnabled": true,
"cSpell.words": [
"devbox",

View File

@@ -1,6 +1,4 @@
# --build-arg BASE_IMAGE=python:3.11-slim-buster
ARG BASE_IMAGE=python:3.13-slim-bookworm
FROM $BASE_IMAGE
FROM python:3.11-slim-bullseye
# https://docs.docker.com/engine/reference/builder/
@@ -12,25 +10,23 @@ FROM $BASE_IMAGE
WORKDIR /extract
COPY requirements*.txt src/ run_pytest.sh pytest.ini tests/ example_*.txt example_*.png example_*.csv example*.json docker/.alias ./
COPY . .
ARG RUN_TESTS=true
RUN apt-get update && apt-get install -y --no-install-recommends \
RUN apt-get update && apt-get install -y \
libgl1 \
libglib2.0-0 \
libsm6 \
libzbar0 \
python3-tk \
&& rm -rf /var/lib/apt/lists/* \
&& pip install --no-cache-dir -U pip -r requirements.txt \
&& if [ "$RUN_TESTS" = "true" ]; then /extract/run_pytest.sh; else echo "Not running tests..."; fi \
&& echo 'test -s /extract/.alias && . /extract/.alias || true' >> ~/.bashrc
&& pip install --no-cache-dir -U -r \
requirements.txt \
&& if [ "$RUN_TESTS" = "true" ]; then /extract/run_pytest.sh; else echo "Not running tests..."; fi
WORKDIR /files
ENTRYPOINT ["python", "/extract/extract_otp_secrets.py"]
ENTRYPOINT ["python", "/extract/src/extract_otp_secrets.py"]
LABEL org.opencontainers.image.source https://github.com/scito/extract_otp_secrets
LABEL org.opencontainers.image.license GPL-3.0+
LABEL maintainer="Scito https://scito.ch, https://github.com/scito"

View File

@@ -1,19 +1,16 @@
ARG BASE_IMAGE=python:3.13-alpine
FROM $BASE_IMAGE
FROM python:3.11-alpine
# https://docs.docker.com/engine/reference/builder/
# For debugging
# docker run --rm -v "$(pwd)":/files:ro extract_otp_secrets_only_txt
# docker build . -t extract_otp_secrets_only_txt -f Dockerfile_only_txt --pull --build-arg RUN_TESTS=false
# docker run --entrypoint /bin/ash -it --rm -v "$(pwd)":/files:ro extract_otp_secrets_only_txt -l
# docker run --entrypoint /extract/run_pytest.sh --rm -v "$(pwd)":/files:ro extract_otp_secrets_only_txt extract_otp_secrets_test.py -k "not qreader" --relaxed -vvv -s
# https://github.com/pypa/manylinux/blob/main/docker/Dockerfile
# docker run --entrypoint /bin/sh -it --rm -v "$(pwd)":/files:ro extract_otp_secrets_only_txt
# docker run --entrypoint /extract/run_pytest.sh --rm -v "$(pwd)":/files:ro extract_otp_secrets_only_txt tests/extract_otp_secrets_test.py -k "not qreader" --relaxed -vvv -s
WORKDIR /extract
COPY requirements*.txt src/ run_pytest.sh pytest.ini tests/ example_*.txt example_*.png example_*.csv example*.json docker/.alias ./
COPY . .
ARG RUN_TESTS=true
@@ -30,19 +27,16 @@ RUN apk add --no-cache \
zlib-dev \
; fi \
&& pip install --no-cache-dir -U \
pip \
colorama \
Pillow \
protobuf \
qrcode \
&& if [[ "$(apk --print-arch)" == "aarch64" ]]; then apk del .build-deps; fi \
&& if [[ "$RUN_TESTS" == "true" ]]; then /extract/run_pytest.sh extract_otp_secrets_test.py -k "not qreader" --relaxed; else echo "Not running tests..."; fi \
&& echo 'test -s /extract/.alias && . /extract/.alias || true' >> ~/.profile
&& if [[ "$RUN_TESTS" == "true" ]]; then /extract/run_pytest.sh tests/extract_otp_secrets_test.py -k "not qreader" --relaxed; else echo "Not running tests..."; fi
WORKDIR /files
ENTRYPOINT ["python", "/extract/extract_otp_secrets.py"]
ENTRYPOINT ["python", "/extract/src/extract_otp_secrets.py"]
LABEL org.opencontainers.image.source https://github.com/scito/extract_otp_secrets
LABEL org.opencontainers.image.license GPL-3.0+
LABEL maintainer="Scito https://scito.ch, https://github.com/scito"

27
Pipfile
View File

@@ -4,29 +4,24 @@ verify_ssl = true
name = "pypi"
[packages]
colorama = ">=0.4.6"
opencv-contrib-python = "*"
# for macOS: opencv-contrib-python = "<=4.7.0"
pillow = "*"
pyzbar = "*"
protobuf = "*"
qrcode = "*"
qreader = "<2.0.0"
pillow = "*"
qreader = "*"
opencv-contrib-python = "*"
colorama = ">=0.4.6"
# for macOS: opencv-contrib-python = "<=4.7.0"
# for PYTHON <= 3.7: typing_extensions = "*"
[dev-packages]
build = "*"
flake8 = "*"
gfm-toc = "*"
mypy = "*"
mypy-protobuf = "*"
nuitka = "*"
pylint = "*"
pytest = "*"
pytest-cov = "*"
pytest-mock = "*"
setuptools-git-versioning = "*"
types-protobuf = "*"
pytest-cov = "*"
wheel = "*"
flake8 = "*"
pylint = "*"
mypy = "*"
types-protobuf = "*"
[requires]
python_version = "3.11"

914
Pipfile.lock generated

File diff suppressed because it is too large Load Diff

468
README.md
View File

@@ -1,202 +1,31 @@
# Extract TOTP/HOTP secrets from QR codes exported by two-factor authentication apps
# Extract secrets from QR codes exported by two-factor authentication apps
[![CI tests](https://github.com/scito/extract_otp_secrets/actions/workflows/ci.yml/badge.svg)](https://github.com/scito/extract_otp_secrets/actions/workflows/ci.yml)
![coverage](https://img.shields.io/badge/coverage-96%25-brightgreen)
[![CI docker](https://github.com/scito/extract_otp_secrets/actions/workflows/ci_docker.yml/badge.svg)](https://github.com/scito/extract_otp_secrets/actions/workflows/ci_docker.yml)
![coverage](https://img.shields.io/badge/coverage-94%25-brightgreen)
[![License](https://img.shields.io/github/license/scito/extract_otp_secrets)](https://github.com/scito/extract_otp_secrets/blob/master/LICENSE)
[![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/scito/extract_otp_secrets?sort=semver)](https://github.com/scito/extract_otp_secrets/releases/latest)
![python versions](https://img.shields.io/badge/python-3.8%20%7C%203.9%20%7C%203.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)
[![Docker image](https://img.shields.io/badge/docker-image-blue)](https://hub.docker.com/repository/docker/scit0/extract_otp_secrets/general)
[![Linux](https://img.shields.io/badge/os-linux-yellow)](https://github.com/scito/extract_otp_secrets/releases/latest)
[![Windows](https://img.shields.io/badge/os-windows-yellow)](https://github.com/scito/extract_otp_secrets/releases/latest)
[![MacOS](https://img.shields.io/badge/os-macos-yellow)](https://github.com/scito/extract_otp_secrets/releases/latest)
[![Download executable](https://img.shields.io/badge/download-exe-blue)](https://github.com/scito/extract_otp_secrets/releases/latest)
[![Stand With Ukraine](https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/badges/StandWithUkraine.svg)](https://stand-with-ukraine.pp.ua)
<!-- ![PyPI - Python Version](https://img.shields.io/pypi/pyversions/protobuf)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/protobuf)
[![GitHub Pipenv locked Python version](https://img.shields.io/github/pipenv/locked/python-version/scito/extract_otp_secrets)](https://github.com/scito/extract_otp_secrets/blob/master/Pipfile.lock)
![protobuf version](https://img.shields.io/badge/protobuf-5.29.0-informational)-->
<!-- [![Github all releases](https://img.shields.io/github/downloads/scito/extract_otp_secrets/total.svg)](https://GitHub.com/scito/extract_otp_secrets/releases/) -->
![protobuf version](https://img.shields.io/badge/protobuf-4.21.12-informational)
[![License](https://img.shields.io/github/license/scito/extract_otp_secrets)](https://github.com/scito/extract_otp_secrets/blob/master/LICENSE)
[![GitHub tag (latest SemVer)](https://img.shields.io/github/v/tag/scito/extract_otp_secrets?sort=semver&label=version)](https://github.com/scito/extract_otp_secrets/tags)
[![Stand With Ukraine](https://raw.githubusercontent.com/vshymanskyy/StandWithUkraine/main/badges/StandWithUkraine.svg)](https://stand-with-ukraine.pp.ua)
---
The Python script `extract_otp_secrets.py` extracts one time password (OTP) secrets from QR codes exported by two-factor authentication (2FA) apps such as "Google Authenticator".
The exported QR codes from authentication apps can be read in three ways:
The export QR codes from authentication apps can be provided in three ways to this script:
1. Capture the QR codes with the system camera using a GUI, 🆕
2. Read image files containing the QR codes, and 🆕
3. Read text files containing the QR code data generated by third-party QR readers.
1. Capture from the system camera in a GUI,
2. Image files containing the QR codes, and
3. Text files containing the QR code data generated by QR readers.
The secrets can be exported to JSON or CSV, or printed as QR codes to console or saved as PNG.
The secret and otp values can be exported to json or csv files, as well as printed or saved to PNG images.
**This project/script was renamed from `extract_otp_secret_keys` to `extract_otp_secrets`.**
This script/project was renamed from extract_otp_secret_keys to extract_otp_secrets in version 2.0.0.
<details>
<summary>Table of contents</summary>
## Installation
## Table of contents
- [Table of contents](#table-of-contents)
- [Download and run binary executable (🆕 since v2.1)](#download-and-run-binary-executable--since-v21)
- [MacOS](#macos)
- [Usage](#usage)
- [Capture QR codes from camera (🆕 since version 2.0)](#capture-qr-codes-from-camera--since-version-20)
- [With builtin QR decoder from image files (🆕 since version 2.0)](#with-builtin-qr-decoder-from-image-files--since-version-20)
- [With external QR decoder app from text files](#with-external-qr-decoder-app-from-text-files)
- [Installation of Python script (recommended for developers or advanced users)](#installation-of-python-script-recommended-for-developers-or-advanced-users)
- [Installation of optional shared system libraries (recommended)](#installation-of-optional-shared-system-libraries-recommended)
- [Program help: arguments and options](#program-help-arguments-and-options)
- [Examples](#examples)
- [Printing otp secrets from text file](#printing-otp-secrets-from-text-file)
- [Printing otp secrets from image file](#printing-otp-secrets-from-image-file)
- [Writing otp secrets to csv file](#writing-otp-secrets-to-csv-file)
- [Writing otp secrets to json file](#writing-otp-secrets-to-json-file)
- [Printing otp secrets multiple files](#printing-otp-secrets-multiple-files)
- [Printing otp secrets from stdin (text)](#printing-otp-secrets-from-stdin-text)
- [Printing otp secrets from stdin (image)](#printing-otp-secrets-from-stdin-image)
- [Printing otp secrets csv to stdout](#printing-otp-secrets-csv-to-stdout)
- [Printing otp secrets csv to stdout without header line](#printing-otp-secrets-csv-to-stdout-without-header-line)
- [Reading from stdin and printing to stdout](#reading-from-stdin-and-printing-to-stdout)
- [Features](#features)
- [KeePass](#keepass)
- [How to export otp secrets from Google Authenticator app](#how-to-export-otp-secrets-from-google-authenticator-app)
- [Glossary](#glossary)
- [Alternative installation methods](#alternative-installation-methods)
- [pip using github](#pip-using-github)
- [local pip](#local-pip)
- [pipenv](#pipenv)
- [Visual Studio Code Remote - Containers / VSCode devcontainer](#visual-studio-code-remote---containers--vscode-devcontainer)
- [venv](#venv)
- [devbox 1](#devbox-1)
- [devbox 2](#devbox-2)
- [docker](#docker)
- [More docker examples](#more-docker-examples)
- [Tests](#tests)
- [PyTest](#pytest)
- [unittest](#unittest)
- [VSCode Setup](#vscode-setup)
- [Development](#development)
- [Build](#build)
- [Upgrade pip Packages](#upgrade-pip-packages)
- [Build docker images](#build-docker-images)
- [Create executables with pyinstaller](#create-executables-with-pyinstaller)
- [Full local build (bash)](#full-local-build-bash)
- [Technical background](#technical-background)
- [References](#references)
- [Issues](#issues)
- [Problems and Troubleshooting](#problems-and-troubleshooting)
- [Windows error message](#windows-error-message)
- [Related projects](#related-projects)
</details>
## Download and run binary executable (🆕 since v2.1)
1. Download executable for your platform from [latest release](https://github.com/scito/extract_otp_secrets/releases/latest), see assets
2. Linux and macOS: Set executable bit for the downloaded file, e.g in terminal with `chmod +x extract_otp_secrets_X.Y.Z_OS_ARCH`
3. Start executable by clicking or from command line (macOS: startable only from command line, see [below](#macos))
:heavy_check_mark: Everything is just packed in one executable.
:heavy_check_mark: No installation needed, neither Python nor any dependencies have to be installed.
:heavy_check_mark: Easy and convenient
:information_source: There is a delay after starting the executable since the files have internally to be unpacked.
:information_source: If you are a developer, you might prefer to run the Python script directly, see [Installation](#installation-of-python-script-recommended-for-developers-or-advanced-users)
> :warning: Some antivirus tools may show a virus or trojan alert for the executable.
> This alert is a false positive.
> This is a known problem for executables generated by PyInstaller.
> If you have any doubt, please use directly the [Python script](#installation-of-python-script-recommended-for-developers-or-advanced-users).
:information_source: The executables are not signed. Thus, the operating system may show a warning about download from unknown source.
### MacOS
> Beginning in macOS 10.15, all software built after June 1, 2019, and distributed with Developer ID must be notarized. However, you arent required to notarize software that you distribute through the Mac App Store because the App Store submission process already includes equivalent security checks. <small>[developer.apple.com](https://developer.apple.com/documentation/security/notarizing_macos_software_before_distribution)</small>
:x: Unfortunately, I cannot provide a signed and notarized installable application for macOS as .dmg or .pkg. Apple is not Open Source friendly and requires a yearly Developer ID subscription. I am not willing to pay [USD 99 per year](https://developer.apple.com/support/compare-memberships/) to Apple for this little open source tool.
However, the bare executable can be executed from the command line:
1. Download executable for macOS platform from [latest release](https://github.com/scito/extract_otp_secrets/releases/latest), see assets
2. Open `Terminal` application
3. Change to Downloads folder in Terminal: `cd $HOME/Downloads`
4. Remove quarantine bit for the downloaded file: `xattr -r -d com.apple.quarantine extract_otp_secrets_X.Y.Z_macos_x86_64`
5. Set executable bit for the downloaded file: `chmod +x extract_otp_secrets_X.Y.Z_macos_x86_64`
6. Start executable from command line for the first time: `./extract_otp_secrets_X.Y.Z_macos_x86_64`
7. Wait approximately 30 seconds to 1 minute on the first run. Terminal will display the following error:
```
OpenCV: not authorized to capture video (status 0), requesting...
OpenCV: camera failed to properly initialize!
```
8. macOS will then prompt to request camera access.
9. After allowing camera access, rerun the program.
10. On the second run, the GUI prompt shows correctly and is fully operable: `./extract_otp_secrets_X.Y.Z_macos_x86_64`
:information_source: Replace `X.Y.Z` in above commands with the version number of your downloaded file, e.g. `extract_otp_secrets_2.4.0_macos_x86_64`
:information_source: If Rosetta2 emulation is installed, these steps work also for M1 and M2 Apple Silicon processors and the program can be executed directly.
Tested with extract_otp_secrets_2.8.1_macos_x86_64 on macOS Sequoia 15.1 beta. Source: [#283](https://github.com/scito/extract_otp_secrets/issues/283)
## Usage
### Capture QR codes from camera (🆕 since version 2.0)
1. Open "Google Authenticator" app on the mobile phone
2. Export the QR codes from "Google Authenticator" app (see [how to export](#how-to-export-otp-secrets-from-google-authenticator-app))
3. Point the exported QR codes to the camera of your computer
4. Run the program without infile parameters:
```
extract_otp_secrets
```
![CV2 Capture from camera screenshot](docs/cv2_capture_screenshot.png)
Detected QR codes are surrounded with a frame. The color of the frame indicates the extracting result:
* Green: The QR code is detected, decoded and the OTP secret was successfully extracted.
* Red: The QR code is detected and decoded, but could not be successfully extracted. This is the case if a QR code not containing OTP data is captured.
* Magenta: The QR code is detected, but could not be decoded. The QR code should be presented better to the camera or another QR reader could be used.
Key commands:
* Space: change QR code reader
* C: save as csv file (🆕 since v2.2)
* J: save as json file (🆕 since v2.2)
* K: save as KeePass csv file (🆕 since v2.2)
* ESC, ENTER, Q: quit the program
The secrets are printed by default to the console. [Set program parameters](#program-help-arguments-and-options) for other types of output, e.g. `--csv exported_secrets.csv`.
### With builtin QR decoder from image files (🆕 since version 2.0)
1. Open "Google Authenticator" app on the mobile phone
2. Export the QR codes from "Google Authenticator" app (see [how to export](#how-to-export-otp-secrets-from-google-authenticator-app))
3. Save the QR code as image file, e.g. example_export.png
4. Transfer the images files to the computer where his script is installed.
5. Call this script with the file as input:
```
extract_otp_secrets example_export.png
```
6. Remove unencrypted files with secrets from your computer and mobile.
### With external QR decoder app from text files
1. Open "Google Authenticator" app on the mobile phone
2. Export the QR codes from "Google Authenticator" app (see [how to export](#how-to-export-otp-secrets-from-google-authenticator-app))
3. Read QR codes with a third-party QR code reader (e.g. from another phone)
4. Save the captured QR codes from the QR code reader to a text file, e.g. example_export.txt. Save each QR code on a new line. (The captured QR codes look like `otpauth-migration://offline?data=…`)
5. Transfer the file to the computer where his script is installed.
6. Call this script with the file as input:
```
extract_otp_secrets example_export.txt
```
7. Remove unencrypted files with secrets from your computer and mobile.
## Installation of Python script (recommended for developers or advanced users)
```bash
git clone https://github.com/scito/extract_otp_secrets.git
cd extract_otp_secrets
pip install -U -r requirements.txt
@@ -206,14 +35,14 @@ python src/extract_otp_secrets.py example_export.txt
In case this script is not starting properly, the debug mode can be activated by adding parameter `-d` in the command line.
### Installation of optional shared system libraries (recommended)
### Installation of shared system libraries
For reading QR codes with `ZBAR` QR reader, the zbar library must be installed.
If you do not use the `ZBAR` QR reader, you do not need to install the zbar shared library. Note: The `ZBAR` QR reader is the showed for me the best results and is thus default QR Reader.
For a detailed installation documentation of [pyzbar](https://github.com/NaturalHistoryMuseum/pyzbar#installation).
#### Linux (Debian, Ubuntu, )
#### Linux (Debian, Ubuntu, ...)
sudo apt-get install libzbar0
@@ -223,11 +52,7 @@ For a detailed installation documentation of [pyzbar](https://github.com/Natural
#### Linux (Fedora)
sudo dnf install zbar
#### Linux (Arch Linux)
pacman -S zbar
sudo dnf install libzbar0
#### Mac OS X
@@ -239,13 +64,47 @@ For a detailed installation documentation of [pyzbar](https://github.com/Natural
The zbar DLLs are included with the Windows Python wheels. However, you might need additionally to install [Visual C++ Redistributable Packages for Visual Studio 2013](https://www.microsoft.com/en-US/download/details.aspx?id=40784). Install `vcredist_x64.exe` if using 64-bit Python, `vcredist_x86.exe` if using 32-bit Python. For more information see [pyzbar](https://github.com/NaturalHistoryMuseum/pyzbar)
##### OpenCV (CV2)
##### OpenCV
OpenCV requires [Visual C++ redistributable 2015](https://www.microsoft.com/en-us/download/details.aspx?id=48145). For more information see [opencv-python](https://pypi.org/project/opencv-python/)
## Usage
### Capture QR codes from camera (since version 2.0.0)
1. Open "Google Authenticator" app on the mobile phone
2. Export the QR codes from "Google Authenticator" app
3. Point the QR codes to the camera of your computer
4. Call this script without infile parameters:
python src/extract_otp_secrets.py
![CV2 Capture from camera screenshot](cv2_capture_screenshot.png)
### With builtin QR decoder from image files (since version 2.0.0)
1. Open "Google Authenticator" app on the mobile phone
2. Export the QR codes from "Google Authenticator" app
4. Save the QR code as image file, e.g. example_export.png
5. Transfer the images files to the computer where his script is installed.
6. Call this script with the file as input:
python src/extract_otp_secrets.py example_export.png
### With external QR decoder app from text files
1. Open "Google Authenticator" app on the mobile phone
2. Export the QR codes from "Google Authenticator" app
3. Read QR codes with a QR code reader (e.g. from another phone)
4. Save the captured QR codes in the QR code reader to a text file, e.g. example_export.txt. Save each QR code on a new line. (The captured QR codes look like `otpauth-migration://offline?data=...`)
5. Transfer the file to the computer where his script is installed.
6. Call this script with the file as input:
python src/extract_otp_secrets.py example_export.txt
## Program help: arguments and options
<pre>usage: extract_otp_secrets.py [-h] [--csv FILE] [--keepass FILE] [--json FILE] [--txt FILE] [--urls FILE] [--printqr] [--saveqr DIR] [--camera NUMBER] [--qr {ZBAR,QREADER,QREADER_DEEP,CV2,CV2_WECHAT}] [-i] [--no-color] [--version] [-d | -v | -q] [infile ...]
<pre>usage: extract_otp_secrets.py [-h] [--csv FILE] [--keepass FILE] [--json FILE] [--printqr] [--saveqr DIR] [--camera NUMBER] [--qr {ZBAR,QREADER,QREADER_DEEP,CV2,CV2_WECHAT}] [-i] [--no-color] [-d | -v | -q] [infile ...]
Extracts one time password (OTP) secrets from QR codes exported by two-factor authentication (2FA) apps
If no infiles are provided, a GUI window starts and QR codes are captured from the camera.
@@ -256,19 +115,16 @@ positional arguments:
options:
-h, --help show this help message and exit
--csv FILE, -c FILE export csv file, or - for stdout
--csv FILE, -c FILE export csv file or - for stdout
--keepass FILE, -k FILE export totp/hotp csv file(s) for KeePass, - for stdout
--json FILE, -j FILE export json file or - for stdout
--txt FILE, -t FILE export txt file or - for stdout
--urls FILE, -u FILE export file with list of otpauth urls, or - for stdout
--printqr, -p print QR code(s) as text to the terminal
--saveqr DIR, -s DIR save QR code(s) as images to directory
--printqr, -p print QR code(s) as text to the terminal (requires qrcode module)
--saveqr DIR, -s DIR save QR code(s) as images to the given folder (requires qrcode module)
--camera NUMBER, -C NUMBER camera number of system (default camera: 0)
--qr {ZBAR,QREADER,QREADER_DEEP,CV2,CV2_WECHAT}, -Q {ZBAR,QREADER,QREADER_DEEP,CV2,CV2_WECHAT}
QR reader (default: ZBAR)
-i, --ignore ignore duplicate otps
--no-color, -n do not use ANSI colors in console output
--version, -V print version and quit
-d, --debug enter debug mode, do checks and quit
-v, --verbose verbose output
-q, --quiet no stdout output, except output set by -
@@ -282,7 +138,7 @@ python extract_otp_secrets.py = < example_export.png</pre>
## Examples
### Printing otp secrets from text file
### Printing otp secrets form text file
python src/extract_otp_secrets.py example_export.txt
@@ -290,14 +146,6 @@ python extract_otp_secrets.py = < example_export.png</pre>
python src/extract_otp_secrets.py example_export.png
### Writing otp secrets to csv file
python src/extract_otp_secrets.py -q --csv extracted_secrets.csv example_export.txt
### Writing otp secrets to json file
python src/extract_otp_secrets.py -q --json extracted_secrets.json example_export.txt
### Printing otp secrets multiple files
python src/extract_otp_secrets.py example_*.txt
@@ -329,17 +177,13 @@ python extract_otp_secrets.py = < example_export.png</pre>
* Free and open source
* Supports Google Authenticator exports (and compatible apps like Aegis Authenticator)
* Captures the the QR codes directly from the camera using different QR code libraries (based on OpenCV) (🆕 since v2.0)
* ZBAR: [pyzbar](https://github.com/NaturalHistoryMuseum/pyzbar) - fast and reliable, good for images and video capture (default and recommended) [if [libzbar](#installation-of-optional-shared-system-libraries-recommended) is installed]
* QREADER: [QReader](https://github.com/Eric-Canas/QReader) [if [libzbar](#installation-of-optional-shared-system-libraries-recommended) is installed]
* QREADER_DEEP: [QReader](https://github.com/Eric-Canas/QReader) - very slow in GUI [if [libzbar](#installation-of-optional-shared-system-libraries-recommended) is installed]
* Captures the the QR codes directly from the camera using different QR code libraries (based on OpenCV)
* ZBAR: [pyzbar](https://github.com/NaturalHistoryMuseum/pyzbar) - fast and reliable, good for images and video capture (default and recommended)
* QREADER: [QReader](https://github.com/Eric-Canas/QReader)
* QREADER_DEEP: [QReader](https://github.com/Eric-Canas/QReader) - very slow in GUI
* CV2: [QRCodeDetector](https://docs.opencv.org/4.x/de/dc3/classcv_1_1QRCodeDetector.html)
* CV2_WECHAT: [WeChatQRCode](https://docs.opencv.org/4.x/dd/d63/group__wechat__qrcode.html)
* Program usable as pure GUI application without any command line switches (🆕 since v2.2)
* Save otp secrets as csv file (🆕 since v2.2)
* Save otp secrets as json file (🆕 since v2.2)
* Save otp secrets as KeePass csv file(s) (🆕 since v2.2)
* Supports [TOTP](https://www.ietf.org/rfc/rfc6238.txt) and [HOTP](https://www.ietf.org/rfc/rfc4226.txt) standards
* Supports TOTP and HOTP standards
* Generates QR codes
* Exports to various formats:
* CSV
@@ -347,8 +191,7 @@ python extract_otp_secrets.py = < example_export.png</pre>
* Dedicated CSV for KeePass
* QR code images
* Supports reading from stdin and writing to stdout, thus pipes can be used
* Handles multiple input files (🆕 since v2.0)
* Reads QR codes images: (See [OpenCV docu](https://docs.opencv.org/4.x/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56)) (🆕 since v2.0)
* Reads QR codes images: (See [OpenCV docu](https://docs.opencv.org/4.x/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56))
* Portable Network Graphics - *.png
* WebP - *.webp
* JPEG files - *.jpeg, *.jpg, *.jpe
@@ -356,16 +199,8 @@ python extract_otp_secrets.py = < example_export.png</pre>
* Windows bitmaps - *.bmp, *.dib
* JPEG 2000 files - *.jp2
* Portable image format - *.pbm, *.pgm, *.ppm *.pxm, *.pnm
* Prints errors and warnings to stderr (🆕 since v2.0)
* Prints colored output (🆕 since v2.0)
* Startable as executable (script, Python, and all dependencies packed in one executable) (🆕 since v2.1)
* extract_otp_secrets_linux_x86_64 (requires glibc >= 2.31)
* extract_otp_secrets_linux_arm64 (requires glibc >= 2.31)
* extract_otp_secrets_win_x86_64.exe
* extract_otp_secrets_macos_x86_64 (optional [libzbar](#installation-of-optional-shared-system-libraries-recommended) needs to be installed manually if needed)
* extract_otp_secrets_macos_x86_64.dmg N/A, see [why](#macos)
* extract_otp_secrets_macos_x86_64.pkg N/A, see [why](#macos)
* Prebuilt Docker images provided for amd64 and arm64 (🆕 since v2.0)
* Prints errors and warnings to stderr
* Prints colored output
* Many ways to run the script:
* Native Python
* pipenv
@@ -374,13 +209,13 @@ python extract_otp_secrets.py = < example_export.png</pre>
* Docker
* VSCode devcontainer
* devbox
* Prebuilt Docker images provided for amd64 and arm64
* Compatible with major platforms:
* Linux
* macOS
* Windows
* Uses UTF-8 on all platforms
* Supports Python >= 3.8
* Installation of shared system libraries is optional (🆕 since v2.3)
* Supports Python >= 3.7
* Provides a debug mode (-d) for analyzing import problems
* Written in modern Python using type hints and following best practices
* All these features are backed by tests ran nightly
@@ -424,18 +259,23 @@ Import CSV with HOTP entries in KeePass as
KeePass can be used as a backup for one time passwords (second factor) from the mobile phone.
## How to export otp secrets from Google Authenticator app
## Technical background
1. Open "Google Authenticator" app
2. Select "Transfer accounts" in the three dot menu of the app.
![Transfer accounts option in the Google Authenticator.](docs/Transfer-accounts-option-in-the-Google-Authenticator_300px.webp)
3. Select "Export accounts"
![Export account option in the Google Authenticator.](docs/Export-account-option-in-the-Google-Authenticator_300px.webp)
4. Pass the verification by password or fingerprint.
5. Select your accounts
6. Press "Next" button
7. The exported QR code(s) ready for extraction are shown.
![Exported Google Authenticator QR codes](docs/Exported-QR-codes_300px.webp)
The export QR code of "Google Authenticator" contains the URL `otpauth-migration://offline?data=...`.
The data parameter is a base64 encoded proto3 message (Google Protocol Buffers).
Command for regeneration of Python code from proto3 message definition file (only necessary in case of changes of the proto3 message definition or new protobuf versions):
protoc --plugin=protoc-gen-mypy=path/to/protoc-gen-mypy --python_out=src/protobuf_generated_python --mypy_out=src/protobuf_generated_python src/google_auth.proto
The generated protobuf Python code was generated by protoc 21.12 (https://github.com/protocolbuffers/protobuf/releases/tag/v21.12).
For Python type hint generation the [mypy-protobuf](https://github.com/nipunn1313/mypy-protobuf) package is used.
## References
* Proto3 documentation: https://developers.google.com/protocol-buffers/docs/pythontutorial
* Template code: https://github.com/beemdevelopment/Aegis/pull/406
## Glossary
@@ -448,16 +288,10 @@ KeePass can be used as a backup for one time passwords (second factor) from the
## Alternative installation methods
### pip using github
### pip
```
pip install -U git+https://github.com/scito/extract_otp_secrets
extract_otp_secrets
```
or run it
```
python -m extract_otp_secrets
```
@@ -465,7 +299,7 @@ or from a specific tag
```
pip install -U git+https://github.com/scito/extract_otp_secrets.git@v2.0.0
extract_otp_secrets
python -m extract_otp_secrets
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.txt | python -m extract_otp_secrets -
```
@@ -474,13 +308,7 @@ curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/examp
```
git clone https://github.com/scito/extract_otp_secrets.git
pip install -U -e extract_otp_secrets
extract_otp_secrets extract_otp_secrets/example_export.txt
```
or run it
```
python -m extract_otp_secrets extract_otp_secrets/example_export.txt
python -m extract_otp_secrets example_export.txt
```
### pipenv
@@ -519,7 +347,7 @@ Alternatively, you can use a python virtual env for the dependencies:
The requirements\*.txt files contain all the dependencies (also the optional ones).
To leave the python virtual env just call `deactivate`.
### devbox 1
### devbox
Install [devbox](https://github.com/jetpack-io/devbox), which is a wrapper for nix. Then enter the environment with Python and the packages installed with:
@@ -527,14 +355,6 @@ Install [devbox](https://github.com/jetpack-io/devbox), which is a wrapper for n
devbox shell
```
### devbox 2
Install [devbox](https://devenv.sh), which is a wrapper for nix. Then enter the environment with Python and the packages installed with:
```
devenv shell
```
### docker
Install [Docker](https://docs.docker.com/get-docker/).
@@ -544,35 +364,36 @@ Prebuilt docker images are available for amd64 and arm64 architectures on [Docke
Extracting from an QR image file:
```
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.png | docker run --network none --pull always -i --rm -v "$(pwd)":/files:ro scit0/extract_otp_secrets =
docker login -u USERNAME
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.png | docker run --pull always -i --rm -v "$(pwd)":/files:ro scit0/extract_otp_secrets =
```
Capturing from camera in GUI window (X Window system required on host):
Capturing from camera in GUI (X Window system required on host):
```
docker run --network none --pull always --rm -v "$(pwd)":/files:ro -i --device="/dev/video0:/dev/video0" --env="DISPLAY" -v /tmp/.X11-unix:/tmp/.X11-unix:ro scit0/extract_otp_secrets
docker run --pull always --rm -v "$(pwd)":/files:ro -i --device="/dev/video0:/dev/video0" --env="DISPLAY" -v /tmp/.X11-unix:/tmp/.X11-unix:ro scit0/extract_otp_secrets
```
If only text processing is required, there is a small Image based on Alpine Linux:
```
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.txt | docker run --network none --pull always -i --rm -v "$(pwd)":/files:ro scit0/extract_otp_secrets:latest-only-txt -
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.txt | docker run --pull always -i --rm -v "$(pwd)":/files:ro scit0/extract_otp_secrets:latest-only-txt -
```
Docker image from GitHub:
```
docker login ghcr.io -u USERNAME
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.png | docker run --network none --pull always -i --rm -v "$(pwd)":/files:ro ghcr.io/scito/extract_otp_secrets =
curl -s https://raw.githubusercontent.com/scito/extract_otp_secrets/master/example_export.png | docker run --pull always -i --rm -v "$(pwd)":/files:ro ghcr.io/scito/extract_otp_secrets =
```
### More docker examples
docker run --network none --pull always --rm -v "$(pwd)":/files:ro scit0/extract_otp_secrets example_export.png
docker run --pull always --rm -v "$(pwd)":/files:ro scit0/extract_otp_secrets example_export.png
docker run --network none --pull always --rm -i -v "$(pwd)":/files:ro scit0/extract_otp_secrets_only_txt - < example_export.txt
docker run --pull always --rm -i -v "$(pwd)":/files:ro scit0/extract_otp_secrets_only_txt - < example_export.txt
cat example_export.txt | docker run --network none --pull always --rm -i -v "$(pwd)":/files:ro scit0/extract_otp_secrets:latest_only_txt - -c - > example_out.csv
cat example_export.txt | docker run --pull always --rm -i -v "$(pwd)":/files:ro scit0/extract_otp_secrets:latest_only_txt - -c - > example_out.csv
## Tests
@@ -626,7 +447,6 @@ Setup for running the tests in VSCode.
### Build
```
cd extract_otp_secrets/
pip install -U -e .
python src/extract_otp_secrets.py
@@ -643,12 +463,14 @@ pip install -U -r requirements.txt
### Build docker images
#### Debian (full functionality)
Build and run the app within the container:
```bash
docker build . -t extract_otp_secrets --pull -f docker/Dockerfile --build-arg RUN_TESTS=false
docker build . -t extract_otp_secrets --pull --build-arg RUN_TESTS=false
```
```bash
docker build . -t extract_otp_secrets_only_txt --pull -f Dockerfile_only_txt --build-arg RUN_TESTS=false
```
Run tests in docker container:
@@ -657,90 +479,13 @@ Run tests in docker container:
docker run --entrypoint /extract/run_pytest.sh --rm -v "$(pwd)":/files:ro extract_otp_secrets
```
#### Alpine (only text file processing)
```bash
docker build . -t extract_otp_secrets:only_txt --pull -f docker/Dockerfile_only_txt --build-arg RUN_TESTS=false
```
Run tests in docker container:
```bash
docker run --entrypoint /extract/run_pytest.sh --rm -v "$(pwd)":/files:ro extract_otp_secrets_only_txt extract_otp_secrets_test.py -k "not qreader" --relaxed
```
### Create executables with pyinstaller
#### Linux
```bash
pyinstaller -y --add-data $pythonLocation/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile src/extract_otp_secrets.py
```
Output is executable `dist/extract_otp_secrets`
#### Windows
```
pyinstaller -y --add-data "%pythonLocation%\__yolo_v3_qr_detector;__yolo_v3_qr_detector" --add-binary "%pythonLocation%\pyzbar\libiconv.dll;pyzbar" --add-binary "%pythonLocation%\pyzbar\libzbar-64.dll;pyzbar" --add-binary "%windir%\system32\msvcr120.dll;pyzbar" --add-binary "%windir%\system32\msvcp120.dll;pyzbar" --add-binary "%windir%\system32\vcamp120.dll;pyzbar" --add-binary "%windir%\system32\vcomp120.dll;pyzbar" --add-binary "%windir%\system32\vccorlib120.dll;pyzbar" --add-binary "%windir%\system32\mfc120.dll;pyzbar" --add-binary "%windir%\system32\mfc120u.dll;pyzbar" --add-binary "%windir%\system32\mfc120chs.dll;pyzbar" --add-binary "%windir%\system32\mfc120cht.dll;pyzbar" --add-binary "%windir%\system32\mfc120deu.dll;pyzbar" --add-binary "%windir%\system32\mfc120enu.dll;pyzbar" --add-binary "%windir%\system32\mfc120esn.dll;pyzbar" --add-binary "%windir%\system32\mfc120fra.dll;pyzbar" --add-binary "%windir%\system32\mfc120ita.dll;pyzbar" --add-binary "%windir%\system32\mfc120jpn.dll;pyzbar" --add-binary "%windir%\system32\mfc120kor.dll;pyzbar" --add-binary "%windir%\system32\mfc120rus.dll;pyzbar" --onefile --version-file build\file_version_info.txt src\extract_otp_secrets.py
```
Output is `dist\extract_otp_secrets.exe`
### Full local build (bash)
There is a Bash script for a full local build including linting and type checking.
```bash
./build.sh
```
The options of the build script:
```
Build extract_otp_secrets project
./build.sh [options]
Options:
-i Interactive mode, all steps must be confirmed
-C Ignore version check of protobuf/protoc
-e Build exe
-n Build nuitka exe
-L Do not build local (exes)
-d Build docker
-a Build arm
-X Do not build x86_64
-B Do not build base
-V Do not run pipenv
-g Start extract_otp_secrets.py in GUI mode
-c Clean everything
-r Generate result files
-h, --help Show help and quit
```
## Technical background
The export QR code of "Google Authenticator" contains the URL `otpauth-migration://offline?data=…`.
The data parameter is a base64 encoded proto3 message (Google Protocol Buffers).
Command for regeneration of Python code from proto3 message definition file (only necessary in case of changes of the proto3 message definition or new protobuf versions):
protoc --plugin=protoc-gen-mypy=path/to/protoc-gen-mypy --python_out=src/protobuf_generated_python --mypy_out=src/protobuf_generated_python src/google_auth.proto
The generated protobuf Python code was generated by protoc 29.0 (https://github.com/protocolbuffers/protobuf/releases/tag/v29.0).
For Python type hint generation the [mypy-protobuf](https://github.com/nipunn1313/mypy-protobuf) package is used.
## References
* Proto3 documentation: https://developers.google.com/protocol-buffers/docs/pythontutorial
* Template code: https://github.com/beemdevelopment/Aegis/pull/406
## Issues
* Segmentation fault on macOS with CV2 4.7.0: https://github.com/opencv/opencv/issues/23072 (fixed)
* CV2 window does not show icons: https://github.com/opencv/opencv-python/issues/585
* Known issue for macOS: https://github.com/opencv/opencv/issues/23072
## Problems and Troubleshooting
@@ -772,12 +517,11 @@ FileNotFoundError: Could not find module 'libiconv.dll' (or one of its dependenc
## Related projects
* [ZBar](https://github.com/mchehab/zbar) is an open source software suite for reading bar codes from various sources, including webcams.
* [Aegis Authenticator](https://github.com/beemdevelopment/Aegis) is a free, secure and open source 2FA app for Android. This app can scan Google export QR codes and export the secrets, e.g. as JSON. However, a second device is required.
* [pyzbar](https://github.com/NaturalHistoryMuseum/pyzbar) is a good QR code reader Python module
* [OpenCV](https://docs.opencv.org/4.x/) (CV2) Open Source Computer Vision library with [opencv-python](https://github.com/opencv/opencv-python)
* [Python QReader](https://github.com/Eric-Canas/QReader) Python QR code readers
* [Aegis Authenticator](https://github.com/beemdevelopment/Aegis) is a free, secure and open source 2FA app for Android.
* [Android OTP Extractor](https://github.com/puddly/android-otp-extractor) can extract your tokens from popular Android OTP apps and export them in a standard format or just display them as QR codes for easy importing. [Requires a _rooted_ Android phone.]
* [Google Authenticator secret extractor](https://github.com/krissrex/google-authenticator-exporter) is similar project written in JavaScript. It also extracts otp secrets from Google Authenticator.
* [Python QReader](https://github.com/Eric-Canas/QReader)
* [pyzbar](https://github.com/NaturalHistoryMuseum/pyzbar)
* [OpenCV](https://docs.opencv.org/4.x/) (CV2) Open Source Computer Vision library with [opencv-python](https://github.com/opencv/opencv-python)
***

738
build.sh
View File

@@ -73,47 +73,36 @@ askContinueYn() {
# Reference: https://gist.github.com/steinwaywhw/a4cd19cda655b8249d908261a62687f8
echo "Checking Protoc version..."
VERSION=$(curl -sL https://github.com/protocolbuffers/protobuf/releases/latest | grep -E "<title>" | perl -pe's%.*Protocol Buffers v(\d+\.\d+(\.\d+)?).*%\1%')
BASEVERSION=4
echo
interactive=false
ignore_version_check=true
clean=false
clean_flag=""
build_base=true
build_arm=false
build_x86_64=true
build_docker=false
build_local=true
build_exe=false
build_nuitka_exe=false
run_pipenv=true
run_gui=false
build_docker=true
run_gui=true
generate_result_files=false
PYTHONHASHSEED=31
while test $# -gt 0; do
case $1 in
-h|--help)
echo "Build extract_otp_secrets project"
echo "Upgrade Protoc"
echo
echo "$0 [options]"
echo
echo "Options:"
echo "-i Interactive mode, all steps must be confirmed"
echo "-C Ignore version check of protobuf/protoc"
echo "-e Build exe"
echo "-n Build nuitka exe"
echo "-L Do not run protoc and base build locally incl. exes"
echo "-d Build docker"
echo "-a Build arm"
echo "-X Do not build x86_64"
echo "-B Do not build base"
echo "-V Do not run pipenv"
echo "-g Start extract_otp_secrets.py in GUI mode"
echo "-c Clean everything"
echo "-i Interactive"
echo "-C Ignore version check"
echo "-D No docker build"
echo "-G No not run gui"
echo "-c Clean"
echo "-r Generate result files"
echo "-h, --help Show help and quit"
echo "-h, --help Help"
quit
;;
-i)
-a)
interactive=true
shift
;;
@@ -121,41 +110,12 @@ while test $# -gt 0; do
ignore_version_check=false
shift
;;
-B)
build_base=false
-D)
build_docker=false
shift
;;
-L)
build_local=false
build_base=false
shift
;;
-a)
build_arm=true
shift
;;
-X)
build_x86_64=false
shift
;;
-e)
build_exe=true
shift
;;
-n)
build_nuitka_exe=true
shift
;;
-d)
build_docker=true
shift
;;
-V)
run_pipenv=false
shift
;;
-g)
run_gui=true
-G)
run_gui=false
shift
;;
-r)
@@ -164,7 +124,6 @@ while test $# -gt 0; do
;;
-c)
clean=true
clean_flag="--clean"
shift
;;
esac
@@ -179,34 +138,16 @@ PIPENV="$PYTHON -m pipenv"
FLAKE8="$PYTHON -m flake8"
MYPY="$PYTHON -m mypy"
# sudo ln -s /usr/bin/python3.11 /usr/bin/python
# Upgrade protoc
DEST="protoc"
OLDVERSION=$(cat $BIN/$DEST/.VERSION.txt || echo "")
echo -e "\nProtoc remote version $VERSION\n"
echo -e "Protoc local version: $OLDVERSION\n"
if $clean; then
cmd="docker image prune -f || echo 'No docker image pruned'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PIP uninstall -y extract-otp-secrets || echo nothing done"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PIP freeze | grep -v -E '^-e|^#' | xargs sudo $PIP uninstall -y || echo nothing done"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PIP freeze --user | grep -v -E '^-e|^#' | xargs $PIP uninstall -y || echo nothing done"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PIP freeze | cut -d \"@\" -f1 | xargs pip uninstall -y || echo Nothing to do"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="sudo rm -rf dist/ build/ dist_*/ *.whl extracted_*.csv extracted_*.json pytest.xml pytest-coverage.txt .coverage tests/reports || true; find . -name '*.pyc' -type f -delete; find . -name '__pycache__' -type d -exec rm -r {} \; || true; find . -name '*.egg-info' -type d -exec rm -r {} \; || true; find . -name '*_cache' -type d -exec rm -r {} \; || true; mkdir -p tests/reports;"
cmd="rm -r dist/ build/ *.whl pytest.xml pytest-coverage.txt .coverage tests/reports || true; find . -name '*.pyc' -type f -delete; find . -name '__pycache__' -type d -exec rm -r {} \; || true; find . -name '*.egg-info' -type d -exec rm -r {} \; || true; find . -name '*_cache' -type d -exec rm -r {} \; || true; mkdir -p tests/reports;"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
@@ -217,436 +158,232 @@ if $clean; then
cmd="sudo pipenv --rm || true"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
cmd="mkdir -p dist"
if [ "$OLDVERSION" != "$VERSION" ] || ! $ignore_version_check; then
echo "Upgrade protoc from $OLDVERSION to $VERSION"
NAME="protoc-$VERSION"
ARCHIVE="$NAME.zip"
mkdir -p $DOWNLOADS
# https://github.com/protocolbuffers/protobuf/releases/download/v21.6/protoc-21.6-linux-x86_64.zip
cmd="wget --trust-server-names https://github.com/protocolbuffers/protobuf/releases/download/v$VERSION/protoc-$VERSION-linux-x86_64.zip -O $DOWNLOADS/$ARCHIVE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="echo -e '\nSize [Byte]'; stat --printf='%s\n' $DOWNLOADS/$ARCHIVE; echo -e '\nMD5'; md5sum $DOWNLOADS/$ARCHIVE; echo -e '\nSHA256'; sha256sum $DOWNLOADS/$ARCHIVE;"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="mkdir -p $BIN/$NAME; unzip $DOWNLOADS/$ARCHIVE -d $BIN/$NAME"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="echo $VERSION > $BIN/$NAME/.VERSION.txt; echo $VERSION > $BIN/$NAME/.VERSION_$VERSION.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="[ -d $BIN/$DEST.old ] && rm -rf $BIN/$DEST.old || echo 'No old dir to delete'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="[ -d $BIN/$DEST ] && mv -iT $BIN/$DEST $BIN/$DEST.old || echo 'No previous dir to keep'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="mv -iT $BIN/$NAME $BIN/$DEST"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="rm $DOWNLOADS/$ARCHIVE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$BIN/$DEST/bin/protoc --plugin=protoc-gen-mypy=$HOME/.local/bin/protoc-gen-mypy --python_out=src/protobuf_generated_python --mypy_out=src/protobuf_generated_python --proto_path=src google_auth.proto"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Update README.md
cmd="perl -i -pe 's%proto(buf|c)([- ])(\d\.)?$OLDVERSION%proto\$1\$2\${3}$VERSION%g' README.md && perl -i -pe 's%(protobuf/releases/tag/v)$OLDVERSION%\${1}$VERSION%g' README.md"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
else
echo -e "\nVersion has not changed. Quit"
fi
# Upgrade pip requirements
cmd="sudo pip install -U pip"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
$PIP --version
cmd="$PIP install --use-pep517 -U -r requirements.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PIP install --use-pep517 -U -r requirements-dev.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Lint
LINT_OUT_FILE="tests/reports/flake8_results.txt"
cmd="$FLAKE8 . --count --select=E9,F63,F7,F82 --show-source --statistics | tee $LINT_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$FLAKE8 . --count --exit-zero --max-complexity=10 --max-line-length=200 --statistics --exclude=.git,__pycache__,docs/source/conf.py,old,build,dist,protobuf_generated_python | tee -a $LINT_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Type checking
TYPE_CHECK_OUT_FILE="tests/reports/mypy_results.txt"
cmd="$MYPY --install-types --non-interactive src/*.py tests/*.py"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# change to src as python -m mypy adds the current dir Python sys.path
# execute in a subshell in order not to loose the exit code and not to change the dir in the currrent shell
cmd="$MYPY --strict src/*.py tests/*.py | tee $TYPE_CHECK_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Test
cmd="$PYTHON src/extract_otp_secrets.py example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PYTHON src/extract_otp_secrets.py - < example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
COVERAGE_OUT_FILE="tests/reports/pytest-coverage.txt"
cmd="pytest --cov=extract_otp_secrets_test --junitxml=tests/reports/pytest.xml --cov-report html:tests/reports/html --cov-report=term-missing tests/ | tee $COVERAGE_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Pipenv
cmd="$PIP install -U pipenv"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
$PIPENV --version
cmd="$PIPENV update && $PIPENV --rm && $PIPENV install"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
$PIPENV run python --version
cmd="$PIPENV run pytest --cov=extract_otp_secrets_test tests/"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# sudo pip
cmd="sudo $PIP install --use-pep517 -U -r requirements.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="sudo $PIP install --use-pep517 -U -r requirements-dev.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="sudo $PIP install -U pipenv"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# pip -e install (must be after other pip installs in order to have this environment for development)
cmd="$PIP install -U -e ."
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="extract_otp_secrets example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="extract_otp_secrets - < example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build wheel
cmd="$PIP wheel .
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Generate results files
if $generate_result_files; then
cmd="for color in '' '-n'; do for level in '' '-v' '-vv' '-vvv'; do $PYTHON src/extract_otp_secrets.py example_export.txt $color $level > tests/data/print_verbose_output$color$level.txt; done; done"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
if $build_local; then
cmd="$PIP install --use-pep517 -U -r requirements-dev.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Update Code Coverage in README.md
echo -e "\n\nChecking Protoc version..."
VERSION=$(curl -sL https://github.com/protocolbuffers/protobuf/releases/latest | grep -E "<title>" | perl -pe's%.*Protocol Buffers v(\d+\.\d+(\.\d+)?).*%\1%')
BASEVERSION=4
echo
OLDVERSION=$(cat $BIN/$DEST/.VERSION.txt || echo "")
echo -e "\nProtoc remote version $VERSION\n"
echo -e "Protoc local version: $OLDVERSION\n"
if [ "$OLDVERSION" != "$VERSION" ] || ! $ignore_version_check; then
echo "Upgrade protoc from $OLDVERSION to $VERSION"
NAME="protoc-$VERSION"
ARCHIVE="$NAME.zip"
mkdir -p $DOWNLOADS
# https://github.com/protocolbuffers/protobuf/releases/download/v21.6/protoc-21.6-linux-x86_64.zip
cmd="wget --trust-server-names https://github.com/protocolbuffers/protobuf/releases/download/v$VERSION/protoc-$VERSION-linux-x86_64.zip -O $DOWNLOADS/$ARCHIVE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="echo -e '\nSize [Byte]'; stat --printf='%s\n' $DOWNLOADS/$ARCHIVE; echo -e '\nMD5'; md5sum $DOWNLOADS/$ARCHIVE; echo -e '\nSHA256'; sha256sum $DOWNLOADS/$ARCHIVE;"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="mkdir -p $BIN/$NAME; unzip $DOWNLOADS/$ARCHIVE -d $BIN/$NAME"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="echo $VERSION > $BIN/$NAME/.VERSION.txt; echo $VERSION > $BIN/$NAME/.VERSION_$VERSION.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="[ -d $BIN/$DEST.old ] && rm -rf $BIN/$DEST.old || echo 'No old dir to delete'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="[ -d $BIN/$DEST ] && mv -iT $BIN/$DEST $BIN/$DEST.old || echo 'No previous dir to keep'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="mv -iT $BIN/$NAME $BIN/$DEST"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="rm $DOWNLOADS/$ARCHIVE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$BIN/$DEST/bin/protoc --plugin=protoc-gen-mypy=$HOME/.local/bin/protoc-gen-mypy --python_out=src/protobuf_generated_python --mypy_out=src/protobuf_generated_python --proto_path=src google_auth.proto"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Update README.md
cmd="perl -i -pe 's%proto(buf|c)([- ])(\d\.)?$OLDVERSION%proto\$1\$2\${3}$VERSION%g' README.md && perl -i -pe 's%(protobuf/releases/tag/v)$OLDVERSION%\${1}$VERSION%g' README.md"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
else
echo -e "\nVersion has not changed. Quit"
fi
# Upgrade pip requirements
cmd="pip install -U pip"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
$PIP --version
cmd="$PIP install --use-pep517 -U -r requirements.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
if $build_base; then
# Lint
LINT_OUT_FILE="tests/reports/flake8_results.txt"
cmd="$FLAKE8 . --count --select=E9,F63,F7,F82 --show-source --statistics | tee $LINT_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$FLAKE8 . --count --exit-zero --max-complexity=10 --max-line-length=200 --statistics --exclude=.git,__pycache__,docs/source/conf.py,old,build,dist,protobuf_generated_python | tee -a $LINT_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Type checking
TYPE_CHECK_OUT_FILE="tests/reports/mypy_results.txt"
cmd="$MYPY --install-types --non-interactive src/*.py tests/*.py"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# change to src as python -m mypy adds the current dir Python sys.path
# execute in a subshell in order not to loose the exit code and not to change the dir in the currrent shell
cmd="$MYPY --strict src/*.py tests/*.py | tee $TYPE_CHECK_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Generate results files
if $generate_result_files; then
cmd="for color in '' '-n'; do for level in '' '-v' '-vv' '-vvv'; do $PYTHON src/extract_otp_secrets.py example_export.txt \$color \$level > tests/data/print_verbose_output\$color\$level.txt; done; done"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
# pip -e install
cmd="$PIP install -U -e ."
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="extract_otp_secrets example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="extract_otp_secrets - < example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Test (needs module)
cmd="$PYTHON src/extract_otp_secrets.py example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="$PYTHON src/extract_otp_secrets.py - < example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
COVERAGE_OUT_FILE="tests/reports/pytest-coverage.txt"
cmd="pytest --cov=extract_otp_secrets_test --junitxml=tests/reports/pytest.xml --cov-report html:tests/reports/html --cov-report=term-missing tests/ | tee $COVERAGE_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Update Code Coverage in README.md
# https://github.com/marketplace/actions/pytest-coverage-comment
# Coverage-95%25-yellowgreen
echo -e "Update code coverage in README.md"
TOTAL_COVERAGE=$(cat $COVERAGE_OUT_FILE | grep 'TOTAL' | perl -ne 'print "$&" if /\b(\d{1,3})%/') && perl -i -pe "s/coverage-(\d{1,3}%)25-/coverage-${TOTAL_COVERAGE}25-/" README.md
# Pipenv
if $run_pipenv; then
cmd="$PIP install -U pipenv"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
$PIPENV --version
cmd="$PIPENV --rm && $PIPENV update && $PIPENV install"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
$PIPENV run python --version
cmd="$PIPENV run pytest --cov=extract_otp_secrets_test tests/"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
# Build wheel
cmd="$PIP wheel ."
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
# Build executable
if $build_exe; then
cmd="LOCAL_GLIBC_VERSION=$(ldd --version | sed '1!d' | sed -E 's/.* ([[:digit:]]+\.[[:digit:]]+)$/\1/')"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
echo "local glibc: $LOCAL_GLIBC_VERSION"
cmd="time pyinstaller -y --specpath installer --add-data $HOME/.local/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile $clean_flag src/extract_otp_secrets.py"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets -h"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets --qr ZBAR example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
# Build compiled executable
if $build_nuitka_exe; then
cmd="LOCAL_GLIBC_VERSION=$(ldd --version | sed '1!d' | sed -E 's/.* ([[:digit:]]+\.[[:digit:]]+)$/\1/')"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
echo "local glibc: $LOCAL_GLIBC_VERSION"
cmd="$PIP install -U pyqt5"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="time $PYTHON -m nuitka --enable-plugin=tk-inter --enable-plugin=pyqt5 --noinclude-default-mode=nofollow --clang --include-data-dir=$HOME/.local/__yolo_v3_qr_detector/=__yolo_v3_qr_detector/ --onefile --output-dir=build/nuitka --output-filename=extract_otp_secrets_compiled src/extract_otp_secrets.py"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cp build/nuitka/extract_otp_secrets_compiled dist/"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets_compiled -h"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets_compiled --qr ZBAR example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
# Generate README.md TOC
cmd="gfm-toc -s 2 -e 3 -t -o README.md > docs/README_TOC.md"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# create Windows win_file_version_info.txt
cmd="VERSION_STR=$(setuptools-git-versioning) VERSION_MAJOR=$(cut -d '.' -f 1 <<< "$(setuptools-git-versioning)") VERSION_MINOR=$(cut -d '.' -f 2 <<< "$(setuptools-git-versioning)") VERSION_PATCH=$(cut -d '.' -f 3 <<< "$(setuptools-git-versioning)") VERSION_BUILD=$(($(git rev-list --count $(git tag | sort -V -r | sed '1!d')..HEAD))) COPYRIGHT_YEARS='2020-2023' envsubst < installer/win_file_version_info_template.txt > build/win_file_version_info.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# create macOS extract_otp_secrets_macos.spec from extract_otp_secrets_macos_template.spec
cmd="VERSION_STR=$(setuptools-git-versioning) COPYRIGHT_YEARS='2020-2023' envsubst < installer/extract_otp_secrets_macos_template.spec > build/extract_otp_secrets_macos.spec"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
# https://github.com/marketplace/actions/pytest-coverage-comment
# Coverage-95%25-yellowgreen
echo -e "Update code coverage in README.md"
TOTAL_COVERAGE=$(cat $COVERAGE_OUT_FILE | grep 'TOTAL' | perl -ne 'print "$&" if /\b(\d{1,3})%/') && perl -i -pe "s/coverage-(\d{1,3}%)25-/coverage-${TOTAL_COVERAGE}25-/" README.md
if $build_docker; then
# Build docker
if $build_x86_64; then
# Build Dockerfile_only_txt (Alpine)
cmd="docker build . -t extract_otp_secrets_only_txt -t extract_otp_secrets:only-txt -t extract_otp_secrets:alpine -f docker/Dockerfile_only_txt --pull --build-arg RUN_TESTS=false"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build Dockerfile_only_txt (Alpine)
cmd="docker build . -t extract_otp_secrets_only_txt -f Dockerfile_only_txt --pull --build-arg RUN_TESTS=false"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --network none --rm -v \"$(pwd)\":/files:ro extract_otp_secrets_only_txt example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --rm -v \"$(pwd)\":/files:ro extract_otp_secrets_only_txt example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --network none --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets_only_txt - < example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets_only_txt - < example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --entrypoint /extract/run_pytest.sh --rm -v \"$(pwd)\":/files:ro extract_otp_secrets_only_txt extract_otp_secrets_test.py -k 'not qreader' -vvv --relaxed"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --entrypoint /extract/run_pytest.sh --rm -v \"$(pwd)\":/files:ro extract_otp_secrets_only_txt tests/extract_otp_secrets_test.py -k 'not qreader' -vvv --relaxed"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build extract_otp_secrets (Debian Bookworm)
cmd="docker build . -t extract_otp_secrets -t extract_otp_secrets:bookworm --pull -f docker/Dockerfile --build-arg RUN_TESTS=false"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --network none --rm -v \"$(pwd)\":/files:ro extract_otp_secrets example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build extract_otp_secrets (Debian)
cmd="docker build . -t extract_otp_secrets --pull --build-arg RUN_TESTS=false"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cat example_export.txt | docker run --network none --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets - -c - > example_output.csv"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --rm -v \"$(pwd)\":/files:ro extract_otp_secrets example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --network none --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets = < example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cat example_export.txt | docker run --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets - -c - > example_output.csv"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --entrypoint /extract/run_pytest.sh --rm -v \"$(pwd)\":/files:ro extract_otp_secrets"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets = < example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build extract_otp_secrets (Debian Bullseye)
cmd="docker build . -t extract_otp_secrets:bullseye -t extract_otp_secrets:bullseye-x86_64 --pull -f docker/Dockerfile --build-arg RUN_TESTS=false --build-arg BASE_IMAGE=python:3.12-slim-bullseye"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --entrypoint /extract/run_pytest.sh --rm -v \"$(pwd)\":/files:ro extract_otp_secrets"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --network none --rm -v \"$(pwd)\":/files:ro extract_otp_secrets:bullseye example_export.txt"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker image prune -f || echo 'No docker image pruned'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cat example_export.txt | docker run --network none --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets:bullseye - -c - > example_output.csv"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --network none --rm -i -v \"$(pwd)\":/files:ro extract_otp_secrets:bullseye = < example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="docker run --entrypoint /extract/run_pytest.sh --rm -v \"$(pwd)\":/files:ro extract_otp_secrets:bullseye"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build executable from Docker latest
# sed "1!d" is workaround for head -n 1 since it head procduces exit code != 0
BOOKWORM_GLIBC_VERSION=$(docker run --network none --entrypoint /bin/bash --rm extract_otp_secrets -c 'ldd --version | sed "1!d" | sed -E "s/.* ([[:digit:]]+\.[[:digit:]]+)$/\1/"')
echo "Bookworm glibc: $BOOKWORM_GLIBC_VERSION"
fi
if $build_arm; then
# build linux/arm64
cmd="docker run --network none --pull always --rm --privileged multiarch/qemu-user-static --reset -p yes"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build extract_otp_secrets (Debian Bullseye)
cmd="docker buildx build --platform=linux/arm64 . -t extract_otp_secrets:bullseye-arm64 --pull -f docker/Dockerfile --build-arg RUN_TESTS=false --build-arg BASE_IMAGE=python:3.12-slim-bullseye"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
if $build_exe; then
if $build_x86_64; then
cmd="docker run --platform linux/amd64 --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets -c 'apt-get update && apt-get -y install binutils && pip install -U pip && pip install -U -r /files/requirements.txt && pip install pyinstaller && PYTHONHASHSEED=31 && pyinstaller -y --specpath installer --add-data /usr/local/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile --name extract_otp_secrets_linux_x86_64_bookworm --distpath /files/dist/ /files/src/extract_otp_secrets.py'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets_linux_x86_64_bookworm -h || echo 'Could not run exe; see error message above'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets_linux_x86_64_bookworm --qr ZBAR example_export.png || echo 'Could not run exe; see error message above'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build executable from Docker bullseye
BULLSEYE_GLIBC_VERSION=$(docker run --network none --entrypoint /bin/bash --rm extract_otp_secrets:bullseye -c 'ldd --version | sed "1!d" | sed -E "s/.* ([[:digit:]]+\.[[:digit:]]+)$/\1/"')
echo "Bullseye glibc: $BULLSEYE_GLIBC_VERSION"
cmd="docker run --platform linux/amd64 --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets:bullseye -c 'apt-get update && apt-get -y install binutils && pip install -U pip && pip install -U -r /files/requirements.txt && pip install pyinstaller && PYTHONHASHSEED=31 && pyinstaller -y --specpath installer --add-data /usr/local/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile --name extract_otp_secrets_linux_x86_64 --distpath /files/dist/ /files/src/extract_otp_secrets.py'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets_linux_x86_64 -h"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="dist/extract_otp_secrets_linux_x86_64 --qr ZBAR example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
if $build_arm; then
# build linux/arm64
cmd="docker run --platform linux/arm64 --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets:bullseye-arm64 -c 'apt-get update && apt-get -y install binutils && pip install -U pip && pip install -U -r /files/requirements.txt && pip install pyinstaller && PYTHONHASHSEED=31 && pyinstaller --specpath installer -y --add-data /usr/local/__yolo_v3_qr_detector/:__yolo_v3_qr_detector/ --onefile --name extract_otp_secrets_linux_arm64 --distpath /files/dist/ /files/src/extract_otp_secrets.py'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="PLATFORM='linux/arm64' && EXE='dist/extract_otp_secrets_linux_arm64' && docker run --platform \"\$PLATFORM\" --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets:bullseye-arm64 -c \"\$EXE -V && \$EXE -h && \$EXE example_export.png && \$EXE - < example_export.txt && \$EXE --qr ZBAR example_export.png && \$EXE --qr QREADER example_export.png && \$EXE --qr QREADER_DEEP example_export.png && \$EXE --qr CV2 example_export.png && \$EXE --qr CV2_WECHAT example_export.png\""
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
fi
if $build_nuitka_exe; then
if $build_x86_64; then
cmd="docker run --platform linux/amd64 --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets -c 'apt-get update && apt-get -y install binutils build-essential patchelf && pip install -U pip && pip install -U -r /files/requirements.txt && pip install nuitka pyqt5 && PYTHONHASHSEED=31 && python -m nuitka --enable-plugin=tk-inter --enable-plugin=pyqt5 --include-data-dir=/usr/local/__yolo_v3_qr_detector/=__yolo_v3_qr_detector/ --onefile --output-dir=/files/build/docker/nuitka --output-filename=extract_otp_secrets_linux_x86_64_bookworm_compiled /files/src/extract_otp_secrets.py'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="build/docker/nuitka/extract_otp_secrets_linux_x86_64_bookworm_compiled -h"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="build/docker/nuitka/extract_otp_secrets_linux_x86_64_bookworm_compiled --qr ZBAR example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cp build/docker/nuitka/extract_otp_secrets_linux_x86_64_bookworm_compiled dist/"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
# Build executable from Docker bullseye
BULLSEYE_GLIBC_VERSION=$(docker run --network none --entrypoint /bin/bash --rm extract_otp_secrets:bullseye -c 'ldd --version | sed "1!d" | sed -E "s/.* ([[:digit:]]+\.[[:digit:]]+)$/\1/"')
echo "Bookworm glibc: $BULLSEYE_GLIBC_VERSION"
cmd="docker run --platform linux/amd64 --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets:bullseye -c 'apt-get update && apt-get -y install binutils build-essential patchelf && pip install -U pip && pip install -U -r /files/requirements.txt && pip install nuitka pyqt5 && PYTHONHASHSEED=31 && python -m nuitka --enable-plugin=tk-inter --enable-plugin=pyqt5 --include-data-dir=/usr/local/__yolo_v3_qr_detector/=__yolo_v3_qr_detector/ --onefile --output-dir=/files/build/docker/nuitka --output-filename=extract_otp_secrets_linux_x86_64_bullseye_compiled /files/src/extract_otp_secrets.py'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="build/docker/nuitka/extract_otp_secrets_linux_x86_64_bullseye_compiled -h"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="build/docker/nuitka/extract_otp_secrets_linux_x86_64_bullseye_compiled --qr ZBAR example_export.png"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cp build/docker/nuitka/extract_otp_secrets_linux_x86_64_bullseye_compiled dist/"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
if $build_arm; then
# build linux/arm64
cmd="docker run --platform linux/arm64 --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets:bullseye-arm64 -c 'apt-get update && apt-get -y install binutils build-essential patchelf qt5-default && pip install -U pip && pip install -U -r /files/requirements.txt && pip install nuitka pyqt5 && PYTHONHASHSEED=31 && python -m nuitka --enable-plugin=tk-inter --enable-plugin=pyqt5 --include-data-dir=/usr/local/__yolo_v3_qr_detector/=__yolo_v3_qr_detector/ --onefile --output-dir=/files/build/docker/nuitka --output-filename=extract_otp_secrets_linux_arm64_compiled /files/src/extract_otp_secrets.py'"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="PLATFORM='linux/arm64' && EXE='build/docker/nuitka/extract_otp_secrets_linux_arm64_compiled' && docker run --platform \"\$PLATFORM\" --entrypoint /bin/bash --rm -v \"$(pwd)\":/files -w /files extract_otp_secrets:bullseye-arm64 -c \"\$EXE -V && \$EXE -h && \$EXE example_export.png && \$EXE - < example_export.txt && \$EXE --qr ZBAR example_export.png && \$EXE --qr QREADER example_export.png && \$EXE --qr QREADER_DEEP example_export.png && \$EXE --qr CV2 example_export.png && \$EXE --qr CV2_WECHAT example_export.png\""
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
cmd="cp build/docker/nuitka/extract_otp_secrets_linux_arm64_compiled dist/"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
fi
# Run GUI from Docker
if $build_x86_64 && $run_gui; then
cmd="docker run --network none --rm -v "$(pwd)":/files:ro --device=\"/dev/video0:/dev/video0\" --env=\"DISPLAY\" -v /tmp/.X11-unix:/tmp/.X11-unix:ro extract_otp_secrets &"
if $run_gui; then
cmd="docker run --rm -v "$(pwd)":/files:ro --device=\"/dev/video0:/dev/video0\" --env=\"DISPLAY\" -v /tmp/.X11-unix:/tmp/.X11-unix:ro extract_otp_secrets &"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
@@ -658,18 +395,13 @@ if $run_gui; then
eval "$cmd"
fi
if $build_base; then
line=$(printf '#%.0s' $(eval echo {1..$(( ($COLUMNS - 10) / 2))}))
echo -e "\n${blueBold}$line RESULTS $line${reset}"
cmd="cat $TYPE_CHECK_OUT_FILE $LINT_OUT_FILE $COVERAGE_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
fi
line=$(printf '#%.0s' $(eval echo {1..$(( ($COLUMNS - 10) / 2))}))
echo -e "\n${greenBold}$line SUCCESS $line${reset}"
echo -e "\n${blueBold}$line RESULTS $line${reset}"
git status
cmd="cat $TYPE_CHECK_OUT_FILE $LINT_OUT_FILE $COVERAGE_OUT_FILE"
if $interactive ; then askContinueYn "$cmd"; else echo -e "${cyan}$cmd${reset}";fi
eval "$cmd"
echo -e "\n${greenBold}SUCCESS${reset}"
quit

View File

Before

Width:  |  Height:  |  Size: 528 KiB

After

Width:  |  Height:  |  Size: 528 KiB

View File

@@ -1,122 +0,0 @@
{
"nodes": {
"devenv": {
"locked": {
"dir": "src/modules",
"lastModified": 1726050924,
"owner": "cachix",
"repo": "devenv",
"rev": "cf471f691c3765ed431199f61b8bd70530bc4305",
"treeHash": "04a5d566820f8fb1955c7c49ccbd3ff95d9129d7",
"type": "github"
},
"original": {
"dir": "src/modules",
"owner": "cachix",
"repo": "devenv",
"type": "github"
}
},
"flake-compat": {
"flake": false,
"locked": {
"lastModified": 1696426674,
"owner": "edolstra",
"repo": "flake-compat",
"rev": "0f9255e01c2351cc7d116c072cb317785dd33b33",
"treeHash": "2addb7b71a20a25ea74feeaf5c2f6a6b30898ecb",
"type": "github"
},
"original": {
"owner": "edolstra",
"repo": "flake-compat",
"type": "github"
}
},
"gitignore": {
"inputs": {
"nixpkgs": [
"pre-commit-hooks",
"nixpkgs"
]
},
"locked": {
"lastModified": 1709087332,
"owner": "hercules-ci",
"repo": "gitignore.nix",
"rev": "637db329424fd7e46cf4185293b9cc8c88c95394",
"treeHash": "ca14199cabdfe1a06a7b1654c76ed49100a689f9",
"type": "github"
},
"original": {
"owner": "hercules-ci",
"repo": "gitignore.nix",
"type": "github"
}
},
"nixpkgs": {
"locked": {
"lastModified": 1716977621,
"owner": "cachix",
"repo": "devenv-nixpkgs",
"rev": "4267e705586473d3e5c8d50299e71503f16a6fb6",
"treeHash": "6d9f1f7ca0faf1bc2eeb397c78a49623260d3412",
"type": "github"
},
"original": {
"owner": "cachix",
"ref": "rolling",
"repo": "devenv-nixpkgs",
"type": "github"
}
},
"nixpkgs-stable": {
"locked": {
"lastModified": 1725826545,
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "f4c846aee8e1e29062aa8514d5e0ab270f4ec2f9",
"treeHash": "8fc49deaed3f2728a7147c38163cc468a117570a",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-24.05",
"repo": "nixpkgs",
"type": "github"
}
},
"pre-commit-hooks": {
"inputs": {
"flake-compat": "flake-compat",
"gitignore": "gitignore",
"nixpkgs": [
"nixpkgs"
],
"nixpkgs-stable": "nixpkgs-stable"
},
"locked": {
"lastModified": 1725513492,
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"rev": "7570de7b9b504cfe92025dd1be797bf546f66528",
"treeHash": "4b46d77870afecd8f642541cb4f4927326343b59",
"type": "github"
},
"original": {
"owner": "cachix",
"repo": "pre-commit-hooks.nix",
"type": "github"
}
},
"root": {
"inputs": {
"devenv": "devenv",
"nixpkgs": "nixpkgs",
"pre-commit-hooks": "pre-commit-hooks"
}
}
},
"root": "root",
"version": 7
}

View File

@@ -1,24 +0,0 @@
{
pkgs,
lib,
config,
inputs,
...
}:
{
cachix.enable = false;
languages.python = {
enable = true;
venv = {
enable = true;
requirements = ./requirements.txt;
};
};
packages = [
pkgs.git
pkgs.zbar
];
}

View File

@@ -1,14 +0,0 @@
inputs:
nixpkgs:
url: github:cachix/devenv-nixpkgs/rolling
# If you're using non-OSS software, you can set allowUnfree to true.
# allowUnfree: true
# If you're willing to use a package that's vulnerable
# permittedInsecurePackages:
# - "openssl-1.1.1w"
# If you have more than one devenv you can merge them
#imports:
# - ./backend

View File

@@ -1,4 +0,0 @@
alias ll='ls -lh'
alias la='ls -lha'
alias l='ls -alhF'
alias ls-l='ls -lh'

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.5 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 38 KiB

View File

@@ -1,57 +0,0 @@
Generate from file: README.md
## Table of contents
- [Table of contents](#table-of-contents)
- [Download and run binary executable (🆕 since v2.1)](#download-and-run-binary-executable--since-v21)
- [MacOS](#macos)
- [Usage](#usage)
- [Capture QR codes from camera (🆕 since version 2.0)](#capture-qr-codes-from-camera--since-version-20)
- [With builtin QR decoder from image files (🆕 since version 2.0)](#with-builtin-qr-decoder-from-image-files--since-version-20)
- [With external QR decoder app from text files](#with-external-qr-decoder-app-from-text-files)
- [Installation of Python script (recommended for developers or advanced users)](#installation-of-python-script-recommended-for-developers-or-advanced-users)
- [Installation of optional shared system libraries (recommended)](#installation-of-optional-shared-system-libraries-recommended)
- [Program help: arguments and options](#program-help-arguments-and-options)
- [Examples](#examples)
- [Printing otp secrets from text file](#printing-otp-secrets-from-text-file)
- [Printing otp secrets from image file](#printing-otp-secrets-from-image-file)
- [Writing otp secrets to csv file](#writing-otp-secrets-to-csv-file)
- [Writing otp secrets to json file](#writing-otp-secrets-to-json-file)
- [Printing otp secrets multiple files](#printing-otp-secrets-multiple-files)
- [Printing otp secrets from stdin (text)](#printing-otp-secrets-from-stdin-text)
- [Printing otp secrets from stdin (image)](#printing-otp-secrets-from-stdin-image)
- [Printing otp secrets csv to stdout](#printing-otp-secrets-csv-to-stdout)
- [Printing otp secrets csv to stdout without header line](#printing-otp-secrets-csv-to-stdout-without-header-line)
- [Reading from stdin and printing to stdout](#reading-from-stdin-and-printing-to-stdout)
- [Features](#features)
- [KeePass](#keepass)
- [How to export otp secrets from Google Authenticator app](#how-to-export-otp-secrets-from-google-authenticator-app)
- [Glossary](#glossary)
- [Alternative installation methods](#alternative-installation-methods)
- [pip using github](#pip-using-github)
- [local pip](#local-pip)
- [pipenv](#pipenv)
- [Visual Studio Code Remote - Containers / VSCode devcontainer](#visual-studio-code-remote---containers--vscode-devcontainer)
- [venv](#venv)
- [devbox 1](#devbox-1)
- [devbox 2](#devbox-2)
- [docker](#docker)
- [More docker examples](#more-docker-examples)
- [Tests](#tests)
- [PyTest](#pytest)
- [unittest](#unittest)
- [VSCode Setup](#vscode-setup)
- [Development](#development)
- [Build](#build)
- [Upgrade pip Packages](#upgrade-pip-packages)
- [Build docker images](#build-docker-images)
- [Create executables with pyinstaller](#create-executables-with-pyinstaller)
- [Full local build (bash)](#full-local-build-bash)
- [Technical background](#technical-background)
- [References](#references)
- [Issues](#issues)
- [Problems and Troubleshooting](#problems-and-troubleshooting)
- [Windows error message](#windows-error-message)
- [Related projects](#related-projects)
Table of contents generated.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.1 KiB

View File

@@ -1,3 +0,0 @@
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=scito/extract_otp_secrets&type=Date)](https://star-history.com/#scito/extract_otp_secrets&Date)

View File

@@ -1,22 +0,0 @@
#!/bin/sh
# Create a folder (named dmg) to prepare our DMG in (if it doesn't already exist).
# https://www.pythonguis.com/tutorials/packaging-pyqt5-applications-pyinstaller-macos-dmg/
mkdir -p dist/dmg
# Empty the dmg folder.
rm -r dist/dmg/*
# Copy the app bundle to the dmg folder.
cp -r "dist/extract_otp_secrets.app" dist/dmg
# If the DMG already exists, delete it.
test -f "dist/extract_otp_secrets.dmg" && rm "dist/extract_otp_secrets.dmg"
create-dmg \
--volname "Extract OTP Secrets" \
--window-pos 200 120 \
--window-size 600 300 \
--icon-size 100 \
--hide-extension "extract_otp_secrets.app" \
--app-drop-link 425 120 \
"dist/extract_otp_secrets.dmg" \
"dist/dmg/"

View File

@@ -1,111 +0,0 @@
# -*- mode: python ; coding: utf-8 -*-
# https://www.pythonguis.com/tutorials/packaging-pyqt5-applications-pyinstaller-macos-dmg/
# https://developer.apple.com/library/archive/documentation/CoreFoundation/Conceptual/CFBundles/BundleTypes/BundleTypes.html
block_cipher = None
a = Analysis(
['src/extract_otp_secrets.py'],
pathex=[],
binaries=[],
datas=[('$macos_python_path/__yolo_v3_qr_detector/', '__yolo_v3_qr_detector/')],
hiddenimports=[],
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
exe = EXE(
pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
name='extract_otp_secrets',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
console=True,
disable_windowed_traceback=False,
argv_emulation=True,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
app = BUNDLE(
exe,
name='extract_otp_secrets.app',
icon=None,
bundle_identifier='ch.scito.tools.extract_otp_secrets',
version='$VERSION_STR',
info_plist={
'NSPrincipalClass': 'NSApplication',
'NSAppleScriptEnabled': False,
'NSHumanReadableCopyright': 'Copyright © $COPYRIGHT_YEARS Scito.',
'CFBundleDocumentTypes': [
# Reference: https://chromium.googlesource.com/chromium/src/+/lkgr/chrome/app/app-Info.plist
# https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/understanding_utis/understand_utis_conc/understand_utis_conc.html#//apple_ref/doc/uid/TP40001319-CH202-SW6
# https://developer.apple.com/documentation/uniformtypeidentifiers/system-declared_uniform_type_identifiers
{
'CFBundleTypeName': 'GIF image',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['com.compuserve.gif'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
{
'CFBundleTypeName': 'JPEG image',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['public.jpeg'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
{
'CFBundleTypeName': 'PNG image',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['public.png'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
{
'CFBundleTypeName': 'WebP image',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['org.webmproject.webp'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
{
'CFBundleTypeName': 'Tiff image',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['public.tiff'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
{
'CFBundleTypeName': 'Bmp image',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['com.microsoft.bmp'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
{
'CFBundleTypeName': 'Plain text document',
'CFBundleTypeIconFile': 'document.icns',
'LSItemContentTypes': ['public.plain-text'],
'LSHandlerRank': 'Alternate',
'NSExportableTypes': ['public.json','public.comma-separated-values-text','public.plain-text'],
},
],
},
)

View File

@@ -1,46 +0,0 @@
# UTF-8
#
# For more details about fixed file info 'ffi' see:
# http://msdn.microsoft.com/en-us/library/ms646997.aspx
VSVersionInfo(
ffi=FixedFileInfo(
# The elements of each tuple represent 16-bit values from most-significant to least-significant. For example the value (2, 0, 4, 0) resolves to 0002000000040000 in hex.
# filevers and prodvers should be always a tuple with four items: (1, 2, 3, 4)
# Set not needed items to zero 0.
filevers=($VERSION_MAJOR, $VERSION_MINOR, $VERSION_PATCH, $VERSION_BUILD),
prodvers=($VERSION_MAJOR, $VERSION_MINOR, $VERSION_PATCH, $VERSION_BUILD),
# Contains a bitmask that specifies the valid bits 'flags'r
mask=0x3f,
# Contains a bitmask that specifies the Boolean attributes of the file.
flags=0x0,
# The operating system for which this file was designed.
# 0x4 - NT and there is no need to change it.
OS=0x4,
# The general type of file.
# 0x1 - the file is an application.
fileType=0x1,
# The function of the file.
# 0x0 - the function is not defined for this fileType
subtype=0x0,
# Creation date and time stamp.
date=(0, 0)
),
kids=[
StringFileInfo(
[
StringTable(
# 0x0409 (U.S. English) + 04B0 (1200 = Unicode), https://learn.microsoft.com/en-us/windows/win32/menurc/stringfileinfo-block
'040904B0',
[StringStruct('CompanyName', 'scito'),
StringStruct('FileDescription', 'extract_otp_secrets'),
StringStruct('FileVersion', '$VERSION_STR'),
StringStruct('InternalName', 'extract_otp_secrets'),
StringStruct('LegalCopyright', 'Copyright © $YEARS Scito.'),
StringStruct('OriginalFilename', 'extract_otp_secrets.exe'),
StringStruct('ProductName', 'extract_otp_secrets'),
StringStruct('ProductVersion', '$VERSION_STR')])
]),
# 1033 (0x0409 = U.S. English), 1200 (Unicode)
VarFileInfo([VarStruct('Translation', [0, 1200])])
]
)

View File

@@ -1 +1,4 @@
[mypy]
[mypy-protobuf_generated_python.*]
ignore_errors = True

View File

@@ -1,11 +1,8 @@
[build-system]
requires = [
"pip",
"nuitka",
"setuptools>=64.0.0", "wheel>=0.37.0", "pip",
# https://setuptools-git-versioning.readthedocs.io/en/latest/differences.html
"setuptools>=64.0.0",
"setuptools-git-versioning",
"wheel>=0.37.0",
]
build-backend = "setuptools.build_meta"
@@ -19,33 +16,28 @@ classifiers = [
"Environment :: Win32 (MS Windows)",
"Topic :: System :: Archiving :: Backup",
"Topic :: Utilities",
"Topic :: Security",
"Topic :: Multimedia :: Graphics :: Capture :: Digital Camera",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Intended Audience :: End Users/Desktop",
"Intended Audience :: Developers",
"Intended Audience :: System Administrators",
"Programming Language :: Python",
"Operating System :: POSIX :: Linux",
"Operating System :: Microsoft :: Windows :: Windows 10",
"Operating System :: Microsoft :: Windows :: Windows 11",
"Operating System :: MacOS",
"Natural Language :: English",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"Typing :: Typed",
]
dependencies = [
"colorama>=0.4.6",
"opencv-contrib-python",
"Pillow",
"protobuf",
"pyzbar",
"qrcode",
"qreader<2.0.0",
"Pillow",
"qreader",
"pyzbar",
"opencv-contrib-python<=4.7.0; sys_platform == 'darwin'",
"opencv-contrib-python; sys_platform != 'darwin'",
"typing_extensions; python_version<='3.7'",
"colorama>=0.4.6",
]
description = "Extracts one time password (OTP) secrets from QR codes exported by two-factor authentication (2FA) apps such as 'Google Authenticator'"
dynamic = ["version"]
@@ -54,7 +46,7 @@ license = {text = "GNU General Public License v3 (GPLv3)"}
readme = "README.md"
authors = [{name = "scito", email = "info@scito.ch"}]
maintainers = [{name = "scito", email = "info@scito.ch"}]
requires-python = ">=3.8, <4"
requires-python = ">=3.7, <4"
scripts = {extract_otp_secrets = "extract_otp_secrets:sys_main"}
urls = {Project-URL = "https://github.com/scito/extract_otp_secrets", Bug-Reports = "https://github.com/scito/extract_otp_secrets/issues", Source = "https://github.com/scito/extract_otp_secrets"}

View File

@@ -1,15 +1,10 @@
build
flake8
gfm-toc
mypy
mypy-protobuf
nuitka
pyinstaller
types-protobuf
pylint
pytest
pytest-cov
pytest-mock
pytest-cov
setuptools
setuptools-git-versioning
types-protobuf
wheel
build

View File

@@ -1,7 +1,9 @@
colorama>=0.4.6
opencv-contrib-python
Pillow
protobuf
pyzbar
qrcode
qreader<2.0.0
Pillow
qreader
opencv-contrib-python<=4.7.0; sys_platform == 'darwin'
opencv-contrib-python; sys_platform != 'darwin'
pyzbar
typing_extensions; python_version<='3.7'
colorama>=0.4.6

View File

@@ -1,5 +1,3 @@
#!/bin/sh
cd /extract
mkdir -p tests
ln -sf /extract/data tests/data
pip install -U pytest pytest-mock && pytest "$@"
pip install -U pytest pytest-mock && pip install --no-deps . && pytest "$@"

View File

@@ -2,14 +2,10 @@
name = extract_otp_secrets
[options]
python_requires = >=3.8, <4
python_requires = >=3.7, <4
py_modules = extract_otp_secrets, protobuf_generated_python.google_auth_pb2
package_dir =
=src
platforms =
Linux
Windows
MacOS
# packages=find:
# [options.packages.find]

View File

@@ -5,7 +5,7 @@
# Source code available on https://github.com/scito/extract_otp_secrets
#
# Technical background:
# The export QR code from "Google Authenticator" contains the URL "otpauth-migration://offline?data=".
# The export QR code from "Google Authenticator" contains the URL "otpauth-migration://offline?data=...".
# The data parameter is a base64 encoded proto3 message (Google Protocol Buffers).
#
# Command for regeneration of Python code from proto3 message definition file (only necessary in case of changes of the proto3 message definition):
@@ -36,53 +36,42 @@ import argparse
import base64
import csv
import fileinput
import glob
import json
import os
import platform
import re
import sys
import urllib.parse as urlparse
from enum import Enum, IntEnum
from importlib.metadata import PackageNotFoundError, version
from typing import (Any, Final, List, Optional, Sequence, TextIO, Tuple,
TypedDict, Union)
from typing import Any, List, Optional, TextIO, Tuple, Union
# workaround for PYTHON <= 3.7: compatibility
if sys.version_info >= (3, 8):
from typing import Final, TypedDict
else:
from typing_extensions import Final, TypedDict
import colorama
from qrcode import QRCode # type: ignore
import protobuf_generated_python.google_auth_pb2 as pb
import colorama
debug_mode = '-d' in sys.argv[1:] or '--debug' in sys.argv[1:]
quiet = '-q' in sys.argv[1:] or '--quiet' in sys.argv[1:]
headless: bool = False
try:
import cv2
import numpy as np
import cv2.typing
import cv2 # type: ignore # TODO use cv2 types if available
try:
import tkinter
import tkinter.filedialog
import tkinter.messagebox
except ImportError:
headless = True
import numpy as np # TODO use numpy types if available
try:
import pyzbar.pyzbar as zbar # type: ignore
from qreader import QReader # type: ignore
zbar_available = True
except Exception as e:
if not quiet:
print(f"""
WARN: Cannot import pyzbar module. This problem is probably due to the missing zbar shared library. (The zbar library is optional.)
See in README.md for the installation of the zbar shared library.
except ImportError as e:
print(f"""
ERROR: Cannot import QReader module. This problem is probably due to the missing zbar shared library.
On Linux and macOS libzbar0 must be installed.
See in README.md for the installation of the libzbar0.
Exception: {e}\n""", file=sys.stderr)
zbar_available = False
if debug_mode:
raise e
raise e
# Types
# workaround for PYTHON <= 3.9: Final[tuple[int]]
@@ -94,8 +83,9 @@ Exception: {e}\n""", file=sys.stderr)
FONT_SCALE: Final[float] = 1.3
FONT_THICKNESS: Final[int] = 1
FONT_LINE_STYLE: Final[int] = cv2.LINE_AA
FONT_COLOR: Final[ColorBGR] = 255, 0, 0
FONT_COLOR: Final[ColorBGR] = (255, 0, 0)
BOX_THICKNESS: Final[int] = 5
# workaround for PYTHON <= 3.7: must use () for assignments
WINDOW_X: Final[int] = 0
WINDOW_Y: Final[int] = 1
WINDOW_WIDTH: Final[int] = 2
@@ -104,28 +94,28 @@ Exception: {e}\n""", file=sys.stderr)
TEXT_HEIGHT: Final[int] = 1
BORDER: Final[int] = 5
START_Y: Final[int] = 20
START_POS_TEXT: Final[Point] = BORDER, START_Y
NORMAL_COLOR: Final[ColorBGR] = 255, 0, 255
SUCCESS_COLOR: Final[ColorBGR] = 0, 255, 0
FAILURE_COLOR: Final[ColorBGR] = 0, 0, 255
START_POS_TEXT: Final[Point] = (BORDER, START_Y)
NORMAL_COLOR: Final[ColorBGR] = (255, 0, 255)
SUCCESS_COLOR: Final[ColorBGR] = (0, 255, 0)
FAILURE_COLOR: Final[ColorBGR] = (0, 0, 255)
CHAR_DX: Final[int] = (lambda text: cv2.getTextSize(text, FONT, FONT_SCALE, FONT_THICKNESS)[0][TEXT_WIDTH] // len(text))("28 QR codes capturedMMM")
FONT_DY: Final[int] = cv2.getTextSize("M", FONT, FONT_SCALE, FONT_THICKNESS)[0][TEXT_HEIGHT] + 5
WINDOW_NAME: Final[str] = "Extract OTP Secrets: Capture QR Codes from Camera"
TextPosition = Enum('TextPosition', ['LEFT', 'RIGHT'])
cv2_available = True
qreader_available = True
except ImportError as e:
cv2_available = False
qreader_available = False
if debug_mode:
raise e
# Workaround for PYTHON <= 3.9: Generally Union[int, None] used instead of int | None
# Workaround for PYTHON <= 3.9: Union[int, None] used instead of int | None
# Types
Args = argparse.Namespace
OtpUrl = str
# Workaround for PYTHON <= 3.9: Otp = TypedDict('Otp', {'name': str, 'secret': str, 'issuer': str, 'type': str, 'counter': int | None, 'url': OtpUrl})
# workaround for PYTHON <= 3.7: Otp = TypedDict('Otp', {'name': str, 'secret': str, 'issuer': str, 'type': str, 'counter': int | None, 'url': OtpUrl})
Otp = TypedDict('Otp', {'name': str, 'secret': str, 'issuer': str, 'type': str, 'counter': Union[int, None], 'url': OtpUrl})
# workaround for PYTHON <= 3.9: Otps = list[Otp]
Otps = List[Otp]
@@ -138,14 +128,11 @@ LogLevel = IntEnum('LogLevel', ['QUIET', 'NORMAL', 'VERBOSE', 'MORE_VERBOSE', 'D
# Constants
CAMERA: Final[str] = 'camera'
CV2_QRMODES: List[str] = [QRMode.CV2.name, QRMode.CV2_WECHAT.name]
# Global variable declaration
verbose: IntEnum = LogLevel.NORMAL
quiet: bool = False
colored: bool = True
executable: bool = False
__version__: str
tk_root: tkinter.Tk
def sys_main() -> None:
@@ -153,7 +140,6 @@ def sys_main() -> None:
def main(sys_args: list[str]) -> None:
global executable, tk_root, headless
# allow to use sys.stdout with with (avoid closing)
sys.stdout.close = lambda: None # type: ignore
# set encoding to utf-8, needed for Windows
@@ -164,157 +150,63 @@ def main(sys_args: list[str]) -> None:
# StringIO in tests do not have all attributes, ignore it
pass
# https://pyinstaller.org/en/stable/runtime-information.html#run-time-information
executable = getattr(sys, 'frozen', False) and hasattr(sys, '_MEIPASS')
if cv2_available and not headless:
try:
tk_root = tkinter.Tk()
tk_root.withdraw()
except tkinter.TclError:
headless = True
args = parse_args(sys_args)
if colored:
colorama.just_fix_windows_console()
if verbose >= LogLevel.DEBUG:
print(f"Version: {get_full_version()}\n")
if args.debug:
sys.exit(0 if do_debug_checks() else 1)
otps = extract_otps(args)
write_csv(args.csv, otps)
write_keepass_csv(args.keepass, otps)
write_json(args.json, otps)
write_txt(args.txt, otps, True)
write_urls(args.urls, otps)
# workaround for PYTHON <= 3.9 use: pb.MigrationPayload | None
def get_payload_from_otp_url(otp_url: str, i: int, source: str) -> Optional[pb.MigrationPayload]:
'''Extracts the otp migration payload from an otp url. This function is the core of the this appliation.'''
if not is_opt_url(otp_url, source):
return None
parsed_url = urlparse.urlparse(otp_url)
if verbose >= LogLevel.DEBUG: log_debug(f"parsed_url={parsed_url}")
try:
params = urlparse.parse_qs(parsed_url.query, strict_parsing=True)
except Exception: # workaround for PYTHON <= 3.10
params = {}
if verbose >= LogLevel.DEBUG: log_debug(f"querystring params={params}")
if 'data' not in params:
log_error(f"could not parse query parameter in input url\nsource: {source}\nurl: {otp_url}")
return None
data_base64 = params['data'][0]
if verbose >= LogLevel.DEBUG: log_debug(f"data_base64={data_base64}")
data_base64_fixed = data_base64.replace(' ', '+')
if verbose >= LogLevel.DEBUG: log_debug(f"data_base64_fixed={data_base64_fixed}")
data = base64.b64decode(data_base64_fixed, validate=True)
payload = pb.MigrationPayload()
try:
payload.ParseFromString(data)
except Exception as e:
abort(f"Cannot decode otpauth-migration migration payload.\n"
f"data={data_base64}", e)
if verbose >= LogLevel.DEBUG: log_debug(f"\n{i}. Payload Line", payload, sep='\n')
return payload
def extract_otp_from_otp_url(otpauth_migration_url: str, otps: Otps, urls_count: int, infile: str, args: Args) -> int:
'''Converts the otp migration payload into a normal Python dictionary. This function is the core of the this appliation.'''
payload = get_payload_from_otp_url(otpauth_migration_url, urls_count, infile)
if not payload:
return 0
new_otps_count = 0
# pylint: disable=no-member
for raw_otp in payload.otp_parameters:
if verbose: print(f"\n{len(otps) + 1}. Secret")
secret = convert_secret_from_bytes_to_base32_str(raw_otp.secret)
if verbose >= LogLevel.DEBUG: log_debug('OTP enum type:', get_enum_name_by_number(raw_otp, 'type'))
otp_type = get_otp_type_str_from_code(raw_otp.type)
otp_url = build_otp_url(secret, raw_otp)
otp: Otp = {
"name": raw_otp.name,
"secret": secret,
"issuer": raw_otp.issuer,
"type": otp_type,
"counter": raw_otp.counter if raw_otp.type == 1 else None,
"url": otp_url
}
if otp not in otps or not args.ignore:
otps.append(otp)
new_otps_count += 1
if not quiet:
print_otp(otp)
if args.printqr:
print_qr(otp_url)
if args.saveqr:
save_qr_image(otp, args.saveqr, len(otps))
if not quiet:
print()
elif args.ignore and not quiet:
eprint(f"Ignored duplicate otp: {otp['name']}", f" / {otp['issuer']}\n" if otp['issuer'] else '\n', sep='')
return new_otps_count
write_csv(args, otps)
write_keepass_csv(args, otps)
write_json(args, otps)
def parse_args(sys_args: list[str]) -> Args:
global verbose, quiet, colored
cmd = f"python {name}" if (name := os.path.basename(sys.argv[0])).endswith('.py') else f"{name}"
description_text = "Extracts one time password (OTP) secrets from QR codes exported by two-factor authentication (2FA) apps"
if cv2_available:
if qreader_available:
description_text += "\nIf no infiles are provided, a GUI window starts and QR codes are captured from the camera."
example_text = f"""examples:
{cmd}
{cmd} example_*.txt
{cmd} - < example_export.txt
{cmd} --csv - example_*.png | tail -n+2
{cmd} = < example_export.png"""
example_text = """examples:
python extract_otp_secrets.py
python extract_otp_secrets.py example_*.txt
python extract_otp_secrets.py - < example_export.txt
python extract_otp_secrets.py --csv - example_*.png | tail -n+2
python extract_otp_secrets.py = < example_export.png"""
arg_parser = argparse.ArgumentParser(formatter_class=lambda prog: argparse.RawTextHelpFormatter(prog, max_help_position=32),
description=description_text,
epilog=example_text)
arg_parser.add_argument('infile', help="""a) file or - for stdin with 'otpauth-migration://...' URLs separated by newlines, lines starting with # are ignored;
b) image file containing a QR code or = for stdin for an image containing a QR code""", nargs='*' if cv2_available else '+')
arg_parser.add_argument('--csv', '-c', help='export csv file, or - for stdout', metavar=('FILE'))
b) image file containing a QR code or = for stdin for an image containing a QR code""", nargs='*' if qreader_available else '+')
arg_parser.add_argument('--csv', '-c', help='export csv file or - for stdout', metavar=('FILE'))
arg_parser.add_argument('--keepass', '-k', help='export totp/hotp csv file(s) for KeePass, - for stdout', metavar=('FILE'))
arg_parser.add_argument('--json', '-j', help='export json file or - for stdout', metavar=('FILE'))
arg_parser.add_argument('--txt', '-t', help='export txt file or - for stdout', metavar=('FILE'))
arg_parser.add_argument('--urls', '-u', help='export file with list of otpauth urls, or - for stdout', metavar=('FILE'))
arg_parser.add_argument('--printqr', '-p', help='print QR code(s) as text to the terminal', action='store_true')
arg_parser.add_argument('--saveqr', '-s', help='save QR code(s) as images to directory', metavar=('DIR'))
if cv2_available:
arg_parser.add_argument('--printqr', '-p', help='print QR code(s) as text to the terminal (requires qrcode module)', action='store_true')
arg_parser.add_argument('--saveqr', '-s', help='save QR code(s) as images to the given folder (requires qrcode module)', metavar=('DIR'))
if qreader_available:
arg_parser.add_argument('--camera', '-C', help='camera number of system (default camera: 0)', default=0, type=int, metavar=('NUMBER'))
if not zbar_available:
arg_parser.add_argument('--qr', '-Q', help=f'QR reader (default: {QRMode.CV2.name})', type=str, choices=[QRMode.CV2.name, QRMode.CV2_WECHAT.name], default=QRMode.CV2.name)
else:
arg_parser.add_argument('--qr', '-Q', help=f'QR reader (default: {QRMode.ZBAR.name})', type=str, choices=[mode.name for mode in QRMode], default=QRMode.ZBAR.name)
arg_parser.add_argument('--qr', '-Q', help=f'QR reader (default: {QRMode.ZBAR.name})', type=str, choices=[mode.name for mode in QRMode], default=QRMode.ZBAR.name)
arg_parser.add_argument('-i', '--ignore', help='ignore duplicate otps', action='store_true')
arg_parser.add_argument('--no-color', '-n', help='do not use ANSI colors in console output', action='store_true')
arg_parser.add_argument('--version', '-V', help='print version and quit', action=PrintVersionAction)
output_group = arg_parser.add_mutually_exclusive_group()
output_group.add_argument('-d', '--debug', help='enter debug mode, do checks and quit', action='count')
output_group.add_argument('-v', '--verbose', help='verbose output', action='count')
output_group.add_argument('-q', '--quiet', help='no stdout output, except output set by -', action='store_true')
args = arg_parser.parse_args(sys_args)
colored = not args.no_color
if args.csv == '-' or args.json == '-' or args.keepass == '-' or args.txt == '-' or args.urls == '-':
if args.csv == '-' or args.json == '-' or args.keepass == '-':
args.quiet = args.q = True
verbose = args.verbose if args.verbose else LogLevel.NORMAL
if args.debug:
verbose = LogLevel.DEBUG
log_debug('Debug mode start')
log_debug(args)
quiet = True if args.quiet else False
if verbose: print(f"QReader installed: {cv2_available}")
if cv2_available:
if verbose: print(f"QReader installed: {qreader_available}")
if qreader_available:
if verbose >= LogLevel.VERBOSE: print(f"CV2 version: {cv2.__version__}")
if verbose: print(f"QR reading mode: {args.qr}\n")
@@ -328,6 +220,42 @@ def extract_otps(args: Args) -> Otps:
return extract_otps_from_files(args)
def get_color(new_otps_count: int, otp_url: str) -> ColorBGR:
if new_otps_count:
return SUCCESS_COLOR
else:
if otp_url:
return FAILURE_COLOR
else:
return NORMAL_COLOR
# TODO use cv2 types if available
def cv2_draw_box(img: Any, raw_pts: Any, color: ColorBGR) -> Any:
pts = np.array([raw_pts], np.int32)
pts = pts.reshape((-1, 1, 2))
cv2.polylines(img, [pts], True, color, BOX_THICKNESS)
return pts
# TODO use cv2 types if available
def cv2_print_text(img: Any, text: str, line_number: int, position: TextPosition, color: ColorBGR, opposite_len: Optional[int] = None) -> None:
window_dim = cv2.getWindowImageRect(WINDOW_NAME)
out_text = text
if opposite_len:
text_dim, _ = cv2.getTextSize(out_text, FONT, FONT_SCALE, FONT_THICKNESS)
actual_width = text_dim[TEXT_WIDTH] + opposite_len * CHAR_DX + 4 * BORDER
if actual_width >= window_dim[WINDOW_WIDTH]:
out_text = out_text[:(window_dim[WINDOW_WIDTH] - actual_width) // CHAR_DX] + '.'
text_dim, _ = cv2.getTextSize(out_text, FONT, FONT_SCALE, FONT_THICKNESS)
if position == TextPosition.LEFT:
pos = BORDER, START_Y + line_number * FONT_DY
else:
pos = window_dim[WINDOW_WIDTH] - text_dim[TEXT_WIDTH] - BORDER, START_Y + line_number * FONT_DY
cv2.putText(img, out_text, pos, FONT, FONT_SCALE, color, FONT_THICKNESS, FONT_LINE_STYLE)
def extract_otps_from_camera(args: Args) -> Otps:
if verbose: print("Capture QR codes from camera")
otp_urls: OtpUrls = []
@@ -338,8 +266,7 @@ def extract_otps_from_camera(args: Args) -> Otps:
cam = cv2.VideoCapture(args.camera)
cv2.namedWindow(WINDOW_NAME, cv2.WINDOW_AUTOSIZE)
if zbar_available:
qreader = QReader()
qreader = QReader()
cv2_qr = cv2.QRCodeDetector()
cv2_qr_wechat = cv2.wechat_qrcode.WeChatQRCode()
while True:
@@ -358,9 +285,9 @@ def extract_otps_from_camera(args: Args) -> Otps:
if otp_url:
new_otps_count = extract_otps_from_otp_url(otp_url, otp_urls, otps, args)
if found:
cv2_draw_box(img, [(bbox[0], bbox[1]), (bbox[2], bbox[1]), (bbox[2], bbox[3]), (bbox[0], bbox[3])], get_color(new_otps_count, otp_url))
cv2.rectangle(img, (bbox[0], bbox[1]), (bbox[2], bbox[3]), get_color(new_otps_count, otp_url), BOX_THICKNESS)
elif qr_mode == QRMode.ZBAR:
for qrcode in zbar.decode(img, symbols=[zbar.ZBarSymbol.QRCODE]):
for qrcode in zbar.decode(img):
otp_url = qrcode.data.decode('utf-8')
new_otps_count = extract_otps_from_otp_url(otp_url, otp_urls, otps, args)
cv2_draw_box(img, [qrcode.polygon], get_color(new_otps_count, otp_url))
@@ -368,7 +295,7 @@ def extract_otps_from_camera(args: Args) -> Otps:
if QRMode.CV2:
otp_url, raw_pts, _ = cv2_qr.detectAndDecode(img)
else:
otp_url, raw_pts = cv2_qr_wechat.detectAndDecode(img) # type: ignore # use proper cv2 types
otp_url, raw_pts = cv2_qr_wechat.detectAndDecode(img)
if raw_pts is not None:
if otp_url:
new_otps_count = extract_otps_from_otp_url(otp_url, otp_urls, otps, args)
@@ -380,16 +307,15 @@ def extract_otps_from_camera(args: Args) -> Otps:
qr_mode = next_qr_mode(qr_mode)
continue
cv2_print_text(img, f"Mode: {qr_mode.name} (Hit SPACE to change)", 0, TextPosition.LEFT, FONT_COLOR, 20)
cv2_print_text(img, "Press ESC to quit", 1, TextPosition.LEFT, FONT_COLOR, 17)
cv2_print_text(img, "Press c,j,k,t,u to save as csv/json/keepass/txt/urls file", 2, TextPosition.LEFT, FONT_COLOR, None)
cv2_print_text(img, f"Mode: {qr_mode.name} (Hit space to change)", 0, TextPosition.LEFT, FONT_COLOR, 20)
cv2_print_text(img, "Hit ESC to quit", 1, TextPosition.LEFT, FONT_COLOR, 17)
cv2_print_text(img, f"{len(otp_urls)} QR code{'s'[:len(otp_urls) != 1]} captured", 0, TextPosition.RIGHT, FONT_COLOR)
cv2_print_text(img, f"{len(otps)} otp{'s'[:len(otps) != 1]} extracted", 1, TextPosition.RIGHT, FONT_COLOR)
cv2.imshow(WINDOW_NAME, img)
quit, qr_mode = cv2_handle_pressed_keys(qr_mode, otps)
quit, qr_mode = cv2_handle_pressed_keys(qr_mode)
if quit:
break
@@ -399,109 +325,14 @@ def extract_otps_from_camera(args: Args) -> Otps:
return otps
def get_color(new_otps_count: int, otp_url: str) -> ColorBGR:
if new_otps_count:
return SUCCESS_COLOR
else:
if otp_url:
return FAILURE_COLOR
else:
return NORMAL_COLOR
# TODO use proper cv2 types if available
def cv2_draw_box(img: cv2.typing.MatLike, raw_pts: cv2.typing.MatLike | list[tuple[Any, Any]], color: ColorBGR) -> np.ndarray[Any, np.dtype[np.int32]]:
pts = np.array([raw_pts], np.int32)
pts = pts.reshape((-1, 1, 2))
cv2.polylines(img, [pts], True, color, BOX_THICKNESS)
return pts
def cv2_print_text(img: cv2.typing.MatLike, text: str, line_number: int, position: TextPosition, color: ColorBGR, opposite_len: Optional[int] = None) -> None:
window_dim = cv2.getWindowImageRect(WINDOW_NAME)
out_text = text
if opposite_len:
text_dim, _ = cv2.getTextSize(out_text, FONT, FONT_SCALE, FONT_THICKNESS)
actual_width = text_dim[TEXT_WIDTH] + opposite_len * CHAR_DX + 4 * BORDER
if actual_width >= window_dim[WINDOW_WIDTH]:
out_text = out_text[:(window_dim[WINDOW_WIDTH] - actual_width) // CHAR_DX] + '.'
text_dim, _ = cv2.getTextSize(out_text, FONT, FONT_SCALE, FONT_THICKNESS)
if position == TextPosition.LEFT:
pos = BORDER, START_Y + line_number * FONT_DY
else:
pos = window_dim[WINDOW_WIDTH] - text_dim[TEXT_WIDTH] - BORDER, START_Y + line_number * FONT_DY
cv2.putText(img, out_text, pos, FONT, FONT_SCALE, color, FONT_THICKNESS, FONT_LINE_STYLE)
def cv2_handle_pressed_keys(qr_mode: QRMode, otps: Otps) -> Tuple[bool, QRMode]:
def cv2_handle_pressed_keys(qr_mode: QRMode) -> Tuple[bool, QRMode]:
key = cv2.waitKey(1) & 0xFF
quit = False
if key == 27 or key == ord('q') or key == ord('Q') or key == 13:
if key == 27 or key == ord('q') or key == 13:
# ESC or Enter or q pressed
quit = True
elif (key == ord('c') or key == ord('C')) and is_not_headless():
if has_no_otps_show_warning(otps):
pass
else:
file_name = tkinter.filedialog.asksaveasfilename(
title="Save extracted otp secrets as CSV",
defaultextension='.csv',
filetypes=[('CSV', '*.csv'), ('All', '*.*')]
)
tk_root.update()
if len(file_name) > 0:
write_csv(file_name, otps)
elif (key == ord('j') or key == ord('J')) and is_not_headless():
if has_no_otps_show_warning(otps):
pass
else:
file_name = tkinter.filedialog.asksaveasfilename(
title="Save extracted otp secrets as JSON",
defaultextension='.json',
filetypes=[('JSON', '*.json'), ('All', '*.*')]
)
tk_root.update()
if len(file_name) > 0:
write_json(file_name, otps)
elif (key == ord('k') or key == ord('K')) and is_not_headless():
if has_no_otps_show_warning(otps):
pass
else:
file_name = tkinter.filedialog.asksaveasfilename(
title="Save extracted otp secrets as KeePass CSV file(s)",
defaultextension='.csv',
filetypes=[('CSV', '*.csv'), ('All', '*.*')]
)
tk_root.update()
if len(file_name) > 0:
write_keepass_csv(file_name, otps)
elif (key == ord('t') or key == ord('T')) and is_not_headless():
if has_no_otps_show_warning(otps):
pass
else:
file_name = tkinter.filedialog.asksaveasfilename(
title="Save extracted otp secrets as text",
defaultextension='.txt',
filetypes=[('Text', '*.txt'), ('All', '*.*')]
)
tk_root.update()
if len(file_name) > 0:
write_txt(file_name, otps, True)
elif (key == ord('u') or key == ord('U')) and is_not_headless():
if has_no_otps_show_warning(otps):
pass
else:
file_name = tkinter.filedialog.asksaveasfilename(
title="Save extracted otp secrets as list of urls",
defaultextension='.txt',
filetypes=[('Text', '*.txt'), ('All', '*.*')]
)
tk_root.update()
if len(file_name) > 0:
write_urls(file_name, otps)
elif key == 32:
qr_mode = next_valid_qr_mode(qr_mode, zbar_available)
qr_mode = next_qr_mode(qr_mode)
if verbose >= LogLevel.MORE_VERBOSE: print(f"QR reading mode: {qr_mode}")
if cv2.getWindowProperty(WINDOW_NAME, cv2.WND_PROP_VISIBLE) < 1:
# Window close clicked
@@ -528,20 +359,14 @@ def extract_otps_from_files(args: Args) -> Otps:
files_count = urls_count = otps_count = 0
if verbose: print(f"Input files: {args.infile}")
for infile_raw in args.infile:
expanded_infiles = glob.glob(infile_raw)
if not expanded_infiles:
expanded_infiles = [infile_raw]
if verbose >= LogLevel.DEBUG: log_debug(f"Could not expand input files, fallback to infile")
if verbose >= LogLevel.DEBUG: log_debug(f"Expanded input files: {expanded_infiles}")
for infile in expanded_infiles:
if verbose >= LogLevel.MORE_VERBOSE: log_verbose(f"Processing infile {infile}")
files_count += 1
for line in get_otp_urls_from_file(infile, args):
if verbose >= LogLevel.MORE_VERBOSE: log_verbose(line)
if line.startswith('#') or line == '': continue
urls_count += 1
otps_count += extract_otp_from_otp_url(line, otps, urls_count, infile, args)
for infile in args.infile:
if verbose >= LogLevel.MORE_VERBOSE: log_verbose(f"Processing infile {infile}")
files_count += 1
for line in get_otp_urls_from_file(infile, args):
if verbose >= LogLevel.MORE_VERBOSE: log_verbose(line)
if line.startswith('#') or line == '': continue
urls_count += 1
otps_count += extract_otp_from_otp_url(line, otps, urls_count, infile, args)
if verbose: print(f"Extracted {otps_count} otp{'s'[:otps_count != 1]} from {urls_count} otp url{'s'[:urls_count != 1]} by reading {files_count} infile{'s'[:files_count != 1]}")
return otps
@@ -555,7 +380,7 @@ def get_otp_urls_from_file(filename: str, args: Args) -> OtpUrls:
return lines
# could not process text file, try reading as image
if filename != '-' and cv2_available:
if filename != '-' and qreader_available:
return convert_img_to_otp_urls(filename, args)
return []
@@ -589,6 +414,45 @@ def read_lines_from_text_file(filename: str) -> list[str]:
return lines
def extract_otp_from_otp_url(otpauth_migration_url: str, otps: Otps, urls_count: int, infile: str, args: Args) -> int:
payload = get_payload_from_otp_url(otpauth_migration_url, urls_count, infile)
if not payload:
return 0
new_otps_count = 0
# pylint: disable=no-member
for raw_otp in payload.otp_parameters:
if verbose: print(f"\n{len(otps) + 1}. Secret")
secret = convert_secret_from_bytes_to_base32_str(raw_otp.secret)
if verbose >= LogLevel.DEBUG: log_debug('OTP enum type:', get_enum_name_by_number(raw_otp, 'type'))
otp_type = get_otp_type_str_from_code(raw_otp.type)
otp_url = build_otp_url(secret, raw_otp)
otp: Otp = {
"name": raw_otp.name,
"secret": secret,
"issuer": raw_otp.issuer,
"type": otp_type,
"counter": raw_otp.counter if raw_otp.type == 1 else None,
"url": otp_url
}
if otp not in otps or not args.ignore:
otps.append(otp)
new_otps_count += 1
if not quiet:
print_otp(otp)
if args.printqr:
print_qr(args, otp_url)
if args.saveqr:
save_qr(otp, args, len(otps))
if not quiet:
print()
elif args.ignore and not quiet:
eprint(f"Ignored duplicate otp: {otp['name']}", f" / {otp['issuer']}\n" if otp['issuer'] else '\n', sep='')
return new_otps_count
def convert_img_to_otp_urls(filename: str, args: Args) -> OtpUrls:
if verbose: print(f"Reading image {filename}")
try:
@@ -599,7 +463,7 @@ def convert_img_to_otp_urls(filename: str, args: Args) -> OtpUrls:
stdin = sys.stdin.buffer.read()
except AttributeError:
# Workaround for pytest, since pytest cannot monkeypatch sys.stdin.buffer
stdin = sys.stdin.read()
stdin = sys.stdin.read() # type: ignore # Workaround for pytest fixtures
if not stdin:
log_warn("stdin is empty")
try:
@@ -634,7 +498,7 @@ def decode_qr_img_otp_urls(img: Any, qr_mode: QRMode) -> OtpUrls:
otp_url, _ = cv2.wechat_qrcode.WeChatQRCode().detectAndDecode(img)
otp_urls += list(otp_url)
elif qr_mode == QRMode.ZBAR:
qrcodes = zbar.decode(img, symbols=[zbar.ZBarSymbol.QRCODE])
qrcodes = zbar.decode(img)
otp_urls += [qrcode.data.decode('utf-8') for qrcode in qrcodes]
else:
assert False, f"Wrong QReader mode {qr_mode.name}"
@@ -642,6 +506,37 @@ def decode_qr_img_otp_urls(img: Any, qr_mode: QRMode) -> OtpUrls:
return otp_urls
# workaround for PYTHON <= 3.9 use: pb.MigrationPayload | None
def get_payload_from_otp_url(otp_url: str, i: int, source: str) -> Optional[pb.MigrationPayload]:
'''Extracts the otp migration payload from an otp url. This function is the core of the this appliation.'''
if not is_opt_url(otp_url, source):
return None
parsed_url = urlparse.urlparse(otp_url)
if verbose >= LogLevel.DEBUG: log_debug(f"parsed_url={parsed_url}")
try:
params = urlparse.parse_qs(parsed_url.query, strict_parsing=True)
except Exception: # workaround for PYTHON <= 3.10
params = {}
if verbose >= LogLevel.DEBUG: log_debug(f"querystring params={params}")
if 'data' not in params:
log_error(f"could not parse query parameter in input url\nsource: {source}\nurl: {otp_url}")
return None
data_base64 = params['data'][0]
if verbose >= LogLevel.DEBUG: log_debug(f"data_base64={data_base64}")
data_base64_fixed = data_base64.replace(' ', '+')
if verbose >= LogLevel.DEBUG: log_debug(f"data_base64_fixed={data_base64_fixed}")
data = base64.b64decode(data_base64_fixed, validate=True)
payload = pb.MigrationPayload()
try:
payload.ParseFromString(data)
except Exception as e:
abort(f"Cannot decode otpauth-migration migration payload.\n"
f"data={data_base64}", e)
if verbose >= LogLevel.DEBUG: log_debug(f"\n{i}. Payload Line", payload, sep='\n')
return payload
def is_opt_url(otp_url: str, source: str) -> bool:
if not otp_url.startswith('otpauth-migration://'):
msg = f"input is not a otpauth-migration:// url\nsource: {source}\ninput: {otp_url}"
@@ -675,31 +570,28 @@ def build_otp_url(secret: str, raw_otp: pb.MigrationPayload.OtpParameters) -> st
return otp_url
def print_otp(otp: Otp, out: Optional[TextIO] = None) -> None:
print(f"Name: {otp['name']}", file=out)
print(f"Secret: {otp['secret']}", file=out)
if otp['issuer']: print(f"Issuer: {otp['issuer']}", file=out)
print(f"Type: {otp['type']}", file=out)
def print_otp(otp: Otp) -> None:
print(f"Name: {otp['name']}")
print(f"Secret: {otp['secret']}")
if otp['issuer']: print(f"Issuer: {otp['issuer']}")
print(f"Type: {otp['type']}")
if otp['type'] == 'hotp':
print(f"Counter: {otp['counter']}", file=out)
print(f"Counter: {otp['counter']}")
if verbose:
print(otp['url'], file=out)
print(otp['url'])
def write_url(otp: Otp, out: Optional[TextIO] = None) -> None:
print(otp['url'], file=out)
def save_qr_image(otp: Otp, dir: str, j: int) -> str:
def save_qr(otp: Otp, args: Args, j: int) -> str:
dir = args.saveqr
if not (os.path.exists(dir)): os.makedirs(dir, exist_ok=True)
pattern = re.compile(r'[\W_]+')
file_otp_name = pattern.sub('', otp['name'])
file_otp_issuer = pattern.sub('', otp['issuer'])
save_qr_image_file(otp['url'], f"{dir}/{j}-{file_otp_name}{'-' + file_otp_issuer if file_otp_issuer else ''}.png")
save_qr_file(args, otp['url'], f"{dir}/{j}-{file_otp_name}{'-' + file_otp_issuer if file_otp_issuer else ''}.png")
return file_otp_name
def save_qr_image_file(otp_url: OtpUrl, name: str) -> None:
def save_qr_file(args: Args, otp_url: OtpUrl, name: str) -> None:
qr = QRCode()
qr.add_data(otp_url)
img = qr.make_image(fill_color='black', back_color='white')
@@ -707,46 +599,28 @@ def save_qr_image_file(otp_url: OtpUrl, name: str) -> None:
img.save(name)
def print_qr(otp_url: str, out: Optional[TextIO] = None) -> None:
def print_qr(args: Args, otp_url: str) -> None:
qr = QRCode()
qr.add_data(otp_url)
qr.print_ascii(out)
qr.print_ascii()
def write_txt(file: str, otps: Otps, write_qr: bool = False) -> None:
if file and len(file) > 0 and len(otps) > 0:
with open_file_or_stdout(file) as outfile:
for otp in otps:
print_otp(otp, outfile)
if write_qr:
print_qr(otp['url'], outfile)
print(file=outfile)
def write_urls(file: str, otps: Otps) -> None:
if file and len(file) > 0 and len(otps) > 0:
with open_file_or_stdout(file) as outfile:
for otp in otps:
write_url(otp, outfile)
if not quiet: print(f"Exported {len(otps)} otp{'s'[:len(otps) != 1]} to otpauth url list file {file}")
def write_csv(file: str, otps: Otps) -> None:
if file and len(file) > 0 and len(otps) > 0:
with open_file_or_stdout_for_csv(file) as outfile:
def write_csv(args: Args, otps: Otps) -> None:
if args.csv and len(otps) > 0:
with open_file_or_stdout_for_csv(args.csv) as outfile:
writer = csv.DictWriter(outfile, otps[0].keys())
writer.writeheader()
writer.writerows(otps)
if not quiet: print(f"Exported {len(otps)} otp{'s'[:len(otps) != 1]} to csv {file}")
if not quiet: print(f"Exported {len(otps)} otp{'s'[:len(otps) != 1]} to csv {args.csv}")
def write_keepass_csv(file: str, otps: Otps) -> None:
if file and len(file) > 0 and len(otps) > 0:
def write_keepass_csv(args: Args, otps: Otps) -> None:
if args.keepass and len(otps) > 0:
has_totp = has_otp_type(otps, 'totp')
has_hotp = has_otp_type(otps, 'hotp')
if file != '-':
otp_filename_totp = file if has_totp != has_hotp else add_pre_suffix(file, "totp")
otp_filename_hotp = file if has_totp != has_hotp else add_pre_suffix(file, "hotp")
if args.keepass != '-':
otp_filename_totp = args.keepass if has_totp != has_hotp else add_pre_suffix(args.keepass, "totp")
otp_filename_hotp = args.keepass if has_totp != has_hotp else add_pre_suffix(args.keepass, "hotp")
else:
otp_filename_totp = otp_filename_hotp = '-'
if has_totp:
@@ -754,13 +628,13 @@ def write_keepass_csv(file: str, otps: Otps) -> None:
if has_hotp:
count_hotp_entries = write_keepass_htop_csv(otp_filename_hotp, otps)
if not quiet:
if has_totp and count_totp_entries: print(f"Exported {count_totp_entries} totp entrie{'s'[:count_totp_entries != 1]} to keepass csv file {otp_filename_totp}")
if has_hotp and count_hotp_entries: print(f"Exported {count_hotp_entries} hotp entrie{'s'[:count_hotp_entries != 1]} to keepass csv file {otp_filename_hotp}")
if count_totp_entries: print(f"Exported {count_totp_entries} totp entrie{'s'[:count_totp_entries != 1]} to keepass csv file {otp_filename_totp}")
if count_hotp_entries: print(f"Exported {count_hotp_entries} hotp entrie{'s'[:count_hotp_entries != 1]} to keepass csv file {otp_filename_hotp}")
def write_keepass_totp_csv(file: str, otps: Otps) -> int:
def write_keepass_totp_csv(otp_filename: str, otps: Otps) -> int:
count_entries = 0
with open_file_or_stdout_for_csv(file) as outfile:
with open_file_or_stdout_for_csv(otp_filename) as outfile:
writer = csv.DictWriter(outfile, ["Title", "User Name", "TimeOtp-Secret-Base32", "Group"])
writer.writeheader()
for otp in otps:
@@ -775,9 +649,9 @@ def write_keepass_totp_csv(file: str, otps: Otps) -> int:
return count_entries
def write_keepass_htop_csv(file: str, otps: Otps) -> int:
def write_keepass_htop_csv(otp_filename: str, otps: Otps) -> int:
count_entries = 0
with open_file_or_stdout_for_csv(file) as outfile:
with open_file_or_stdout_for_csv(otp_filename) as outfile:
writer = csv.DictWriter(outfile, ["Title", "User Name", "HmacOtp-Secret-Base32", "HmacOtp-Counter", "Group"])
writer.writeheader()
for otp in otps:
@@ -793,11 +667,11 @@ def write_keepass_htop_csv(file: str, otps: Otps) -> int:
return count_entries
def write_json(file: str, otps: Otps) -> None:
if file and len(file) > 0:
with open_file_or_stdout(file) as outfile:
def write_json(args: Args, otps: Otps) -> None:
if args.json:
with open_file_or_stdout(args.json) as outfile:
json.dump(otps, outfile, indent=4)
if not quiet: print(f"Exported {len(otps)} otp{'s'[:len(otps) != 1]} to json {file}")
if not quiet: print(f"Exported {len(otps)} otp{'s'[:len(otps) != 1]} to json {args.json}")
def has_otp_type(otps: Otps, otp_type: str) -> bool:
@@ -834,13 +708,6 @@ def check_file_exists(filename: str) -> None:
f"\ninput file: {filename}")
def has_no_otps_show_warning(otps: Otps) -> bool:
if len(otps) == 0:
tkinter.messagebox.showinfo(title="No data", message="There are no otp secrets to write")
tk_root.update() # dispose dialog
return len(otps) == 0
def is_binary(line: str) -> bool:
try:
line.startswith('#')
@@ -849,14 +716,6 @@ def is_binary(line: str) -> bool:
return True
def next_valid_qr_mode(qr_mode: QRMode, with_zbar: bool = True) -> QRMode:
ok = False
while not ok:
qr_mode = next_qr_mode(qr_mode)
ok = True if with_zbar else qr_mode.name in CV2_QRMODES
return qr_mode
def next_qr_mode(qr_mode: QRMode) -> QRMode:
return QRMode((qr_mode.value + 1) % len(QRMode))
@@ -867,70 +726,16 @@ def do_debug_checks() -> bool:
import cv2 # noqa: F401 # This is only a debug import
log_debug('Try: import numpy as np')
import numpy as np # noqa: F401 # This is only a debug import
log_debug('Try: import pyzbar.pyzbar as zbar')
import pyzbar.pyzbar as zbar # noqa: F401 # This is only a debug import
log_debug('Try: from qreader import QReader')
from qreader import QReader # noqa: F401 # This is only a debug import
print(color('\nDebug checks passed', colorama.Fore.GREEN))
return True
def is_not_headless() -> bool:
if headless:
log_warn("Cannot open dialog in headless mode")
return not headless
class PrintVersionAction(argparse.Action):
def __init__(self, option_strings: Sequence[str], dest: str, nargs: int = 0, **kwargs: Any) -> None:
super().__init__(option_strings, dest, nargs, **kwargs)
def __call__(self, parser: argparse.ArgumentParser, namespace: Args, values: Union[str, Sequence[Any], None], option_string: Optional[str] = None) -> None:
print_version()
parser.exit()
def print_version() -> None:
print(get_full_version())
def get_full_version() -> str:
version = get_raw_version()
meta = [
platform.python_implementation()
]
if executable: meta.append('exe')
meta.append(f"called as {'package' if __package__ else 'script'}")
return (
f"extract_otp_secrets {version} {platform.system()} {platform.machine()}"
f" Python {platform.python_version()}"
f" ({'/'.join(meta)})"
)
# https://setuptools-git-versioning.readthedocs.io/en/stable/runtime_version.html
def get_raw_version() -> str:
global __version__
try:
__version__ = version("extract_otp_secrets")
return __version__
except PackageNotFoundError:
# package is not installed
return ''
# workaround for PYTHON <= 3.9 use: BaseException | None
def log_debug(*values: object, sep: Optional[str] = ' ') -> None:
if os.name == 'nt':
# Workaround "Windows fatal exception: access violation"
print(f"\nDEBUG: {str(values[0])}")
return
if colored:
print(f"{colorama.Fore.CYAN}\nDEBUG: {str(values[0])}", *values[1:], colorama.Fore.RESET, sep=sep)
print(f"{colorama.Fore.CYAN}\nDEBUG: {str(values[0])}", *values[1:], colorama.Fore.RESET, sep)
else:
print(f"\nDEBUG: {str(values[0])}", *values[1:], sep=sep)
print(f"\nDEBUG: {str(values[0])}", *values[1:], sep)
# workaround for PYTHON <= 3.9 use: BaseException | None

View File

@@ -1,22 +1,11 @@
# -*- coding: utf-8 -*-
# Generated by the protocol buffer compiler. DO NOT EDIT!
# NO CHECKED-IN PROTOBUF GENCODE
# source: google_auth.proto
# Protobuf Python Version: 5.29.0
"""Generated protocol buffer code."""
from google.protobuf.internal import builder as _builder
from google.protobuf import descriptor as _descriptor
from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import runtime_version as _runtime_version
from google.protobuf import symbol_database as _symbol_database
from google.protobuf.internal import builder as _builder
_runtime_version.ValidateProtobufRuntimeVersion(
_runtime_version.Domain.PUBLIC,
5,
29,
0,
'',
'google_auth.proto'
)
# @@protoc_insertion_point(imports)
_sym_db = _symbol_database.Default()
@@ -26,17 +15,17 @@ _sym_db = _symbol_database.Default()
DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x11google_auth.proto\"\xb7\x03\n\x10MigrationPayload\x12\x37\n\x0eotp_parameters\x18\x01 \x03(\x0b\x32\x1f.MigrationPayload.OtpParameters\x12\x0f\n\x07version\x18\x02 \x01(\x05\x12\x12\n\nbatch_size\x18\x03 \x01(\x05\x12\x13\n\x0b\x62\x61tch_index\x18\x04 \x01(\x05\x12\x10\n\x08\x62\x61tch_id\x18\x05 \x01(\x05\x1a\xb7\x01\n\rOtpParameters\x12\x0e\n\x06secret\x18\x01 \x01(\x0c\x12\x0c\n\x04name\x18\x02 \x01(\t\x12\x0e\n\x06issuer\x18\x03 \x01(\t\x12.\n\talgorithm\x18\x04 \x01(\x0e\x32\x1b.MigrationPayload.Algorithm\x12\x0e\n\x06\x64igits\x18\x05 \x01(\x05\x12\'\n\x04type\x18\x06 \x01(\x0e\x32\x19.MigrationPayload.OtpType\x12\x0f\n\x07\x63ounter\x18\x07 \x01(\x03\",\n\tAlgorithm\x12\x10\n\x0c\x41LGO_INVALID\x10\x00\x12\r\n\tALGO_SHA1\x10\x01\"6\n\x07OtpType\x12\x0f\n\x0bOTP_INVALID\x10\x00\x12\x0c\n\x08OTP_HOTP\x10\x01\x12\x0c\n\x08OTP_TOTP\x10\x02\x62\x06proto3')
_globals = globals()
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals)
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'google_auth_pb2', _globals)
if not _descriptor._USE_C_DESCRIPTORS:
DESCRIPTOR._loaded_options = None
_globals['_MIGRATIONPAYLOAD']._serialized_start=22
_globals['_MIGRATIONPAYLOAD']._serialized_end=461
_globals['_MIGRATIONPAYLOAD_OTPPARAMETERS']._serialized_start=176
_globals['_MIGRATIONPAYLOAD_OTPPARAMETERS']._serialized_end=359
_globals['_MIGRATIONPAYLOAD_ALGORITHM']._serialized_start=361
_globals['_MIGRATIONPAYLOAD_ALGORITHM']._serialized_end=405
_globals['_MIGRATIONPAYLOAD_OTPTYPE']._serialized_start=407
_globals['_MIGRATIONPAYLOAD_OTPTYPE']._serialized_end=461
_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'google_auth_pb2', globals())
if _descriptor._USE_C_DESCRIPTORS == False:
DESCRIPTOR._options = None
_MIGRATIONPAYLOAD._serialized_start=22
_MIGRATIONPAYLOAD._serialized_end=461
_MIGRATIONPAYLOAD_OTPPARAMETERS._serialized_start=176
_MIGRATIONPAYLOAD_OTPPARAMETERS._serialized_end=359
_MIGRATIONPAYLOAD_ALGORITHM._serialized_start=361
_MIGRATIONPAYLOAD_ALGORITHM._serialized_end=405
_MIGRATIONPAYLOAD_OTPTYPE._serialized_start=407
_MIGRATIONPAYLOAD_OTPTYPE._serialized_end=461
# @@protoc_insertion_point(module_scope)

View File

@@ -2,7 +2,6 @@
@generated by mypy-protobuf. Do not edit manually!
isort:skip_file
"""
import builtins
import collections.abc
import google.protobuf.descriptor
@@ -19,7 +18,7 @@ else:
DESCRIPTOR: google.protobuf.descriptor.FileDescriptor
@typing.final
@typing_extensions.final
class MigrationPayload(google.protobuf.message.Message):
"""Copied from: https://github.com/beemdevelopment/Aegis/blob/master/app/src/main/proto/google_auth.proto"""
@@ -29,7 +28,7 @@ class MigrationPayload(google.protobuf.message.Message):
ValueType = typing.NewType("ValueType", builtins.int)
V: typing_extensions.TypeAlias = ValueType
class _AlgorithmEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[MigrationPayload._Algorithm.ValueType], builtins.type):
class _AlgorithmEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[MigrationPayload._Algorithm.ValueType], builtins.type): # noqa: F821
DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor
ALGO_INVALID: MigrationPayload._Algorithm.ValueType # 0
ALGO_SHA1: MigrationPayload._Algorithm.ValueType # 1
@@ -42,7 +41,7 @@ class MigrationPayload(google.protobuf.message.Message):
ValueType = typing.NewType("ValueType", builtins.int)
V: typing_extensions.TypeAlias = ValueType
class _OtpTypeEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[MigrationPayload._OtpType.ValueType], builtins.type):
class _OtpTypeEnumTypeWrapper(google.protobuf.internal.enum_type_wrapper._EnumTypeWrapper[MigrationPayload._OtpType.ValueType], builtins.type): # noqa: F821
DESCRIPTOR: google.protobuf.descriptor.EnumDescriptor
OTP_INVALID: MigrationPayload._OtpType.ValueType # 0
OTP_HOTP: MigrationPayload._OtpType.ValueType # 1
@@ -53,7 +52,7 @@ class MigrationPayload(google.protobuf.message.Message):
OTP_HOTP: MigrationPayload.OtpType.ValueType # 1
OTP_TOTP: MigrationPayload.OtpType.ValueType # 2
@typing.final
@typing_extensions.final
class OtpParameters(google.protobuf.message.Message):
DESCRIPTOR: google.protobuf.descriptor.Descriptor
@@ -82,19 +81,19 @@ class MigrationPayload(google.protobuf.message.Message):
type: global___MigrationPayload.OtpType.ValueType = ...,
counter: builtins.int = ...,
) -> None: ...
def ClearField(self, field_name: typing.Literal["algorithm", b"algorithm", "counter", b"counter", "digits", b"digits", "issuer", b"issuer", "name", b"name", "secret", b"secret", "type", b"type"]) -> None: ...
def ClearField(self, field_name: typing_extensions.Literal["algorithm", b"algorithm", "counter", b"counter", "digits", b"digits", "issuer", b"issuer", "name", b"name", "secret", b"secret", "type", b"type"]) -> None: ...
OTP_PARAMETERS_FIELD_NUMBER: builtins.int
VERSION_FIELD_NUMBER: builtins.int
BATCH_SIZE_FIELD_NUMBER: builtins.int
BATCH_INDEX_FIELD_NUMBER: builtins.int
BATCH_ID_FIELD_NUMBER: builtins.int
@property
def otp_parameters(self) -> google.protobuf.internal.containers.RepeatedCompositeFieldContainer[global___MigrationPayload.OtpParameters]: ...
version: builtins.int
batch_size: builtins.int
batch_index: builtins.int
batch_id: builtins.int
@property
def otp_parameters(self) -> google.protobuf.internal.containers.RepeatedCompositeFieldContainer[global___MigrationPayload.OtpParameters]: ...
def __init__(
self,
*,
@@ -104,6 +103,6 @@ class MigrationPayload(google.protobuf.message.Message):
batch_index: builtins.int = ...,
batch_id: builtins.int = ...,
) -> None: ...
def ClearField(self, field_name: typing.Literal["batch_id", b"batch_id", "batch_index", b"batch_index", "batch_size", b"batch_size", "otp_parameters", b"otp_parameters", "version", b"version"]) -> None: ...
def ClearField(self, field_name: typing_extensions.Literal["batch_id", b"batch_id", "batch_index", b"batch_index", "batch_size", b"batch_size", "otp_parameters", b"otp_parameters", "version", b"version"]) -> None: ...
global___MigrationPayload = MigrationPayload

View File

@@ -2,6 +2,8 @@ from typing import Any
import pytest
from extract_otp_secrets import QRMode
def pytest_addoption(parser: pytest.Parser) -> None:
parser.addoption("--relaxed", action='store_true', help="run tests in relaxed mode")
@@ -15,7 +17,6 @@ def relaxed(request: pytest.FixtureRequest) -> Any:
def pytest_generate_tests(metafunc: pytest.Metafunc) -> None:
if "qr_mode" in metafunc.fixturenames:
all_qr_modes = ['ZBAR', 'QREADER', 'QREADER_DEEP', 'CV2', 'CV2_WECHAT']
number = 2 if metafunc.config.getoption("fast") else len(all_qr_modes)
qr_modes = [mode for mode in all_qr_modes]
number = 2 if metafunc.config.getoption("fast") else len(QRMode)
qr_modes = [mode.name for mode in QRMode]
metafunc.parametrize("qr_mode", qr_modes[0:number])

View File

@@ -1,3 +0,0 @@
# comment 1
# comment 2

View File

@@ -1,5 +1,5 @@
QReader installed: True
CV2 version: 4.10.0
CV2 version: 4.7.0
QR reading mode: ZBAR
Input files: ['example_export.txt']

View File

@@ -1,5 +1,5 @@
QReader installed: True
CV2 version: 4.10.0
CV2 version: 4.7.0
QR reading mode: ZBAR
Input files: ['example_export.txt']

View File

@@ -1,12 +1,8 @@
QReader installed: True
CV2 version: 4.10.0
CV2 version: 4.7.0
QR reading mode: ZBAR
Version: extract_otp_secrets 2.8.4.post4+git.7ce765dd.dirty Linux x86_64 Python 3.11.10 (CPython/called as script)
Input files: ['example_export.txt']
DEBUG: Expanded input files: ['example_export.txt']
Processing infile example_export.txt
Reading lines of example_export.txt
# 2FA example from https://www.raspberrypi.org/blog/setting-up-two-factor-authentication-on-your-raspberry-pi/
@@ -34,17 +30,16 @@ otpauth-migration://offline?data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8O
# otpauth://totp/pi@raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspberrypi
otpauth-migration://offline?data=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK%2B%2F%2F%2F%2F%2F8B
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK%2B%2F%2F%2F%2F%2F8B', fragment='')
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK%2B%2F%2F%2F%2F%2F8B', fragment='')
DEBUG: querystring params={'data': ['CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B']}
DEBUG: querystring params={'data': ['CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B']}
DEBUG: data_base64=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B
DEBUG: data_base64=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B
DEBUG: data_base64_fixed=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B
DEBUG: data_base64_fixed=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B
DEBUG:
1. Payload Line
otp_parameters {
1. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "pi@raspberrypi"
issuer: "raspberrypi"
@@ -55,11 +50,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -1320898453
1. Secret
DEBUG: OTP enum type: OTP_TOTP
DEBUG: OTP enum type: OTP_TOTP
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Issuer: raspberrypi
@@ -70,17 +66,16 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspber
# otpauth://totp/pi@raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
otpauth-migration://offline?data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4%2F%2F%2F%2F%2FwE%3D
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4%2F%2F%2F%2F%2FwE%3D', fragment='')
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4%2F%2F%2F%2F%2FwE%3D', fragment='')
DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=']}
DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=']}
DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=
DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=
DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=
DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=
DEBUG:
2. Payload Line
otp_parameters {
2. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "pi@raspberrypi"
algorithm: ALGO_SHA1
@@ -90,11 +85,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -2094403140
2. Secret
DEBUG: OTP enum type: OTP_TOTP
DEBUG: OTP enum type: OTP_TOTP
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: totp
@@ -105,17 +101,16 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
# otpauth://totp/pi@raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
otpauth-migration://offline?data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa%2Bf%2F%2F%2F%2F8B
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa%2Bf%2F%2F%2F%2F8B', fragment='')
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa%2Bf%2F%2F%2F%2F8B', fragment='')
DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B']}
DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B']}
DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B
DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B
DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B
DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B
DEBUG:
3. Payload Line
otp_parameters {
3. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "pi@raspberrypi"
algorithm: ALGO_SHA1
@@ -133,11 +128,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -1822886384
3. Secret
DEBUG: OTP enum type: OTP_TOTP
DEBUG: OTP enum type: OTP_TOTP
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: totp
@@ -146,7 +142,7 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
4. Secret
DEBUG: OTP enum type: OTP_TOTP
DEBUG: OTP enum type: OTP_TOTP
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Issuer: raspberrypi
@@ -157,17 +153,16 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspber
# otpauth://hotp/hotp%20demo?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&counter=4
otpauth-migration://offline?data=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6%2F%2F%2F%2F%2FwE%3D
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6%2F%2F%2F%2F%2FwE%3D', fragment='')
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6%2F%2F%2F%2F%2FwE%3D', fragment='')
DEBUG: querystring params={'data': ['CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=']}
DEBUG: querystring params={'data': ['CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=']}
DEBUG: data_base64=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=
DEBUG: data_base64=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=
DEBUG: data_base64_fixed=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=
DEBUG: data_base64_fixed=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=
DEBUG:
4. Payload Line
otp_parameters {
4. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "hotp demo"
algorithm: ALGO_SHA1
@@ -178,11 +173,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -1558849573
5. Secret
DEBUG: OTP enum type: OTP_HOTP
DEBUG: OTP enum type: OTP_HOTP
Name: hotp demo
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: hotp
@@ -194,17 +190,16 @@ otpauth://hotp/hotp%20demo?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&counter=4
# Name: "encoding: ¿äÄéÉ? (demo)"
otpauth-migration://offline?data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv%2F%2F%2F%2F%2F%2FAQ%3D%3D
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv%2F%2F%2F%2F%2F%2FAQ%3D%3D', fragment='')
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv%2F%2F%2F%2F%2F%2FAQ%3D%3D', fragment='')
DEBUG: querystring params={'data': ['CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==']}
DEBUG: querystring params={'data': ['CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==']}
DEBUG: data_base64=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==
DEBUG: data_base64=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==
DEBUG: data_base64_fixed=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==
DEBUG: data_base64_fixed=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==
DEBUG:
5. Payload Line
otp_parameters {
5. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "encoding: ¿äÄéÉ? (demo)"
algorithm: ALGO_SHA1
@@ -214,11 +209,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -171198419
6. Secret
DEBUG: OTP enum type: OTP_TOTP
DEBUG: OTP enum type: OTP_TOTP
Name: encoding: ¿äÄéÉ? (demo)
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: totp

View File

@@ -1,5 +1,5 @@
QReader installed: True
CV2 version: 4.10.0
CV2 version: 4.7.0
QR reading mode: ZBAR
Input files: ['example_export.txt']

View File

@@ -1,5 +1,5 @@
QReader installed: True
CV2 version: 4.10.0
CV2 version: 4.7.0
QR reading mode: ZBAR
Input files: ['example_export.txt']

View File

@@ -1,12 +1,8 @@
QReader installed: True
CV2 version: 4.10.0
CV2 version: 4.7.0
QR reading mode: ZBAR
Version: extract_otp_secrets 2.8.4.post4+git.7ce765dd.dirty Linux x86_64 Python 3.11.10 (CPython/called as script)
Input files: ['example_export.txt']

DEBUG: Expanded input files: ['example_export.txt'] 
Processing infile example_export.txt
Reading lines of example_export.txt
# 2FA example from https://www.raspberrypi.org/blog/setting-up-two-factor-authentication-on-your-raspberry-pi/
@@ -34,17 +30,16 @@ Reading lines of example_export.txt
# otpauth://totp/pi@raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspberrypi
otpauth-migration://offline?data=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK%2B%2F%2F%2F%2F%2F8B

DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK%2B%2F%2F%2F%2F%2F8B', fragment='') 
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK%2B%2F%2F%2F%2F%2F8B', fragment='') 

DEBUG: querystring params={'data': ['CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B']} 
DEBUG: querystring params={'data': ['CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B']} 

DEBUG: data_base64=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B 
DEBUG: data_base64=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B 

DEBUG: data_base64_fixed=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B 
DEBUG: data_base64_fixed=CjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACjr4JKK+/////8B 

DEBUG:
1. Payload Line
otp_parameters {
1. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "pi@raspberrypi"
issuer: "raspberrypi"
@@ -55,12 +50,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -1320898453


1. Secret

DEBUG: OTP enum type: OTP_TOTP 
DEBUG: OTP enum type: OTP_TOTP 
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Issuer: raspberrypi
@@ -71,17 +66,16 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspber
# otpauth://totp/pi@raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
otpauth-migration://offline?data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4%2F%2F%2F%2F%2FwE%3D

DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4%2F%2F%2F%2F%2FwE%3D', fragment='') 
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4%2F%2F%2F%2F%2FwE%3D', fragment='') 

DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=']} 
DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE=']} 

DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE= 
DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE= 

DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE= 
DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACEAEYASAAKLzjp5n4/////wE= 

DEBUG:
2. Payload Line
otp_parameters {
2. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "pi@raspberrypi"
algorithm: ALGO_SHA1
@@ -91,12 +85,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -2094403140


2. Secret

DEBUG: OTP enum type: OTP_TOTP 
DEBUG: OTP enum type: OTP_TOTP 
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: totp
@@ -107,17 +101,16 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
# otpauth://totp/pi@raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
otpauth-migration://offline?data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa%2Bf%2F%2F%2F%2F8B

DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa%2Bf%2F%2F%2F%2F8B', fragment='') 
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa%2Bf%2F%2F%2F%2F8B', fragment='') 

DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B']} 
DEBUG: querystring params={'data': ['CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B']} 

DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B 
DEBUG: data_base64=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B 

DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B 
DEBUG: data_base64_fixed=CigKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpIAEoATACCjUKEPqlBekzoNEukL7qlsjBCDYSDnBpQHJhc3BiZXJyeXBpGgtyYXNwYmVycnlwaSABKAEwAhABGAEgACiQ7OOa+f////8B 

DEBUG:
3. Payload Line
otp_parameters {
3. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "pi@raspberrypi"
algorithm: ALGO_SHA1
@@ -135,12 +128,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -1822886384


3. Secret

DEBUG: OTP enum type: OTP_TOTP 
DEBUG: OTP enum type: OTP_TOTP 
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: totp
@@ -149,7 +142,7 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
4. Secret

DEBUG: OTP enum type: OTP_TOTP 
DEBUG: OTP enum type: OTP_TOTP 
Name: pi@raspberrypi
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Issuer: raspberrypi
@@ -160,17 +153,16 @@ otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspber
# otpauth://hotp/hotp%20demo?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&counter=4
otpauth-migration://offline?data=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6%2F%2F%2F%2F%2FwE%3D

DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6%2F%2F%2F%2F%2FwE%3D', fragment='') 
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6%2F%2F%2F%2F%2FwE%3D', fragment='') 

DEBUG: querystring params={'data': ['CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=']} 
DEBUG: querystring params={'data': ['CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE=']} 

DEBUG: data_base64=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE= 
DEBUG: data_base64=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE= 

DEBUG: data_base64_fixed=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE= 
DEBUG: data_base64_fixed=CiUKEPqlBekzoNEukL7qlsjBCDYSCWhvdHAgZGVtbyABKAEwATgEEAEYASAAKNuv15j6/////wE= 

DEBUG:
4. Payload Line
otp_parameters {
4. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "hotp demo"
algorithm: ALGO_SHA1
@@ -181,12 +173,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -1558849573


5. Secret

DEBUG: OTP enum type: OTP_HOTP 
DEBUG: OTP enum type: OTP_HOTP 
Name: hotp demo
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: hotp
@@ -198,17 +190,16 @@ otpauth://hotp/hotp%20demo?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&counter=4
# Name: "encoding: ¿äÄéÉ? (demo)"
otpauth-migration://offline?data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv%2F%2F%2F%2F%2F%2FAQ%3D%3D

DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv%2F%2F%2F%2F%2F%2FAQ%3D%3D', fragment='') 
DEBUG: parsed_url=ParseResult(scheme='otpauth-migration', netloc='offline', path='', params='', query='data=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv%2F%2F%2F%2F%2F%2FAQ%3D%3D', fragment='') 

DEBUG: querystring params={'data': ['CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==']} 
DEBUG: querystring params={'data': ['CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ==']} 

DEBUG: data_base64=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ== 
DEBUG: data_base64=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ== 

DEBUG: data_base64_fixed=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ== 
DEBUG: data_base64_fixed=CjYKEPqlBekzoNEukL7qlsjBCDYSHGVuY29kaW5nOiDCv8Okw4TDqcOJPyAoZGVtbykgASgBMAIQARgBIAAorfCurv//////AQ== 

DEBUG:
5. Payload Line
otp_parameters {
5. Payload Line otp_parameters {
secret: "\372\245\005\3513\240\321.\220\276\352\226\310\301\0106"
name: "encoding: ¿äÄéÉ? (demo)"
algorithm: ALGO_SHA1
@@ -218,12 +209,12 @@ otp_parameters {
version: 1
batch_size: 1
batch_id: -171198419


6. Secret

DEBUG: OTP enum type: OTP_TOTP 
DEBUG: OTP enum type: OTP_TOTP 
Name: encoding: ¿äÄéÉ? (demo)
Secret: 7KSQL2JTUDIS5EF65KLMRQIIGY
Type: totp

Binary file not shown.

Before

Width:  |  Height:  |  Size: 478 B

View File

@@ -1,6 +0,0 @@
otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspberrypi
otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY
otpauth://totp/pi%40raspberrypi?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&issuer=raspberrypi
otpauth://hotp/hotp%20demo?secret=7KSQL2JTUDIS5EF65KLMRQIIGY&counter=4
otpauth://totp/encoding%3A%20%C2%BF%C3%A4%C3%84%C3%A9%C3%89%3F%20%28demo%29?secret=7KSQL2JTUDIS5EF65KLMRQIIGY

View File

@@ -26,8 +26,7 @@ import pathlib
import re
import sys
import time
from enum import Enum
from typing import Any, List, Optional, Tuple
from typing import Optional
import colorama
import pytest
@@ -38,18 +37,11 @@ from utils import (count_files_in_dir, file_exits, read_binary_file_as_stream,
import extract_otp_secrets
try:
import cv2
from extract_otp_secrets import SUCCESS_COLOR, FAILURE_COLOR, FONT, FONT_SCALE, FONT_COLOR, FONT_THICKNESS, FONT_LINE_STYLE
except ImportError:
# ignore
pass
cv2_available: bool = extract_otp_secrets.cv2_available
qreader_available: bool = extract_otp_secrets.qreader_available
# Quickfix comment
# @pytest.mark.skipif(sys.platform.startswith("win") or not cv2 or sys.implementation.name == 'pypy' or sys.version_info >= (3, 10), reason="Quickfix")
# @pytest.mark.skipif(sys.platform.startswith("win") or not qreader_available or sys.implementation.name == 'pypy' or sys.version_info >= (3, 10), reason="Quickfix")
def test_extract_stdout(capsys: pytest.CaptureFixture[str]) -> None:
@@ -107,22 +99,8 @@ def test_extract_stdin_empty(capsys: pytest.CaptureFixture[str], monkeypatch: py
assert captured.err == '\nWARN: stdin is empty\n'
def test_extract_stdin_only_comments(capsys: pytest.CaptureFixture[str], monkeypatch: pytest.MonkeyPatch) -> None:
# Arrange
monkeypatch.setattr('sys.stdin', io.StringIO("\n\n# comment 1\n\n\n#comment 2"))
# Act
extract_otp_secrets.main(['-n', '-'])
# Assert
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == ''
def test_extract_empty_file_no_qreader(capsys: pytest.CaptureFixture[str]) -> None:
if cv2_available:
if qreader_available:
# Act
with pytest.raises(SystemExit) as e:
extract_otp_secrets.main(['-n', 'tests/data/empty_file.txt'])
@@ -147,17 +125,6 @@ def test_extract_empty_file_no_qreader(capsys: pytest.CaptureFixture[str]) -> No
assert captured.out == ''
def test_extract_only_comments_file_no_qreader(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-n', 'tests/data/only_comments.txt'])
# Assert
captured = capsys.readouterr()
assert captured.err == ''
assert captured.out == ''
@pytest.mark.qreader
def test_extract_stdin_img_empty(capsys: pytest.CaptureFixture[str], monkeypatch: pytest.MonkeyPatch) -> None:
# Arrange
@@ -173,24 +140,6 @@ def test_extract_stdin_img_empty(capsys: pytest.CaptureFixture[str], monkeypatch
assert captured.err == '\nWARN: stdin is empty\n'
@pytest.mark.qreader
def test_extract_stdin_img_garbage(capsys: pytest.CaptureFixture[str], monkeypatch: pytest.MonkeyPatch) -> None:
# Arrange
monkeypatch.setattr('sys.stdin', io.BytesIO("garbage".encode('utf-8')))
# Act
with pytest.raises(SystemExit) as e:
extract_otp_secrets.main(['-n', '='])
# Assert
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == '\nERROR: Unable to open file for reading.\ninput file: =\n'
assert e.type == SystemExit
assert e.value.code == 1
def test_extract_csv(capsys: pytest.CaptureFixture[str], tmp_path: pathlib.Path) -> None:
# Arrange
output_file = str(tmp_path / 'test_example_output.csv')
@@ -226,19 +175,6 @@ def test_extract_csv_stdout(capsys: pytest.CaptureFixture[str]) -> None:
assert captured.err == ''
def test_extract_csv_stdout_only_comments(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-c', '-', 'tests/data/only_comments.txt'])
# Assert
assert not file_exits('test_example_output.csv')
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == ''
def test_extract_stdin_and_csv_stdout(capsys: pytest.CaptureFixture[str], monkeypatch: pytest.MonkeyPatch) -> None:
# Arrange
monkeypatch.setattr('sys.stdin', io.StringIO(read_file_to_str('example_export.txt')))
@@ -282,18 +218,6 @@ def test_keepass_csv(capsys: pytest.CaptureFixture[str], tmp_path: pathlib.Path)
assert captured.err == ''
def test_keepass_empty(capsys: pytest.CaptureFixture[str], tmp_path: pathlib.Path) -> None:
# Act
extract_otp_secrets.main(['-k', '-', 'tests/data/only_comments.txt'])
# Assert
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == ''
def test_keepass_csv_stdout(capsys: pytest.CaptureFixture[str]) -> None:
'''Two csv files .totp and .htop are generated.'''
# Act
@@ -364,91 +288,6 @@ def test_extract_json_stdout(capsys: pytest.CaptureFixture[str]) -> None:
assert captured.err == ''
def test_extract_json_stdout_only_comments(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-j', '-', 'tests/data/only_comments.txt'])
# Assert
assert not file_exits('test_example_output.json')
captured = capsys.readouterr()
assert captured.out == '[]'
assert captured.err == ''
def test_extract_txt(capsys: pytest.CaptureFixture[str], tmp_path: pathlib.Path) -> None:
# Arrange
output_file = str(tmp_path / 'test_example_output.txt')
# Act
extract_otp_secrets.main(['-q', '-t', output_file, 'example_export.txt'])
# Assert
expected_txt = read_file_to_str('tests/data/printqr_output.txt')
actual_txt = read_file_to_str(output_file)
assert actual_txt == expected_txt
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == ''
def test_extract_txt_stdout(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-t', '-', 'example_export.txt'])
# Assert
expected_txt = read_file_to_str('tests/data/printqr_output.txt')
captured = capsys.readouterr()
assert captured.out == expected_txt
assert captured.err == ''
def test_extract_txt_stdout_only_comments(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-t', '-', 'tests/data/only_comments.txt'])
# Assert
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == ''
def test_extract_urls(capsys: pytest.CaptureFixture[str], tmp_path: pathlib.Path) -> None:
# Arrange
output_file = str(tmp_path / 'test_example_url_list.txt')
# Act
extract_otp_secrets.main(['-q', '-u', output_file, 'example_export.txt'])
# Assert
expected_txt = read_file_to_str('tests/data/url_list_output.txt')
actual_txt = read_file_to_str(output_file)
assert actual_txt == expected_txt
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == ''
def test_extract_urls_stdout(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-u', '-', 'example_export.txt'])
# Assert
expected_txt = read_file_to_str('tests/data/url_list_output.txt')
captured = capsys.readouterr()
assert captured.out == expected_txt
assert captured.err == ''
def test_extract_not_encoded_plus(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['tests/data/test_plus_problem_export.txt'])
@@ -559,7 +398,7 @@ def test_normalize_bytes() -> None:
# Generate verbose output:
# for color in '' '-n'; do for level in '' '-v' '-vv' '-vvv'; do python3.11 src/extract_otp_secrets.py example_export.txt $color $level > tests/data/print_verbose_output$color$level.txt; done; done
# workaround for PYTHON <= 3.10
@pytest.mark.skipif(sys.version_info < (3, 10) or sys.platform.startswith("win"), reason="fileinput.input encoding exists since PYTHON 3.10 OR Windows fatal exception: access violation")
@pytest.mark.skipif(sys.version_info < (3, 10), reason="fileinput.input encoding exists since PYTHON 3.10")
@pytest.mark.parametrize("verbose_level", ['', '-v', '-vv', '-vvv'])
@pytest.mark.parametrize("color", ['', '-n'])
def test_extract_verbose(verbose_level: str, color: str, capsys: pytest.CaptureFixture[str], relaxed: bool) -> None:
@@ -582,8 +421,8 @@ def test_extract_verbose(verbose_level: str, color: str, capsys: pytest.CaptureF
def normalize_verbose_text(text: str, relaxed: bool) -> str:
normalized = re.sub('^.*version: .+$', '', text, flags=re.MULTILINE | re.IGNORECASE)
if not cv2_available:
normalized = re.sub('^.+ version: .+$', '', text, flags=re.MULTILINE | re.IGNORECASE)
if not qreader_available:
normalized = normalized \
.replace('QReader installed: True', 'QReader installed: False') \
.replace('\nQR reading mode: ZBAR\n\n', '')
@@ -622,22 +461,8 @@ def test_extract_help(capsys: pytest.CaptureFixture[str]) -> None:
assert e.value.code == 0
def test_extract_version(capsys: pytest.CaptureFixture[str]) -> None:
with pytest.raises(SystemExit) as e:
# Act
extract_otp_secrets.main(['--version'])
# Assert
captured = capsys.readouterr()
assert captured.out.startswith('extract_otp_secrets ')
assert captured.err == ''
assert e.type == SystemExit
assert e.value.code == 0
def test_extract_no_arguments(capsys: pytest.CaptureFixture[str], mocker: MockerFixture) -> None:
if cv2_available:
if qreader_available:
# Arrange
otps = read_json('example_output.json')
mocker.patch('extract_otp_secrets.extract_otps_from_camera', return_value=otps)
@@ -669,113 +494,6 @@ def test_extract_no_arguments(capsys: pytest.CaptureFixture[str], mocker: Mocker
assert e.type == SystemExit
MockMode = Enum('MockMode', ['REPEAT_FIRST_ENDLESS', 'LOOP_LIST'])
class MockCam:
read_counter: int = 0
read_files: List[str] = []
mock_mode: MockMode
def __init__(self, files: List[str] = ['example_export.png'], mock_mode: MockMode = MockMode.REPEAT_FIRST_ENDLESS):
self.read_files = files
self.image_mode = mock_mode
def read(self) -> Tuple[bool, Any]:
if self.image_mode == MockMode.REPEAT_FIRST_ENDLESS:
file = self.read_files[0]
elif self.image_mode == MockMode.LOOP_LIST:
file = self.read_files[self.read_counter]
self.read_counter += 1
if file:
img = cv2.imread(file)
return True, img
else:
return False, None
def release(self) -> None:
# ignore
pass
@pytest.mark.parametrize("qr_reader,file,success", [
(None, 'example_export.png', True),
('ZBAR', 'example_export.png', True),
('QREADER', 'example_export.png', True),
('QREADER_DEEP', 'example_export.png', True),
('CV2', 'example_export.png', True),
('CV2_WECHAT', 'example_export.png', True),
(None, 'tests/data/qr_but_without_otp.png', False),
('ZBAR', 'tests/data/qr_but_without_otp.png', False),
('QREADER', 'tests/data/qr_but_without_otp.png', False),
('QREADER_DEEP', 'tests/data/qr_but_without_otp.png', False),
('CV2', 'tests/data/qr_but_without_otp.png', False),
('CV2_WECHAT', 'tests/data/qr_but_without_otp.png', False),
(None, 'tests/data/lena_std.tif', None),
('ZBAR', 'tests/data/lena_std.tif', None),
('QREADER', 'tests/data/lena_std.tif', None),
('QREADER_DEEP', 'tests/data/lena_std.tif', None),
('CV2', 'tests/data/lena_std.tif', None),
('CV2_WECHAT', 'tests/data/lena_std.tif', None),
])
def test_extract_otps_from_camera(qr_reader: Optional[str], file: str, success: bool, capsys: pytest.CaptureFixture[str], mocker: MockerFixture) -> None:
if cv2_available:
# Arrange
mockCam = MockCam([file])
mocker.patch('cv2.VideoCapture', return_value=mockCam)
mocker.patch('cv2.namedWindow')
mocked_polylines = mocker.patch('cv2.polylines')
mocker.patch('cv2.imshow')
mocker.patch('cv2.getTextSize', return_value=([8, 200], False))
mocked_putText = mocker.patch('cv2.putText')
mocker.patch('cv2.getWindowImageRect', return_value=[0, 0, 640, 480])
mocker.patch('cv2.waitKey', return_value=27)
mocker.patch('cv2.getWindowProperty', return_value=False)
mocker.patch('cv2.destroyAllWindows')
args = []
if qr_reader:
args.append('-Q')
args.append(qr_reader)
# Act
extract_otp_secrets.main(args)
# Assert
captured = capsys.readouterr()
if success:
assert captured.out == EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT_PNG
assert captured.err == ''
mocked_polylines.assert_called_with(mocker.ANY, mocker.ANY, True, SUCCESS_COLOR, mocker.ANY)
mocked_putText.assert_called_with(mocker.ANY, "3 otps extracted", mocker.ANY, FONT, FONT_SCALE, FONT_COLOR, FONT_THICKNESS, FONT_LINE_STYLE)
elif success is None:
assert captured.out == ''
assert captured.err == ''
mocked_polylines.assert_not_called()
mocked_putText.assert_called_with(mocker.ANY, "0 otps extracted", mocker.ANY, FONT, FONT_SCALE, FONT_COLOR, FONT_THICKNESS, FONT_LINE_STYLE)
else:
assert captured.out == ''
assert captured.err != ''
mocked_polylines.assert_called_with(mocker.ANY, mocker.ANY, True, FAILURE_COLOR, mocker.ANY)
mocked_putText.assert_called_with(mocker.ANY, "0 otps extracted", mocker.ANY, FONT, FONT_SCALE, FONT_COLOR, FONT_THICKNESS, FONT_LINE_STYLE)
else:
# Act
with pytest.raises(SystemExit) as e:
extract_otp_secrets.main([])
# Assert
captured = capsys.readouterr()
expected_err_msg = 'error: the following arguments are required: infile'
assert expected_err_msg in captured.err
assert captured.out == ''
assert e.value.code == 2
assert e.type == SystemExit
def test_verbose_and_quiet(capsys: pytest.CaptureFixture[str]) -> None:
with pytest.raises(SystemExit) as e:
# Act
@@ -797,10 +515,8 @@ def test_verbose_and_quiet(capsys: pytest.CaptureFixture[str]) -> None:
('-k', 'outfile', False, False),
('-k', '-', True, False),
('-j', 'outfile', False, False),
('-j', '-', True, False),
('-t', 'outfile', False, False),
('-t', '-', True, False),
('-s', 'outfile', False, False),
('-j', '-', True, False),
('-i', None, False, False),
('-p', None, True, False),
('-Q', 'CV2', False, False),
@@ -808,7 +524,7 @@ def test_verbose_and_quiet(capsys: pytest.CaptureFixture[str]) -> None:
('-n', None, False, False),
])
def test_quiet(parameter: str, parameter_value: Optional[str], stdout_expected: bool, stderr_expected: bool, capsys: pytest.CaptureFixture[str], tmp_path: pathlib.Path) -> None:
if parameter in ['-Q', '-C'] and not cv2_available:
if parameter in ['-Q', '-C'] and not qreader_available:
return
# Arrange
@@ -849,14 +565,7 @@ data=XXXX
Exception: unpack requires a buffer of 4 bytes
'''
#
third_expected_stderr = '''
ERROR: Cannot decode otpauth-migration migration payload.
data=XXXX
Exception: Error parsing message with type 'MigrationPayload'
'''
assert captured.err == first_expected_stderr or captured.err == second_expected_stderr or captured.err == third_expected_stderr
assert captured.err == first_expected_stderr or captured.err == second_expected_stderr
assert captured.out == ''
assert e.value.code == 1
assert e.type == SystemExit
@@ -869,8 +578,19 @@ def test_wrong_content(capsys: pytest.CaptureFixture[str]) -> None:
# Assert
captured = capsys.readouterr()
expected_stderr = '''
WARN: input is not a otpauth-migration:// url
source: tests/data/test_export_wrong_content.txt
input: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
Maybe a wrong file was given
ERROR: could not parse query parameter in input url
source: tests/data/test_export_wrong_content.txt
url: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
'''
assert captured.out == ''
assert captured.err == EXPECTED_STDERR_OTP_URL_WRONG
assert captured.err == expected_stderr
def test_one_wrong_file(capsys: pytest.CaptureFixture[str]) -> None:
@@ -880,8 +600,19 @@ def test_one_wrong_file(capsys: pytest.CaptureFixture[str]) -> None:
# Assert
captured = capsys.readouterr()
expected_stderr = '''
WARN: input is not a otpauth-migration:// url
source: tests/data/test_export_wrong_content.txt
input: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
Maybe a wrong file was given
ERROR: could not parse query parameter in input url
source: tests/data/test_export_wrong_content.txt
url: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
'''
assert captured.out == EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT
assert captured.err == EXPECTED_STDERR_OTP_URL_WRONG
assert captured.err == expected_stderr
def test_one_wrong_file_colored(capsys: pytest.CaptureFixture[str]) -> None:
@@ -891,8 +622,19 @@ def test_one_wrong_file_colored(capsys: pytest.CaptureFixture[str]) -> None:
# Assert
captured = capsys.readouterr()
expected_stderr = f'''{colorama.Fore.RED}
WARN: input is not a otpauth-migration:// url
source: tests/data/test_export_wrong_content.txt
input: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
Maybe a wrong file was given{colorama.Fore.RESET}
{colorama.Fore.RED}
ERROR: could not parse query parameter in input url
source: tests/data/test_export_wrong_content.txt
url: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.{colorama.Fore.RESET}
'''
assert captured.out == EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT
assert captured.err == EXPECTED_STDERR_COLORED_OTP_URL_WRONG
assert captured.err == expected_stderr
def test_one_wrong_line(capsys: pytest.CaptureFixture[str], monkeypatch: pytest.MonkeyPatch) -> None:
@@ -964,46 +706,6 @@ def test_img_qr_reader_from_file_happy_path(capsys: pytest.CaptureFixture[str])
assert captured.err == ''
@pytest.mark.qreader
def test_img_qr_reader_but_no_otp_from_file(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-n', 'tests/data/qr_but_without_otp.png'])
# Assert
captured = capsys.readouterr()
assert captured.out == ''
assert captured.err == EXPECTED_STDERR_NO_OTP_URL
@pytest.mark.qreader
def test_img_qr_reader_from_wildcard(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-n', 'tests/data/*.png'])
# Assert
captured = capsys.readouterr()
assert captured.out == EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT_PNG
assert normalize_testfile_path(captured.err) == EXPECTED_STDERR_NO_OTP_URL
def normalize_testfile_path(text: str):
return text.replace('tests/data\\', 'tests/data/') if sys.platform.startswith("win") else text
@pytest.mark.qreader
def test_img_qr_reader_from_multiple_files(capsys: pytest.CaptureFixture[str]) -> None:
# Act
extract_otp_secrets.main(['-n', 'tests/data/test_googleauth_export.png', 'tests/data/text_masquerading_as_image.jpeg'])
# Assert
captured = capsys.readouterr()
assert captured.out == EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT_PNG
assert captured.err == EXPECTED_STDERR_BAD_IMAGE
@pytest.mark.qreader
def test_img_qr_reader_by_parameter(capsys: pytest.CaptureFixture[str], qr_mode: str) -> None:
# Act
@@ -1048,7 +750,24 @@ def test_img_qr_reader_from_stdin(capsys: pytest.CaptureFixture[str], monkeypatc
# Assert
captured = capsys.readouterr()
assert captured.out == EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT_PNG
expected_stdout = '''Name: Test1:test1@example1.com
Secret: JBSWY3DPEHPK3PXP
Issuer: Test1
Type: totp
Name: Test2:test2@example2.com
Secret: JBSWY3DPEHPK3PXQ
Issuer: Test2
Type: totp
Name: Test3:test3@example3.com
Secret: JBSWY3DPEHPK3PXR
Issuer: Test3
Type: totp
'''
assert captured.out == expected_stdout
assert captured.err == ''
@@ -1133,15 +852,19 @@ def test_non_image_file(capsys: pytest.CaptureFixture[str]) -> None:
# Assert
captured = capsys.readouterr()
expected_stderr = '''
WARN: input is not a otpauth-migration:// url
source: tests/data/text_masquerading_as_image.jpeg
input: This is just a text file masquerading as an image file.
Maybe a wrong file was given
ERROR: could not parse query parameter in input url
source: tests/data/text_masquerading_as_image.jpeg
url: This is just a text file masquerading as an image file.
'''
assert captured.err == expected_stderr
assert captured.out == ''
assert captured.err == EXPECTED_STDERR_BAD_IMAGE
def test_next_valid_qr_mode() -> None:
assert extract_otp_secrets.next_valid_qr_mode(extract_otp_secrets.QRMode.CV2, True) == extract_otp_secrets.QRMode.CV2_WECHAT
assert extract_otp_secrets.next_valid_qr_mode(extract_otp_secrets.QRMode.CV2_WECHAT, True) == extract_otp_secrets.QRMode.ZBAR
assert extract_otp_secrets.next_valid_qr_mode(extract_otp_secrets.QRMode.CV2_WECHAT, False) == extract_otp_secrets.QRMode.CV2
EXPECTED_STDOUT_FROM_EXAMPLE_EXPORT = '''Name: pi@raspberrypi
@@ -1189,47 +912,3 @@ Issuer: Test3
Type: totp
'''
EXPECTED_STDERR_OTP_URL_WRONG = '''
WARN: input is not a otpauth-migration:// url
source: tests/data/test_export_wrong_content.txt
input: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
Maybe a wrong file was given
ERROR: could not parse query parameter in input url
source: tests/data/test_export_wrong_content.txt
url: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
'''
EXPECTED_STDERR_COLORED_OTP_URL_WRONG = f'''{colorama.Fore.RED}
WARN: input is not a otpauth-migration:// url
source: tests/data/test_export_wrong_content.txt
input: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.
Maybe a wrong file was given{colorama.Fore.RESET}
{colorama.Fore.RED}
ERROR: could not parse query parameter in input url
source: tests/data/test_export_wrong_content.txt
url: Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua.{colorama.Fore.RESET}
'''
EXPECTED_STDERR_NO_OTP_URL = '''
WARN: input is not a otpauth-migration:// url
source: tests/data/qr_but_without_otp.png
input: NOT A otpauth-migration:// URL
Maybe a wrong file was given
ERROR: could not parse query parameter in input url
source: tests/data/qr_but_without_otp.png
url: NOT A otpauth-migration:// URL
'''
EXPECTED_STDERR_BAD_IMAGE = '''
WARN: input is not a otpauth-migration:// url
source: tests/data/text_masquerading_as_image.jpeg
input: This is just a text file masquerading as an image file.
Maybe a wrong file was given
ERROR: could not parse query parameter in input url
source: tests/data/text_masquerading_as_image.jpeg
url: This is just a text file masquerading as an image file.
'''