Python discovers its inner PHP and JavaScript personae

Did you recently switch from PHP or JavaScript to Python, and are missing the fun of being bitten by your programming language?

The collection of surprising Python snippets and lesser-known features is your ultimate guide for provoking Python to bite you in the arse.

Security and Obscurity

If you do not know the password but know how to use the password to gain access to something that was secured using this password, then this is security by obscurity.

On the other hand, if you know the password but do not know how to use the password, then this is obscurity by security.

(Sources of inspiration: The Butterfly DreamCategory Theory’s reversal of rows.)

Do material design icons fail to show in your Cordova/Vue/Vuetify/Android application?

When developing an Android application using Cordova 8.0.0 with Vue and Vuetify, I noticed that the beautiful material design icons do not show as expected.

There were two problems.

Problem 1: ligature transformations in Android’s WebView

Ligature transformations did not work in Android’s WebView, so instead of icons one sees text strings.

To fix it, I had to substitute '' for 'keyboard_arrow_up' and similar substitutions for other icons. This list of icon names and the corresponding codepoints can help.

Problem 2: missing glyph indicators

After getting rid of the text strings, instead of beautiful icons I got missing glyph indicators.

Turns out that the css file generated by the build process (npm run build followed by cordova build android) expects the font files containing the icon glyphs to be found in android_asset/www/dist/static/css/static/fonts, but they were actually located in android_asset/www/dist/static/fonts.

The fix is to use a Cordova hook script to transform the relevant links in the css file.

There are two possible locations for the hook script.

  1. Drop it into the ./hooks/before_build/ subdirectory.
  2. Specify its location in ./config.xml.

I am not very experienced with the platform, so I chose the first location.
Due to a similar reason (inexperience with node scripts), I wrote it quickly as a /bin/bash shell script.
Here it is in its full glory:

#/bin/bash
echo ============================================
echo Fix paths to font files in Android css files.
echo ============================================
CSSFILES=$1/www/dist/static/css/*.css
echo The relevant css files are: ${CSSFILES}
mkdir -p /tmp/fixcssfiles
rm -v /tmp/fixcssfiles/*
for cssfname in ${CSSFILES} ; do
  #cp ${cssfname} /tmp/fixcssfiles/`basename ${cssfname}`
  #                            # (Save a backup css file)
  sed 's=url(static/fonts/=url(../fonts/=g' < ${cssfname} \
                             > /tmp/fixcssfiles/tmpcssfname
  mv /tmp/fixcssfiles/tmpcssfname ${cssfname}
  echo Procssed ${cssfname}
done
echo ============================================

After adding the script, I got to see the material design icons when running the Android application.

Android unit testing and Mazer Rackham

כבר אמר מייזר רקהאם (“המשחק של אנדר”) שאין מורה כמו האוייב.
נזכרתי בזה במהלך המלחמה שלי בבניית בדיקות יחידה לאפליקציה לאנדרואיד בסביבת הבדיקה של API 24 והלאה.
Mazer Rackham (“Ender’s Game”) said: There is no teacher but the enemy.
I was reminded of this during my war of building unit tests for an Android application in the testing environment of API 24 and later.

I got the ability to work with Heroku using my Debian Stretch system

The other day I found that:

  1. Heroku needs Python 3.6 or later to work (as of June 22, 2018). See: Getting Started on Heroku with Python.
  2. Debian Stretch (Debian Stable as of June 22, 2018) and its backports have only Python 3.5.

The solution was to build a Docker image based upon Ubuntu 18.04, which does have Python 3.6. See the project https://gitlab.com/TDDPirate/heroku_on_debian in GitLab.

July 15, 2018 update:

After I complained about flakiness of Selenium-based tests when the Selenium server is running outside of the Docker container while the application runs inside the container, Udi Oron suggested another way to run Python 3.6 on a Debian Stretch system: use pyenv.

Turns out that pyenv solves the pain point of running Python 3.6 on Debian Stretch without having to use a container. So Selenium-based tests are now stable.

The following is an excellent article about using pyenv:
Pyenv – Python Version Management Made Easier

And the following is a link to the GitHub repository:
https://github.com/pyenv/pyenv

I suspect that pyenv is the reason why people are not in a hurry to backport new Python versions to Debian.

How to visually compare two PDF files? (cont’d)

When I asked the above question in a Telegram group, people proposed also other tools, which I am summarizing below.
Amiad Bareli, Amit Aronovitch, Meir Gil and Yehuda Deutsch – thanks.

  1. ImageMagick compare
  2. matplotlib testing framework – supports also PDF:
    >>> import matplotlib.testing.compare
    >>> matplotlib.testing.compare.comparable_formats()
    ['png', 'eps', 'svg', 'pdf']
  3.  pHash – The open source perceptual hash library.

To get more PDF tips and updates, subscribe to my mailing list

How to visually compare two PDF files?

I have an application written in Python, which uses the ReportLab package for exporting PDF files.

Of course, the application needs to be tested. Among other tests, the PDF export function needs to be tested to ensure that the visual rendering of PDF files did not unexpectedly change.

Since it is possible to create and save an expected-results PDF file using fabricated test data, the above implies the need to compare two PDF files. It turns out that two PDF files created from the same data at two different dates – are different, due to embedded timestamps.

Hence, the need to compare visual renderings of the PDF files. ImageMagick’s convert knows how to convert PDF files into PNG. However, one needs to set the background and remove the alpha channel.

convert knows also to perform bitwise XOR on two image files, but it must be told how to compute the bitwise XOR. This is documented in StackOverflow: Searching for a way to do Bitwise XOR on images.

The script in  https://gitlab.com/TDDPirate/compare_pdfs implements all the above.

To get more PDF tips and updates, subscribe to my mailing list

The “Unable to locate WordPress root directory.” error when trying to update WordPress or install a plugin

If you encounter the “Unable to locate WordPress root directory.” error when trying to update your WordPress installation and/or plugins, and your blog is running on a PHP 7 version, installing the SSH SFTP Updater Support plugin may solve your problem. It worked for me.

Reference: DigitalOcean: WordPress SSH2 Updates, msmarcal’s answer .

Outputting PDFs with your own fonts from your Django application

Do you use reportlab for creating PDF files from your Django application, as instructed in Outputting PDFs with Django?
Do you wish to add your own font files?
Do you need to render Hebrew text?

If yes, the following information will help you.

When installed in a virtualenv, as recommended by good working practices, reportlab searches for font files it uses in your virtualenv/lib/python3.x/site-packages/reportlab/fonts and it is not good idea to add your own font files there.

Instead, add your font files to your_project_root_directory/reportlab_extra_fonts where your_project_root_directory is where your project’s manage.py is located. Add the following to any Python script that uses reportlab (usually views.py), after all regular reportlab import and configuration statements.
# The following configures extra reportlab fonts.
import os
from django.conf import settings
reportlab.rl_config.TTFSearchPath.append(os.path.join(settings.BASE_DIR, 'reportlab_extra_fonts'))
from reportlab.pdfbase import pdfmetrics
from reportlab.pdfbase.ttfonts import TTFont
HEBREW_FONT_NAME = 'your_font_name'
pdfmetrics.registerFont(TTFont(HEBREW_FONT_NAME, 'your_font_file_name.ttf'))
# The above configures extra reportlab fonts.

If you want to properly display Hebrew in your PDF file (the probable reason why you needed to add your own fonts in the first place), you need to convert the text yourself from logical ordering to visual ordering, because reportlab (as of version 3.4.0) does not currently process BiDi text. For this purpose, install the python-bidi package in your virtualenv (using pip install python-bidi) and add the following import statement to your views.py script:
from bidi.algorithm import get_display
Now, get_display() will reorder your BiDi text.

To get more PDF tips and updates, subscribe to my mailing list

A proposed new language construct: do/undo

There are several cases, in which a program has to do operations A,B,C. If an error occurs while doing A, the program needs to undo A. If there is an error while doing B, the program needs to undo B then A. Similarly, an error in C requires undoing of C,B,A in this sequence.
Continue reading “A proposed new language construct: do/undo”