How to visually compare two PDF files?

I have an application written in Python, which uses the ReportLab package for exporting PDF files.

Of course, the application needs to be tested. Among other tests, the PDF export function needs to be tested to ensure that the visual rendering of PDF files did not unexpectedly change.

Since it is possible to create and save an expected-results PDF file using fabricated test data, the above implies the need to compare two PDF files. It turns out that two PDF files created from the same data at two different dates – are different, due to embedded timestamps.

Hence, the need to compare visual renderings of the PDF files. ImageMagick’s convert knows how to convert PDF files into PNG. However, one needs to set the background and remove the alpha channel.

convert knows also to perform bitwise XOR on two image files, but it must be told how to compute the bitwise XOR. This is documented in StackOverflow: Searching for a way to do Bitwise XOR on images.

The script in  https://gitlab.com/TDDPirate/compare_pdfs implements all the above.

The “Unable to locate WordPress root directory.” error when trying to update WordPress or install a plugin

If you encounter the “Unable to locate WordPress root directory.” error when trying to update your WordPress installation and/or plugins, and your blog is running on a PHP 7 version, installing the SSH SFTP Updater Support plugin may solve your problem. It worked for me.

Reference: DigitalOcean: WordPress SSH2 Updates, msmarcal’s answer .

Outputting PDFs with your own fonts from your Django application

Do you use reportlab for creating PDF files from your Django application, as instructed in Outputting PDFs with Django?
Do you wish to add your own font files?
Do you need to render Hebrew text?

If yes, the following information will help you.

When installed in a virtualenv, as recommended by good working practices, reportlab searches for font files it uses in your virtualenv/lib/python3.x/site-packages/reportlab/fonts and it is not good idea to add your own font files there.

Instead, add your font files to your_project_root_directory/reportlab_extra_fonts where your_project_root_directory is where your project’s manage.py is located. Add the following to any Python script that uses reportlab (usually views.py), after all regular reportlab import and configuration statements.
# The following configures extra reportlab fonts.
import os
from django.conf import settings
reportlab.rl_config.TTFSearchPath.append(os.path.join(settings.BASE_DIR, 'reportlab_extra_fonts'))
from reportlab.pdfbase import pdfmetrics
from reportlab.pdfbase.ttfonts import TTFont
HEBREW_FONT_NAME = 'your_font_name'
pdfmetrics.registerFont(TTFont(HEBREW_FONT_NAME, 'your_font_file_name.ttf'))
# The above configures extra reportlab fonts.

If you want to properly display Hebrew in your PDF file (the probable reason why you needed to add your own fonts in the first place), you need to convert the text yourself from logical ordering to visual ordering, because reportlab (as of version 3.4.0) does not currently process BiDi text. For this purpose, install the python-bidi package in your virtualenv (using pip install python-bidi) and add the following import statement to your views.py script:
from bidi.algorithm import get_display
Now, get_display() will reorder your BiDi text.

A proposed new language construct: do/undo

There are several cases, in which a program has to do operations A,B,C. If an error occurs while doing A, the program needs to undo A. If there is an error while doing B, the program needs to undo B then A. Similarly, an error in C requires undoing of C,B,A in this sequence.
Continue reading “A proposed new language construct: do/undo”

A better way to run Android Studio from a Docker container

Did you want to build a Docker container, which will run Android Studio for you, for example because your PC does not run GNU/Linux Ubuntu 14.04 but another Linux distribution?

You can find suitable scripts in the GitHub repository tddpirate/androiddocker

האם רצית לבנות Docker container שיריץ לך את Android Studio, למשל מפני שמחשבך אינו מריץ GNU/Linux Ubuntu 14.04 אלא הפצת לינוקס אחרת?

ניתן למצוא סקריפטים מתאימים בגיטהאב: tddpirate/androiddocker

The Python module for file type identification, called ‘magic’, is not standardized

I found the hard way that the API exported by the Python module ‘magic’ differs among different versions of the module.

The version installed when installing the Debian package ‘python-magic’ expects the following API:

import magic
mymagic = magic.open(magic.MAGIC_MIME_TYPE)
mymagic.load()
mtype = mymagic.file(inpfname)
print("The MIME type of the file %s is %s" % (inpfname,mtype))

The version installed using ‘pip install python-magic’ expects the following API:

import magic
mymagic = magic.Magic(mime=True)
mtype = mymagic.from_file(inpfname)
print("The MIME type of the file %s is %s" % (inpfname,mtype))

The following code allows the rest of the script to work the same way with either version of ‘magic’:

import magic
def build_magic():
  try:
    mymagic = magic.open(magic.MAGIC_MIME_TYPE)
    mymagic.load()
  except AttributeError,e:
    mymagic = magic.Magic(mime=True)
    mymagic.file = mymagic.from_file
  return(mymagic)
mymagic = build_magic()
mtype = mymagic.file(inpfname)
print("The MIME type of the file %s is %s" % (inpfname,mtype))

Learn to develop for the Android, contribute to Free Software and advertise yourself

During the last few weeks I developed an Android application for helping people prepare for the Israeli driving theory test.  The application uses the questions database available from http://data.gov.il/dataset/249 (and the corresponding ones for Russian and Arabic).  It was a nice and fun exercise, which helped me master more parts of the Android development platform.

The application is now at version 0.1 and is working.  Its look and feel is not polished, yet; and some missing features have been identified by comparing it to three other applications available from Google Play doing the same thing.

Today the source code has been released under the GPLv3 and is available from https://github.com/tddpirate/teuria and if you are looking for an Android developer with few weeks worth of experience and known quality of work, you may have a look at the above.

Installing Slackware (a Linux distribution) in a VirtualBox virtual machine

After not having dealt with Slackware for several years, I needed to create a virtual machine with Slackware.  I had to solve two problems, and am writing below how they were solved for the benefit of future Slackware based virtual machine builders.

  • Slackware version: 13.37 for 32-bit x86, installed from DVD
  • VirtualBox version: 4.1.18 r78361, Debian Squeeze amd64

Virtual machine parameters:

  • Operating system: Linux
  • Version: Linux 2.6 (32-bit being implied) – the combobox list had no entry for Slackware.
  • Base memory: 1.5GB
  • Hard disk: 30GB
  • Host I/O caching: on (to silence a warning).

Problem 1
When trying to boot from the DVD in order to install Slackware 13.37, the hugesmp.s kernel failed to boot.
I found the following bug reports:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/246067
http://www.linuxquestions.org/questions/slackware-14/kernel-panic-on-virtual-box-installation-its-driving-me-nuts-901223/
https://forums.virtualbox.org/viewtopic.php?f=3&t=44598

Some of the above bug reports had the suggestion to turn on VT-x/AMD-V acceleration.  However it was already turned on in my virtual machine.  I tried several combinations of flags in Settings/System/Processor and Settings/System/Acceleration but of no avail.

The problem was solved by changing the IDE controller type from PIIX4 to PIIX3 in Settings/Storage/IDE Controller/Type.

Problem 2
The initial reboot after installation failed.  From the stack trace it seemed that the problem was due to unknown kernel options.  I found some bug reports, but they were not directly related to this.

I successfully rebooted the virtual machine when entering the following as LILO prompt:

Linux pci=routeirq acpi=off pci=noacpi

Then, the problem was solved by editing /etc/lilo.conf as follows.  The line

append=” vt.default_utf8=0″

was changed to

append=” vt.default_utf8=0 pci=routeirq acpi=off pci=noacpi”

And then /sbin/lilo was run (without arguments).

Computerized elections in Israel – the threat is still there

Time has passed since the Nov. 2007 council elections (with no publicity to the results of the pilot of computerized elections), and since the Dec. 2008 Taldor fiasco.

And it turns out that in their infinite wisdom, the Israeli government decided to continue to cater to the impatient, “want to know the results right NOW!!!” segment of the electorate in future elections.  Having learned from the experience of other countries, a system based upon near-field RFID tags was chosen for development and deployment.

However, according to http://www.eng.tau.ac.il/%7Eyash/evoting-relay-rfid2010.pdf, the chosen system suffers from serious vulnerabilities.

It seems that the real agenda of the politicians here, like everywhere, is still to get the populace to accept ballot systems, which allow the elections to be rigged.

Web 2.0 Best Practices

Suppose you have been contacted by someone with a great Web 2.0 idea and he wants you to join his startup.
You need to know whether he knows what he is talking about.
The following checklist may help you tell the clueful apart from clueless.
I hope people will be able to contribute advice concerning each item in the checklist as well as more items I missed.

  1. Dealing with bad content:
    • Spam
    • Trolling
    • Off-topic user-contributed content
  2. Vandalism (and in general, content backup/restore).
  3. Legal:
    • Acceptable use guidelines
    • Copyright violations and other issues
  4. Content ownership/lockdown policies – will a contributor be able to export his contributions into file/s in his own PC?
  5. How will the network effect be overcome (if another Web 2.0 site already exists serving the same need, how to get people to use your Web site instead of the other site, if they already have stuff).
  6. Business model (i.e. how to actually get people to pay for the stuff).
  7. Scaling with demand (nowadays, thanks to cloud computing services availability, the required scaling is not that of servers but that of customer service personnel and maybe other critical resources).
  8. Are there standards (such as XML schema) relevant to the kind of content to be served by the site?