Updated jwhois.conf File for CentOS for New gTLDs

The whois command on CentOS 6.x and 7.x doesn’t handle queries for many domains in new Top Level Domains (TLDs) that were added by ICANN in the last few years.

Domains from many of these new TLDs are selling as cheap as $0.99 a pop, making them attractive to snowshoe spammers who create them in large numbers. As a spam researcher, I see lots of new spam domains from TLDs such as .xyz, .online, .top. .club, .services, .win, .site, .bid, .life and .trade.

WHOIS is an important tool for me to track the domain registrants. CentOS uses jwhois as its WHOIS client, which relies on a configuration file to tell it what servers to query for detailed information. The configuration file that comes with recent CentOS versions is woefully out of date.

I have gone through the currently existing TLDs and counted 466 of them that are not supported by jwhois but appear to have a valid WHOIS server. I have been able to verify for about half of these TLDs that the WHOIS server works and have added them to my configuraion file, which you can download here.

Many of the rest of the new TLDs are hosted on Neustar, which performs rate limiting on lookups. Because of that I didn’t fully verify functioning of all those hosts, but I verified that CNAMEs exist for the WHOIS hosts that redirect to Neustar WHOIS servers and tested a small sample of those TLDs.

Getting Rid of the EMUI Launcher on the Huawei P9 Lite

Last time I switched mobile provider here in Japan, I signed up for a contract that included a Huawei P9 Lite. My biggest grip about it is its non-standard EMUI interface that runs on top of Android 6.0.1.

Previously I was using a Nexus 5, which had worked OK for me, though the picture quality of its camera was rather mediocre. One nice thing about the Nexus 5 was that it runs stock Android, with no customization. Its user interface is identical to that of my other phone, a Nexus 6P.

I really prefer stock Android without OEM customization. For one, stock Android means you can get version upgrades sooner and for longer (or at all!).

I found the EMUI launcher confusing. For example, I did not see any easy way to launch an app that didn’t have a desktop link.

It’s possible to switch from EMUI to the standard Google launcher. Here are the steps I performed:

1) Install “Google Now Launcher” via Play Store.

2) Swipe down, select Shortcuts and then Settings

3) Enter “def” into the search box at the top (may have to scroll up first)

4) Select “Default app settings”

5) Select “Launcher” and pick “Google” instead of “Huawei Home”. Ignore the warning that tries to scare you into sticking with EMUI (you can always change back by following the same steps and selecting “Huawei Home” again).

6) There you go!

The irritating long push home button

Another irritation that seemed to happen more on the Huawei than on my other phones was the Google screen that pops up (seemingly randomly) when I just try to go to the home screen. It has a “Want answers before you ask?” prompt at the bottom and a Google search box with voice search option at the top. I really don’t need this screen because the standard Android home screen already has a Google search bar at the top. I’d rather have the home screen with all my app shortcuts come up reliably whenever I push the Home button!

It took me a while to figure out that this Google search screen comes up on what the phone thinks is a long push of the Home button, which has a different meaning from a regular short tap. If that happens, just tap again and it will go to the home screen. Or just make it your habit to double tap the home screen to go to the home screen, then this should never happen 🙂

Adding Free SSL Certificates for HTTPS To Your Websites

I recently received a warning email from Google:

“Starting October 2017, Chrome (version 62) will show a ‘NOT SECURE’ warning when users enter text in a form on an HTTP page, and for all HTTP pages in Incognito mode.

The recommended solution was to migrate the affected website(s) to HTTPS. This requires an SSL certificate. There are many companies selling those for hundreds of dollars. I didn’t really want to spend that money.

It turns out there is a free alternative: The Let’s Encrypt project (https://letsencrypt.org/) provides free SSL certificates with just enough functionality to run SSL with current browsers. It also provides automated tools that greatly assist you in obtaining and installing those certificates.

I had a default SSL host configured on my Apache 2.4 installation (inherited from a different server running Ubuntu) that I had to manually remove.

Then, when all virtual hosts only had port 80 (HTTP) enabled, I could run the certbot tool as root:

# certbot --apache

It enumerates all host names supported by your Apache installation. I ran it repeatedly, for each domain and the corresponding www. host name (e.g. joewein.net, www.joewein.net) in my installation and verified the results, one at a time. It will create a new virtual host file in /etc/httpd/hosts-enabled for those hosts for port 443 (HTTPS). I appended the content of that file to my existing port 80 (HTTP) virtual host file in /etc/httpd/hosts-available for that host name and deleted the new file created by certbot. That way I can track all configuration details for each website for both HTTP and HTTPS in a single file, but this purely a personal choice.

All it takes is an Apache restart to enable the new configuration.

You can test if SSL is working as expected by accessing the website with a browser using https:// instead of http:// at the start of the URI.

If you have iptables rules for port 80, you may want to replicate those for port 443 or the certificate generation / renewal may fail. Also, you want to make sure that SSLv3 is turned off on your Apache installation, to protect against the POODLE vulnerability. This required the following setting in ssl.conf:

/etc/httpd/conf.d/ssl.conf:SSLProtocol all -SSLv2 -SSLv3

The free certificates will expire in 90 days, but it’s recommended to add a daily cron job that requests renewals so that an updated key will be downloaded after 60 days, long before the old key expires. Once that is in place, maintenance of SSL keys will be totally automatic.

JWHOIS uses 100% of CPU on CentOS

Occasionally we hit a bug where the ‘whois’ command hangs on one of our CentOS servers and goes CPU-bound. This has been happening on several CentOS versions, including 6.8. Specifically, this is a problem in jwhois, the whois client included in CentOS.

Apparently, CentOS (and RHEL, on whose source code it’s based) is missing a number of fixes that have been added to other Linux versions including Fedora over the last couple of years. So the problem is actually known and a fix has been available for years, it’s just not included in the product.

Comparing the change logs for jwhois between CentOS and Fedora, everything matches up to and including build 4.0-18 in September 2009, but then the two diverge.

On Jan 26, 2010, Fedora received a fix (“Use select to wait for input (patch by Joshua Roys <joshua.roys AT gtri.gatech.edu>)”) for a new 4.0-19 build that resolved bug #469412 for precisely this issue. There are many more changes in Fedora’s jwhois after that, unlike its RHEL and CentOS equivalent, which in all the years since then received only a single update. This is also called 4.0-19, but it was made on Jun 23, 2011 and it includes only two fixes for unrelated issues that were fixed in Fedora’s jwhois updates 4.0-24 (Dec 20, 2010) and 4.0-26 (Mar 15, 2011), but not the earlier select fix or fixes for any of the other issues. CentOS is missing the “jwhois-4.0-select.patch” and that’s why WHOIS hangs.

Google broke Picasa uploads

Having used Google Picasaweb for picture-hosting for many years, Google’s transition to Google Photos has been a frustrating experience. The original Picasaweb has always worked better for me than its supposed replacement. Several friends of mine who had also been using Picasaweb have already switched to other services, including Facebook.

The latest nail in the coffin came a few days ago, when the Picasa 3 application failed to upload new albums to Google Photo. The error message was:

Error: Request failed
Click here to View Errors

The link revealed that it was a server error:

HTTP Error 400 – https://picasaweb.google.com/post?tok={long-token-here}[156]

It looks like someone at Google broke the upload servers. When they announced the transition a year ago, Google wrote:

Desktop application
As of March 15, 2016, we will no longer be supporting the Picasa desktop application. For those who have already downloaded this—or choose to do so before this date—it will continue to work as it does today, but we will not be developing it further, and there will be no future updates.

For now, the workaround appears to be to use “File | Export Picture to Folder” in the Desktop application to create files no wider than 1600 pixels (below the limit for unlimited free uploads) and then upload those file sets to Google Photo using its web interface.

At the moment it is still possible to share Google Photo images in blog and forum posts but for how much longer? First you must share the album, for example by clicking on the “share” icon in the album in Google Photo, then “Get Link” to generate a link, which you don’t actually have to use. Then you can view an image, right-click on it and select “Open image in new tab”. The URI above the new tab that opens can be used for embedding images in blogs and forums. If or more precisely, when Google also breaks this feature then Google Photos will become unusable for me.

I am looking for a good solution to be hosted on one of my own servers that will replace Google Photos, without size limits and without any hassle for resizing images for public sharing, that will let me control who can see what images like the old Picasaweb did.

Nexus 6P Flashing Charge Icon

Recently the USB-C quick charger that came with my Huawei Nexus 6P appeared to stop working. I normally leave the phone charging over night, but one morning I found its battery charge was low and it hadn’t been charging. When I disconnected and reconnected it to the cable (which is reversible with USB-C connectors on both ends), the lightning bolt inside the battery charge indicator kept blinking (flashing on and off), rather than being solid on as it normally would while the device is charging.

Disconnecting and reconnecting the device or unplugging and reconnecting the charger to the wall socket made no difference whatsoever. Reversing the cable direction did not help either.

My only way to still charge the phone was to use a USB-A to USB-C cable that draws power from a PC USB socket, which is a much slower way to charge the phone. So I decided that the charger must have failed after less than a year of use. I already started looking for USB-C quick chargers on Amazon (they exist but are much more pricey than USB-A chargers), but didn’t order one yet.

Today I decided to Google for the problem and found others who had the exact same issue. It turned out that simply powering down the phone and powering it up again will fix the problem: Yepp, it will charge again!

I’m not sure what the fundamental issue is, but it seems I won’t have to rush out and buy a new charger (which wouldn’t have helped anyway!), as the issue is on the phone side.

If a simple phone reboot fixes it and it doesn’t happen too often, I guess I can live with that. The Nexus 6P has worked great for me so far.

Removing “Suggested for you” category from Google News

In November 2014 Google added a “Suggested for you” category to Google News which includes articles on topics that it thinks are of interest to you.

Now, Google has some pretty smart algorithms for rating websites with content that people are searching for to give you the most relevant information, but even 18 months after this particular feature was launched in Google News I find its results pretty poor. They merely distract from news items I am really interested in.

For example, if you search for the lyrics of a song by a particular artist because a family member asks you to then you’re likely to be seeing articles about that artist popup in your feed daily for weeks and months…

The solution is easy:

  • Open Google News while you’re logged in to Google
  • Click on the “Personalize” button on the top right
  • Look for a “Suggested for you” category with a slider next to it.
  • Move the mouse to that category and click the trash can that appears next to the slider
  • Save the changes

Should you ever want to restore the category, you can click “Personalize” and “Reset” to undo all personalization.

Upgrading to 14.04.1 LTS or If It Ain’t Broke, Don’t Fix it

I should have left my Ubuntu 12.04 LTS well alone. Yes, it is over 2 years old, but it worked rock solid and I’ve been good about installing updates on it.

I don’t know what devil rode me last Friday, but when the system informed me that an upgrade to 14.04.1 LTS was available, I went ahead and gave it a try. I should have known better.

When the upgrade finished many hours later, POP access to the dovecot server was no longer working and rsync using modules was broken (rsync daemon not running). I had accepted all the defaults to keep existing configuration files during the upgrade. It turned out that dovecot needed some changes for namespace inbox:

namespace inbox {
...
inbox=yes
}

The rsync daemon needed to be manually enabled again via

sudo vi /etc/default/rsync

RSYNC_ENABLE=true

Hopefully I won’t stumble across more problems that will need fixing, but the experience was a reminder not to needlessly mess with a working system.

Google Maps Engine brings back custom routes

Last year I stopped updating Google Maps on my Android phone because Google had dropped important functionality with Google Maps 7.x. Google Maps 6.x for Android was a great tool for following mapped routes on long bicycle rides, especially randonnes of 200 km and more. After an update I had to revert to Google Maps 6.x to get it back. This also meant I could no longer allow Android to install all available updates in one go. I always had to manually confirm all updates except Maps to not lose 6.x again.

Finally Google has brought this functionality back. There are still missing bits, but at least the product seems usable again for my purposes.

On Android there is an app called Google Maps Engine, which supports loading custom maps. Select “Open a map” in the menu. You’ll get a list of maps created by you or shared with you.

This menu can be populated from a desktop machine. There you can import existing maps created for Maps 6.x. Go to https://mapsengine.google.com/map/ and select “Open a map” (you need to be logged in to your Google account). Select “Classic My Maps”. You’ll be able to select one of your existing maps and import it in to Maps Engine. After that it will become available to the Google Maps Engine app on your Android and you can use it for navigation. The route will show as a blue line and special locations, such as my brevet PCs (“points de controle”, route check points) will show marked with a pin.

One drawback of Maps Engine on the Android compared to the old Google Maps 6.x is that it doesn’t seem to support displaying a ruler on a map yet. Thus when you zoom in or out you won’t be able to tell how far you are from any point you see on the map, whether one cm on the screen corresponds to 100 m or 10 km on the map. This is the same problem that Google Maps 7.x had when it was launched last year. Hopefully it will be fixed soon. Still, it is disconcerting that Google misses out such basic functionality when launching products. Are all their eyes on monetization these days?

Adding sudo on Debian Linux

For a long time I had been using the sudo command on Ubuntu and other Linux versions, but my main server did not have it installed. I always had to use ‘su’ with the root password to do be able to do administrative jobs. It turns out it was really easy to fix. Simply follow these steps as root (using your actual user name in place of jsmith):

apt-get install sudo
adduser jsmith sudo

This installs the sudo package, creates a sudo user group and the /etc/sudoers configuration file. It then adds your user to the user group sudo, which per the default /etc/sudoers file is permitted to run sudo.

Note that these changes do not take effect for any ssh sessions already open. If you have a running session logged in as the user you just added to the sudoers list and you attempt to use sudo from there, it will ask for your password and then fail with this error message:

jsmith is not in the sudoers file. This incident will be reported.

The fix is simple: log out and log back in again. On the new login, the new configuration will be picked up and you will be able to use sudo as intended.

If you would like to do multiple commands from sudo like you could from su, it’s very easy. Simply use sudo to launch a copy of bash and exit after you’re done:

sudo bash